Linux Console Scrollback
We refer here to the GNU/Linux text console - which is not what you have in a terminal window of a X windows manager or desktop environment. There is so much confusion on this point that I'm going to go into trivially-nitpicky detail: a text console is what you get when you press 'Alt-Ctrl-F1' or 'F2' and so on while the focus is on the desktop.
0. What are we talking about?
You are in a text console and text flies by. There is a way to recall it and have a look by pressing 'Shift-PgUp' or 'Shift-PgDn' or possibly other keys if you have modified the default keymap (no, not the X keymap.) You are then using the scrollback buffer (scrollbuffer, for short.)
When trying to put the scrollbuffer to good use, a couple of things quickly become apparent:
- only part of the boot messages can be recalled
- the file /var/log/messages catches a lot less of what you are looking for
At this point, the web search starts and preliminary results emerge:
- scrollback issues are very old in Linux, they are traceable to the early 90s.
- You can forget everything published prior to kernel version 2.6.x.
Indeed, you have to be selective with what you read: if you find any advice prior to 2.6.x it is likely to confuse you and lead you astray. Believe me - I have put in the time so you don't have to.
The scrollback behaviour is determined in 'vc.c', a nasty little file in the kernel source, where 'vc' stands for virtual console.
First, note that when you switch consoles, the scrollbuffer is lost - so the entire size of the scrollbuffer is available for the current console. (Here especially, there is a pile of junk from pre-2.6.x times.)
The default scrollbuffer size is 32K. With that, you get 4 key presses - each scrolling half a screen, 25 lines per screen. Figure it out: that's about 50 lines. You will not get more than that, even if you have 30 or 50 lines per screen.
However, 50 lines are just a small fraction of what flies by at boot time. So two questions need to be answered:
- How can the scrollbuffer be increased?
- Why are the log files not catching some of the messages you see at boot time?
1. How can the scrollback buffer be increased?
One solution to (1) would be to change the default size in the kernel source and recompile. Let me assume you are as inclined to do so as I am (NOT) and would rather look for something more flexible.
Well, there is a way - and it goes through a device called framebuffer console, 'fbcon' for short. This device has a documentation file by the name of 'fbcon.txt'; if you have installed the kernel documentation, you have it. Sniff it out somewhere in the '/usr/share' branch (I am not stating a path because it may differ from distro to distro.) Anyway, you can get it here as a single file:
At this point, sorry: detour! We have got to talk briefly about the framebuffer.
A framebuffer is a buffer between the display and the graphic adapter. The beauty of it is that this buffer can be manipulated: it allows for tricks that would be impossible with the adapter acting directly on the display.
One of the tricks concerns the scrollbuffer; for instance, you can tell the framebuffer to allocate more memory for it.
This is done via the kernel boot parameters. First, you request a framebuffer; then you request a bigger scrollbuffer.
The following example refers to GRUB but can be easily adapted to LILO. In GRUB's 'menu.lst', find the appropriate kernel line, and:
- delete option 'vga=xxx', if present
- append option 'video=vesabf' or whatever fits your hardware
- append option 'fbcon=scrollback:128'
The kernel line would then look something like the following;
kernel /vmlinuz root=/dev/sdb5 video=radeonfb fbcon=scrollback:128
Why are we deleting the option 'vga=xxx'? Because it may conflict with the video option. On my ATI adapter, I cannot change the scrollbuffer if 'vga=xxx' is present. This may not apply in your case.
The options just listed do work but what if you want more lines and a smaller font on the screen? You used to have that with the 'vga=xxx' option - and that option is gone.
Take heart - it can still be done with fbcon parameters as detailed in 'fbcon.txt' (but not detailed here.) Under Ubuntu and Debian there is a more comfortable way:
It makes sense to use this command because it also adjusts the initial ram disk (initrd) to your new settings.
2. Pushing up the cap
The option 'fbcon=scrollback:128' gives you 12-13 key presses to traverse the scrollbuffer. That is approx. 150 lines - quite a bit, but possibly still not enough to catch all the boot messages. Can we say more than 128, e.g. 256?
The cap of 128 is set in 'vc.c'. If you want more, you can edit it and recompile.
For myself, I decided that it is not worth it. Before that I almost believed that I found a higher cap - I quote from 'fbcon.txt':
fbcon=scrollback:<value>[k] ... The 'k' suffix is optional, and will multiply the 'value' by 1024.
I rushed to try it out... and whatever the author was thinking, rest assured that 128 and 128k will give you exactly the same result. The default cap is 128KB of memory and that's the end of it.
Finally, note that using an extended scrollbuffer implies writes to both the graphic adapter hardware and the buffer in memory. If you use the default of 32KB, there is only a write to the hardware. In practice, I was unable to notice any slowdown.
3. What's missing in the logs?
In Ubuntu and Debian and other distros all system messages are logged to the file '/var/log/messages'. This applies even if the log service (daemon) is not the old syslog but syslog-ng ('ng': "new generation").
In both cases, you can view the messages by issuing 'dmesg' at the command prompt. It still doesn't help, though: you saw messages at boot time and they are certainly not in the log. How come?
It is a feature, not a bug! The messages logged to the file come from various subsystems of the operating system. The contributing subsystems are referred to as 'facilities', and there are 8 of them. If, at boot time, scripts or applications are run that do not belong to any facility, you will see their messages rushing past your eyes - but nothing will be written to the log file!
As an example you will not see the messages produced by 'loadkeys' (openSUSE) or 'consolechars' (Ubuntu and Debian) when loading keymaps at boot time. Another example: when running a text console editor you can scroll back its display (including colour) using the scrollbuffer but what the editor produced will never go to the system log file. The reason is, of course, that 'loadkeys', 'consolechars', and the editor do not belong to any facility.
Can this be changed? Yes - "just" customize and recompile those applications you want logged. And/or modify the boot scripts. And force logging of messages outside the system facilities.
My bet is you are not going to do it.
(And your most likely response is: "nothing is missing from my log files." Oh well...)
A. N. Onymous has been writing for LG since the early days - generally by
sneaking in at night and leaving a variety of articles on the Editor's
desk. A man (woman?) of mystery, claiming no credit and hiding in
darkness... probably something to do with large amounts of treasure in an
ancient Mayan temple and a beautiful dark-eyed woman with a snake tattoo
winding down from her left hip. Or maybe he just treasures his privacy. In
any case, we're grateful for his contribution.
-- Editor, Linux Gazette