By Bri Hatch.
Summary: Upgrading your software is a constant task. But when does it require a reboot, and when can you get by without?
Every operating system in the world requires software updates from time to time. If you're used to the world of Windows, you have probably become accustomed to a simple fact of windows life: you need to reboot your machine after any system change, such as upgrading software, installing patches, installing new software, moving your mouse, and so on. In the Linux world, this is not the case. You can keep your machine running for months or years without rebooting as long as you properly handle software upgrades.
There are four main types of software upgrades that you could require. I'll point out the actions you should take after performing an upgrade to be sure that your machine is running smoothly and you're not accidentally running old versions.
User-level software
User level software are your standard programs in places like
/bin and /usr/bin that are executed
from the command line interactively, from system scripts,
cron jobs, you name it. Things like awk,
emacs, grep, mutt, passwd, perl,
ping, vi, and so on. These programs are run
for short periods of time, rather than hanging around forever like a
webserver.
These programs, since they are not running continually, do not require
any reboot of the machine after an upgrade. The next time they're called,
the new version will automatically be run. On the oft chance you have
extremely long lived processes (say you've had mutt
running on a screen session since the beginning of time) then
you should stop and restart them, but this is rare.
Daemon software
Daemon software is anything that runs continually unattended, such as
a webserver, mail server, database server, network time server, log analyser,
etc. Often you can identify these by looking at their parent process ID
in ps output - it will usually be '1', the init process. For example:
$ ps -fC syslog-ng UID PID PPID C STIME TTY TIME CMD root 221 1 0 Jun23 ? 00:04:53 /sbin/syslog-ng
These processes are usually controlled by a script in /etc/init.d.
After you upgrade this sort of software, you should stop and restart the daemon.
For example
# /etc/init.d/syslog-ng restart # /etc/init.d/apache restart or # /etc/init.d/ntpd stop # /etc/init.d/ntpd start
Doing this after an upgrade assures your daemon is the newly upgraded version. The service will be down for a brief moment while it's restarting, but that's small change compared to the time required for a reboot of the computer.
Inetd-style Daemon software
Some daemon software, particularly network daemons, is started by
super servers like inetd, xinetd, or Bernstein's
tcpserver. Each time a client connects to one of these
services, the super server will accept the inbound connection and launch
a program to handle it, for example an IMAP or POP server. These programs
are only running for the time that the connection is active, after which
they stop. For example, here's a snippet of ps output
which shows a POP server that's controlled by xinetd:
UID PID PPID C STIME TTY TIME CMD root 1128 1 0 Jun23 ? 00:00:25 xinetd -stayalive -reuse -pidfil ryan 10169 1128 1 12:30 ? 00:00:01 [ipop3d] reegen 32015 1128 1 12:14 ? 00:00:10 [ipop3d] maddie 32015 1128 1 12:31 ? 00:00:05 [ipop3d] chris 32015 1128 1 12:28 ? 00:00:03 [ipop3d]
All the [ipop3d] processes were started by
xinetd and when they are done servicing the request,
they will terminate.
If you were to upgrade the ipop3d software, the
processes that are still running would be the vulnerable version,
but all new connections will be the bug free version. To stop
the buggy versions rudely, you could simply
# killall ipop3d
This will stop the old version, and the clients would need to reconnect. Again, no reboot is needed.
Libraries
A library is a file that contains compiled code that can be used
by more than one program. Most Linux programs are compiled to use
shared libraries, as opposed to being compiled 'statically', which would
mean that all the code is contained in the binary itself. Using existing
libraries is good - it means that should a bug be found in the implementation
of printf, the library needs to be upgraded, but the rest of
the software doesn't need to be recompiled.
Different Linux distributions may handle library upgrades slightly differently, however they all do share one common feature: the existing library is deleted, and the new one is installed. The problem is that any program that is running has already mapped the old buggy library, so it keeps using the old one, even though the new one is available.
Let's take a look at this. Say on my Debian system I have slapd
(the OpenLDAP server) running. Turns out, there's an update for the ldap
shared libraries -- the libldap2[1] package -- that I need to apply.
$ dpkg -L libldap2 /usr/lib/libldap.so.2.0.15 /usr/lib/libldap_r.so.2.0.15 /usr/lib/libldap.so.2 /usr/lib/libldap_r.so.2
So, it looks like the current (buggy) version has files named /usr/lib/libldap*. Let's see what programs are using these:
$ lsof /usr/lib/libldap* COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME slapd 29401 root mem REG 3,5 159648 350748 /usr/lib/libldap_r.so.2.0.15 slapd 29402 root mem REG 3,5 159648 350748 /usr/lib/libldap_r.so.2.0.15 slapd 29403 root mem REG 3,5 159648 350748 /usr/lib/libldap_r.so.2.0.15 slapd 29445 root mem REG 3,5 159648 350748 /usr/lib/libldap_r.so.2.0.15
Ok, slapd is using those libs, unsurprisingly. Let's upgrade and see what happens afterwards:
# apt-get install libldap2 ... upgraded package installs ... $ lsof /usr/lib/libldap* COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME slapd 29401 root mem DEL 3,5 350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new slapd 29402 root mem DEL 3,5 350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new slapd 29403 root mem DEL 3,5 350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new slapd 29445 root mem DEL 3,5 350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
What happened here? The way the Debian upgrade worked was to rename the old libraries to
oldname.dpkg-new, then install the new versions and delete the old
ones. Slapd was already using the old ones, and thus those files are still available
to it, even though you can't see them in the directory itself -- notice "DEL" in the
"TYPE" column, and the empty "SIZE" column.[2].
In order to have slapd start using the new libraries you should -- you guessed it -- restart
slapd
# /etc/init.d/slapd restart
Again, no reboot necessary, but there will be momentary downtime as the program restarts.
The Kernel
So you want to upgrade the kernel to patch the most recent ptrace bug, for
example? Great idea. Does this require a reboot? You betcha. No two ways about it,
to run a new kernel you need to reboot. Sorry for the bad news.
Summary
So, when do you need to reboot? Technically, the only time you need to reboot is if you're installing a new kernel. Any other process can be stopped and restarted without much worry.[3] However you may still want to schedule a reboot for a convenient time anyway, perhaps at 3am on a weekend. This allows you to test any new changes and make sure that the computer comes up again properly, and have time to fix it when it's not much needed. You'd much rather test then, than have a power outage take down your machine while you're on vacation and find that it won't start up your software properly.
NOTES:
[1] I've
taken the liberty of trimming out some of the extraneous files to make this
explanation cleaner. There are actually more files in the libldap2
package.
[2] The Linux kernel won't actually remove a file from disk until all files using it close the file, and all hard links to the file on disk are removed from the filesystem. You might want to check out a previous article about finding deleted files at http://www.hackinglinuxexposed.com/articles/20020507.html
[3] Ok, /sbin/init doesn't restart in
the normal way - it requires a telinit u to restart, but when's the last time
you needed to upgrade init?
Bri Hatch is Chief Hacker at Onsight, Inc and author of Hacking Linux Exposed and Building Linux VPNs. He changes his bio in these newsletters every week, and wonders if anyone notices. Bri can be reached at bri@hackinglinuxexposed.com.
Copyright Bri Hatch, 2003
This is the April 09, 2003 issue of the Linux Security: Tips, Tricks, and Hackery newsletter. If you wish to subscribe, visit http://lists.onsight.com/ or send email to Linux_Security-request@lists.onsight.com.