<chapter id="backups-intro">
<title>Backups</title>

	<blockquote><para><literallayout>
Hardware is indeterministically reliable. 
Software is deterministically unreliable.
People are indeterministically unreliable.
Nature is deterministically reliable.
</literallayout></para></blockquote>

	<para> This chapter explains about why, how, and when to make
	backups, and how to restore things from backups.</para>

<sect1 id="backups">
<title>On the importance of being backed up</title>

	<para> Your data is valuable.  It will cost you time and effort
	re-create it, and that costs money or at least personal grief
	and tears; sometimes it can't even be re-created, e.g., if it
	is the results of some experiments.  Since it is an investment,
	you should protect it and take steps to avoid losing it.  </para>

	<para> There are basically four reasons why you might lose data:
	hardware failures, software bugs, human action, or natural
	disasters.  Although modern hardware tends to be quite reliable, it 
	can still break seemingly spontaneously.  The most critical piece
	of hardware for storing data is the hard disk, which relies on
	tiny magnetic fields remaining intact in a world filled with
	electromagnetic noise.	Modern software doesn't even tend to
	be reliable; a rock solid program is an exception, not a rule.
	Humans are quite unreliable, they will either make a mistake, or
	they will be malicious and destroy data on purpose.  Nature might
	not be evil, but it can wreak havoc even when being good.  All in
	all, it is a small miracle that anything works at all.	</para>

	<para> Backups are a way to protect the investment in data.
	By having several copies of the data, it does not matter as much
	if one is destroyed (the cost is only that of the restoration
	of the lost data from the backup).  </para>

	<para> It is important to do backups properly.	Like everything
	else that is related to the physical world, backups will fail
	sooner or later.  Part of doing backups well is to make sure
	they work; you don't want to notice that your backups didn't work.
	Adding insult to injury, you might have a bad crash just as
	you're making the backup; if you have only one backup medium,
	it might destroyed as well, leaving you with the smoking ashes
	of hard work.
	Or you might notice, when trying to restore, that you forgot to
	back up something important, like the user database on a 15000
	user site.  Best of all, all your backups might be working
	perfectly, but the last known tape drive reading the kind of
	tapes you used was the one that now has a bucketful of water
	in it.	</para>

	<para> When it comes to backups, paranoia is in the job
	description.  </para>

</sect1>

<sect1 id="backup-media">
<title>Selecting the backup medium</title>

	<para> The most important decision regarding backups is the choice
	of backup medium.  You need to consider cost, reliability, speed,
	availability, and usability.  </para>

	<para> Cost is important, since you should preferably have
	several times more backup storage than what you need for the data.
	A cheap medium is usually a must.  </para>

	<para> Reliability is extremely important, since a broken
	backup can make a grown man cry.  A backup medium must be able
	to hold data without corruption for years.  The way you use the
	medium affects it reliability as a backup medium.  A hard disk
	is typically very reliable, but as a backup medium it is not
	very reliable, if it is in the same computer as the disk you
	are backing up.  </para>

	<para> Speed is usually not very important, if backups can be done
	without interaction.  It doesn't matter if a backup takes two
	hours, as long as it needs no supervision.  On the other hand,
	if the backup can't be done when the computer would otherwise
	be idle, then speed is an issue.  </para>

	<para> Availability is obviously necessary, since you can't
	use a backup medium if it doesn't exist.  Less obvious is the
	need for the medium to be available even in the future, and on
	computers other than your own.	Otherwise you may not be able
	to restore your backups after a disaster.  </para>

	<para> Usability is a large factor in how often backups are made.
	The easier it is to make backups, the better.  A backup medium
	mustn't be hard or boring to use.  </para>

	<para> The typical alternatives are floppies and tapes.
	Floppies are very cheap, fairly reliable, not very fast,
	very available, but not very usable for large amounts of data.
	Tapes are cheap to somewhat expensive, fairly reliable, fairly
	fast, quite available, and, depending on the size of the tape,
	quite comfortable.  </para>

	<para> There are other alternatives.  They are usually not very
	good on availability, but if that is not a problem, they can
	be better in other ways.  For example, magneto-optical disks
	can have good sides of both floppies (they're random access,
	making restoration of a single file quick) and tapes (contain
	a lot of data).  </para>

</sect1>

<sect1 id="backup-tools">
<title>Selecting the backup tool</title>

	<para> There are many tools that can be used to make
	backups.  The traditional UNIX tools used for backups
	are <command>tar</command>, <command>cpio</command>, and
	<command>dump</command>.  In addition, there are large number
	of third party packages (both freeware and commercial) that
	can be used.  The choice of backup medium can affect the choice
	of tool.  </para>

	<para> <command>tar</command> and <command>cpio</command> are
	similar, and mostly equivalent from a backup point of view.
	Both are capable of storing files on tapes, and retrieving
	files from them.  Both are capable of using almost any media,
	since the kernel device drivers take care of the low level
	device handling and the devices all tend to look alike to user
	level programs.  Some UNIX versions of <command>tar</command>
	and <command>cpio</command> may have problems with unusual files
	(symbolic links, device files, files with very long pathnames, and
	so on), but the Linux versions should handle all files correctly.
	</para>

	<para> <command>dump</command> is different in that it reads
	the filesystem directly and not via the filesystem.  It is
	also written specifically for backups; <command>tar</command>
	and <command>cpio</command> are really for archiving files,
	although they work for backups as well.  </para>

	<para> Reading the filesystem directly has some advantages.
	It makes it possible to back files up without affecting their time
	stamps; for <command>tar</command> and <command>cpio</command>,
	you would have to mount the filesystem read-only first.
	Directly reading the filesystem is also more effective, if
	everything needs to be backed up, since it can be done with
	much less disk head movement.  The major disadvantage is that
	it makes the backup program specific to one filesystem type;
	the Linux <command>dump</command> program understands the ext2
	filesystem only.  </para>

	<para> <command>dump</command> also directly supports
	backup levels (which we'll be discussing below); with
	<command>tar</command> and <command>cpio</command> this has to
	be implemented with other tools.  </para>

	<para> A comparison of the third party backup tools is beyond
	the scope of this book.  The Linux Software Map lists many of
	the freeware ones.  </para>

</sect1>

<sect1 id="simple-backups">
<title>Simple backups</title>

	<para> A simple backup scheme is to back up everything once,
	then back up everything that has been modified since the
	previous backup.  The first backup is called a <glossterm>full
	backup</glossterm>, the subsequent ones are <glossterm>incremental
	backups</glossterm>.  A full backup is often more laborious
	than incremental ones, since there is more data to write to the
	tape and a full backup might not fit onto one tape (or floppy).
	Restoring from incremental backups can be many times more work
	than from a full one.  Restoration can be optimized so that
	you always back up everything since the previous full backup;
	this way, backups are a bit more work, but there should never
	be a need to restore more than a full backup and an incremental
	backup.  </para>

	<para> If you want to make backups every day and have six
	tapes, you could use tape 1 for the first full backup (say, on
	a Friday), and tapes 2 to 5 for the incremental backups (Monday
	through Thursday).  Then you make a new full backup on tape 6
	(second Friday), and start doing incremental ones with tapes 2
	to 5 again.  You don't want to overwrite tape 1 until you've got
	a new full backup, lest something happens while you're making
	the full backup.  After you've made a full backup to tape 6,
	you want to keep tape 1 somewhere else, so that when your other
	backup tapes are destroyed in the fire, you still have at least
	something left.  When you need to make the next full backup,
	you fetch tape 1 and leave tape 6 in its place.  </para>

	<para> If you have more than six tapes, you can use the extra
	ones for full backups.	Each time you make a full backup, you
	use the oldest tape.  This way you can have full backups from
	several previous weeks, which is good if you want to find an old,
	now deleted file, or an old version of a file.	</para>

<sect2 id="tar-backups">
<title>Making backups with <command>tar</command></title>

	<para>
	A full backup can easily be made with <command>tar</command>:

<screen>
<prompt>#</prompt> <userinput>tar --create --file /dev/ftape 
/usr/src</userinput>
<computeroutput>tar: Removing leading / from absolute path names in 
the archive</computeroutput>
<prompt>#</prompt>
</screen>

	The example above uses the GNU version of <command>tar</command>
	and its long option names.  The traditional version of
	<command>tar</command> only understands single character
	options.  The GNU version can also handle backups that don't
	fit on one tape or floppy, and also very long paths; not all
	traditional versions can do these things.  (Linux only uses
	GNU <command>tar</command>.)  </para>
	
	<para> If your backup doesn't fit on one tape, you need to use
	the <option>--multi-volume</option> (<option>-M</option>) option:

<screen>
<prompt>#</prompt> <userinput>tar -cMf /dev/fd0H1440 
/usr/src</userinput>
<computeroutput>tar: Removing leading / from absolute path names in 
the archive
Prepare volume #2 for /dev/fd0H1440 and hit return:</computeroutput>
<prompt>#</prompt>
</screen>

	Note that you should format the floppies before you begin the
	backup, or else use another window or virtual terminal and do
	it when <command>tar</command> asks for a new floppy.  </para>

	<para> After you've made a backup, you should check that it is OK,
	using the <option>--compare</option> (<option>-d</option>) option:

<screen>
<prompt>#</prompt> <userinput>tar --compare --verbose -f 
/dev/ftape</userinput>
<computeroutput>usr/src/
usr/src/linux
usr/src/linux-1.2.10-includes/
....</computeroutput>
<prompt>#</prompt>
</screen>

	Failing to check a backup means that you will not notice that your
	backups aren't working until after you've lost the original data.
	</para>
	
	<para> An incremental backup can be done with
	<command>tar</command> using the <option>--newer</option>
	(<option>-N</option>) option:

<screen>
<prompt>#</prompt> <userinput>tar --create --newer '8 Sep 1995' 
--file /dev/ftape /usr/src 
--verbose</userinput>
<computeroutput>tar: Removing leading / from absolute path names in 
the archive
usr/src/
usr/src/linux-1.2.10-includes/
usr/src/linux-1.2.10-includes/include/
usr/src/linux-1.2.10-includes/include/linux/
usr/src/linux-1.2.10-includes/include/linux/modules/
usr/src/linux-1.2.10-includes/include/asm-generic/
usr/src/linux-1.2.10-includes/include/asm-i386/
usr/src/linux-1.2.10-includes/include/asm-mips/
usr/src/linux-1.2.10-includes/include/asm-alpha/
usr/src/linux-1.2.10-includes/include/asm-m68k/
usr/src/linux-1.2.10-includes/include/asm-sparc/
usr/src/patch-1.2.11.gz</computeroutput>
<prompt>#</prompt>
</screen>

	Unfortunately, <command>tar</command> can't notice when a file's
	inode information has changed, for example, that its permission
	bits have been changed, or when its name has been changed.
	This can be worked around using <command>find</command> and
	comparing current filesystem state with lists of files that have
	been previously backed up.  Scripts and programs for doing this
	can be found on Linux ftp sites.  </para>
	
</sect2>

<sect2 id="tar-restore">
<title>Restoring files with <command>tar</command></title>

	<para> The <option>--extract</option> (<option>-x</option>)
	option for <command>tar</command> extracts files:

<screen>
<prompt>#</prompt> <userinput>tar --extract --same-permissions 
--verbose --file 
/dev/fd0H1440</userinput>
<computeroutput>usr/src/
usr/src/linux
usr/src/linux-1.2.10-includes/
usr/src/linux-1.2.10-includes/include/
usr/src/linux-1.2.10-includes/include/linux/
usr/src/linux-1.2.10-includes/include/linux/hdreg.h
usr/src/linux-1.2.10-includes/include/linux/kernel.h
...</computeroutput>
<prompt>#</prompt>
</screen>

	You also extract only specific files or directories (which
	includes all their files and subdirectories) by naming on the
	command line:

<screen>
<prompt>#</prompt> <userinput>tar xpvf /dev/fd0H1440 
usr/src/linux-1.2.10-includes/include/linux/hdreg.h</userinput>
<computeroutput>usr/src/linux-1.2.10-includes/include/linux/hdreg.h</computeroutput>
<prompt>#</prompt>
</screen>

	Use the <option>--list</option> (<option>-t</option>) option,
	if you just want to see what files are on a backup volume:

<screen>
<prompt>#</prompt> <userinput>tar --list --file 
/dev/fd0H1440</userinput>
<computeroutput>usr/src/
usr/src/linux
usr/src/linux-1.2.10-includes/
usr/src/linux-1.2.10-includes/include/
usr/src/linux-1.2.10-includes/include/linux/
usr/src/linux-1.2.10-includes/include/linux/hdreg.h
usr/src/linux-1.2.10-includes/include/linux/kernel.h
...</computeroutput>
<prompt>#</prompt>
</screen>

	Note that <command>tar</command> always reads the backup volume
	sequentially, so for large volumes it is rather slow.  It is not
	possible, however, to use random access database techniques when
	using a tape drive or some other sequential medium.  </para>
	
	<para> <command>tar</command> doesn't handle deleted files
	properly. If you need to restore a filesystem from a full and
	an incremental backup, and you have deleted a file between
	the two backups, it will exist again after you have done the
	restore. This can be a big problem, if the file has sensitive
	data that should no longer be available.  </para>

</sect2>

</sect1>

<sect1 id="multi-level-backups">
<title>Multilevel backups</title>

	<para> The simple backup method outlined in the previous section
	is often quite adequate for personal use or small sites.  For more
	heavy duty use, multilevel backups are more appropriate.  </para>

	<para> The simple method has two backup levels: full and
	incremental backups.  This can be generalized to any number of
	levels.  A full backup would be level 0, and the different levels
	of incremental backups levels 1, 2, 3, etc.  At each incremental
	backup level you back up everything that has changed since the
	previous backup at the same or a previous level.  </para>

	<para> The purpose for doing this is that it allows a longer
	<glossterm>backup history</glossterm> cheaply.	In the example in
	the previous section, the backup history went back to the previous
	full backup.  This could be extended by having more tapes, but
	only a week per new tape, which might be too expensive.  A longer
	backup history is useful, since deleted or corrupted files are
	often not noticed for a long time.  Even a version of a file that
	is not very up to date is better than no file at all.  </para>

	<para> With multiple levels the backup history can be extended
	more cheaply.  For example, if we buy ten tapes, we could use
	tapes 1 and 2 for monthly backups (first Friday each month),
	tapes 3 to 6 for weekly backups (other Fridays; note that there
	can be five Fridays in one month, so we need four more tapes),
	and tapes 7 to 10 for daily backups (Monday to Thursday).
	With only four more tapes, we've been able to extend the backup
	history from two weeks (after all daily tapes have been used)
	to two months.	It is true that we can't restore every version
	of each file during those two months, but what we can restore
	is often good enough.  </para>

	<para><xref linkend="backup-history-timeline"> shows which backup
	level is used each day, and which backups can be restored from
	at the end of the month.  </para>

		<figure id="backup-history-timeline" float="1">
		<title>A sample multilevel backup schedule.</title>
		<graphic fileref="backup-timeline.png">
		</figure>

	<para> Backup levels can also be used to keep filesystem
	restoration time to a minimum.	If you have many incremental
	backups with monotonously growing level numbers, you need to
	restore all of them if you need to rebuild the whole filesystem.
	Instead you can use level numbers that aren't monotonous, and
	keep down the number of backups to restore.  </para>

	<para> To minimize the number of tapes needed to restore, you
	could use a smaller level for each incremental tape.  However,
	then the time to make the backups increases (each backup copies
	everything since the previous full backup).  A better scheme is
	suggested by the <command>dump</command> manual page and described
	by the table XX (efficient-backup-levels).  Use the following
	succession of backup levels: 3, 2, 5, 4, 7, 6, 9, 8, 9, etc.
	This keeps both the backup and restore times low.  The most you
	have to backup is two day's worth of work.  The number of tapes
	for a restore depends on how long you keep between full backups,
	but it is less than in the simple schemes.  </para>

<table id="efficient-backup-levels">
<title>Efficient backup scheme using many backup levels</title>
<tgroup cols=4>
<thead>
<row><entry>Tape</entry> <entry>Level</entry> <entry>Backup 
(days)</entry> <entry>Restore 
tapes</entry></row>
</thead>
<tbody>
<row><entry>1</entry> <entry>0</entry> <entry>n/a</entry> 
<entry>1</entry></row>
<row><entry>2</entry> <entry>3</entry> <entry>1</entry> <entry>1, 
2</entry></row>
<row><entry>3</entry> <entry>2</entry> <entry>2</entry> <entry>1, 
3</entry></row>
<row><entry>4</entry> <entry>5</entry> <entry>1</entry> <entry>1, 2, 
4</entry></row>
<row><entry>5</entry> <entry>4</entry> <entry>2</entry> <entry>1, 2, 
5</entry></row>
<row><entry>6</entry> <entry>7</entry> <entry>1</entry> <entry>1, 2, 
5, 6</entry></row>
<row><entry>7</entry> <entry>6</entry> <entry>2</entry> <entry>1, 2, 
5, 7</entry></row>
<row><entry>8</entry> <entry>9</entry> <entry>1</entry> <entry>1, 2, 
5, 7, 8</entry></row>
<row><entry>9</entry> <entry>8</entry> <entry>2</entry> <entry>1, 2, 
5, 7, 9</entry></row>
<row><entry>10</entry> <entry>9</entry> <entry>1</entry> <entry>1, 2, 
5, 7, 9, 10</entry></row>
<row><entry>11</entry> <entry>9</entry> <entry>1</entry> <entry>1, 2, 
5, 7, 9, 10, 
11</entry></row>
<row><entry>...</entry> <entry>9</entry> <entry>1</entry> <entry>1, 
2, 5, 7, 9, 10, 11, 
...</entry></row>
</tbody>
</tgroup>
</table>
			
	<para> A fancy scheme can reduce the amount of labor needed, but
	it does mean there are more things to keep track of.  You must
	decide if it is worth it.  </para>

	<para> <command>dump</command> has built-in support for backup
	levels.  For <command>tar</command> and <command>cpio</command>
	it must be implemented with shell scripts.  </para>

</sect1>

<sect1 id="what-to-backup">
<title>What to back up</title>

	<para> You want to back up as much as possible.  The major
	exception is software that can be easily reinstalled,
	but even they may have configuration files that it is
	important to back up, lest you need to do all the work to
	configure them all over again.	Another major exception is
	the <filename>/proc</filename> filesystem; since that only
	contains data that the kernel always generates automatically,
	it is never a good idea to back it up.	Especially the
	<filename>/proc/kcore</filename> file is unnecessary, since it
	is just an image of your current physical memory; it's pretty
	large as well.	</para>

	<para> Gray areas include the news spool, log files, and many
	other things in <filename>/var</filename>.  You must decide what
	you consider important.  </para>

	<para> The obvious things to back up are user files
	(<filename>/home</filename>) and system configuration files
	(<filename>/etc</filename>, but possibly other things scattered
	all over the filesystem).  </para>

</sect1>

<sect1 id="compressed-backups">
<title>Compressed backups</title>

	<para> Backups take a lot of space, which can cost quite
	a lot of money.  To reduce the space needed, the backups
	can be compressed.  There are several ways of doing this.
	Some programs have support for compression built in; for
	example, the <option>--gzip</option> (<option>-z</option>)
	option for GNU <command>tar</command> pipes the whole backup
	through the <command>gzip</command> compression program, before
	writing it to the backup medium.  </para>
	
	<para> Unfortunately, compressed backups can cause trouble.
	Due to the nature of how compression works, if a single bit is
	wrong, all the rest of the compressed data will be unusable.
	Some backup programs have some built in error correction, but no
	method can handle a large number of errors.  This means that if
	the backup is compressed the way GNU <command>tar</command> does
	it, with the whole output compressed as a unit, a single error
	makes all the rest of the backup lost.	Backups must be reliable,
	and this method of compression is not a good idea.  </para>
	
	<para> An alternative way is to compress each file separately.
	This still means that the one file is lost, but all other files
	are unharmed.  The lost file would have been corrupted anyway,
	so this situation is not much worse than not using compression
	at all.  The <command>afio</command> program (a variant of
	<command>cpio</command>) can do this.  </para>
	
	<para>
	Compression takes some time, which may make the backup program
	unable to write data fast enough for a tape drive.
	This can be avoided by buffering the output (either internally, if
	the backup program if smart enough, or by using another program),
	but even that might not work well enough.  This should only be
	a problem on slow computers.  </para>

</sect1>

</chapter>
