How to Backup Your System in linux

• BackupYourSystem
• TAR

 

 

Introduction to tar

 

Contents

1. Introduction to tar
2. Preparing for Backup
3. Backing Up
   1. Additional Tips
   2. Archive Splitting
   3. Backup Over a Network
      1. Netcat
      2. SSH
4. Restoring
   1. Restoring Over a Network
5. Reformatted Partitions
   1. Restoring RAID1 to EXT3 for import into VM
   2. Restoring GRUB
6. Additional Resources

 

This page is part of the BackupYourSystemarticle, as such, ensure you've read that prior to continuing. Thissubpage will acquaint a user with the tar archival program, a CLIsolution to the creation of compressed archival backups. It will detailthe creation and restoration of archives, including operation over anetwork.

Before continuing users are encouraged to read the TerminalHowto page which explains many basic concepts related to working with a terminal.

Improper usage of any archival program can cause unintended data loss.Read the entire tutorial before proceeding and understand what you aredoing.

 

Preparing for Backup

 

Inpreparation for a complete backup of the system, it is a good idea toempty the trash and remove any unwanted files and programs from yourcurrent installation. This includes the home folder which can be filledwith many files not needed. Doing so will reduce the size of thearchive created in relation to how much space is liberated.

A quick list of examples is below, decide for yourself what applies:

• Delete all your emails.
• Wipe your saved browser personal details and search history.

  • If you are not worried about the security concerns, this step is not necessary. Many users explicitly want backups of their email and browser settings.

• Unmount any external drives and remove any optical media such as CDs or DVDs that you do not want to include in the backup.
  • This will reduce the amount of exclusions you need to type later in the process.

• Go through the contents of your user folder in /home and delete all unwanted files in the subdirectories, often people download files and forget about them for instance.

 

Backing Up

 

To get started, please open up a terminal, in Ubuntu this can be done by Applications Menu -> Accessories -> Terminal. Some directories require root or superuser permissions to read and write (needed for backup), for an explanation on why see FilePermissions. To gain temporary root permission, simply preface any command you want to issue with sudo, as explained in RootSudo.

Forthis example, we will change directories to root. This is where thebackup will be made. This is an arbitrary decision, you should createthe backup elsewhere. For instance to a mounted external hard drive,another partition or disk connected internally, even a folder in yourhome directory could be used. In all cases, ensure the location yoursaving the archive to has enough space. Simply use the cd command tonavigate there.

cd / 

 

The following is an exemplary command of how to archive your system.

tar -cvpzf backup.tar.gz -–exclude=/backup.tar.gz --exclude=/proc --exclude=/lost+found --exclude=/sys --exclude=/mnt --exclude=/media / 

 

To understand what is going on, we will dissect each part of the command.

• tar - is the command that creates the archive. It is modified by each letter immediately following, each is explained bellow.

  • c - create a new backup archive.

  • v - verbose mode, tar will print what it's doing to the screen.

  • p - preserves the permissions of the files put in the archive for restoration later.

  • z - compress the backup file with 'gzip' to make it smaller.

  • f <filename> - specifies where to store the backup, backup.tar.gzis the filename used in this example. It will be stored in the currentworking directory, the one you set when you used the cd command.

• --exclude=/example/path - The options following this model instruct tar what directories NOTto backup. We don't want to backup everything since some directoriesaren't very useful to include. The first exclusion rule directs tar notto back itself up, this is important to avoid errors during theoperation. After, we proceed to exclude the /proc, /lost+found, /sys, /mnt and /media directories in root.

  • Itis important to note that these exclusions are recursive. This meansthat all folders located within the one excluded will be ignored aswell. In the example, excluding the /media folder excludes all mounted drives and media there.

  • If there are certain partitions you wish to backup located in /media,simply remove the exclusion and write a new one excluding thepartitions you don't want backed up stored within it. For example:

    • --exclude=/media/unwanted_partition 

• / - After all options is the directory to backup. Since we want to backup everything on the system we use /for the root directory. Like exclusions, this recursively includesevery folder below root not listed in the exclusions or other options.

See tips before operation for additional information.

Oncesatisfied with the command, execute it and wait until it has completed.The duration of the operation depends on the amount of files andcompression choses. Once completed, check the directory you set to findthe archive. In our example, backup.tar.gz would be located in the / directory once completed. This archive can then be moved to any other directory for long term storage.

Note:At the end of the process you might get a message along the lines of'tar: Error exit delayed from previous errors' or something, but inmost cases you can just ignore that.

 

Additional Tips

 

• To keep good records, you should include the date and a description of backup in the filename.

• Anotheroption would be to use bzip2 to compress your backup instead of gzip.Bzip2 provides a higher compression ratio at the expense of speed. Ifcompression is important to you, just substitute the z in the command with j, and change the file name to .tar.bz2. The rest of this guides examples use gzip, make the subsequent changes to the examples before using them.

• Ifyou want to exclude all other mounts other than the current - by this Imean partitions mounted to directories - then use the --one-file-systemoption appended before the exclusion rules. This has the effect ofstopping tar from crossing into any other mounts in any directoryincluding /mnt or /media. For instance, many users create a separatemount for /home to keep user folders separate from root, adding this option to our original example would exclude home's contents entirely.

 

Archive Splitting

 

If you wantto burn the archive to discs, or transfer them to a filesystem with alimited max filesize (say FAT32 with a limit of 4GB per file) then youwill have to split the file either during or after archive creation. Asimple means is to use the split command. Below are examples of both scenarios. More information than conveyed here can be found in the man pages of split, use man splitin a terminal to read. Ensure you keep these archives all together in adirectory you label for extraction at a later date. Once the archivesare split to a desirable size, they can be burned one at a time todisc.

To Split During Creation

tar -cvpz <put options here> / | split -d -b 3900m - /name/of/backup.tar.gz. 

 

• Thefirst half until the pipe (|) is identical to our earlier example,except for the omission of the f option. Without this, tar will outputthe archive to standard output, this is then piped to the splitcommand.

• -d- This option means that the archive suffix will be numerical insteadof alphabetical, each split will be sequential starting with 01 andincreasing with each new split file.

• -b - This option designates the size to split at, in this example I've made it 3900mB to fit into a FAT32 partition.

• -- The hyphen is a placeholder for the input file (normally an actualfile already created) and directs split to use standard input.

• /name/of/backup.tar.gz.Is the prefix that will be applied to all generated split files. Itshould direct to the folder you want the archives to end up. In ourexample, the first split archive will be in the directory /name/of/ andbe named backup.tar.gz.01 .

To Split After Creation

split -d -b 3900m /path/to/backup.tar.gz /name/of/backup.tar.gz. 

 

• Here instead of using standard input, we are simply splitting an existing file designated by /path/to/backup.tar.gz .

To Reconstitute the Archive
Reconstructing the complete archive is easy, first cd into the directory holding the split archives. Then simply use cat to write all the archives into one and send over standard output to tar to extract to the specified directory.

 

cat *tar.gz* | tar -xvpzf - -C /  

 

• Theuse of * as a wild card before and after tar.gz tells cat to start withfirst matching file and add every other that matches, a process knownas catenation, how the command got its name.
• Afterwards, it simply passes all that through standard output to tar to be extracted into root in this example.

• For a more complete explanation of restoration, see Restoring.

 

Backup Over a Network

 

The commandtar does not include network support within itself, but when used inconjunction with other programs this can be achieved. Two commonoptions are netcat (nc) and ssh.

 

Netcat

 

The command ncis designed to be a general purpose networking tool. It sets up asimple connection between two networked machines. This connectionsurvives until the user manually disconnects it, unlike normalconnections such as tcp which terminate upon completion of a file.

Receiving Computer
Onthe receiving end you'll setup netcat to write the backup file as inthe following example. This command will setup a machine to receivestandard input from network to port 1024 then write it to the filebackup.tar.gz. The choice of port is entirely up to the user, as longas it is 1024 or larger. A simple example:

nc -l -p 1024 > backup.tar.gz 

 

Sending Computer
Onthe machine to be backed up, the tar command will be piped to nc whichwill then send the backup over the network to the port in question tobe written in the file. Take note, where it says <receiving host>replace with the name of the computer on the network. The f option wasomitted since we are not writing to a local file, but moving thearchive through standard output. The following is an example:

tar -cvpz <all those other options like above> / | nc -q 0 <receiving host> 1024 

 

If all goes well the backup will be piped through the network without touching the file system being read.

 

SSH

 

You can also use SSH. For a complete explanation of its proper use see SSH. The command below is an example of what is possible.

tar -cvpz <all those other options like above> / | ssh <backuphost> "( cat > ssh_backup.tar.gz )"

 

In the example:

• Thetar half of the command is the same as above, with the omission of thef option to pipe the archive via standard output to ssh and onto thenetworked computer.

• ssh_backup.tar.gz Is the name of the file that will be created on the machine indicated.

• <backuphost> - Should be replaced with the name of the computer in question on the network.

 

Restoring

 

Be careful!If you don't understand what you are doing here you might end upoverwriting stuff that is important to you. If you are restoring theroot directory while booted and using root, you will encounter trouble.Operate such a restoration from a Live CD, simply mount the drive inquestion and correct the path as needed.

For the purpose of this tutorial we will assume your backup file is stored in the directory /home/testand named backup.tar.gz. If you are restoring files to directoriesowned by root, gain superuser access as above by prefacing commandswith sudo. Ensure your archive is NOT stored in the directories to berestored, or it will be overwritten during the operation.

Restore Your Backup

tar -xvpzf /home/test/backup.tar.gz -C / 

 

A brief explanation:

• x - Tells tar to extract the file designated by the f option immediately after. In this case, the archive is /home/test/backup.tar.gz

• -C <directory>- This option tells tar to change to a specific directory beforeextracting. In this example, we are restoring to the root (/)directory.

This will overwrite every single file and directory on the designatedmount with the one in the archive. Any file created after the archive,will have no equivalent stored in the archive and thus will remainuntouched

Allowthe restoration the time it needs to complete. Once extraction iscompleted, recreate directories that were not included in the originalarchive. In our example, these would include /proc, /lost+found, /sys, /mnt and /media. This can be done with the following command in our example:

mkdir /proc /lost+found /sys /mnt /media 

 

Once finished, reboot and everything should be restored to the state of your system when you made the backup.

 

Restoring Over a Network

 

This short guide, assumes you employed nc to make the original backup as described above.

Receiving Computer
Ensurethe disk has been mounted and use the following command to accept inputover the network that will then be extracted to the path indicated. Inthis example, the directory /mnt/disk will be extracted to.

nc -l -p 1024 | tar -xvpjf - -C /mnt/disk 

 

Sending Computer
On the computer with the archive to send, use the following command:

cat backup.tar.gz | nc -q 0 <receiving host> 1024 

 

A few comments:

• The -character in the first command will tell tar to accept input fromstandard input rather than a file. In this case, input comes from thepipe.

• The backup file will be expanded without being saved on the disk of receiving computer, the same as when the backup was made.

 

Reformatted Partitions

 

If you had to format partitions or are restoring to different discs, update the /etc/fstab and /boot/grub/menu.lst files after restoring the backup.

1. Mount the reformatted partitions on a LiveCD.

2. Open a terminal and type

   blkid

3. Making note of the UUIDs listed, edit the /etc/fstab and /boot/grub/menu.lst files in the restored root partition to match. Use the following commands to edit the files respectively.

 

sudo nano /mnt/disk/etc/fstab 

 

 

sudo nano /mnt/disk/boot/grub/menu.lst 

Change the UUIDs to match the results of your blkid command.

 

Restoring RAID1 to EXT3 for import into VM

 

Usethis up procedure for logical transfert to disk into VM system bootupwith live-cd and create first your file system with one Ext2/3partition and Swap partition. The procedure for restore RAID1 or RAID5 to VM engine request this additional step for regenerate image for upload with Grub

cd /media/disk/etc/
mv mdadm.conf mdadm.conf.old
mkinitrd /media/disk/boot/initrd-noraid-'uname -r'.img 'uname -r'
cd /media/disk/grub

 

If not available mkinitrd command use $sudo apt-get install initrd-tools. Nowedit grub.conf and change kernel boot parameter root= to reflectpartionwithout raid e.g. change from /dev/md0 to /dev/sda1, also change theinitrd= line to /boot/initrd=noraid-x.x.x.img

Now system is ready to next setup GRUB (step below) and re-boot of virtual machine without Raid.

 

Restoring GRUB

 

In most cases restoring GRUB should not be necessary since we are only restoring files.

If you find yourself unable to boot GRUB properly, follow the instructions on GrubHowto

 

Additional Resources

 

• "Backing Up Ubuntu"

• "Wei’s world: HOWTO: backup my Ubuntu"