Backup via shell command

  9 years ago

Who has not yet lost some data, due to some silly error, virus, or simply being a "sleepy head"?

Note: All of the following details were used on my machine and I do use this way regulary to make a safty copy of my data. But in case of troubles I do not feel responsible for eventually data losses. Therefore you are adviced to try this method first with sample data to make yourself familar with the method.

There are various way of prevent this. The term redundancy refers to making a back up of some kind. There are ready tools, e.g. 'Backup tool', available for Linux Mint. For the people, who still feel more attracted to the shell the following approach using the unix command dd might be more interesting than using a ready application.

dd is an unix command used to copy and/or cut of data streams or entire paths. Its' general syntax is the following:

                                              dd if=/dev/sda of=dev/sdb

At this point is might be very handy to emphasis that if is describing the data source. The target can be a hard-disc or a file and is described via of.

The names or ids' of hard discs or partitions can be found with the command

                                              fdisk –l

In the following we assume your (data)source drive is called: /dev/sda

Thetarget file could be for instance on a USB-drive called:/media/.../LM_mirror.img

So now you have all needed details to store your data. Simply type:

                                              dd if=/dev/sda of=/media/.../LM_mirror.img

In order to restore the data from a file onto the inital source location (or somewhere else) on has to switch the source and target location for the dd-command. That could look as follows:

                                              dd if=/media/.../LM_mirror.img of=/dev/sda

Attention if the target has less free space than the source data needs one has the option to pack the source data just when applying the dd-command

Storing and packing data would look as follows:

                                             dd if=/dev/sda | gzip –c > /media/.../LM_mirror.img.gz

Restore and unpack the data works as follows:

                                             gunzip –c /media/Science/LM_mirror.img.gz | dd if=/dev/sda

Ofcourse time does matter nowadays same as size does matter! Therefore one may use buffer space while storing or restoring. For that one may define the buffer size with the flag bs=10M , which stands for 10 MByte buffer size. The buffer size can be modified but unless you have a really great machine I would not exceed 100 MByte and better stay below 50 MByte for smooth operation. This option may be add at any point behind the dd-command.

Storing, buffering and packing data would look as follows:

                                           dd bs=10M if=/dev/sda | gzip –c > /media/…/LM_mirror.img.gz

Restore, buffering and unpack the data works as follows:

                                           gunzip –c /media/…/LM_mirror.img.gz | dd if=/dev/sdabs=10M

Finally, you may face the problem that your target drive is a MS-Windows artifact and is still format as FAT32. That means for you that the file size on the target drive is not allowed to exceed 4 GByte! Then again your source data may easily exceed 4 GByte and no packing in the world will make them smaller than 4 GByte.

The answer to your problem will be splitting of target file into many target files. You may wish to define the target file size with 1 GByte by using the flag -b. Also the ending of your target file needs an index of some kind. For that you can use the flag -a. The integer value behind -a is determine how many index fields you wish to use. In this case we choose 3. That will look like aaa aab aac and so on.

Storing and splitting data would look as follows:

First change to the target directory!

                                           cd /media/…/…

                                           dd if=/dev/sda | split –a 3 –b 1GB - LM_mirror_img_

Restore and splitting the data works as follows:

                                           cd /media/…/…

                                           cat LM_mirror* | dd of=/dev/sda

linux1492 9 years ago

@timequake: thank you for the comments. Certainly it had to be gzip instead of gunzip. Good that you noticed that. At the last two statements I was not initially intending to do the packing via g(un)zip. It would be an option, but I was rejecting it for myself, due to the extended time consumption of the packing routine. Instead, having enough disc space available I considered a quick run as more beneficial. However, if one intends to save disc space it would be a good idea to use a packing routine, e.g. g(un)zip on top.

For the incremental backups I would not use the 'dd' command. From my personal experiences the 'rsync' (remote synchronization) command would be more useful. One can use it to rsync to an other computer, as many people do rsync for daily changes to a fellows server. Or, one may rsync to an other disc drive or even directory. For most incremental backups this is good enough. But I faced the dilemma of having the /media directory with a backup drive within the tree to be backup. That causes of cause a recursive issue ... )-: At this point I learned to use dd command.

timequake 9 years ago

@linux1492: thank you for this nice base manual on how to backup with simple bash commands. i think the only disadvantage is that there is no way to do incremental backups.

i don't want to hang out the wise guy, but i think i spotted a little mistake, maybe you can correct it:

"Storing, buffering and packing data would look as follows:
dd bs=10M if=/dev/sda | gunzip –c > /media/…/LM_mirror.img.gz"

and do i see this right that you are missing the g(un)zip command in the last two statements?

besides that really good work. cya