Linux Lite Forums

Software - Support => Tutorials => Topic started by: m654321 on March 27, 2018, 06:50:26 AM

Title: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on March 27, 2018, 06:50:26 AM
This tutorial is for those of you who:
Reasons I use 'dd' for cloning and what I use it for.

It's a very simple tool for cloning LL, or any other distro, or  Windows OS, including any dual- or multi-booted systems.  The reason I started using 'dd' for cloning, is that I found other dedicated cloning software (e.g. clonezilla) too complicated to use:  the 'dd' command only takes up one line of code in the terminal.
I have LL installed on a 250GB drive. About once-a-week, I clone it to an identical drive (same size & make), via a USB port (the important thing here is that the backup drive has a  storage size that is either the same or bigger than the written space on the drive you want to clone from).  I do this as I have accumulated several virtual machines, important data, packages, symlinks, etc which, if lost, would cost me a huge amount of time and energy to reinstall - doesn't bear thinking about  :-[ .  Should LL on the internal drive on my laptop crash irretrievably (it has done at least a couple of times due to my tinkering around and the odd silly mistake), I just take the drive out of the PC, and replace it with the backup drive.  The backup drive is instantly recognised and works from the get-go.

How long will cloning it take?

Using setup 1 in my signature below as an example (an i7 laptop), where I'm cloning from a 250GB SSD  to an identical external SSD (via USB 3.0 connection)  it takes just under 3 hours, with a copy rate of around 23 to 27 MB/s. With older CPUs this will take longer, and it could be an activity you could just leave to run overnight.  Sometimes, I clone from a 250GB SSD to a 500GB SSD - the cloning time is the same - in this case what you end up with is 250GB of cloned space and 250GB of unallocated space. I have cloned to HHDs, but the copy rate is significantly slower, as you'd expect.  The backup drive doesn't need to be formatted for cloning - you can just leave it as unallocated space - cloning will overwrite it anyway. Also, don't forget to transfer any valuable data files on your backup drive before cloning to it, otherwise these will be lost !
Here's what you do to clone the whole drive:
Code: [Select]
sudo dd if=/dev/sdx of=/dev/sdy
So, if you are cloning from /dev/sda to /dev/sdb, you would enter:
Code: [Select]
sudo dd if=/dev/sda of=/dev/sdb status=progress
The term status=progress, means that you are given a live on-screen progress of cloning, i.e. the cumulative number of bytes transferred, the time passed from the start of cloning (seconds) and the transfer speed (MB/s). If you leave this term out, cloning will go ahead but without any indication of progress until cloning has completed.

After entering this you'll be asked for your password - type this then hit the ENTER key to start the cloning process. When cloning is completed, the output from the terminal will have the following format:
Code: [Select]
[mike@G750 ~]$ sudo dd if=/dev/sdb of=/dev/sdc status=progress
[sudo] password for mike: 
488397168+0 records in
488397168+0 records out
250059350016 bytes (250 GB, 233 GiB) copied, 10865.2 s, 23.0 MB/s

In the output shown above, the internal drive that has LL installed on it (the drive that's being copied from) has the designation /dev/sdb - I just want to make the point that the designation of the drive, containing the operating system, is not always going to be /dev/sda, though it usually is.


Here's what you do if you want to clone just a specific partition, not the whole driveThe method is the same - you just need to quote the specific partition you want to clone 'from' and 'to' in the dd command.For example, if Linux Lite is installed as root on the partition /dev/sda1 on your PC, and you want to clone this to a partition on another drive named /dev/sdc1, then the command is as follows:
Code: [Select]
[mike@G750-mj ~]$ sudo dd if=/dev/sda1 of=/dev/sdc1 status=progress[sudo] password for mike: 48315625984 bytes (48 GB, 45 GiB) copied, 2200 s, 22.0 MB/s[[B94373887+0 records in94373887+0 records out
48319430144 bytes (48 GB, 45 GiB) copied, 2217.25 s, 21.8 MB/s

A WORD OF CAUTION (don't say you haven't been warned)! 

Should you enter the wrong /dev/sdx designation, you could risk wiping your computer clean - this is probably the reason that 'dd' is sometimes nicknamed 'disk-destroyer'. So, my advice is don't do this if you're drinking beers, distracted, or overly tired, otherwise ..! That said, I've been using 'dd' for the last three years and haven't had a problem with it. Just make sure the USB connection of the backup drive is secure (as a precaution don't move the PC or external drive during cloning). If the USB disconnects during cloning, even just momentarily, you'll have to restart the cloning process.
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: bitsnpcs on March 29, 2018, 07:08:45 PM
Thank You for the tutorial 8) , bookmarked.
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: rijnsma on March 30, 2018, 05:46:21 AM
So you copy the whole disk? I like to clone only what is relevant, so for example the OS or data. The way Clonezilla does it. Livecd/-usb is also nice.
Or maybe I don't read you right?  :)
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on March 30, 2018, 11:24:07 AM
So you copy the whole disk? I like to clone only what is relevant, so for example the OS or data. The way Clonezilla does it. Livecd/-usb is also nice.
Or maybe I don't read you right?  :)

Thanks rijnsma.
Yes I copy the whole disk. The objective of the tutorial was to simply clone the whole OS drive as, in my case, LL occupies the whole drive. Sorry for not making that a bit clearer at the outset, though the title does refer to cloning a drive - not a partition.

Apparently, you can use 'dd' to clone specific partitions but I haven't had the need to do this yet.
I assume (NB: this is an assumption as I haven't checked it yet !)  that if you wanted to clone a specific partition (let's call it /dev/sda7 on drive /dev/sda) to drive /dev/sdb, then it would probably be as follows:

Code: [Select]
sudo dd if=/dev/sda7 of=/dev/sdb
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: torreydale on March 30, 2018, 10:43:15 PM
You might also want to add the following to your command:


Code: [Select]
status=progress
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on March 31, 2018, 05:01:02 AM
You might also want to add the following to your command:
Code: [Select]
status=progress

Many thanks Torreydale.
That looks very useful, but could you add some detail about how to use it, i.e. do you add it to the 'dd' line of code as:

sudo dd if=/dev/sda of=/dev/sdb && sudo status=progress ?
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: justme2 on March 31, 2018, 02:33:57 PM
I have used this method to clone one SDHC to another kept as a standby sysytem, where both cards are the same size. If I copy from my working system card to a partition on a hard disk, does the partition size have to be the same size as the card? E.g. 32GB sdhc  copied to 500GB HDD, do I have to create a 32GB partition on the HDD or can I just 'dd' without worrying about the HDD size? I'm confused by conflicting info from google.
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on March 31, 2018, 04:06:30 PM
If I copy from my working system card to a partition on a hard disk, does the partition size have to be the same size as the card?
E.g. 32GB sdhc  copied to 500GB HDD, do I have to create a 32GB partition on the HDD or can I just 'dd' without worrying about the HDD size? I'm confused by conflicting info from google.

Thanks for your question justme2.
No, the HDD partition size doesn't have to be the same size as the card (i.e. 32GB), unless the card has, or almost has, 32GB of written space on it.

The partition on the 500GB HDD that you're cloning to,  must either be the same size or larger (never smaller) than the written space you're cloning from on your 32GB SDHC.

Let's take a hypothetical example ...

The HDD could have say just a single partition spanning the whole drive.  If  the written space on your SDHC only takes up  say 20 out of the 32GB, then it will take up 20GB on the HDD following cloning, with 480 GB either left unused or, if your HDD is unformatted, then the remaining 480GB will be left as unallocated space.

Using the same example above, you could of course clone the 20GB of written space on your SDHC, to a partition smaller than 500GB on your HDD. The important thing here is  that the partition you're cloning to is  at least the same size or larger than 20GB. Just because the device you're cloning from has a 32GB capacity, doesn' t mean your partition on the HDD has to be 32GB. 

The key thing is that the size of the written space that you're copying from your SDHC will determine the minimum size  needed for the partition receiving the cloned copy on your HDD.

Hope this helps  :)

Mike
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: justme2 on April 01, 2018, 04:57:32 AM
Mike, thanks for clarifying, the partition I wish to clone to is much larger than the data size on the SDHC. A further thought has raised its head though, if I want to clone FROM a 500GB HD would I then need a destination device not less than 500GB. I think I need to make a much smaller partition on the HDD for linux lite (the total size on the SDHC is only about 26GB including data and lots of additional apps). So if I create a 32GB partition on the HDD, that will allow me to create clones from HDD back to SDHC. Does that sound feasible?

Some descriptions of the use of dd indicate that the whole drive is copied, including empty space, so I'm still not certain.

Thanks for your time,
another UK Mike!
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on April 01, 2018, 08:46:32 AM
if I want to clone FROM a 500GB HD would I then need a destination device not less than 500GB
Yes, that's true if you're selecting the whole of the 500GB drive to be cloned

Quote
I think I need to make a much smaller partition on the HDD for linux lite (the total size on the SDHC is only about 26GB including data and lots of additional apps). So if I create a 32GB partition on the HDD, that will allow me to create clones from HDD back to SDHC. Does that sound feasible?
Yes, that's sounds feasible. To be on the safe side you could make your partition on the HDD a bit smaller than your destination drive: maybe 30 or 31GB. It may well work with 32GB - perhaps I'm just being over-cautious ...

Have a good Easter Sunday  & don't eat too many chocolate eggs  8)
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: justme2 on April 01, 2018, 09:15:05 AM
Yes, that's sounds feasible. To be on the safe side you could make your partition on the HDD a bit smaller than your destination drive: maybe 30 or 31GB. It may well work with 32GB - perhaps I'm just being over-cautious ...

Have a good Easter Sunday  & don't eat too many chocolate eggs  8)

Thank you again, I feel confident to try it now.

Chocolate is generally banned in our household but I have 'sneaked in' some 70% coco choc which supposedly has some health benefits!  ;D
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on April 02, 2018, 05:55:52 AM
Thank You for the tutorial 8) , bookmarked.

A pleasure to share  :)
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: justme2 on April 02, 2018, 04:39:53 PM
OK now I 'dd' cloned my LL3.8 system from SDHC to a new partition on the HDD to give dual boot with win**ws, (which I have to retain for my wife). Although I could see LL folders and files on the 2nd partition, no way could I choose to boot LL3.8
After lots of googling I found 'Boot-Repair' and installed it to a SD card, booted from it and allowed it to have its own way! Eventually it reported its findings which looked ok. Rebooting from the HDD then gave me a grub menu and I was able to boot to either system (win or LL3.8).

So the moral of the tale is that maybe your Tutorial could have a reference to 'Boot-Repair' for 'dders' who are confronted with a no-boot situation.

Thanks Mike for your assistance.  :)
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on April 04, 2018, 08:51:35 AM
maybe your Tutorial could have a reference to 'Boot-Repair' for 'dders' who are confronted with a no-boot situation.
Glad you got everything working  :)

I didn't have a no-boot situation, nor did I foresee one, which is the reason 'boot-repair' was not included. I just simply wanted to share what has routinely worked for me. On the repair note, I discovered recently that gparted GUI has a repair disk function which turned out to be really useful & easy to use.
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: justme2 on April 04, 2018, 09:29:15 AM
After some thought, I decided to limit the LL3.8 HDD partition to 32GB and keep a seperate partition for 'big' data like video and music files, the reason being that it only takes around 50 minutes (on my i5 PC) to clone the full system including all the extra apps I installed. My total system size is only 26GB. So 'dd' will now be my default system backup method henceforth.

A further problem raised its head - long boot time due to sitting on the yellow feather boot screen for 1m 30s. That was due to not finding the swap partition during booting. For some unknown reason the UUID for the swap partition was not being accepted, even though I double checked it was correct, so I edited fstab to replace the UUID with the correct device id, sda5 and boot time was much reduced.

I think I can now say I have a 'perfect' system for my needs.
Power to Linux Lite (Thanks Jerry and team).

Thanks Mike,  :)
Title: Re: Tutorial: cloning your Linux Lite drive using the 'dd' command.
Post by: m654321 on May 03, 2018, 05:19:19 AM
UPDATE 1: status=progress added to 'dd' command line in tutorial in post #1.

You might also want to add the following to your command:
Code: [Select]
status=progress

Many thanks to torreydale for this suggestion 8)
It has now been added to the tutorial.  I just needed to test it before posting, and can confirm it works perfectly giving you have an ongoing idea of how cloning is going.