Migrate Fedora OS via partition cloning efficiently over network

How to migrate Fedora OS from one system to another, across network or (better) a Thunderbolt connection, without cloning the whole disk contents, saving SSD life.

Background

When migrating a Linux installation from a source drive to a target drive of a different size, traditional block-by-block tools like dd don’t allow to migrate to a smaller drive, and also waste SSD life by writing unused blocks. This guide describes a simple approach in which different disk sizes can be used, and the process is performed over network (or Thunderbolt), so that it’s not necessary to place two physical disks in the same device. The target system is exactly the same as the original, with all partition and filesystem UUIDs staying the same, which means no post-migration OS configuration is necessary. The time and SSD wear is minimized by pre-shrinking partitions and skipping unallocated disk space entirely. The default Fedora Workstation disk layout is supported, including LUKS encryption of the system partition.

This guide expects the following disk layout:

/dev/nvme0n1p1 -- ESP (most often FAT)
/dev/nvme0n1p2 -- Boot partition (most often Ext4)
/dev/nvme0n1p3 -- System partition (most often Btrfs, optionally encrypted using LUKS)

For simplicity, this will not try to skip copying all unused blocks, just most of them. The first two partitions will be cloned in raw mode. The third partition will be shrunk first (thus saving lots of blocks from being copied, and also supporting smaller target drives), and then cloned in raw mode.

Migration steps

Shrink the system partition on the source system

On the source system:

1. Delete all necessary data, prune cash dirs, empty the trash, etc.
2. Boot to a Fedora Workstation Live system from a USB drive.
3. Install necessary tools: sudo dnf install gparted pv
4. Run Gparted. In it, unlock the system partition, if encrypted. Then shrink the partition to a reasonable degree. For example, if the partition has 950 GB (from a 1 TB disk), but only 300 GB is used, you can shrink it e.g. to 350 GB. That still leaves enough free space for regular usage if you had to boot it for any reason, and saves 600 GB from being copied needlessly. This operation might take some time, because any data blocks currently present after the new size limit will need to be moved (therefore don’t be too aggressive in setting the limit). If your partition was encrypted, make sure that both the LUKS encryption container and the filesystem inside it correctly show up as resized.
5. Unmount all partitions (if any), lock the encrypted partition (if applicable) and close GParted.

Erase the target system

On the target system:

1. Ensure that this is really the system you want to completely erase (and replace with the OS from the source system).
2. Ensure that the disk size is sufficient to fit all partitions from the source system (now that the system partition on the source system has been shrunk).
3. Boot to a Fedora Workstation Live system from a USB drive.
4. Install necessary tools: sudo dnf install sfdisk sgdisk
5. Unmount all partitions (if any). You can use e.g. GNOME Disks for this.
6. Erase the whole drive (make sure you’re using the correct system): sudo wipefs -a /dev/nvme0n1
7. Discard all blocks on your drive (better for your SSD longevity): sudo blkdiscard -v /dev/nvme0n1

Establish an Ethernet/Thunderbolt network link

If you have both systems connected to the network and intend to use it, skip the Thunderbolt setup section below. Instead figure out the IP addresses of both systems (using the ip address command) and go to the Connectivity test.

Thunderbolt setup

If you want something faster than a regular network, have Thunderbolt ports on both systems and a fast USB-C cable ready, you can use that instead. With Thunderbolt, you can transfer data with 10 Gb/s speeds or more, compared to the common 1 Gb/s of a standard Ethernet.

1. On both laptops, after connecting them through a Thunderbolt cable, locate the network interface name: ip link
   Look for an interface like thunderbolt0.
2. Assign static IP addresses to them:

• Source system:
  bash sudo ip addr add 192.168.5.1/24 dev <interface_name>
sudo ip link set <interface_name> up
• Target system:
  bash sudo ip addr add 192.168.5.2/24 dev <interface_name>
sudo ip link set <interface_name> up

Connectivity test

1. Let’s use a well-named variable for both numbers, so that we don’t mix them up later. On both systems, run:

   export SOURCE_IP=192.168.5.1
   export TARGET_IP=192.168.5.2

2. Test connectivity between your systems.

• From the source system: ping $TARGET_IP
• From the target system: ping $SOURCE_IP

  You should see both systems pinging the other system correctly.

Replicate partition table from source to target

This will make an exact copy of the source system partition table (including partition UUIDs) on the target system. Then we will adjust it to the different disk size.

1. On target system, start listening for data and save it:

   nc -l -p 2000 > table.txt

2. On source system, dump the partition table and send it over:

   sudo sfdisk --dump /dev/nvme0n1 | nc $TARGET_IP 2000

3. On target system, apply the partition layout to the disk:

   sudo sfdisk /dev/nvme0n1 < table.txt

And then fix the GPT backup header location (because it’s unlikely that you have both disks of exactly the same size):

   sudo sgdisk --move-second-header /dev/nvme0n1

Copy all partitions

Everything should be ready now to transfer all partitions data.

1. Clone the ESP partition (p1):

• On target system, listen and write the data:

   nc -l -p 2000 | sudo dd of=/dev/nvme0n1p1 bs=4M

• On source system, send the data:

   sudo dd if=/dev/nvme0n1p1 bs=4M | pv | nc $TARGET_IP 2000

2. Clone the Boot partition (p2):

• On target system, listen and write the data:

   nc -l -p 2000 | sudo dd of=/dev/nvme0n1p2 bs=4M

• On source system, send the data:

   sudo dd if=/dev/nvme0n1p2 bs=4M | pv | nc $TARGET_IP 2000

3. Clone the System partition (p3):

• On target system, listen and write the data:

   nc -l -p 2000 | sudo dd of=/dev/nvme0n1p3 bs=4M

• On source system, send the data:

   sudo dd if=/dev/nvme0n1p3 bs=4M | pv | nc $TARGET_IP 2000

Expand the System partition on the target system

Now that the target system is a complete clone of the source system, you can expand the System partition to fully utilize the remaining disk space:

1. Install GParted on the target system: sudo dnf install gparted
2. In Gparted, unlock the system partition, if encrypted. Then enlarge it according to your preferences. If your partition was encrypted, make sure that both the LUKS encryption container and the filesystem inside it correctly show up as resized.

Reboot the target system

You can now reboot the target system, and you should see your OS and data exactly the same as on your source system. The process is now complete.

Boot problems

In case the target system has a very different hardware from the source system (e.g. a different brand of a graphics card), the drivers might be missing from the current kernel initramfs and you can see a black screen or kernel errors. In that case, reboot the system again, press F8 repeatedly during boot to get into the GRUB bootloader menu, and boot the rescue Fedora option, instead of the default one. That should hopefully boot using the generic image (which should contain all known drivers), and then you can regenerate all the kernel images using your current hardware setup with: sudo dracut --regenerate-all --force

Mentions of other cloning approaches

1. Using dd
   This works fine when cloning to a larger disk (if you fix the partition table afterwards), but unused blocks are written needlessly (wearing down SSD), and migrating to a smaller drive is problematic.
2. Native Btrfs streams using btrfs send and btrfs receive
   This perfectly copies only used blocks of the main partition, but it only submits a single subvolume. Which means there’s some extra work needed to make sure exactly the same subvolumes are replicated from source to target. Also, this works above the LUKS encryption, which also needs to be created manually on the target (while keeping IDs, etc).
3. Using Clonezilla
   Clonezilla will copy LUKS partitions as raw data, because it can’t see inside. Furthermore, it doesn’t support cloning to a smaller drive by default (there are some tweaks in Expert Mode, but you need to repair the GPT backup table manually anyway).

Source: Kparal's Fedora Blog