HOWTO Upgrade or Install Linux Mint by Keeping Your Data and Settings

MagicMint
  9 years ago
  5

Scope

The 17.x releases of Linux Mint which began with Qiana (will) share with each other not only the long term support until 2019, but also the easy upgradability from such a point release to another: as the software base remains the same (namely Ubuntu 14.04 Trusty Tahr), so do your user environment, and therefore all of your data will remain in place. In other words, you’ll have to adapt to the new features in Cinnamon or MATE only.

But next year at the latest, there will be an updated software base for Linux Mint, so we’ll have to face again that nasty problem of how to avoid copying (maybe) gigabytes of data from users and losing their beloved settings — a problem that everyone switching from another (working) distro or from an older, major release of Linux Mint is still confronted with.

That is, we want to save us from as much work as possible, as opposed to try to save and then to restore lots of data and personal settings which might be very time consuming and might also cause permission issues within the new system… This tutorial simply presumes that you made conscientiously your backups any time in the past wink, just in case something would go wrong, but we don’t want to bother ourselves with that while we’re installing an exciting new version of Linux Mint

Step 1

As you’d probably guess, you must have an external boot medium in order to install a new Linux system. This is the easiest thing in the world:

  1. First of all, download the latest release of Linux Mint to a folder named iso in your home folder. This way, we can use “~/iso” (i.e. “/home/your_user_name/iso”) as a reference location for the “transfer of knowledge” from the old system to the new one.

  2. Check if the download has gone smoothly by right clicking on the ISO file› (in Linux Mint) and then selecting Check MD5. You can do this from the command line (in a terminal) too:
      …$ md5sum ~/iso/*.iso
       
  3. The result must match the checksum displayed on the website of course, otherwise you have to download the ISO image again crying! It’s also a good idea to save that checksum by pasting it into the following file:
      …$ gedit ~/iso/checksum.md5
       
  4. Now, it’s time to “burn” the ISO to a medium, i.e. to a USB stick or a blank DVD. The first kind of medium is faster and better integrated into the file manager of Linux Mint: a right click on the ISO file› or a left click in nemo’s menu on File› Make bootable USB stick — that’s it. If your distro doesn’t allow for that, you can still abide by Clem’s tutorial.

    As far as the live DVD is concerned, you have to ensure that the ISO image is transferred to the disk by your preferred burner application, e.g. brasero, xfburn, etc.

  5. Once finished, eject the live medium, and then re-insert it again. In the folder it opens up or which appears when you click on the symbol of the medium, start a terminal. Therein, you should verify the checksum of your live USB or DVD by typing:
      …$ md5sum -c MD5SUMS
       

 

Step 2

We must now collect some bits of information that we will need in order to conserve the current state of our data before we can definitely freeze the old system for the installation process.

  1. First, you must save your software selection with Mint’s Backup Tool (under Administration› in the menu) to the ~/iso folder, albeit older versions of mintBackup might store it in another folder, e.g. ”/” or ”/home”. Anyway, open the destination folder as Root, and make the backup over to yourself by changing software_selection_…_package.list› Properties› Permissions. Then, you can easily drag it to ~/iso.

  2. On top of that, you must also backup the sources of your additional packages (if any). This is much easier in the terminal:
      …$ mkdir ~/iso/apt
    …$ cp -R /etc/apt/sources* ~/iso/apt
       
  3. As we want to preserve the users’ data, we must know about their whereabouts after installation cheeky. That’s why we need some more command-line magic (assuming you’ve got less than 10 users on your machine):
      …$ cat /etc/passwd | grep 100.: > ~/iso/users.list
       

    Each line of of the resulting ~/iso/users.list will contain one line for each real user account on the system, with seven fields delimited by colons (‘:’) which store the user’s login name, an ’x’, his numerical user ID, his numerical group ID, his full name, the path to his home folder and his command interpreter. Please write down these data for the very first user, as you will need them during the installation, just in a moment when you won’t be able to access that file.


  4. It’s equally important to remember the grouping of the users too. So create a new file with:
      …$ gedit ~/iso/old_groups.sh
       
    Fill this command file with the following contents:
      #!/bin/bash
    # iso/old_groups.sh (MagicMint) O0202
    # Writing out old users’ groups
    
    USERS=~/iso/users.list
    echo -n > ~/iso/groups.list
    
    for user in `cat "$USERS" | cut -d':' -f1`; do
     echo -n "$user": >> ~/iso/groups.list
     sudo -u "$user" groups | tee -a ~/iso/groups.list
    done
    #End of script
       
    Save the script and run it by typing:
      …$ source ~/iso/old_groups.sh
       
  5. At this stage, you should have the following items in your ~/iso folder:
    • a folder named apt;
    • the ISO image ending in .iso;
    • its checksum.md5;
    • a list of your selected packages;
    • the script old_groups.sh;
    • a list of the users on your machine and
    • a list of the groups they belong to.

 

Tip

There are certainly some corrections or improvements you made to your old system. If you had respected some rules, you could re-apply your changes to the new system without hassle (if not, keep those rules in mind until the next upgrade devil):

  • Configuration changes affecting the whole system (as opposed to those of a single user) belong into the folder /etc;

  • Non-standard but shared applications (like additional commands, scripts, etc.) and their resources belong into the folder /usr/local (wine applications and virtual machines are not shared by default, hence they’re always installed in a user’s home folder);

  • Everything else is the domain of package managers and the running system, so don’t touch any other folder, unless in case of inevitable necessity, e.g. when changing the boot background;

  • Document any change of yours, for instance by copying the modified files somewhere in your home directory, i.e. by mirroring the directory structure enlightened for those copies under ~/sys/etc and ~/sys/usr/local.

 

Step 3

Now, reboot into a live session with the medium prepared in Step 1. This is the stage where you should thoroughly test whether the new release of Linux Mint works out for you and your hardware, even in the case you had a well working system, since there could be regressions in a new release. Anyway, you should focus on the following points, because in general the newer the hardware, the likely it is to come across something that declines service yet no (but conversely, chances are that old annoyances are gone yes):

  1. You can use your monitor in its maximal resolution and make use of the extended capacities of your graphics card as far as practicable;

  2. The [Fn] keys, the built-in touchpad (in notebooks) and the touchscreen (if available) work acceptably — an external mouse shouldn’t pose any problem most of the time;

  3. All the network interfaces (Bluetooth, WiFi, LAN) still work;

  4. You can avail yourself as usual of your storage and other external devices that you care of (they connect, and they are readable and writable);

  5. If you encounter any problems here, please google around to find out if there is any chance of a remedy — otherwise you should consider to skip the release and wait for a while, even if the support for your old release had expired already heart.

 

Step 4

Once you’ve made your decision, you should freeze the system. Thereafter, you ought not reboot into that old Linux on the hard disk, because there will be be no place where to login to blush:

  1. Still in the live session, (double) click in the Computer folder on the disk that contains the old system, or more exactly the /home directory, where all of your users reside in user_folders named after them;

  2. Therein, right click on one of the user_folders› and Open it as Root;

  3. Go back in the new window with “Elevated privileges” to the /home folder;

  4. Rename each user_folder to user_folder.old;

  5. Close the disk in the file manager’s side panel.

 

Intermediary Step

This step is totally unnecessary indecision, but only if the /home directory aforesaid has its own partition. Although this sounds logical, because you wish to keep all your data and settings apart while at the same time you’re craving for a newer system which would be more state-of-the-art, the tradition of Windows and other poorly bred distros often misleads people to store anything in the same partition on the disk. However, if you want to separate the data from the rest, the best is to put /home on a partition of its own.

  1. Within the live session, start Menu› Administration› GParted;

  2. Once the application finished scanning your disks, find the root partition of your old system, i.e. the one in which /home is hidden. If you’re lucky, then it’s called “ROOT” (however, for such human-readable names to work, you must not forget blush to label the partitions when formatting them or afterwards). Make that partition ca. 24–32 GiB smaller with (Partition›) Resize/Move and rename it to “HOME”;

  3. In that newly freed space, create by clicking (Partition›) New a partition named “ROOT” of the suggested type and with the proposed parameters. Then, press (Edit›) Apply All Operations and quit the application;

  4. Now, open in the Computer folder the “disk” called HOME, and therein, as root, the folder named home. Once the root window (the one with “Elevated privileges”) opened, go back in it to the disk’s folder, where you should delete everything except the home folder;

  5. Open the home folder, and select the *.old pattern with [Ctrl+S]. The items selected should be the user_folder.old from Step 4, so move (i.e. drag) them all to the disk. Go back to the latter once again, and delete the sole home folder. Close that disk, and your computer should be now in a similar state with regard to the data as in the step before, with the essential difference that you had to delete the old Linux system.

 

Caution

Actually, the whole step above is the only one where things can really go wrong broken heart, if you don’t pay attention, so only do it once for good to never have to re-do it again: in the only case you had made the mistake of installing on a single partition. It’s always better to avoid it altogether by making up the following partitioning scheme (beside the partitions possibly reclaimed by other operating systems):

  • a ROOT partition of the size given above (which is more than generous, by the way: I’m using about 9 GiB on that partition right now);

  • a SWAP partition of about 150% the size of your RAM. If the latter is sufficient for your everyday work, you won’t (and ought to) use the this partition for anything else but hibernation (the powerless sleep mode of your computer);

  • a HOME partition which takes up the rest of the free space on your disk.

 

Step 5

Now comes the moment that you had so much longed for: the installation of your new Linux Mint — but you should do it by trying not to erase wink your old data. If you proceed like the following, the tenet of a fresh upgrade won’t be violated, although you’ll preserve your personal data while getting it done way much faster:

  1. On the desktop of the live session, (double) click on Install Linux Mint. When asked for the Installation type, you must choose the one which lets you freely tune the partitions. In Rebecca’s installer, the proper option is called:
      Something else
    You can create or resize partitions yourself, or choose multiple partitions for Linux Mint
       
  2. Provided that you don’t need to resize the partitions — we’re talking about installing a new system in place of a working old one, therefore you only have to Change their Mount points —, the only thing you have to keep in mind is that the ROOT and SWAP partitions must be re-formatted, whereas HOME must not. Under no circumstances!

  3. The name fields in the dialog “Who are you?” must be filled with the full and user (i.e. login) names of that very first user from Step 2. This is mandatory, but the other fields can be chosen freely.

  4. Finish the installation, but do not restart the computer!

  5. From within the live session, you could reinstate those of your previous customizations which you estimate still necessary for the new release, e.g. changes made in /etc/fstab owing to SSD optimization or concerning a Windows drive to be mounted at boot time — always provided that you still can access these customizations, for example by having heeded the advice in Step 2

 

Step 6

Before rebooting, you must implicitly restore the first user’s data yet:

  1. Open the Computer folder again, and therein the (new) HOME disk;

  2. Right click on the only user_folder which has no “.old” suffix in its name, and Open it as Root;

  3. In the root window that pops up, the menu option View› Show Hidden Files must be ticked, in as much as you ought to Edit› Select All or to press [Ctrl+A] in order to delete everything in that folder;

  4. Open up a new tab for the HOME disk, e.g. by right-clicking in the location bar on the disk button› Open in New Tab. Therein, go to the corresponding user_folder.old, select all of its contents, and move (i.e. drag) it to the other folder, the new and now empty user_folder with the same name;

  5. As soon as this (almost instantaneous) operation is completed, i.e. the old folder is emptied, you can safely delete it. After closing the disk, you’re done with the live session.

 

Step 7

After having rebooted into your brand new release of Linux Mint, you can proceed with restoring the old state of your user environment. Again, there is a determinate sequence of moves to follow:

  1. Re-create all the users from ~/iso/users.list if the iso folder belongs to the first user, otherwise from the right /home/user_folder.old/iso/users.list — as you can see, it’s the order of creation of the users which is reflected by their user ID number (beginning with 1000) that matters. This order must be respected in order to avoid any permission issues with data or commands (this also holds for backups made on Linux partitions!).

    First, Add the users one by one in Menu› Administration› Users and Groups› Users. Then, click on each user in turn to verify that their Groups match those in groups.list (from the same backup folder as above). If not, click on the user’s groups list and rectify it — the correct user groups have a meaning inasmuch as certain features like folder sharing or printer administration won’t work without them. After that, you can quit the Users & Groups application, and open My Computer› File System in the file manager.

    Here, Open As Root the /home folder, and as such, make sure that View› Show Hidden Files is still ticked, and click View› Extra Pane or press [F3]. For each user you’ve just created, you must select all the contents in its user_folder, delete it, and replace it with the contents of the corresponding user_folder.old. After which you can delete all the *.old folders in /home (all this is essentially the same procedure as in Step 6 for the first user);

  2. Now, you should turn to Menu› Administration› Software Sources. For your Official repositories, you should select the fastest Mirrors available to you. If you don’t need Source code, don’t enable it, and keep the hands off the backport and romeo packages;

  3. As far as PPAs and Additional repositories are concerned, the best way to restore them is to google the contents of the files in the apt subfolder of your iso folder, and to add them manually to the software sources, since the structure of the /etc/apt system folder and the single repositories on the internet could have been changed from the old software base to the new one.

    The PPAs (Personal Package Archives) are specific to Ubuntu as they depend on its particular releases most of the time, and they need Authentication keys from the Ubuntu key servers;

  4. You should check now if you had really smiley brought your software sources up to date. This is the case only when the following command runs through with no error messages:
      …$ sudo apt-get update
       

    If any of the repositories generates a “Packages 404 Not Found” message, then you ought to correct its address, if you can. If not, then you should deactivate that repository altogether for now.

    Now, click on the Update Manager in the panel, Edit› [its] Preferences, most notably the Auto-Refresh period which is unnecessarily short in general, and Install [the] Updates.

  5. Before getting down to the last thing left to do, the restoration of your old software packages, you should install a small but useful package called localepurge in order to save some space on your ROOT partition by keeping unnecessary language packs from installing (if you don’t know about locales, just skip this):
      …$ sudo apt-get install localepurge
        

    At the very last, start Menu› Administration› Backup Tool, click on Restore software selection, and choose the software_selection_…_package.list from the iso folder. This operation will re-install all of your previously used packages as a whole — and that’s it cool.

 

Notice

Be aware that depending on your formerly installed applications, the very last step above can take a very long time to complete: while the installation from the USB stick has more of a coffee break, the software restoration feels more like installing Windows (without repeated restarts, however wink), i.e. in my case a ¼ of an hour vs. 1½ hours. But you should realize (what software reviews usually don’t) that all things considered, twice (or more times) the contents of the live DVD worth in packages must be downloaded and installed at once, whereas this process was evenly stretched over the whole lifetime of your previous system.

Step 8

As a final touch, you could re-work your customizations from the old system, insofar as you had saved them after Step 2. Remember, however, that we didn’t saved anything from the old /root folder which is the place for custom settings for any administrative task performed as a super user, like system profiles of luckyBackup, e.g. Such settings can only be restored from external backups with the right permissions.

Before you can fully enjoy your shiny new Linux Mint, you should fix as yet the problems you ran into in Step 3. But that’s what the Community is for wink

External sources

This tutorial is just a detailed synthesis from my own experiences, a lengthy discussion on the forums and another resource that didn’t work satisfactorily for me crying.

Comments
BobS540 8 years ago

Great article, but was written a year ago. I am wondering if it will apply to going from Linux Mint 17.3 to 18.1? Th
anks.


DarrenG 9 years ago

Great article, thanks for your hard work in compiling it.

I also agree with http://community.linuxmint.com/idea/view/4728 that mounting /home on different partition during installation of Linux Mint should be the default option. I think it would save people a lot of grief.


Rebel450 9 years ago

its a great howto.

(in my personal opinion, which does not matter at all,
I find it a bit old school - No, dont shoot at me, please.
I imagine, how long it will take by your description given -
and if it is really worth all that time spending.
I love solutions with a (modern) graphical interface doing the job;
I worked with aptik and mint backup - for me it was fine that way.
Of course, on your way as per your description, you have much more control
all over - but you need also a deeper knowledge, which most people does not have ... as said before - just my personal opinion)

In any case: This is a super how2 with deep background: +
.


MagicMint 9 years ago

@zman58: That’s exactly what’s you’re supposed _not_ to do. Here is why:

1. In case of multiple users, you’re more than likely to run into permission issues, as external media are of type FAT32 or NTFS most of the time;
2. Even Clem recommends a separate home (data) partition in his MacBook installation guide (http://community.linuxmint.com/tutorial/view/1643);
3. You must still know the _order_ of creation of your users;
4. This operation might be lengthy if you have lots of data (think of pictures, music, videos, etc.), and awful if you have multiple users. The aim of this tutorial is just to _avoid_ this really unnecessary (and repeated) step, all the more as it’s unwise to mix old and new settings;
5. How do you know the permissions of _each_ file you’ve swapped out and back?


zman58 9 years ago

I personally would just:

1. Move my, and other users, personal data to external storage media.
2. Install Linux Mint 17.1 from scratch. Reclaim all hard drive space.
3. Create users as before.
4. Restore users data from external storage media.
5. Restore proper permissions on imported user data.

Bookmarks and email are easy to move this way.


jrwilson 9 years ago

I'm not skilled with using the terminal. So I will save all I want to keep, on another medium. Flash drive of some type.
I was hoping I could fix my freezing up problem,by using my 17 Qiana DVD.
Guess it don't work that way. Thanks


ConorCork 9 years ago

+1. A lot of work went into presenting this detailed step by step Guide. Thanks.


MagicMint 9 years ago

@Hammer459: That’s why I mourn my past Gentoo days sometimes, although its first and only install wiped out my RH 6 and Windows partitions ;-)


Hammer459 9 years ago

@MagicMint I agree mostly with what you say. I am a strong proponent of incremental install upgrades. Redhat has used that since RH 5. Never had problems. HPUX had that with no problems since before 94. and so on....
But Ubuntu and Mint claims it is too much risk and that the destructive reinstall is a safer method. I have not have any problems with upgrades of Unix/linux between 1985 and 2010. I have not upgraded Mint easily and safely, not even once with Mint.


MagicMint 9 years ago

@Hammer: You should read more carefully. This tutorial is linked in above. The problem with those tutorials from the beginnings of Linux Mint is that they do _not_ treat the problem fixed here: retaining both settings _and_ data. The question of the permissions in case of external backups is an important one, and the restoring of gigabytes of data is _never_ trivial nor fast as lightning. If it was, mintBackup would receive more positive reviews, I think ;-)

By the way, I hate loosing years of work, just because nobody cares about updating data… It’s always a risky job to upgrade, and as far as I saw it, even the installer of LM 17.1 cannot be trusted truly :-(


Hammer459 9 years ago

Or you could use this _Official_ tutorial on the subject...
http://community.linuxmint.com/tutorial/view/2