Some time ago I’ve written about migrating a CentOS 8 machine to Red Hat 8, because CentOS 8 reached end of life last December. We did that for all CentOS 8 installations at that customer and all are running as virtual machines. The very same customer has many EC2 instances running on CentOS 7. Migrating away from CentOS 7 is one of the next tasks to complete, as CentOS 7 will reach end of life in 2024. For doing that, you basically have two options (if you want to stay inside the Red Hat family): Either you deploy your target distribution (either Rocky Linux, Alma Linux, Oracle Linux or Red Hat Enterprise Linux) from an existing AMI and then re-install everything you need and restore the data. Another option would be to directly upgrade from CentOS 7 to whatever Red Hat based Linux version 8. We’ll go for the second option in this post.

Before we start: Of course you need a backup of your EC2 instance before doing this! Of course you really need to test your application after the upgrade because a lot of packages and the kernel will change.

You might wonder how this will work at all, because there is no supported migration path from CentOS 7 to CentOS 8. The people behind Alma Linux stepped into this and created a project called Elevate to support exactly those migrations. You can use that tool to migrate from CentOS 7 to AlmaLinux 8, to Rock Linux 8 and to Oracle Linux 8.

My starting point is a fresh EC2 instance based on the latest CentOS 7 AMI:

[centos@ip-10-0-1-100 ~]$ cat /etc/centos-release
CentOS Linux release 7.7.1908 (Core)
[centos@ip-10-0-1-100 ~]$ uname -a
Linux ip-10-0-1-100.eu-central-1.compute.internal 3.10.0-1062.12.1.el7.x86_64 #1 SMP Tue Feb 4 23:02:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

First thing to do: Update the system to the latest release and reboot:

[centos@ip-10-0-1-100 ~]$ sudo yum update -y
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: download.cf.centos.org
 * extras: download.cf.centos.org
 * updates: download.cf.centos.org
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/4): base/7/x86_64/group_gz                                                                                                                                                                                                                          | 153 kB  00:00:00     
(2/4): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 243 kB  00:00:00     
(3/4): updates/7/x86_64/primary_db                                                                                                                                                                                                                     |  13 MB  00:00:00     
(4/4): base/7/x86_64/primary_db                                                                                                                                                                                                                        | 6.1 MB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package acl.x86_64 0:2.2.51-14.el7 will be updated
---> Package acl.x86_64 0:2.2.51-15.el7 will be an update
...
  sg3_utils-libs.x86_64 1:1.37-19.el7                   shared-mime-info.x86_64 0:1.8-5.el7                     sudo.x86_64 0:1.8.23-10.el7_9.2                         systemd.x86_64 0:219-78.el7_9.5                      systemd-libs.x86_64 0:219-78.el7_9.5           
  systemd-sysv.x86_64 0:219-78.el7_9.5                  teamd.x86_64 0:1.29-3.el7                               tuned.noarch 0:2.11.0-11.el7_9                          tzdata.noarch 0:2021e-1.el7                          util-linux.x86_64 0:2.23.2-65.el7_9.1          
  vim-minimal.x86_64 2:7.4.629-8.el7_9                  virt-what.x86_64 0:1.18-4.el7_9.1                       wpa_supplicant.x86_64 1:2.6-12.el7_9.2                  xfsprogs.x86_64 0:4.5.0-22.el7                       yum.noarch 0:3.4.3-168.el7.centos              
  yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8     yum-utils.noarch 0:1.1.31-54.el7_8                      zlib.x86_64 0:1.2.7-19.el7_9                           

Replaced:
  iwl7265-firmware.noarch 0:22.0.7.0-72.el7                                                                                                                                                                                                                                   

Complete!
[centos@ip-10-0-1-100 ~]$ sudo reboot
...
[centos@ip-10-0-1-100 ~]$ cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
[centos@ip-10-0-1-100 ~]$ uname -a
Linux ip-10-0-1-100.eu-central-1.compute.internal 3.10.0-1160.49.1.el7.x86_64 #1 SMP Tue Nov 30 15:51:32 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Install the elevate repository:

[centos@ip-10-0-1-100 ~]$ sudo yum install -y http://repo.almalinux.org/elevate/elevate-release-latest-el7.noarch.rpm
Loaded plugins: fastestmirror
elevate-release-latest-el7.noarch.rpm                                                                                                                                                                                                                  | 6.9 kB  00:00:00     
Examining /var/tmp/yum-root-NQPt77/elevate-release-latest-el7.noarch.rpm: elevate-release-1.0-1.el7.noarch
Marking /var/tmp/yum-root-NQPt77/elevate-release-latest-el7.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package elevate-release.noarch 0:1.0-1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                         Arch                                                   Version                                                      Repository                                                                          Size
==============================================================================================================================================================================================================================================================================
Installing:
 elevate-release                                                 noarch                                                 1.0-1.el7                                                    /elevate-release-latest-el7.noarch                                                 3.4 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package

Total size: 3.4 k
Installed size: 3.4 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : elevate-release-1.0-1.el7.noarch                                                                                                                                                                                                                           1/1 
  Verifying  : elevate-release-1.0-1.el7.noarch                                                                                                                                                                                                                           1/1 

Installed:
  elevate-release.noarch 0:1.0-1.el7                                                                                                                                                                                                                                          

Complete!

Elevate uses Leapp in the background so we need to install the corresponding packages:


[centos@ip-10-0-1-100 ~]$  sudo yum install -y leapp-upgrade leapp-data-rocky
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: download.cf.centos.org
 * extras: download.cf.centos.org
 * updates: download.cf.centos.org
elevate                                                                                                                                                                                                                                                | 3.0 kB  00:00:00     
elevate/x86_64/primary_db                                                                                                                                                                                                                              | 6.7 kB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package leapp-data-rocky.noarch 0:0.1-2.el7 will be installed
---> Package leapp-upgrade-el7toel8.noarch 0:0.14.0-100.202109271224Z.b7ebfca.master.el7.elevate will be installed
--> Processing Dependency: leapp-repository-dependencies = 6 for package: leapp-upgrade-el7toel8-0.14.0-100.202109271224Z.b7ebfca.master.el7.elevate.noarch
--> Processing Dependency: leapp-framework < 3 for package: leapp-upgrade-el7toel8-0.14.0-100.202109271224Z.b7ebfca.master.el7.elevate.noarch
...
Dependency Installed:
  deltarpm.x86_64 0:3.6-3.el7                                                     dnf.noarch 0:4.0.9.2-2.el7_9                                                            dnf-data.noarch 0:4.0.9.2-2.el7_9                                                                 
  leapp.noarch 0:0.12.1-100.20210924142320684911.master.28.g1f03432.el7           leapp-deps.noarch 0:0.12.1-100.20210924142320684911.master.28.g1f03432.el7              leapp-upgrade-el7toel8-deps.noarch 0:0.14.0-100.202109271224Z.b7ebfca.master.el7.elevate          
  libcomps.x86_64 0:0.1.8-14.el7                                                  libdnf.x86_64 0:0.22.5-2.el7_9                                                          libmodulemd.x86_64 0:1.6.3-1.el7                                                                  
  librepo.x86_64 0:1.8.1-8.el7_9                                                  libreport-filesystem.x86_64 0:2.1.11-53.el7.centos                                      libsolv.x86_64 0:0.6.34-4.el7                                                                     
  pciutils.x86_64 0:3.5.1-3.el7                                                   python-enum34.noarch 0:1.0.4-1.el7                                                      python2-dnf.noarch 0:4.0.9.2-2.el7_9                                                              
  python2-hawkey.x86_64 0:0.22.5-2.el7_9                                          python2-leapp.noarch 0:0.12.1-100.20210924142320684911.master.28.g1f03432.el7           python2-libcomps.x86_64 0:0.1.8-14.el7                                                            
  python2-libdnf.x86_64 0:0.22.5-2.el7_9                                         

Complete!

If you want to migrate to one of the other distributions, replace the “leapp-data-rocky” package with one of these:

  • leapp-data-almalinux
  • leapp-data-oraclelinux
  • leapp-data-rocky

Time for the pre-ugrade check:

[centos@ip-10-0-1-100 ~]$ sudo leapp preupgrade
==> Processing phase `configuration_phase`
====> * ipu_workflow_config
        IPU workflow config actor
==> Processing phase `FactsCollection`
====> * firewalld_facts_actor
        Provide data about firewalld
====> * source_boot_loader_scanner
        Scans the boot loader configuration on the source system.
====> * repository_mapping
        Produces message containing repository mapping based on provided file.
====> * read_openssh_config
        Collect information about the OpenSSH configuration.
====> * scandasd
        In case of s390x architecture, check whether DASD is used.
====> * rpm_scanner
        Provides data about installed RPM Packages.
...
====> * target_userspace_creator
        Initializes a directory to be populated as a minimal environment to run binaries from the target system.
Rocky Linux 8 - PowerTools                      6.8 MB/s | 2.5 MB     00:00    
Rocky Linux 8 - Extras                           42 kB/s |  10 kB     00:00    
Rocky Linux 8 - AppStream                       7.4 MB/s | 8.7 MB     00:01    
Rocky Linux 8 - BaseOS                          4.2 MB/s | 4.6 MB     00:01    
Rocky Linux 8 - HighAvailability                743 kB/s | 545 kB     00:00    
Dependencies resolved.
================================================================================
 Package                     Arch   Version              Repository        Size
================================================================================
Installing:
 dnf                         noarch 4.7.0-4.el8          rocky8-baseos    543 k
 dnf-plugins-core            noarch 4.0.21-3.el8         rocky8-baseos     69 k
...
Check completed.
====> * tmp_actor_to_satisfy_sanity_checks
        The actor does NOTHING but satisfy static sanity checks
====> * check_initramfs_tasks
        Inhibit the upgrade if conflicting "initramfs" tasks are detected
==> Processing phase `Reports`
====> * verify_check_results
        Check all dialogs and notify that user needs to make some choices.
====> * verify_check_results
        Check all generated results messages and notify user about them.

============================================================
                     UPGRADE INHIBITED                      
============================================================

Upgrade has been inhibited due to the following problems:
    1. Inhibitor: Missing required answers in the answer file
Consult the pre-upgrade report for details and possible remediation.

============================================================
                     UPGRADE INHIBITED                      
============================================================


Debug output written to /var/log/leapp/leapp-preupgrade.log

============================================================
                           REPORT                           
============================================================

A report has been generated at /var/log/leapp/leapp-report.json
A report has been generated at /var/log/leapp/leapp-report.txt

============================================================
                       END OF REPORT                        
============================================================

Answerfile has been generated at /var/log/leapp/answerfile

Have a look at the generated answer file and the report. Any issues you need to fix are mentioned there. In my case, this is what I had to do:

[centos@ip-10-0-1-100 ~]$ echo PermitRootLogin yes | sudo tee -a /etc/ssh/sshd_config
PermitRootLogin yes
[centos@ip-10-0-1-100 ~]$ sudo leapp answer --section remove_pam_pkcs11_module_check.confirm=True

Once this is done, kick off the migration:

[centos@ip-10-0-1-100 ~]$ sudo leapp upgrade
==> Processing phase `configuration_phase`
====> * ipu_workflow_config
        IPU workflow config actor
==> Processing phase `FactsCollection`
====> * firewalld_facts_actor
        Provide data about firewalld
...
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
==> Processing phase `InterimPreparation`
====> * efi_interim_fix
        Adjust EFI boot entry for first reboot
====> * upgrade_initramfs_generator
        Creates the upgrade initramfs
====> * add_upgrade_boot_entry
        Add new boot entry for Leapp provided initramfs.
A reboot is required to continue. Please reboot your system.


Debug output written to /var/log/leapp/leapp-upgrade.log

============================================================
                           REPORT                           
============================================================

A report has been generated at /var/log/leapp/leapp-report.json
A report has been generated at /var/log/leapp/leapp-report.txt

============================================================
                       END OF REPORT                        
============================================================

Answerfile has been generated at /var/log/leapp/answerfile

Looks all fine, the final step is a reboot:

[centos@ip-10-0-1-100 ~]$ sudo reboot

Don’t panic, this reboot takes time. If you are using AWS EC2 you can use the Serial Console to check what is going on:

After some minutes the EC2 instance should be back:

[centos@ip-10-0-1-100 ~]$ cat /etc/rocky-release
Rocky Linux release 8.5 (Green Obsidian)
[centos@ip-10-0-1-100 ~]$ uname -a
Linux ip-10-0-1-100.eu-central-1.compute.internal 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Tue Dec 21 19:02:23 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Quite easy and straight forward, but this was an installation without any user data, modifications to the system or third party applications.