This blog is as much for me as anyone else: If it helps anyone else out there in IT land that’s great and I would love to hear from you, if not it works as a great reference for me and I often refer back to the posts when in need.
This particular issue has occurred over the last couple of days and its one of those which drives you crazy as it just worked last time and you can’ t figure out for the life of you why its not working this time.
I still haven’t worked out why it just worked last time but I finally figured out how to get it to work now and I have a sneaking suspicion it has something to do with differences in AWS regions and how the control ami product codes but I don’t have the time to research deeper into that. I do have time to write how I fixed it though….
C5 instances are great: they are faster and cheaper than their older counterparts but not all AMIs are configured to use them: The Centos6 official AMI is one of these.
Fixing this turned out to be a two part solution the first involved installing and enabling the latest ena driver for enhanced network support onto the image , and the second was to remove the marketplace code from the image to stop AWS automatically telling you the image was not configured to be used on a C5 instance:
Part 1: Installing and enabling the latest ENA driver for enhanced network support onto a centos6 instance
1. Launch an instance from official list of CentOS AMIs.
2. Download and install the ENA driver:
sudo -i # Become root. The rest of the steps assume that they are being run by root user. yum install kernel-devel-$(uname -r) gcc git patch rpm-build wget wget https://github.com/amzn/amzn-drivers/archive/master.zip unzip master.zip cd amzn-drivers-master/kernel/linux/ena make cp ena.ko /lib/modules/$(uname -r)/ # Copy the module to the modules directory insmod ena.ko # Insert the module to validate it loads successfully depmod # Regenerate kernel module dependency map files echo 'add_drivers+=" ena "' >> /etc/dracut.conf.d/ena.conf # Append once dracut -f -v # Generate the new initrd image lsinitrd /boot/initramfs-xxx.el6.x86_64.img | grep ena.ko # Validate that the initramfs image contains the ena driver
The output from the lsinitrd command should contain the driver location lib/modules/…./ena.ko. Confirm that the ena.ko is present in the initramfs to make sure that the ENA driver will get initialized at boot.
Note: If the make command gives a “kcompat.h:219:27: error: net/busy_poll.h: No such file or directory” error, then upgrade the kernel, reboot, and re-run the ENA installation commands from the beginning:
yum upgrade kernel && reboot
3. Configure the Dynamic Kernel Module Support (DKMS) program to make sure that the driver is included during future kernel upgrades.
Install one of the following Red Hat Package Manager (rpm) files:
RHEL 6
yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
RHEL 7
yum install https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
4. Install the DKMS:
yum install dkms
VER=$( grep ^VERSION /root/amzn-drivers-master/kernel/linux/rpm/Makefile | cut -d' ' -f2 ) # Detect current version
sudo cp -a /root/amzn-drivers-master /usr/src/amzn-drivers-${VER} # Copy source into the source directory.
cat > /usr/src/amzn-drivers-${VER}/dkms.conf <<EOM # Generate the dkms config file.
PACKAGE_NAME="ena"
PACKAGE_VERSION="$VER"
CLEAN="make -C kernel/linux/ena clean"
MAKE="make -C kernel/linux/ena/ BUILD_KERNEL=\${kernelver}"
BUILT_MODULE_NAME[0]="ena"
BUILT_MODULE_LOCATION="kernel/linux/ena"
DEST_MODULE_LOCATION[0]="/updates"
DEST_MODULE_NAME[0]="ena"
AUTOINSTALL="yes"
EOM
dkms add -m amzn-drivers -v $VER
dkms build -m amzn-drivers -v $VER
dkms install -m amzn-drivers -v $VER
5. Use the modinfo command to confirm that the ENA module is present.
modinfo ena
6. Append net.ifnames=0 to the boot file to disable network interface naming.
RHEL 6
Append net.ifnames=0 to the kernel line in the /boot/grup/menu.lst file.
vi /boot/grub/menu.lst
RHEL 7
Append net.ifnames=0 to GRUB_CMDLINE_LINUX in the /etc/default/grub file as shown in the following example:
GRUB_CMDLINE_LINUX="selinux=0 console=tty0 crashkernel=auto console=ttyS0,115200 nvme_core.io_timeout=4294967295 net.ifnames=0"
You can also use the sed command:
sudo sed -i '/^GRUB\_CMDLINE\_LINUX/s/\"$/\ net\.ifnames\=0\"/' /etc/default/grub
Then, regenerate the configuration files for grub2:
grub2-mkconfig -o /boot/grub2/grub.cfg
7. Run “poweroff” to stop the instance from an SSH terminal, or stop the instance using the AWS Command Line Interface (AWS CLI) or Amazon EC2 Console.
8. Enable Enhanced Network support at the instance level. The following example modifies the instance’s attribute from AWS CLI:
aws ec2 modify-instance-attribute --instance-id i-xxxx --ena-support --region xx-xxxxx-x
9. Change the instance type to one of the ENA supported instance types.
10. Start the instance, SSH into the instance, and run the ethtool command:
ethtool -i eth0
The output should include the ENA driver version.
Part 2: Removing the MarketPlace Code from the instance to be able to create your new image
1) In the AWS EC2 web console, create a new 8GB EBS volume.
1) Attach this secondary volume to your instance with mount point /dev/xvdj
2) SSH into your instance, and format the secondary volume:
mkfs -t ext4 /dev/xvdj
5) Copy the root volume to the new volume:
(Note: your root volume may appear as /dev/xvda instead of xvde)
dd bs=1M if=/dev/xvde of=/dev/xvdj
6) Once the raw copying has completed, shutdown the instance.
7) Detach the original root volume from the instance. Detach the secondary volume, and re-attach it as /dev/sda1
8) In the EC2 web console, right-click on your instance and select “Create Image”. Once the image process is complete, you will now have a new base CentOS image with no AWS Marketplace code attached to it!
Props go to the AWS blog and Jake on the CentOS Bugtracker only replicated here as I found the two part solution together nowhere else
