There are many ways to add additional IP addresses to EC2 in support of various types of projects. And the documentation is pretty good when you want to add additional Elastic Network Interfaces (ENI) or if you are using an Amazon Linux AMI that provides support for ec2-net-utils and/or if you are planning on multihoming/load balancing.
I recently needed to do something much more simple than is typically provided for in the documentation. I had a single Amazon EC2 instance running Red Hat Enterprise Linux 7 (RHEL) and I wanted to add a second public IP address to it. Furthermore I wanted to do it in the most straight-forward way: without adding an additional ENI - which is the equivalent of adding a secondary physical network interface - which would require me to make additions to my routing table I didn't want to bother with. For test cases, think of adding several SSL certificates or a shared hosting web server - several IPs, one subnet, easy.
Unfortunately things are a bit more complex with EC2.
There are good reasons for this additional complexity. For one, NAT has to be a part of this picture because EC2 depends on it for a whole host of reasons; to be able to keep you IP (almost) immediately consistent across multiple virtual machines, for load balancing, for fail over, and many other reasons too exciting to spend time on here. For my use case, this meant I had to configure a new private IP address to go with my public IP address.
The second reason for the extra complexity is that EC2 depends on DHCP (which, in turn, is required for all the reasons we just briefly outlined). Assignment of a static IP address for your primary network interface in EC2 is a big no-no. I haven't taken a look lately but if my memory is correct on a reboot the cloud-init scripts that come pre-packaged in standard Amazon EMI's will blow out static assignments and replace them with DHCP. Needless to say I didn't want to really get into the nitty-gritty of Amazon's network architecture.
I just wanted a damn second IP address.
Typically with Linux the solution to adding multiple IP addresses to the same interface is really quite straight-forward; particularly when you are assigning those IP addresses within the same subnet. The method is called IP aliasing, and involves the creation of "virtual" network interfaces by adding one or more network initialization scripts. In RHEL, those scripts are stored in a series of files within /etc/sysconfig/network-scripts/ (in Ubuntu they are stored in a single /etc/network/interfaces file - but this walkthrough is focused on RHEL because there is already documentation for Ubuntu).
In this scenario, to add additional IPs to my existing NIC, I would just copy the network-script for my NIC - which by default would be /etc/sysconfig/network-scripts/ifcfg-eth0 - to a new file that prependeds ":0" to the end of the file name, like this:
#cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth0:0
Additional IPs can be added simply by incrementing the last digit (ifcfg-eth0:1, ifcfg-eth0:2, ifcfg-eth0:3, etc).
I would have to make some changes inside the new file itself as well. Let's say this was the content of my eth0 file:
BOOTPROTO=static
NETMASK=255.255.255.0
TYPE=Ethernet
ONBOOT=yes
HWADDR=00:10:17:24:bf:77
HWADDR=00:10:17:24:bf:77
GATEWAY=192.168.1.1
IPADDR=192.168.1.2
#systemctl restart network.service
IPADDR=192.168.1.2
Copying it with 'cp' as outlined above would give me a duplicate of this file, but to get it working I would need to change the DEVICE and IPADDR fields to indicate the new IP. The DEVICE field should match the file name assigned to the configuration file, which also indicates the name of the virtual interface. In this example, it would be eth0:0. I also need to change the IPADDR to indicate the new IP I want - let's say I want it to be 192.168.1.3 in this scenario. So this is what the new file would look like:
DEVICE=eth0:0
BOOTPROTO=static
NETMASK=255.255.255.0
TYPE=Ethernet
BOOTPROTO=static
NETMASK=255.255.255.0
TYPE=Ethernet
ONBOOT=yes
HWADDR=00:10:17:24:bf:77
HWADDR=00:10:17:24:bf:77
GATEWAY=192.168.1.1
IPADDR=192.168.1.3
IPADDR=192.168.1.3
Once that is set up, I should test the new interface by trying to activate it individually using the "ifup" command:
#ifup eth0:0
If it works without issue, I'm all set. If errors occur, I should start troubleshooting. Alternatively, restarting the network service would also raise the interface:
#service network restart
or if you are using systemd instead of init:
I could change this behavior by setting the ONBOOT flag to "no" within the configuration file.
Anyway - this is all pretty easy right? IP aliasing! Anyone can do it!
Here's the problem - none of this works with EC2. It doesn't work with EC2 because, as we mentioned, ENIs must be configured to use DHCP. This is what /etc/sysconfig/network-scripts/ifcfg-eth0 typically looks like in EC2:
DEVICE="eth0"
BOOTPROTO="dhcp"
ONBOOT="yes"
TYPE="Ethernet"
USERCTL="yes"
PEERDNS="yes"
IPV6INIT="no"
Unfortunately, it is impossible to use IP aliasing with a primary network interface that is configured to use DHCP. Here is how CentOS elegantly puts it in their documentation:
Trying to configure an Alias will result in an error as soon as the interface attempts to load. So don't even bother.
Before I provide the solution for dealing with this routing issue, let's make sure you can jump through the hoops you need to do with Amazon itself.
Log into your EC2 console, and select Instances. Right click the instance you would like to add an IP to, select Networking and then Manage Private IP Addresses.
A new menu will pop up. Click Assign New IP and enter the Private IP address that you wish to select. This IP should be within the subnet already assigned to your primary interface - which shouldn't be a problem, because by default it is a /20. You will not select your Public IP here, so just click Yes, Update once you have entered your Private IP.
Next we will be selecting Elastic IPs from the Network & Security group on the left menu column. From the Elastic IP menu, select Actions and Allocate New Address.
Your new public Elastic IP (EIP) will appear in the menu. Highlight the radio button next to the new EIP, go to Actions again and this time select Associate Address to launch the menu in the image below.
It is very important that you select a Network interface and not an instance in this menu. Selecting an instance will replace your pre-existing EIP with your new EIP instead of adding onto it!
If you only have one Instance with one ENI, than only one Network Interface will appear here. If you have multiple Instances be sure that you select the correct Network Interface. You can see which interfaces are assigned to which instances in the Instance menu.
Once you select a Network interface you will be able to select the Private IP Address that you assigned earlier. One you select it, click the blue Associate button (leave the Reassociation checkbox blank).
With all of that done, you should be able to see the association between your new public and private IPs in the Elastic IPs menu. However, if you try to ping your public IP from out of the network, or even ping the private IP locally from your instance, you will get timeouts. Let's resolve this by returning to the routing issue we discussed earlier.
From your user's home directory, create a file and add the following text using your favorite editor:
#!/bin/bash
#add routes for secondary IP addresses
MAC_ADDR=$(ifconfig eth0 | sed -n 's/.*ether \([a-f0-9:]*\).*/\1/p')
IP=($(curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/$MAC_ADDR/local-ipv4s))
for ip in ${IP[@]:1}; do
echo "Adding IP: $ip"
ip addr add dev eth0 $ip/20
done
For use cases that involve an interface other than eth0, or a private subnet allocation other than the EC2 default /20, this second to last line will need to be changed as well:
ip addr add dev eth0 $ip/20
For example, let's say I am using an Ubuntu system and wish to add a secondary IP address to an interface named eth2, and I am using a non-default private subnet that is a single class C (/24). I would use this script instead:
#!/bin/bash
#add routes for secondary IP addresses
MAC_ADDR=$(ifconfig eth2 | sed -n 's/.*HWaddr \([a-f0-9:]*\).*/\1/p')
IP=($(curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/$MAC_ADDR/local-ipv4s))
for ip in ${IP[@]:1}; do
echo "Adding IP: $ip"
ip addr add dev eth2 $ip/24
done
Notice the curl command pulling from the 169.* IP address? That is how we call EC2 Instance Metadata and User Data. Using the Instance Data API is incredibly useful for being able to pull information about your instance in situations like ours where statically storing that data is either impossible or inconvenient.
Save the file and add an executable bit. I named by file "ip-script.bash", so to add an executable bit I performed this command from the same directory as the script:
#chmod +x ip-script.bash
I can then execute the script in order to complete the routing configuration for the new secondary IP (NOTE this script will handle multiple secondary IP addresses):
# ./ip-script.bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 25 100 25 0 0 24582 0 --:--:-- --:--:-- --:--:-- 25000
Adding IP: 192.168.1.3
If successful, you should now be able to ping the public IP from outside your Instance and receive a response (provided your firewall and EC2 Security Group policies allow ICMP traffic from the source of the ping). Alternatively, you could use the following commands to confirm everything is as it should be.
This command will return the public IP bound to the private IP provided in the privateipaddress field below. If this command times out or produces an error, something has gone wrong:
# curl --interface privateipaddress ifconfig.me
You will also want to check your routing table:
# route -n
If this method has been performed exactly as described in this walkthrough - on an instance with a single ENI and a single private IP subnet allocation, but with multiple public and private IPs, then your routing table should look something like this:
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 eth0
192.168.1.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0
One of the more common mistakes is to use a different netmask when assigning the secondary private IP address, even though that secondary private IP is part of the existing private IP allocation. When that occurs, it would look something like this (in this example, the user put a /24 netmask on the secondary private IP instead of the correct /20):
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 eth0
192.168.1.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0
192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
So with all of this done there is still one thing left to do. As is, the changes made to the network interface will not persist after a restart of the network service, or a reboot of the instance.
There are several ways to resolve this last problem - I will be adding a second post with a few of those solutions shortly. Stay tuned!
No comments:
Post a Comment