I previously wrote about automatically retrieving ssh host Ids from the AWS EC2 instance console, but what would be really neat is if we could somehow wrap all this up to automatically sync EC2 instances in the ~/.ssh/known_hosts file with AWS EC2 inventory. This allows for this being incorporated in automation (eg. Ansible) to be able to autonomously and securely maintain all your instances.

The good news is that for the past months, armed with a Hybrid Tablet PC, I've raced other train commuters for seats, stood around helpless with delays, grappled with flaky mobile connectivity (tethering is definitely much more reliable than the free WiFi on all the trains I was on!) and managed to code this up as one of my commute projects.

How it works

When an EC2 instance boots for the first time it generates it's ssh host keys and outputs the fingerprints to the console. Retrieving this data can take some time as the console output generally only comes available on the API minutes after boot. This might be a problem where your application requires fast turnaround where you may have to consider other approaches (eg. securing a channel / route rather than relying on authentication over a public channel).

If an instance is shut down then it will almost certainly come back with a different IP which is not something that ssh copes with - the key will no longer match. This means that we need to track instances by instance Id (the i-xxxxxxx number) and if IPs have changed we need to keep the keys (only available on the first boot) but updated the IPs to keep the known_hosts file aligned with the instances.

The script tries to handle all the scenarios I know about but there is always risk of something going wrong so you should backup your ~/.ssh when using this (or anything else similar) in case it goes wrong.

In use

I've tried to make it easy. It uses the config from aws-cli (~/.aws/conf) so there should be no need for anything further unless you need multiple regions or anything like that in which case the script will need some extra work.

The current version will output the new known_hosts content to STDOUT which means you need to handle it yourself (preferably with backing up existing files you replace) but in time I hope to have some time to get all that in this script.

You may do something like:

$ aws_sync_keys.py >~/.ssh/known_hosts.new
$ mv -i ~/.ssh/known_hosts ~/.ssh/known_hosts.bak
$ mv -i ~/.ssh/known_hosts.new ~/.ssh/known_hosts

.... or whatever approach is appropriate for your case.

This is very much part developed at this stage so apply due care when using it.

Download: aws_sync_keys.py is in my Cloud Tooling GitHub repo