Stuart Herbert | Gentoo

My progress towards an AMI for Amazon’s EC2 has hit an unexpected snag. This evening has been one ‘gcc internal error’ after another. Not in the dom0 machine, but in the domU machines. I wasn’t having this problem last week.

I won’t be able to order any replacement RAM until mid-October, so here’s hoping that it’s a software problem.

I did some (very basic) testing of Amazon’s EC2 tools on Gentoo last night. I had no trouble starting or stopping instances, generating keypairs, monitoring instances, or listing the images available on S3. Total cost of last night’s testing was 11 cents US.

Next step is to get a simple ebuild for the EC2 tools done and into Portage. So far, their only DEP is sun-jdk-1.5. I plan on putting them into /opt/ec2-api-tools, and the package will be called app-admin/ec2-api-tools.

Things Learned About EC2

• For computing on demand, EC2 is cheap.
• Although the docs say that it takes minutes to start a virtual machine, I thought they’d done the usual thing of just putting that in there to cover themselves; I thought (especially at the limited beta stage) that a machine would come up in under a minute. Boy was I wrong . I had time to go and make a fresh brew. EC2 looks fine for computing-on-demand in the traditional meaning … but it’ll be an interesting challenge to use it (say) for dynamically increasing webserver capacity.

Things Added To The TODO List

• Create the app-admin/ec2-api-tools package

To reduce the costs involved in developing a working AMI for Amazon’s EC2, I’ve been rebuilding my local dual-Xeon box to run Xen. The instructions on the unofficial Gentoo Wiki worked a treat:

rahvin ~ # uname -a
Linux rahvin 2.6.16.28-xen #3 SMP Mon Sep 11 21:23:30 BST  i686 Intel(R) Xeon(TM) CPU 2.80GHz GNU/Linux
rahvin ~ # xm list
Name                              ID Mem(MiB) VCPUs State  Time(s)
Domain-0                           0      128     4 r-----   903.9

Next step is to getting a working generic Gentoo Xen image built, which can then be used as the template for the LAMP Server image (and more!).

Things Learned About AMI

• The Amazon EC2 domU kernel (the one that AMIs have to work with) has been built with PAE enabled. If you’re building a local Xen test box, make sure that you enable the ‘pae’ USE flag for app-emulation/xen, and that you build your dom0 kernel with PAE enabled.

Things Added To The TODO List

• Package up Amazon’s AMI tools for Gentoo (thanks to Harris)
• Add the kernel modules from Amazon’s domU kernel into the generic Gentoo Xen image
• Learn catalyst, and produce a stage3-i686-2006.1-xen.tgz tarball (anyone interested in doing this?)
• Upgrade the RAM on the Xen test box to 2GB

The online press has been abuzz this week with the news that Amazon’s EC2 service has gone into beta. EC2 is a big Xen server farm in the sky, where you pay by the hour that your Xen virtual machine is running, and by the amount of bandwidth you consume. If you need a server running 24×7, then renting a dedicated server is probably cheaper than using EC2. But … if you only need a server running for part of the day (for example, during office hours), or if you have a service that can scale horizontally, then EC2’s well worth a look.

I’m currently re-installing Gentoo from scratch on my dual-Xeon box, and converting it over to be a local Xen test box, so that I can create some Gentoo AMIs (Amazon Machine Images) locally before uploading them to EC2. I’ll start off with a generic LAMP Server, and see where to go from there.

If you’re interested in lending a hand, give me a shout.