Woo! You're reading part 1 of a trilogy:
- Part 2 - Running LAMP, Ubuntu...
- Part 3 - Running LAMP, Ubuntu... (in progress)
Quickie Introduction to EC2
For those of you that get bored with paragraph-style introductions to blog entries, here's a bulleted list for you:
- End result feels like a VPS, except you can change the specs of your VPS in seconds.
- Ability to launch a bunch of identical servers in seconds:
- Kickass deployment: bring up a 2nd, identical server, push your new site version there, test vigorously, point amazon to use your new instance, shutdown the old instance.
- Play around: Start up an identical server except with faster hardware. Benchmark and playaround. When you're done playing, shutdown the instance - you're only charged by the hour
- By definition, you full server image is always saved to S3, so you have an exact copy of your setup in a very safe place
- Backup your dynamic data (databases, uploads dir) to S3 in seconds
- Basically, you can do a ton of cool stuff - I'm a believer!
The Setup
OS | Ubuntu |
---|---|
Web server | Apache 2.2 with MySQL & PHP5 |
End Use | Symfony 1.2.8 Web Application |
Mail Server | Nope - this is being outsourced to google, no need for the headache or overhead |
Signing Up & Downloading your Private Key and Certificate files
To get started, you'll need both an EC2 account and an S3 account.
- EC2 - The actual cloud computing account
- S3 - Storage, used to save images of your server
After you signup, you'll need to download 2 files: a Private Key and a Certificate file. These are basically the keys to the mansion. Here's a great resource on this simple step:
Whenever you start an instance [turn your "server" on], you must select a fully-functional image that you want that server to use. Fortunately, the ec2 community has published thousands of images for all operating systems and setups that you will use to boot from initially. From there you can customize your server exactly as you need it. When you're done, you need to save your new, customized image somewhere so you can boot from it. This is done easily in EC2 and stores the image on your S3 account. Once there, you can boot any new instances from your full-customized server image.
EC2 Tools
Basically, there are 2 ways to screw around with EC2:
- Command line, via the awesome ec2-tools
- Gui web interface, via the awesome Amazon console
In reality, you'll need to use both tools. First, let's setup your local computer to use the EC2 tools
sudo apt-get install ec2-api-tools sudo apt-get install openjdk-6-jdk
Remember that Private Key and that Certificate file you downloaded from Amazon earlier? You'll need those here. Basically, the ec2 tools look for those files in a certain shell variable. Also, you'll need another variable for JAVA, which ec2 uses. Paste the following into your ~/.bashrc file:
export EC2_PRIVATE_KEY=$HOME/<where your private key is>/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem export EC2_CERT=$HOME/<where your certificate is>/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
Starting up your First Instance
Starting an "instance" basically means that you push the "power" button on one of Amazon's virtual servers. The cool part is that in addition to turning a machine on, you also specify exactly what operating system and setup you want that instance to have. There are literally thousands that you can use as a starting point, which you'll customize afterwards.
Amazon is loaded with images (known as AMI's) ready to be used to start up your machine. To view them, use the ec2-describe-images command. As this will return MANY results, I grep the results here for only ubuntu AMIs.
ec2-describe-images -a | grep ubuntu ... IMAGE ami-5059be39 canonical-cloud-us/ubuntu-intrepid-20090422-i386.manifest.xml 099720109477 available public i386 machine aki-714daa18 ari-6a5bbc03
I chose the above image as it's the latest (at this time) AMI directly created by Canonical, the creators of Ubuntu itself. The important figure here is the ami-######## string, which identifies the AMI. Let's get a machine booted up with that image. From the command line:
# create a keypair, this'll be used all over the place for access ec2-add-keypair ec2-keypair > ec2-keypair.pem chmod 600 ec2-keypair.pem ec2-run-instances ami-5059be39 -k ec2-keypair ec2-describe-instances
That's it! You're EC2 instance is now botting up. The i-XXXXXXXX portion is the id of your instance, which is used to shut it down, among other things. You can continue to run the ec2-describe-instances to see the status of your instance. It usually takes a few minutes for the machine to fully start up, so wait until it says that it is "running".
As soon as you boot up an instance, you're being billed. To shutdown an instance, use the i-XXXXXXXX id from the ec2-describe-instances command.
ec2-terminate-instances i-XXXXXXXX
EC2 Console - GUI goodness and a flex of the power of EC2
EC2 comes with a fantastic gui control panel that allows you to boot up instances, manage your AMI's (images), and do a lot of other things that we'll touch on later. http://console.aws.amazon.com/
Here's a quick run-down of the more important links along the left side:
- INSTANCES -> Instances:
Displays all the actual booted up instances that you have with EC2 - IMAGES -> AMIs (Amazon Machine Images):
Allows you to browse all the images that are available (including ones that we'll create later for our own use). You can start up an instance with any images found here at any time - ELASTIC BLOCK STORE -> Volumes
These are basically hard drives that you'll mount to your server and is used for any dynamic data (databases, logging, web sites). - NETWORKING & SECURITY -> Elastic IPs
Essentially a static ip you'll bind to your instance later - NETWORKING & SECURITY -> Security Groups
Each instance belongs to one or more security groups (your instance belongs to the "default" security group by default). The security groups define which ports are open to the public. By default, your instance won't be open on any ports, but we'll open them up one by one as necessary
Your server is not persistent. In other words, if I boot up my instance, create a file, then shutdown my instance, that file is gone forever. To save changes to your server, you create your own image (AMI) which you can then boot from later. So, other than software changes, apache configurations, etc, your server should stand as a static begin.
What about databases, upload directories and log files? This is where the EBS volume comes in. An EBS volume is essentially a hard drive that exists as a separate entity from your instance, which you then mount to your instance. An EBS is persistent, meaning that any file you copy into it will remain after shutting down your instance. You can even connect to an EBS from multiple instances. This is where everything dynamic - particularly your databases - will live.
SSH into your instance
SSH'ing into your instance is simple. First, however, we need to open up our server on port 22. We do this by telling the "default" security group (read above about Security Groups) to allow connections from port 22. You'll also need to open up other ports later - specifically port 80 when you get your webserver rocking.
ec2-authorize default -p 22
To ssh in, we need to know the internal host that amazon has assigned to us. The host is that one that starts with ec2- and ends with .amazonaws.com. By the way, this will change each time you shutdown and reboot your image. No fear, later we'll bind a static ip address to the instances to solve this problem.
ec2-describe-instances ... INSTANCE i-######## ami-a37190ca ec2-##-##-###-###.compute-#.amazonaws.com ...
While most EC2 images are setup so that you login as root, the one we installed (made by Cononical) uses the user "ubuntu" (you can then sudo easily once you're in). We'll use the ec2-keypair.pem that we created above to authenticate into the server.
ssh -i ec2-keypair.pem ubuntu@ec2-##-##-###-###.compute-#.amazonaws.com
You should now be in your server. As the welcome message explains, you can get a bunch of things setup on your server fairly easily by running the command sudo tasksel --section server. I selected the following:
- Basic Ubuntu server
- LAMP server
- OpenSSH server
Scp your private key and certificate to your cloud
In order to do things such as creating an image from your server setup, you're going to need that pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem and cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem files from earlier on your cloud. To get them up there, we can simply use scp
scp -i /path/to/ec2-keypair.pem /path/to/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem /path/to/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem ubuntu@ec2-##-##-###-###.compute-#.amazonaws.com:
We'll come back and use those in part 2 of this article, so just let them sit there for now.
Where we are and what we're missing still
What we did:
- Signed up for EC2
- Setup ec2 tools
- Picked out an image (AMI) and booted up an instance with it)
- Logged into your new instance
What's still to come:
- Currently, if you shutdown your new instance, any server changes you made are lost. We'll need to customize our server and then save our own AMI. This will take a snapshot of the server (with our customizations) which we can boot from later on.
- Create an EBS volume (a hard disk that mounts to your server, but whos data persists after you shutdown your instance) and move all of the dynamic data to it (Mysql, webserver, logs, etc)
- Associate an elastic ip address with your instance so you can setup your DNS to point at that ip address
- Get a Symfony project rocking on your server
- Explore potential tripping points, like how to properly rsync your symfony project to your EC2 instance
the thing that has deterred me from playing with ec2 is not being able to assess cost. Any info?
Thats a big deception to see that cloud computing dont mâle webhosting more accessible for individuals.
I'll stick on my shared - but with ssh access - at alwaysdata for .... 5€ per month.
Also, if you reserve an instance up front for a year, the price falls from $72 to about $49 a month.
I would like to see a benchmark on how many resquest can be processed with a small instance so we can compare with other hosting methods
@Ryan. Thanks for this excelent post, I'm waiting for the next.
For my current shared hosting (1GB bandwidth,20MB space) I paid 1000 RS / year
(25$)
So for small websites , Cloud is the right place :)
3000Rs/month is too costly for root access :) I perfer to hack my shared server ,lol
My guess is that you were looking at the "reserved" instances. The pricing I quote is for the "On Demand" instances. If you pay a fee up front, you can significantly reduce your monthly fees, which is almost always a good idea, once you know you're going to stick with it.
thanks alot for grinding through this stuff, it was not trivial but finally got it working. Would be nice with some links to valuable Apache/Symfony links for the absolute beginner but google works just as well :-)
If you are getting a "tasksel: aptitude failed (100)" error when calling tasksel you probably just need to run "sudo apt-get update" first.
sudo apt-get install ec2-api-tools
sudo apt-get install openjdk-6-jdk
I keep getting a "command not found" error
Was wondering if there are any special requirements for the local computer to install EC2 tools. Thanks!