Friday, December 16, 2011

Managing EBS in EC2 for Ubuntu Linux

EBS in EC2 means virtual hard drive.  You are given one to start with, which is attached to your instant to run things.  This is exactly the boot drive where your OS is.  The free tier gives you 10GB EBS for free, but the free micro instant comes with only 8GB, which doesn't add up.

Sooner or later you need more disk space (or less).   You can change the size of the boot disk but very inconvenient.  First you create a snapshot (copy) of the boot drive.  From the snapshot you create a volume (new drive), with bigger space.  Then you stop your instant (computer), remove the boot drive and connect the new one. But you still see exactly the same old drive, because it's up to your installed OS to change the partition.  In addition, you can perform a lot of operation only on an unmounted drive.  So you may need another instant (like CDROM drive for the install disk) to modify your boot disk.  If you modify the partitions you can easily brick the whole disk.  Also, when you are working on EC2, you don't normally have GUI.

It's much easier to create a virtual empty disk drive, and mount as any directory on the file system that you desire.

First, you create a new volume on the EC2 control panel, with the capacity you needed.  Then you just attach it to your instant, running or not.  For Linux, the control panel will give you a new device name by default, such as /dev/sdf .  If you specify sdb, the extra disk space will be mounted to /mnt later by default.

The job of visualization is completed.  You need to configure your hard drives via your computer, i.e., your instant.

You need to start your instant (if not running) and then connect to it with ssh.  Then you need to "format" the new hard drive.

First check if the drive is there:
#sudo fdisk -l

You will get errors because these commands are not compatible with the file system or something else on the drive.

To "format"
#sudo mkfs -t ext4 /dev/sdf
The device name should be the same you have chosen at the control panel.  If you have chosen sdb for the lastest Ubuntu, all you have to is reboot at the control panel.  The drive will be mounted at /mnt.  Then you can setup directory links to this space as desired.


For any other device name or any other mount directory, you need to edit /etc/fstab.  And add the line:
/dev/sdf       /your/directory    auto    defaults,nobootwait,noatime     0       0
You can see that sdb is there already, to be mounted to /mnt.

In your boot drive, you should also create an empty directory /your/directory.  Then when you reboot via control panel, this empty directory will be replaced by the disk space of the new device.

2 comments:

webercoder said...

It's also pretty easy to encrypt /dev/sdf with block level encryption! Here's an article that is similar to what I'm doing: http://silvexis.com/2011/11/26/encrypting-your-data-on-amazon-ec2/ I didn't realize that creating the EBS volume could be done from the command line, too.

The Player said...

That's what I needed! I know a thing or two about fstab because I was struggling with the ecryptfs system. Maybe that will be the first thing I will be looking at instead. Truecrypt may be good too. That's the first thing I used on Windows and Ubuntu dual-boot before moving permanently to Ubuntu.

The control panel is good enough for me now. I don'teven have the AWS tools installed. The instant is always on for free. After any service interruption, the previous will be restored. I hardly need to restart daemons so far.