Increase size and type of AWS EBS volume

I was offline for quite a while because shifting from one continent to another. But now regular posts should be rolling in again.

I am running a couple of instances in pre-production requirement mode and changed from a standard EBS volume to a IOPS volume for the DB instance or the volume with the DB files. I could not identify a reasonable increase of performance, maybe a misconception that IOPS volumes will boost performance, rather provide a defined and consistent random access I/O throughput. I must admit I did not use a value higher than 1000.

Billing IOPS

Billing IOPS

Some recommended reading:

I decided to return to a standard ESB volume for my database as its performance did not benefit from the IOPS type (the DB is not overly busy too).
You cant change type and size of an EBS volume on the fly.

Here the steps to achieve the same: Continue reading

Enforce password for Ubuntu user on EC2 instances

Using linux (Ubuntu) instances on Amazon EC2 is a quite safe thing to do, at least measured by the security provided by the platform (security groups, ACL, physical security,..). I recommend reading their security site here. At the end of the day the server is only as secure as you configure it, if you choose to open all ports running services with their default configurations and password settings, Amazon can’t help you.

When connecting to a Ubuntu server with ssh you need to provide the keyfile (somekeyfile.pem) that you can download when creating the key pair.

Key file

Key file

This 2048 bit key is required to login as regular ubuntu user. What I dislike is the fact that this user can sudo all, so once someone manage to get into you user account, he has root access too. I recommend to set a password for the ubuntu user and change the sudoers configuration.

Change the password for user ubuntu

Open the sudoers include file

sudo vi /etc/suderos.d/90-cloudimg-ubuntu or sudo vi /etc/sudoers

change last line from

ubuntu  ALL=(ALL) NOPASSWD:ALL

to

ubuntu ALL=(ALL) ALL

Glassfish and https running secure applications

By default Glassfish listens to http on port 8080 and https on port 8181.
It is better to listen to the default ports 80 for http and 443 for https, usually you dont want the user to enter port numbers as part of the URL.

Even the Glassfish Admin Console allows to change the ports (Configurations/Server Config/Network Config/Network Listener), certain server OS such as Ubuntu do not allow non-root users (you should run Glassfish as separate user !) to ports below 1024. We can achieve this by port rerouting with the iptables command (under Ubuntu)


iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
iptables -A INPUT -p tcp --dport 443 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8181
iptables-save -c > /etc/iptables.rules
iptables-restore < /etc/iptables.rules

vi /etc/network/if-pre-up.d/iptablesload
#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0

Additionally you can get a proper SSL certificate to stop annoying the user with a no proper certificate warning. See previous tutorial here.

SSL Error

SSL Error (Chrome)

If you operate an enterprise application with a known URL to the users, unlike a regular website where the portal should be reached with regular http, I would completely disable regular http.

Disable http

Disable http

Copy EC2 instance to another region

Is it finally possible ? While the AMI import tool is long awaited for but only available for Windows, it is rather a big hazzle to transfer manually (see this) any other OS ( my last attempt in 2010).

Today Amazon announced the EBS Snapshot Copy Feature (across regions). The intention is certainly to allow easy migration of data to another region, as you can copy the snapshot, create a volume and attach it to an instance. I was curious to try if I can migrate my Ubuntu instance to another region and it worked. You can use both command-line as well the AWS web admin.

Amazon S3 plugin for Jenkins CI again

About once a year I revisit (link) this topic again (usually when the plugin causes trouble). Now I get this signature error

AWS Error Code: SignatureDoesNotMatch, AWS Error Message: The request signature we calculated does not match the signature you provided. Check your key and signing method., S3 Extended Request ID:..

The good news first:
The S3 plugin became mainstream, you can install it from the plugin page under Jenkins Administration | Plugin Manager. You dont need to build the plugin any longer by yourself and can skip the rest of this entry.

S3 Plugin

The long version:
It seems the error is caused by a ‘+’ sign in the access key troubling the encoding function used (see issue). The latest build (Sep 2012) should fix this problem.

If you want to build by yourself, you need to get the sourcecode from git and build the plugin file, beware as it requires Maven 3 now. Below instructions apply fro Ubuntu.

Upload plugin

 

 

Running FTP server on EC2 on demand

or ‘How to cut  (even more) cost while running EC2 instances

I am running a FTP server on an EC2 instance (micro if you want), but we dont use it all the time. The server is run on-demand only and auto-shutdown every night. The challenge: on every new start of the instance you will get a new public ip which screws your passive ip address configuration in vsftpd.conf.

  • How to install and run vsftp on an EC2 Ubuntu instance.
  • How to switch off a Ubuntu EC2 instance ? Add this to the crontab:
    login as root
    crontab -e
    add: 0 12 * * * /sbin/shutdown \-h now
    
  • How to update vsftp.conf on start up ?
    pubip=`curl http://169.254.169.254/latest/meta-data/public-ipv4`
    
    sed "s/pasv_address=.*/pasv_address=$pubip/"  /etc/vsftpd.conf > /etc/vsftpdTEMP.conf
    rm /etc/vsftpd.conf
    mv /etc/vsftpdTEMP.conf /etc/vsftpd.conf
    service vsftpd restart
    

    curl http://169.254.169.254/latest/meta-data/public-ipv4 gives you the public IP address of your instance.

Remaining challenge: If you dont want to spend money on an elastic (permanent) IP which costs you while the instance is NOT running, you need a DNS service like dyndns.com and update the dyndns entry on every start too. This can easily done by a shell script using ddclient or Ubuntu’s dyndns command.

Touchscreen Notebooks using Ubuntu

I purchased 2 notebooks with swivel-touch screens last weekend. Both coming with Windows 7 which I clonezilla’d, wiped out and installed Ubuntu immediately. Both are not an iPad killer whatsoever, but it suits my requirements: you can touch it, you can turn it (read books), it comes with a keyboard and I can load almost any application, even do some development work.

  • Asus EEE T101MT
    1.66 GHz Atom N450 CPU with hyperthreading
    10.1 inch screen, multi-touch resistive display with 1024 x 600 pixels resolution
    2 GB RAM and 320 GB HDD at 5400 RPM
    WiFi 802.11n
    4 cell 2400 mAh and 35 Wh battery pack, removable
    0.3 megapixel webcam
    3 USB ports,  VGA output, Ethernet, Kensington Lock, Mic and Headphones jack and SD Card reader 

    Installing Ubuntu: A breeze with 10.10 (Maverick). All info here.

  • Acer Aspire 1825PTZ
    Intel Pentium processor SU4100 (1.3 GHz, 800 MHz FSB)
    2GB Memory
    Graphics Controller: Intel GMA 4500MHD
    11.6″ Acer CineCrystal LED LCD With (capacitive) Multi Touch(1366×768)
    320GB HD
    0.3 megapixel webcam
    3 USB ports,  VGA output, HDMI Port,Ethernet, Kensington Lock, Mic and Headphones jack and SD/XD/MS Card reader 

    Installing Ubuntu: Basic Installation straight forward, but requires some hacking to get the touchscreen properly running and the auto-rotate screen. But you find all answers in this thread. And some more tricks here.