Hudson saves Artifacts to Amazon S3

I recently start to work a lot with Amazon AWS by utilizing EC2 servers and using S3 as online storage. While you should do your maths using the servers to generously (it quickly becomes expensive if you create instances on the fly and keep them running), S3 usage is a rather innocent matter. Currently 1GB costs you 0.15 U$ per month (more details here), but please dont forget to look at data transfer charges as well (not longer free from November 2010). I use the service mainly for backup purpose, while I still rely on local backups once a month, the continuous backup happens on S3 (which might be safer than my backup harddisk). The data volume I transfer is small enough (not maintaining a filesharing server !) to keep the costs low.

I was looking for a way to transfer my Hudson artifacts to my S3 account and could not find a regular promoted plugin, almost creating a manual script as workaround, until I found a plugin, which you still need by yourself, but works perfectly. This tutorial (based on Ubuntu) is a hands-on walk-through from the scratch. Some steps are rather simple.

In this previous tutorial I described my approach of using artifacts as a backup media.

Update 2010-10-04: There is another updater in github who added functionality to the plugin: here the address git://github.com/d6y/hudson-s3.git

Pre-requisites:

  • A running Hudson instance. You can download it from here and run it from on local machine (java -jar hudson.war) for testing purpose.
  • An Amazon AWS account (create it here)
  • A sample project on a svn server (like in the previous tutorial)
  • The Apache Build Manager Maven. Don’t have it ? sudo apt-get install maven2 (up to 270MB depending what packages you have pre-installed)
  • A GIT sourcecode version control client. Don’t have it ? sudo apt-get install git git-core

Tutorial Step 1 (Preparation of S3)

  • You need to activate the S3 service in your AWS account if you haven’t done yet.
  • Create a bucket (kind of a container for files in S3). Note this buckets are NOT public. No one can touch our files.

    Create a bucket

  • Take not of your access key and secret access key (Account|Security Credential|Access Keys)

    Retrieve keys

Tutorial Step 2 (Build the plugin)

  • This is straight forward and simple, following the instructions from http://github.com/dougm/hudson-s3
  • Create a folder and get the sourcecode (git clone git://github.com/d6y/hudson-s3.git , original source clone http://github.com/dougm/hudson-s3.git)
  • Open the folder and move the file and execute mvn

    Build the plugin

  • You should have a s3.hpi file now. Copy it to your ~/.hudson/plugins folder

    s3.hpi

Tutorial Step 3 (Configure Hudson and project)

  • Start Hudson (java -jar hudson.war) and goto Manage Hudson|Configure System
    Enter your S3 access credentials

    Configure plugin

  • Configure your project

    Configure Project

    For the filenames and location the same logic applies as for saving artifacts.

  • Build the project

    Building Console Output

    Amazon bucket

Thanks to Doug MacEachern for the plugin !

Advertisements

6 thoughts on “Hudson saves Artifacts to Amazon S3

  1. Hi, one question on the S3 plugin. When published to S3, are the URLs on the Hudson project page for Build Artifacts now pointing to the files on S3? Or is this S3 plugin simply archiving a copy of the build artifacts?

  2. Pingback: Amazon S3 plugin for Jenkins CI « The JavaDude Weblog

  3. Hi,

    Thanks for putting together such detailed article.
    Is there a way i can make sure all the uploads are publicly available.
    I tried it but the final files are private so my build urls are not shared.
    I could see the destination s3 bucket has all the read permissions.but sub-folders and files are throwing “Access Denied” error.

    Reg,
    Varun

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s