[vc_row][vc_column][vc_column_text]We recently shipped a Rails SAAS product hosted on Elastic Beanstalk, Amazon’s 1-Click Web App Deployment service. While I had worked with EC2 instances several times on previous applications, setting up and configuring a load balanced, scalable environment was something I personally hadn’t done. Elastic Beanstalk aims to take a fair amount of the setup of scalable environments out of the way, to allow for more focus on the software development. But there were a number of steps I had to figure out on my own. So I wanted to share the majority of the steps we took to deploy a Rails app to both a web server and worker tier.
Not only did we want the ability to scale up application servers as needed, we also needed the ability to process jobs in the background as well as facilitate recurring tasks, such as billing.
For recurring tasks in a Unix environment, cron is the way to go. The problem is that cron jobs do not work in a distributed environment. If you added a task to an EC2 instance, and it scaled, you’d suddenly have multiple tasks running. Apparently customers don’t appreciate being billed multiple times.
To meet those needs, we were able to setup a Web Server tier that would serve public requests to the application, and a Worker tier that would facilitate periodic tasks as well receive messages from an Amazon SQS message queue to process tasks in the background on demand.
AWS provides a CLI client for Elastic Beanstalk as well as a web UI through console.aws.amazon.com to provision and configure Elastic Beanstalk.
We opted to use the eb_deployer Ruby gem. It provides the ability to provision multiple environments, configure Blue-Green deployments, rake tasks to execute deployments, and to create CloudFormation templates. Additionally, it didn’t require the entire dev team to install additional clients, and all the configuration would be checked into source control.
Step 1: Set up your AWS account
I won’t go through all the details here, but you’ll need to sign up for an AWS account. Once your account is created, create a separate IAM account and generate an Access Key. You can store your Access Key ID and Secret Access Key in .bashrc, .zshrc, or whatever your shell’s configuration file is as environment variables like this:
If you don’t want to put those keys in your config file, you can install and use the AWS CLI to store these creds as well.
This will prompt you for your keys.
With your creds in place, you won’t have to reference them in code anywhere.
Step 2: Install eb_deployer
Add the eb_deployer to your app’s Gemfile, and run:
This creates a few boilerplate templates that are to be customized as well as an .ebextensions directory.
The two most important files generated are config/eb_deployer.yml and config/rds.json which we’ll discuss in the next step.
Step 3: Configure eb_deployer.yml
eb_deployer.yml has a number of settings, and is well documented.
The main areas to modify are the option_settings, resources, and environments.
This is a pretty straight forward section. As many environments as you want (i.e. production, ci, staging, qa, dev) you can declare here. Each environment can override any of the general option settings set in the rest of the file.
This is where you can set the type of EC2 instance, set environment variables, configure HTTP ports, and pretty much set any option available via Amazon’s API.
I recommend creating and setting an IAM Role as well as setting up a Key Pair so you can access the instances once they are running:
If you are going to be using SSL, add your SSL cert to AWS, and you can use settings like this:
This section is about setting input values that will be used in the CloudFormation template (by default the config/rds.json file) and capturing output values.
eb_deployer creates a separate RDS instance for each environment. This means we can build up and tear down any Elastic Beanstalk environment without losing our persistence layer.
Out of the box, eb_deployer will create a Postgresql RDS instance. We’ll also need a few other resources along with the database layer, namely two SQS queues (a message queue and a dead letter queue), and a DynamoDB table.
The message queue is how we will send messages to the worker tier to process various jobs. The DynamoDB table is what will store our periodic tasks.
The entire resources section should look like this:
Elastic Beanstalk Worker Tiers will create a queue by default unless you specify one. I like to create them so I can have meaningful names associated with them. In order to specify the name, we need to add the inputs for the environment (env) and the queue prefix (QueuePrefix). Then, in the outputs section, we capture the queue names and urls which will be stored as environment variables.
One last useful modification was to set the version_label. Since we use Git tags for each release, it made sense to reference those tags as the version being deployed.
Step 4: Update config/rds.json
This file can be named anything, and doens’t have to be limited to just RDS instance creation.
This JSON file will be the basis for the CloudFormation template. This is where eb_deployer is better than simply using the eb CLI or web interface. It allows us to store the resource settings in a template that can be used to provision and manage the resources for each environment.
CloudFormation templates can be pretty involved, but the have a few main sections.
- Resources – this is where you specify the various AWS products and settings (i.e. RDS, SQS, EC2, ELB)
- Parameters – These relate to the input parameters we set in eb_deployer.yml
- Outputs – These are outputs that are also referenced in eb_deployer.yml
- Mappings – When you need to specify values conditionally, you create mappings to inform CloudFormation.
The primary thing to note is that we generate queues based on the input values and provide output values that get attributes such as the queue name
Step 5: Specify Components in eb_deployer.yml
Usually Elastic Beanstalk applications have separate application code in the Web Server tier from the Worker tier, which makes good sense. In our situation, we didn’t want to invest in making architectural decisions to split out services. It was more important to ship the product and get feedback to begin iterating on the features. So we have a monolithic Rails application that needs to be deployed to both the web server and worker tiers.
To do that, we took advantage of components in eb_deployer. Components are an undocumented feature of eb_deployer. They allow you to identify components that need to be deployed to a given environment. You might have a web component and an API component, or in our case a bg component. Each component will deploy the same code to their respective containers. But components also allow additional customization beyond the global and environmental options.
As a top level section we added the following components:
For the web component, we didn’t need to change any settings, other than to provide a name. For the bg component, we needed to indicate the tier, set an environment variable so we could know in application code if this was a background environment or not. Lastly, we need to tell Elastic Beanstalk the end point that the message queue would post messages to, and the queue url.
Step 6: Add cron.yaml
At the root of your Rails app, create a cron.yaml file. This file will store all the periodic tasks you want to schedule. The format is pretty simple.
Based on the schedule indicated for each task, a POST request will be sent to the Worker tier at the indicated end point.
The name and urls need to be unique across this file.
Step 7: Install the Postgres development headers
In addition to install the postgres lib on each EC2 instance, we’ll need to install the development headers so the pg gem will install.
Simply edit .ebextensions/01_postgres_packages.config to look like:
Step 8: Deploy
With this in place, commit your changes, and run
This will package up the application, send it to S3, and begin to provision all the resources indicated. It will take a good amount of time on the first deploy.
Once it’s finished, it will have created a single EC2 instance for the web and bg components, configured the web and ruby server, load balancers, scaling triggers, RDS instance and migrated the database to the latest schema, scheduled the periodic tasks
You should be able to log into the AWS web console and see the application and various environments running.
To deploy to a different environment (e.g. production) simply run:
- Elastic Beanstalk w/ eb_deployer can create a scalable environment fairly quickly
- Like any “1-click” solution, it’s not exactly 1-click, and there are some limitations that are to be expected for a pre-configured environment
- We don’t plan on utilizing Elastic Beanstalk forever. As our service matures, and hopefully grows, we’ll undoubtedly move to a more customized, sophisticated environment.
- Elastic Beanstalk along with eb_deployer provided just enough customization with enough convention to allow to focus on developing the application and have enough confidence in the infrastructure to handle the early growth stages of our service.
[/vc_column_text][/vc_column][vc_column][us_cta title=”Contact us about more tech help” btn_link=”url:%2Fcontact-us|||” btn_label=”Get Help” btn_style=”4″ btn_size=”15px”][/us_cta][/vc_column][/vc_row]