Building A Twelve Factor Application (Part 1 of 2)
I've spent most of my career writing and architecting large bespoke Java implementations for banks. This used to involve a large tech stack involving application servers (IBM WebSphere), databases (Oracle DB), caching (Coherence) and interoperability (IBM Message Queue and Message Broker)……eugh! That's the way it was for a lot of enterprises, and it still is now. However, the last few years I have been concentrating on Deployment Automation, Configuration Management and talking about all things DevOps. As part of my work, I often talk to customers about how they can re-design and re-architect their solutions to adapt to this new world of IT. Designing systems with service oriented architectures or microservice architectures are key, but so, too, is incorporating the ability to automate the deployment and monitor the health of the system. These are all properties that are encompassed in the twelve-factor app methodology.
I decided that rather than talking about the theory I would go ahead and build one. This blog explains the journey and the learning experience I have followed in creating this app and hopefully it will be useful for you, the reader, to understand what is required to build such an application.
Let's look at the properties of a twelve factor application and see how I can apply this to my application.
1. Codebase - one codebase tracked in revision control, many deploys.
Use a source repository such as Git to store the application code base to track all the changes and deploys of the code to the environments. A deploy is a running instance of the code. With my application, I am using GitHub as the code repository. I could have used Stash, BitBucket or even SVN, better decided on the free and widely adopted solution.
2. Dependencies - Explicitly declare and isolate dependencies.
My application is written in Java and I am using Maven as the dependency management tool to explicitly declare all the application Jar files within the application `pom.xml` file. Using Maven, I pin specific versions of the dependencies to the application, ensuring consistency in the build. There are many dependency management tools to choose from depending on your language of choice, Maven, Gradle and Ivy for (Java), Bundle and Gemfiles for Ruby and Godep and Godm for Golang. I could have written my app in any language but I chose Java with Maven as it's what I'm familiar with.
3. Configuration - Store configuration in the environment.
A twelve factor app requires a strict separation of configuration from the code. Instead of hard-coding configuration into the application this needs to be externalised into environment variables. In my application I could have used environment variables, but I have externalised configuration into Consul and Vault. Consul is a distributed tool used for service discovery, health checking and dynamic configuration. Vault is a distributed tool used for secrets management, where I store the application passwords.
4. Backing Services - Treat backing services as attached resources.
A backing service such as an MySQL DB should be able to be swapped out without any changes to the code. My application does not (yet) have a backing service. However, by using Vault, I can store the application connection details there and dynamically attach to a new DB each time the application loads. Note: this feature will be in later version of my code.
5. Build, Release, Run - Strictly separate build and run stages.
The build stage builds the binaries, the release stage combines the environment configuration with the deployment run configurations and the run stage runs the app in the execution environment. In my solution I build the code with Maven and store the Jar artefact into an AWS S3 bucket. When I provision the environment I pull down the artefact and environment configuration and then I use an Upstart service to launch the application in its runtime. Every release must have a unique release ID. In my application the version is defined by Maven which uses the standard versioning system of developing with `1.0.0-SNAPSHOT` versions and then releasing with `1.0.0` versions.
6. Processes - Execute the app as one or more stateless processes.
Twelve factor applications are stateless and share nothing. Data must be stored to a stateful backing service such as a database. As stated previously, there is no database…yet. So very much so this application is stateless.
7. Port Binding - Export services via port binding.
A twelve-factor app is completely self-contained and does not require runtime injection of a web server into the execution environment. The application should expose HTTP as a service and bind to that port to expose its services on. This example uses Java SpringBoot, which makes it easy to create application the 'just run'. It has an embedded web server (Tomcat in this case) and I just expose the port to bind HTTP requests from within the configuration file.
8. Concurrency - Scale out via the process model.
The application must run one and only one process on the operating system. The application runs in a Java virtual machine which uses threads under the hood to hide away the Unix process. The JVM does block system resources on start-up but this can be configured to meet the requirements of the application (such as JVM heap size). My application uses the OS process manager Upstart to manage its process, output streams and restarts and shutdowns. You could create different Upstart configurations to run this app concurrently on one host, by assigning different ports to use. In the future I plan to use Nomad to schedule the deployment of the application to different nodes within a cluster. Although I could just as easily use Heroku, Pivitol, Docker or OpenShift to do this.
9. Disposability - Maximise robustness with fast start-up and graceful shutdown.
The application should be disposable, which means it can be shut-down and restarted at a moment's notice. Therefore, the application must be designed to handle graceful shutdowns and ensure in-flight requests are handled correctly and no more HTTP requests are processed. The SpringBoot framework has a module called `Actuator` which allows you to shutdown the application gracefully.
10. Dev/Prod Parity - Keep development, staging and production as similar as possible.
The twelve factor application is designed for continuous deployment by keeping the gap between development and production small. Currently I only have one environment, however I am using Packer and Terraform to provision my environment therefore the infrastructure is defined as code and therefore there would be parity between the different environments.
11. Logs - Treat logs as an event stream.
Each application process should write its event stream, unbuffered, to `stdout`. The execution environment should capture the stream and route the stream to the final destination. By using the Upstart service, the process stream is routed to the log file destination and it is responsible for the location of the stream.
12. Admin Processes - Run admin/management tasks as one-off processes.
The twelve-factor application should provide one-off management processes to be run in the same execution environment. My application does not yet have any admin processes, however the Actuator module allows for remote execution of management tasks such as health, restart, shutdown, etc.
That's the end of part one. Click here to read part two as I walk through creating the application itself!
