Shared Nothing Architecture and Short Lived Processes

WARNNG - This post may offend fans of megalithic application servers

After having worked in the Java environment for the past several years and watching Enterprise Java Beans destroy whatever simplicity existed in the Java world, I was ready to explore new development environs with Collective Intellect. Further spurring this curiousity were the thoughts penned by Paul Graham in his enlightening book, Hackers and Painters. Paul is a big fan of Lisp and spends a few chapters talking about the elegance of the language and economy of expression required to implement powerful concepts. While Lisp is all of those things, it does seem to lack a large ecosystem of open source libraries ready made to bake into your current project. I ended up settling on Ruby and Ruby on Rails to implement our burgeoning ideas and business requirements. I will speak to the long and short comings of Ruby/Rails in a future post.

What I wanted to discuss on this post is Shared Nothing Architecture (SNA), coupled with Short Lived Processes (SLP). Shared Nothing Architecture is a concept made popular by the folks at google and is defined in Wikipedia as:

---

A shared nothing architecture is a distributed database architecture without a single point of failure. The term "shared nothing" is by imitation of other terms such as "shared disk", "shared network" and so on.

A typical shared nothing system would have duplicated disks, processors, power supplies, and networks. To make the system truly resilient, it should also have these systems split between different physical sites, and have multiple sources of electrical power and network connections to the outside world through physically diverse paths.

---

The advantage of SNA is that it greatly simplifies application construction as well as increasing the scalability and reliability of applications. I would argue that you don't even need to go so far as creating a distributed database and in some cases don't even require a database at all. Certainly having a distributed database creates a more resilient infrastructure, but if you don't have the time or resource pool to put a distributed database in place, you can still take advantage of SNA with a bit less resiliency at lower cost but still have a warm failover capability.

The implementation that we have at Collective Intellect uses a single mySQL database running on an industrial strength box with RAID 5. We have several other boxes that process information. They all point to this database box and file server to recover the context for the processing that they perform. One of these boxes also contains a hot nightly backup of all of the information in the database, so if the database box dies and the RAID file system completely fails, we can warm failover to the next box. We can do this because we also have an information architecture that will allow us to roll data forward (with a little pain) from the point at which the last backup was run.

This architecture allows us to write independent, specialized applications that each process a small chunk of our processing operation. This is a vast change from using a monolithic application server based architecture. Application server architectures require all applications to run inside of the architecture and manage state for all of those applications as part of that framework. One typical example of this is Enterprise Java Beans. These frameworks tend to carry a lot of baggage which results in two rather severe implications to your app.

1. Bringing up an entire application server infrastructure requires very large memory and load time overhead vs. a non application server based app.
2. All applications reside inside the application server. This means they are long lived and have a fairly large chance of becoming more and more intertwined with either legacy code or code that is meant to support other apps (i.e. spaghetti)

In our shared nothing architecture our apps can be short lived and only carry the code required by the app. Freeing applications from the application server means that these apps can be short lived. We have a number of small specialized Ruby applications that automatically get launched when work exists for them to do. This allows all components of the system to do what they were designed to do.

• The database stores data state used by the applications
• The operating system controls the sharing of hardware resources (network, disk, cpu, memory) as opposed to the application server controlling the sharing of these resources as a proxy or replacement of the OS.
• The application processes implement the specific features required by the business model