InMemoryDatabase

HomeBlogArticlesBooksAbout MeContact MeThoughtWorks
InMemoryDatabasedesign22 November 2005
An in-memory database (aka embedded database) is a database that runs entirely in main memory, without touching a disk. It‘s usually created when a process starts, runs embedded within that process, and is destroyed when the process finishes.
On first blush the idea of such a beast seems ridiculous. The two main purposes of most databases are to persist information between process invocations and to handle concurrent access to data between processes. In-memory databases do neither - so what‘s the point?
The most common case I‘ve run into recently is for testing. When you‘re developing an enterprise application, tests that hit the database can be a huge time drain when running your test suites. Switching to an in-memory database can have an order of magnitude effect which can dramatically reduce build times. Since most ThoughtWorkers get the shakes if they haven‘t had a green bar recently, this makes a big difference to us.
There are two routes people seem to take to a in-memory database for testing. The first one is to use a SQL in-memory database library. In Java-land the popular one seems to beHSQLDB. ElsewhereSQLite andFirebird come up. The nice thing about these tools is that they allow you to use regular SQL to query them. One issue is that they may not support quite the same dialects or have all the features of the target database. You can do something similar by running a file-based database on a ram disk.
Another route is to abstract all the database access behind aRepository. Then you can swap out the database with regular in-memory data structures. Often just a bunch of hash-tables for the entry points to the object graph is enough. One of the strengths of the repository approach is that it gives you a consistent way to access (and stub out) non SQL data sources too. This means that your object-relational mapping system is also hidden inside the repository.
Indeed a few people actively dislike using SQL in-memory databases under the belief that they encourage spreading either SQL or object-relational mapper code around the domain model. Running SQL in-memory may removes much of the pain of slow access but acts as a deoderent to cover the smell of a missing repository.
Testing is the main driver thus far, but I think there‘s more to come from in-memory databases. Memory sizes are now enough that many application databases can be loaded into memory. If you use an approach that keeps an event log of all changes to your application state, you can treat the in-memory database as a cache of the result of applying the log, rebuilding it and snapshotting it as you need. Such styles can be very scalable and have high performance in cases where you have lots of readers and few writers.
Prevayler got a lot of attention for taking this kind of approach. People I know who tried it found it‘s tight coupling to the in-memory objects and lack of migration tools caused serious problems. But I think the approach of persistent change logs as systems of record is a fertile ground to explore in the future.
Links
home
bliki
rss
Japanese
Spanish
Categories
agile
design
leisure
refactoring
thoughtWorks
tools
uml
writing
Blog Roll
ThoughtBlogs
TW Alumni
Steve Cook
Brian Foote
Simon Harris
/\ndy Hunt
Ralph Johnson
Patrick Logan
David Ing
Brian Marick
Jeremy Miller
Jimmy Nilsson
Samuel Pepys
Keith Ray
Johanna Rothman
Jim Shore
Dave Thomas

CopyrightMartin Fowler, all rights reserved