I’ve always liked DBM files as a lightweight, fast, persistent datastore. For those not familiar with the format, imagine something that has an interface like memcached (key/value storage and lookup, aka a giant hashtable) but stores to a single file on disk ala SQLite. I used GDBM in my C and Perl days; these days, Ruby makes it even easier:
require 'dbm' dbm = DBM.new('fruit') dbm['apples'] = "Red and delicious." dbm['bananas'] = "Yellow and nutritious."
After running this program, there will be a file on disk in the current directory called fruit.db. Load a different process to access your stored values:
DBM.new('fruit')['apples'] # => "Red and delicious."
GDBM has been the gold standard for many years (type “locate libgdbm” on any unix system and you’re almost guaranteed to see it). More recently, Tokyo Cabinet has created a more modern implementation. Better yet, Tokyo Tyrant provides a client/server implementation of the same interface.