blue flood

BlueFlood DB

Blueflood is a multi-tenant distributed metric processing system created by engineers at Rackspace. It is used in production by the Cloud Monitoring team to process metrics generated by their monitoring systems. Blueflood is capable of ingesting, rolling up and serving metrics at a massive scale.

Getting Started

The latest code will always be here on Github.

git clone
cd blueflood

You can run the entire suite of tests using Maven:

mvn test integration-test


Build an ‘uber jar’ using maven:

mvn package -P all-modules

The uber jar will be found in ${BLUEFLOOD_DIR}/blueflood-all/target/blueflood-all-${VERSION}-jar-with-dependencies.jar.
This jar contains all the dependencies necessary to run Blueflood with a very simple classpath.


The best place to start is the 10 minute guide.
In a nutshell, you must do this:

java -cp /path/to/uber.jar \
-Dblueflood.config=file:///path/to/blueflood.conf \
-Dlog4j.configuration=file:///path/to/ \

Each configuration option can be found in Each of those can be overridden on the command line by



We anticipate different use cases for Blueflood. For example, at Rackspace it made more sense to create a Thrift layer for ingestion and query. We have chosen not to release that layer because
it contains a lot of code that is specific to our infrastructure and other backend systems.

We decided to release Blueflood with reference HTTP-based ingestion and query layers. These layers may be replaced by
code that works better with your enterprise.

Custom Ingestion

Several things must be done to properly ingest data:
1. Full resolution data must be written via AstyanaxWriter.insertFull().
2. A ScheduleContext object must be update()d regarding that metrics shard and collection time.
3. Shard state must be periodically pushed to the database for each shard that metrics have been collected for. This
can be done by getting the dirty slot information from the ShardStateManager associated with a particular
ScheduleContext object.

HttpMetricsIngestionServer is an example of how to set up a multi-threaded staged ingestion pipeline.

Custom Querying

Thankfully, querying is easier than ingestion. Whatever query service you create should have a handler that extends RollupHandler, which provides a basic wrapping of low level read operations provided by AstyanaxReader.


Blueflood exposes a great deal of internal performance metrics over JMX.
Blueflood respects the standard JMX JVM settings:

You can use any tool that supports JMX to get internal performance metrics out of Blueflood.

Additionally, internal performance metrics can be pushed directly to a Graphite
service by specifying the following in your Blueflood



First, we welcome bug reports and contributions.
If you would like to contribute code, just fork this project and send us a pull request.
If you would like to contribute documentation, you should get familiar with our wiki

Also, we have set up a Google Group to answer questions.
If you prefer IRC, most of the Blueflood developers are in #blueflood on Freenode.


Copyright 2013 Rackspace

Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
See the License for the specific language governing permissions and
limitations under the License.