We are nearing completion of the HA Replication Cluster. There is a new server process (HAJournalServer) that embeds a Journal and exposes an RMI interface (HAGlue) supporting the internal HA protocols and a variety of administrative tasks. The Journal provides a high performance single machine graph database and SPARQL end point for up to 50 billion edges. The HAJournalServer extends this to support a replication cluster with automated full and incremental backups and offers linear scaling for query in the size of the replication cluster. Linear scaling is achieved because each query is answered 100% locally by the HAJournalServer process to which it is directed. The HAJournalServer has exactly the same high performance, low latency behavior as the embedded Journal.
High Availability is based on a quorum model. The HA replication cluster is up and available for reads and writes as long as a majority of the services are up. The quorum size must be an odd number of machines greater than one (3, 5, 7, etc.). Apache River is used for service discovery. Apache Zookeeper is used to manage the state of the quorum and elect the leader. Writes must be directed to the leader, but you can read on any service that is joined with the met quorum. Writes are replicated using a low level protocol along a pipeline to each of the followers. The /status page of the NanoSparqlServer provides extended status information about the HA replication cluster. You can query either the REST API or the RMI API to discover the status of each running server process.
The chart below shows linear scaling on an HA replication cluster consisting of 3 Intel Mac Minis against the BSBM 100M “Explore” reduced query mix (that is, w/o query 5). This is an aggregated throughput of nearly 140,000 QMpH on a highly available cluster built on commodity hardware.
This work is being performed in the READ_CACHE branch. See the src/resources/HAJournal directory if you want a preview of this capability. There is a README, sample configuration file, and some scripts that are designed to execute from an SVN build. At this point, we are green on the test suite for an HA3 cluster. There will be some additional QA before we bring this feature back into the development branch for a release.