Author Archives: Brad Bebee

Blazegraph 2.1.4 Release

SystapSiteBanner_1600x362_150px_wideWe’re very pleased to announce the the release of Blazegraph 2.1.4.  This is a maintenance release of Blazegraph.  See the full details here.

2.1.4 Release Fixes

  • [BLZG-533] – Vector the query engine on the native heap
  • [BLZG-2041] – BigdataSail should not locate the AbstractTripleStore until a connection is requested
  • [BLZG-2053] – Blazegraph Security Reporting Instructions
  • [BLZG-2050] – Fork Colt Libraries to remove hep.aida
  • [BLZG-2065] – Remove Autojar and Unused Ant Scripts

Download it, clone it, have it sent via carrier pigeon (transportation charges may apply).  Find a bug, hit JIRA.  Have a question, try the mailing list or contact-us.

Security Reporting Procedure
Blazegraph has an updated security reporting procedure. Please see the guide for reporting security related issues. This process is monitored on a daily basis. All security reports are acknowledged within 24 hours. Mitigations for reported security issues are made in a reasonable timeframe, which may be as quickly as 24 hours for high-severity issues.

Github

git clone -b BLAZEGRAPH_RELEASE_2_1_4 --single-branch https://github.com/blazegraph/database.git BLAZEGRAPH_RELEASE_2_1_4
cd BLAZEGRAPH_RELEASE_2_1_4
./scripts/mavenInstall.sh
./scripts/startBlazegraph.sh

Maven Central

Blazegraph 2.1.4 is now on Maven Central.    You can also get the Tinkerpop3 API and the new BlazegraphTPFServer.

    
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-core</artifactId>
        <version>2.1.4</version>
    </dependency>
    <!-- Use if Tinkerpop 2.5 support is needed ; See also Tinkerpop3 below. -->
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-blueprints</artifactId>
        <version>2.1.4</version>
    </dependency>

Tinkerpop3

Blazegraph Tinkerpop3

Tinkerpop3 is here!  Get it from Maven Central.

   <dependency>
      <groupId>com.blazegraph</groupId>
      <artifactId>blazegraph-gremlin</artifactId>
      <version>1.0.0</version>
   </dependency>

Mavenization

Everybody loves (and hates) Maven.  Starting with the 2.0.0 release, Blazegraph has been broken into a collection of maven artifacts.    This has enabled us to work on new features like TinkerPop3, which require Java 8 support while keeping the core platform at Java 7 to support users who are still there.   Checkout the Maven Notes on the wiki for full details on the architecture, getting started with development, building snapshots, etc.  If you have a 1.5.3 version checked out in Eclipse, you will want to pay attention to Getting Started with Eclipse and allocate a little extra time for the transition.

Deployers

2.1.4 provides updates for the Debian Deployer, an RPM Deployer, and a Tarball.  In addition to the blazegraph.war and blazegraph.jar archives.

GPU Acceleration

Are you interested in trying out GPU Acceleration for you Blazegraph instance? Contact us for a free trial!

Stay in touch, we’d love to hear from you.

facebooktwittergoogle_pluslinkedin

Blazegraph 2.1.2 Release

SystapSiteBanner_1600x362_150px_wideWe’re very pleased to announce the the release of Blazegraph 2.1.2.  This is a maintenance release of Blazegraph.  See the full details here.

2.1.2 Release Fixes

  • [BLZG-1911] – Blazegraph 2.1 version does not work on Windows (async IO causes file lock errors)
  • [BLZG-1954] – Potential Race Condition in Latched Executor
  • [BLZG-1957] – PipelinedHashJoin defect in combination with VALUES clause

Download it, clone it, have it sent via carrier pigeon (transportation charges may apply).  Find a bug, hit JIRA.  Have a question, try the mailing list or contact-us.

Github

git clone -b BLAZEGRAPH_RELEASE_2_1_2 --single-branch https://github.com/blazegraph/database.git BLAZEGRAPH_RELEASE_2_1_2
cd BLAZEGRAPH_RELEASE_2_1_2
./scripts/mavenInstall.sh
./scripts/startBlazegraph.sh

Maven Central

Blazegraph 2.1.2 is now on Maven Central.    You can also get the Tinkerpop3 API and the new BlazegraphTPFServer.

    
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-core</artifactId>
        <version>2.1.2</version>
    </dependency>
    <!-- Use if Tinkerpop 2.5 support is needed ; See also Tinkerpop3 below. -->
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-blueprints</artifactId>
        <version>2.1.2</version>
    </dependency>

Tinkerpop3

Blazegraph Tinkerpop3

Tinkerpop3 is here!  Get it from Maven Central.

   <dependency>
      <groupId>com.blazegraph</groupId>
      <artifactId>blazegraph-gremlin</artifactId>
      <version>1.0.0</version>
   </dependency>

Blazegraph-Based TDF Server
The Blazegraph-Based TPF Server is a Linked Data Fragment (LDF) server that provides a Triple Pattern Fragment (TPF) interface using the Blazegraph graph database as backend.  It was originally developed by Olaf Hartig and is being released via Blazegraph under the Apache 2 license.   See here to get started.

Mavenization

Everybody loves (and hates) Maven.  Starting with the 2.0.0 release, Blazegraph has been broken into a collection of maven artifacts.    This has enabled us to work on new features like TinkerPop3, which require Java 8 support while keeping the core platform at Java 7 to support users who are still there.   Checkout the Maven Notes on the wiki for full details on the architecture, getting started with development, building snapshots, etc.  If you have a 1.5.3 version checked out in Eclipse, you will want to pay attention to Getting Started with Eclipse and allocate a little extra time for the transition.

Deployers

2.1.2 provides updates for the Debian Deployer, an RPM Deployer, and a Tarball.  In addition to the blazegraph.war and blazegraph.jar archives.

GPU Acceleration

Are you interested in trying out GPU Acceleration for you Blazegraph instance? Contact us for a free trial!

Much, much, more….

There also many other features for improved data loading, building custom vocabularies, and many more. Check it out.

Stay in touch, we’d love to hear from you.

facebooktwittergoogle_pluslinkedin

Blazegraph 2.1.1 Release!

SystapSiteBanner_1600x362_150px_wideWe’re very pleased to announce the the release of Blazegraph 2.1.1.  This is a maintenance release of Blazegraph.  See the full details here.

Download it, clone it, have it sent via carrier pigeon (transportation charges may apply).  Find a bug, hit JIRA.  Have a question, try the mailing list or contact-us.

Github

git clone -b BLAZEGRAPH_RELEASE_2_1_1 --single-branch https://github.com/blazegraph/database.git BLAZEGRAPH_RELEASE_2_1_1
cd BLAZEGRAPH_RELEASE_2_1_1
./scripts/mavenInstall.sh
./scripts/startBlazegraph.sh

 Bug Fixes for the update to Lucene 5.5.0 Version
Blazegraph 2.1.1 provides fixes for Lucene 5.5.0 support. There’s a guide to reindexing with the updated Lucene tokenizers here. It also now includes support for adding a text index to an existing namespace without reloading.

GeoSpatial Searching

Did you try the GeoSpatial features in 2.1.0? 2.1.1 has some important updates and fixes. The full details are on the Wiki. As a quick start, you can configure your namespace to enable geo-spatial:

com.bigdata.rdf.store.AbstractTripleStore.geoSpatial=true

Add some data with geospatial information:

@prefix geoliteral: <http://www.bigdata.com/rdf/geospatial/literals/v1#> .
@prefix example: <http://www.example.com/> .

example:Oktoberfest-2013
    rdf:type example:Fair ;
    rdfs:label "Oktoberfest 2013" ;
    example:happened "48.13188#11.54965#1379714400"^^geoliteral:lat-lon-time ;
                example:city example:Munich .

example:RAR-2013
    rdf:type example:Festival ;
    rdfs:label "Rock am Ring 2013" ;
    example:happened "50.33406#6.94259#1370556000"^^geoliteral:lat-lon-time ;
                example:city example:Nuerburg .

Then issue some queries to get started!

PREFIX geoliteral: <http://www.bigdata.com/rdf/geospatial/literals/v1#>
PREFIX geo: <http://www.bigdata.com/rdf/geospatial#>
PREFIX example: <http://www.example.com/>

SELECT * WHERE {
  SERVICE geo:search {
    ?event geo:search "inCircle" .
    ?event geo:searchDatatype geoliteral:lat-lon-time .
    ?event geo:predicate example:happened .
    ?event geo:spatialCircleCenter "48.13743#11.57549" .
    ?event geo:spatialCircleRadius "100" . # default unit: Kilometers
    ?event geo:timeStart "1356994800" . # 01.01.2013, 00:00:00
    ?event geo:timeEnd "1388530799" .   # 31.12.2013, 23:59:59
  }
}

Maven Central

Blazegraph 2.1.1 is now on Maven Central.    You can also get the Tinkerpop3 API and the new BlazegraphTPFServer.

    
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-core</artifactId>
        <version>2.1.0</version>
    </dependency>
    <!-- Use if Tinkerpop 2.5 support is needed ; See also Tinkerpop3 below. -->
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-blueprints</artifactId>
        <version>2.1.0</version>
    </dependency>

Tinkerpop3

Blazegraph Tinkerpop3

Tinkerpop3 is here!  Get it from Maven Central.

   <dependency>
      <groupId>com.blazegraph</groupId>
      <artifactId>blazegraph-gremlin</artifactId>
      <version>1.0.0</version>
   </dependency>

Blazegraph-Based TPF Server
The TPF Server for Blazegraph (TP4Blazegraph) is a Linked Data Fragment (LDF) server that provides a Triple Pattern Fragment (TPF) interface using the Blazegraph graph database as backend.  It was originally developed by Olaf Hartig and is being released via Blazegraph under the Apache 2 license.   See here to get started.

Mavenization

Everybody loves (and hates) Maven.  Starting with the 2.0.0 release, Blazegraph has been broken into a collection of maven artifacts.    This has enabled us to work on new features like TinkerPop3, which require Java 8 support while keeping the core platform at Java 7 to support users who are still there.   Checkout the Maven Notes on the wiki for full details on the architecture, getting started with development, building snapshots, etc.  If you have a 1.5.3 version checked out in Eclipse, you will want to pay attention to Getting Started with Eclipse and allocate a little extra time for the transition.

Deployers

2.1.1 provides updates for the Debian Deployer, an RPM Deployer, and a Tarball.  In addition to the blazegraph.war and blazegraph.jar archives.

GPU Acceleration

Are you interested in trying out GPU Acceleration for you Blazegraph instance? Contact us for a free trial!

Much, much, more….

There also many other features for improved data loading, building custom vocabularies, and many more. Check it out.

Stay in touch, we’d love to hear from you.

facebooktwittergoogle_pluslinkedin

Blazegraph 2.1.0 Release!

SystapSiteBanner_1600x362_150px_wideWe’re very pleased to announce the the release Blazegraph 2.1.0.  2.1.0 release is a Major release of Blazegraph.    There are some very exciting changes for GeoSpatial Searching, update to Lucene 5.5.0, support for online back-up via the REST API, JSON-LD support, and much more.

Download it, clone it, have it sent via carrier pigeon (transportation charges may apply).  Find a bug, hit JIRA.  Have a question, try the mailing list or contact-us.

Github

We love Sourceforge, but today it’s important to be on github as well.  This release will be on both Github and Sourceforge.  We’ll continue to use Sourceforge for distribution, mailing lists, etc., but will migrate towards Github for the open source releases.

git clone -b BLAZEGRAPH_RELEASE_2_1_0 --single-branch https://github.com/blazegraph/database.git BLAZEGRAPH_RELEASE_2_1_0
cd BLAZEGRAPH_RELEASE_2_1_0
./scripts/mavenInstall.sh
./scripts/startBlazegraph.sh

Update to Lucene 5.5.0 Version
Blazegraph 2.1.0 supports Lucene 5.5.0. There’s a guide to reindexing with the updated Lucene tokenizers here. It also now includes support for adding a text index to an existing namespace without reloading.

Using Pubchem with Blazegraph
Blazegraph 2.1.0 now includes a custom vocabulary for using with Pubchem. You can checkout a quick guide at Blazegraph Pubchem. We’ll be including some feature blog posts on using and tuning with the Pubchem data. If you’re interested in applying Blazegraph to this data or have any questions, we’d love hear from you.

GeoSpatial Searching

Check out our new features for GeoSpatial searching. The full details are on the Wiki. As a quick start, you can configure your namespace to enable geo-spatial:

com.bigdata.rdf.store.AbstractTripleStore.geoSpatial=true

Add some data with geospatial information:

@prefix geoliteral: <http://www.bigdata.com/rdf/geospatial/literals/v1#> .
@prefix example: <http://www.example.com/> .

example:Oktoberfest-2013
    rdf:type example:Fair ;
    rdfs:label "Oktoberfest 2013" ;
    example:happened "48.13188#11.54965#1379714400"^^geoliteral:lat-lon-time ;
                example:city example:Munich .

example:RAR-2013
    rdf:type example:Festival ;
    rdfs:label "Rock am Ring 2013" ;
    example:happened "50.33406#6.94259#1370556000"^^geoliteral:lat-lon-time ;
                example:city example:Nuerburg .

Then issue some queries to get started!

PREFIX geoliteral: <http://www.bigdata.com/rdf/geospatial/literals/v1#>
PREFIX geo: <http://www.bigdata.com/rdf/geospatial#>
PREFIX example: <http://www.example.com/>

SELECT * WHERE {
  SERVICE geo:search {
    ?event geo:search "inCircle" .
    ?event geo:searchDatatype geoliteral:lat-lon-time .
    ?event geo:predicate example:happened .
    ?event geo:spatialCircleCenter "48.13743#11.57549" .
    ?event geo:spatialCircleRadius "100" . # default unit: Kilometers
    ?event geo:timeStart "1356994800" . # 01.01.2013, 00:00:00
    ?event geo:timeEnd "1388530799" .   # 31.12.2013, 23:59:59
  }
}

Maven Central

Blazegraph 2.1.0 is now on Maven Central.    You can also get the Tinkerpop3 API and the new BlazegraphTPFServer.

    
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-core</artifactId>
        <version>2.1.0</version>
    </dependency>
    <!-- Use if Tinkerpop 2.5 support is needed ; See also Tinkerpop3 below. -->
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-blueprints</artifactId>
        <version>2.1.0</version>
    </dependency>

Tinkerpop3

Blazegraph Tinkerpop3

Tinkerpop3 is here!  Get it from Maven Central.

   <dependency>
      <groupId>com.blazegraph</groupId>
      <artifactId>blazegraph-gremlin</artifactId>
      <version>1.0.0</version>
   </dependency>

Blazegraph-Based TDF Server
The Blazegraph-Based TPF Server is a Linked Data Fragment (LDF) server that provides a Triple Pattern Fragment (TPF) interface using the Blazegraph graph database as backend.  It was originally developed by Olaf Hartig and is being released via Blazegraph under the Apache 2 license.   See here to get started.

Mavenization

Everybody loves (and hates) Maven.  Starting with the 2.0.0 release, Blazegraph has been broken into a collection of maven artifacts.    This has enabled us to work on new features like TinkerPop3, which require Java 8 support while keeping the core platform at Java 7 to support users who are still there.   Checkout the Maven Notes on the wiki for full details on the architecture, getting started with development, building snapshots, etc.  If you have a 1.5.3 version checked out in Eclipse, you will want to pay attention to Getting Started with Eclipse and allocate a little extra time for the transition.

Deployers

2.1.0 provides  Debian Deployer, an RPM Deployer, and a Tarball.  In addition to the blazegraph.war and blazegraph.jar archives.

Blazegraph Benchmarking

We will be releasing published benchmarks for LUBM and BSBM for the 2.1.0. release.

Enterprise Features (HA and Scale-out)

Starting in release 2.0.0, the Scale-out and HA capabilities are moved to Enterprise features. These are available to uses with support and/or license subscription. If you are an existing GPLv2 user of these features, we have some easy ways to migrate. Contact us for more information. We’d like to make it as easy as possible.

Much, much, more….

There also many other features for improved data loading, building custom vocabularies, and many more that will be coming out in a series of blog posts over the next month or so.  Please check back.

Stay in touch, we’d love to hear from you.

facebooktwittergoogle_pluslinkedin

Blazegraph 2.0.0 Release!

SystapSiteBanner_1600x362_150px_wideWe’re very pleased to announce the the release Blazegraph 2.0.0.  2.0.0 release is a Major release of Blazegraph.    There are some very exciting changes for query performance, load performance, improvement deployment options, migration to maven, moving to github, and many more.   This lays the foundation for new features with Tinkerpop3 support, GPU Acceleration, etc.      Download it, clone it, have it sent via carrier pigeon (transportation charges may apply).  Find a bug, hit JIRA.  Have a question, try the mailing list or contact-us.

Github

We love Sourceforge, but today it’s important to be on github as well.  This release will be on both Github and Sourceforge.  We’ll continue to use Sourceforge for distribution, mailing lists, etc., but will migrate towards Github for the open source releases.

git clone -b BLAZEGRAPH_RELEASE_2_0_0 --single-branch https://github.com/blazegraph/database.git BLAZEGRAPH_RELEASE_2_0_0
cd BLAZEGRAPH_RELEASE_2_0_0
./scripts/mavenInstall.sh
./scripts/startBlazegraph.sh

Query Performance

We’ve implemented Symmetric Hash Joins  and provided a number of improvements and new features for query performance and optimizations.  For those of you that followed Dr. Michael Schmidt’s excellent posts on the 1.5.2 features.   These improvement represent a continuation of this work.  Let us know how it works for you.  As always, query optimization is a place where we’d love to help you get your application tuned for the best performance.

Maven Central

Blazegraph 2.0.0 is now on Maven Central.    You can also get the Tinkerpop3 API and the new BlazegraphTPFServer.

    
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-core</artifactId>
        <version>2.0.0</version>
    </dependency>
    <!-- Use if Tinkerpop 2.5 support is needed ; See also Tinkerpop3 below. -->
    <dependency>
        <groupId>com.blazegraph</groupId>
        <artifactId>bigdata-blueprints</artifactId>
        <version>2.0.0</version>
    </dependency>

Tinkerpop3

Blazegraph Tinkerpop3

Tinkerpop3 is here!  Get it from Maven Central.

   <dependency>
      <groupId>com.blazegraph</groupId>
      <artifactId>blazegraph-gremlin</artifactId>
      <version>1.0.0</version>
   </dependency>

Blazegraph-Based TDF Server
The Blazegraph-Based TPF Server is a Linked Data Fragment (LDF) server that provides a Triple Pattern Fragment (TPF) interface using the Blazegraph graph database as backend.  It was originally developed by Olaf Hartig and is being released via Blazegraph under the Apache 2 license.   See here to get started.

Mavenization

Everybody loves (and hates) Maven.  Starting with this release, Blazegraph has been broken into a collection of maven artifacts.    This has enabled us to work on new features like TinkerPop3, which require Java 8 support while keeping the core platform at Java 7 to support users who are still there.   Checkout the Maven Notes on the wiki for full details on the architecture, getting started with development, building snapshots, etc.  If you have a 1.5.3 version checked out in Eclipse, you will want to pay attention to Getting Started with Eclipse and allocate a little extra time for the transition.

So long /bigdata, nice to have known you…

Starting with the 2.0 release, default service context is /blazegraph, i.e. http://localhost:9999/blazegraph/.  This is setup to redirect from /bigdata.    We are continuing to provide the bigdata.war and bigdata.jar archives, which by default point to /bigdata as part of the distribution, but this will be discontinued at some point in the future.  We encourage all users to migration to the /blazegraph context with this release.

If you are currently using a serviceURL like http://localhost:9999/bigdata/sparql and want to continue, you’ll need to use bigdata.war, bigdata.jar, or customize the jetty.xml for your deployer.     If not, they new default serviceURL is http://localhost:9999/blazegraph/sparql.

Deployers

2.0.0 provides  Debian Deployer, an RPM Deployer, and a Tarball.  In addition to the blazegraph.war and blazegraph.jar archives.

Blazegraph Benchmarking

We will be releasing published benchmarks for LUBM and BSBM for the 2.0.0. release.

Enterprise Features (HA and Scale-out)

Starting in release 2.0.0, the Scale-out and HA capabilities are moved to Enterprise features. These are available to uses with support and/or license subscription. If you are an existing GPLv2 user of these features, we have some easy ways to migrate. Contact us for more information. We’d like to make it as easy as possible.

Much, much, more….

There also many other features for improved data loading, building custom vocabularies, and many more that will be coming out in a series of blog posts over the next month or so.  Please check back.

Stay in touch, we’d love to hear from you.

facebooktwittergoogle_pluslinkedin

Blazegraph 1.5.3 Release!

We’re very pleased to announce the release of Blazegraph 1.5.3 is now available for download.  1.5.3 is minor release of Blazegraph with some important bug fixes.   The full set of features is available on JIRA.   We recommend upgrading to this release.  If you’re upgrading from prior to 1.5.2, be sure to checkout the updated documentation on SPARQL and Dr. Michael Schmidt‘s excellent blog posts:  [1] and [2] as well as the Wiki documentation:  Sparql Order Matters and Sparql Bottom Up Semantics.

We have some exciting things planned for our 1.6 release coming later this year including GPU acceleration for SPARQL, deployment on Maven Central, and continued improvements in query optimization and performance.  Don’t miss it.  Sign up to stay in touch with Blazegraph.

SystapSiteBanner_1600x362_150px_wideHave you seen our new website? Did you know you can now purchase commercial licensing and support for Blazegraph online?

Will you be at the International Semantic Web Conference (ISWC) this year? We’re a Gold sponsor along with our partner Metaphacts and will be speaking and exhibiting.  Come see us.

Do you have a great success story with Blazegraph? Want to find out more? Check us out below or contact us.  We’d love to hear from you.

Ready to get started?  Check out:

facebooktwittergoogle_pluslinkedin

Blazegraph 1.5.2 Release!

We’re very pleased to announce the release of Blazegraph 1.5.2 is now available for download. This is a significant release of Blazegraph featuring a large number of performance improvements and new features. The full set of features is available on JIRA. Highlights are below:

SystapSiteBanner_1600x362_150px_wideHave you seen our new website? Did you know you can now purchase commercial licensing and support for Blazegraph online?

smartdata2015

Will you be in San Jose at the SmartData or the NoSQLNow conference? We’re a Gold sponsor and will be speaking and exhibiting. Come see us and get a discounted registration.

Do you have a great success story with Blazegraph? Want to find out more? Check us out below or contact us.  We’d love to hear from you.

facebooktwittergoogle_pluslinkedin

Blazegraph 1.5.2 Preview and Bloor Group Briefing Room

Blazegraph 1.5.2 Release Preview

We’re in the final stages of the Blazegraph 1.5.2 release.    This is major release for Blazegraph and brings both exciting new features as well as important performance and query optimizations.  You can see the full set of tickets for the release here (it’s our first one since we’ve migrated to JIRA).

Check it out when the release goes final (we’ll send a mailing list), but here’s some of the new features and improvements:

  • External SOLR Search Integration:  Use the SPARQL Service keyword to search content externally stored in SOLR.
  • Substantial refactoring of the Query Plan Generator in close collaboration with our partner, Metaphacts.  This both fixes a number of bugs in the query optimizer and improves performance.
  • Fixes and improvements for embedded and remote clients including Blueprints and Rexster.
  • Online Backup for the non-HA Blazegraph Server.

Did you see Blazegraph featured in the Bloor Group Briefing room?

Blazegraph was recently featured in the Bloor Group Briefing room. Check out the video presentation below, if you missed it.

We’d love to hear from you.

Do you have a cool new application using Blazegraph or are interested in understanding how to make Blazegraph work best for your application?    Get in touch or send us an email at blazegraph at blazegraph.com.

facebooktwittergoogle_pluslinkedin

What pairs well with Veuve Clicquot and Big Data for Graphs?

Have you ever wondered what to pair with Veuve Clicquot and your Big Data graph challenge? The 2015 Big Data Innovations Summit has the answer:  Blazegraph and Mapgraph.   We won the Big Data Startup Award at the 2015 Big Data Innovations Summit in San Jose.

SYSTAP wins Big Data Startup award at the 2015 Big Data Innovations conference.

SYSTAP wins Big Data Startup award at the 2015 Big Data Innovations conference.

Our solutions for scalable graph technologies were recognized with the award.  Naturally, the champagne pairs perfectly with our Blazegraph database platform and Mapgraph technology for GPU-accelerated graph analytics.  Don’t forget our Blazegraph SWAG (must be present to appreciate…) because In graphs, size matters.

SYSTAP's Brad Bebee receives the Big Data Innovation 2015 award for Big Data Startup.

SYSTAP’s Brad Bebee receives the Big Data Innovation 2015 award for Big Data Startup

Blazegraph™ is our ultra high-performance graph database supporting Blueprints and RDF/SPARQL APIs. It supports up to 50 Billion edges on a single machine and has a High Availability and Scale-out architecture. It is in production use for Fortune 500 customers such as EMC, Autodesk, and many others.  The Wikimedia Foundation recently chose Blazegraph to power the Wikidata Query Service.  Mapgraph™ is our disruptive new technology to use GPUs to accelerate data-parallel graph analytics up to 10,000X faster than other approaches. It can traverse billions of edges in milliseconds.

Whether you need an embedded graph database, a 1 Trillion Edge Graph Database, or the ability to traverse billions of edges in milliseconds, SYSTAP’s award-winning graph solutions can meet your needs.  We’d love to hear about your success with our technology or talk to you about how we can help scale your graph challenge.  Via Twitter:  @Blazegraph or via email.

 

facebooktwittergoogle_pluslinkedin

CPU, Disk, Main Memory for Graphs

Bryan and I were chatting back on the train from NYC on CPU memory bandwidth. Here’s a quick write-up of the discussion.

15 years ago database researchers recognized that CPU memory bandwidth was the limiting factor for relational database performance. This observation was made in the context of relatively wide tables in RDBMS platforms that were heavily oriented to key range scan on a primary key. The resulting architectures are similar to the structure of array (SOA) pattern used by the high performance computing community and within our Mapgraph platform.

Graphs are commonly modeled as 3-column tables. These tables intrinsically have a very narrow stride, similar to column stores for relational data. Whether organized for main memory or disk, the goal of the query planner is to generate an execution strategy that is the most efficient for a given query.

Graph stores that are organized for disk maintain multiple indices with a log structured cost (e.g., a B+Tree) that allow them to jump to the page on the disk that has the relevant tuples for any given access path. Due to the relative cost of memory and disk, main memory systems (such as SPARQLcity) sometimes choose a single index and resort to full table scans when the default index is not useful for a query or access pattern. In making such decisions, main memory databases are trading off memory for selectivity. These designs can consume fewer resources, but it can be much more efficient to have a better index for a complex query plan. Thus, a disk based database can often do as well as or better than a memory based database of the disk based system has a more appropriate index or family of indices. In fact, 90%+ of the performance of a database platform comes from the query optimizer. The difference in performance between a good query plan and a bad query plan for the same database and hardware can be easily 10x, 100x, or 10000x depending on the query. A main memory system with a bad query plan can easily be beaten by a disk -based system with a good query plan. This is why we like to work closely with our customers. For example, one of our long term customers recently upgraded from 1.2.x (under a long term support contract) to 1.5.x and obtained a 100% performance improvement without changing a single line of their code.

Main memory systems become critical when queries much touch a substantial portion of the data. This is true for most graph algorithms that are not hop constrained. For example, an unconstrained breadth first search on a scale free graph will tend to visit all vertices in the graph during the traversal. A page rank or connected components computation will tend to visit all vertices on each iteration and may required up to 50 iterations to converge page rank to a satisfactory epsilon. In such cases, CPU memory architectures will spend most of the wall clock time blocked on memory fetches due to the inherent non-local access patterns during graph traversal. Architectures such as the XMT/XMT-2 (the Urika appliance) handle this problem by using very slow cores, zero latency thread switching, a fast interconnect and hash partitioned memory allocations. The bet of the XMT architecture is that non-locality dominates so you might as well spread all data out everywhere and hide the latency by having a large number of memory transactions in flight. We take a different approach with GPUs and achieve a 10x price/performance benefit over the XMT-2 and a 3x cost savings. This savings will increase substantially when the Pascal GPU is released in Q1 2016 due to an additional 4x gain in memory bandwidth driven by the breadth of the commodity market for GPUs. We obtain this dramatic price/performance and actual performance advantage using zero overhead context switching, fast memory, 1000s of threads to get a large number of in flight memory transactions, and paying attention to locality. The XMT-2 is a beautiful architecture, but locality *always* matters at every level of the memory hierarchy.

facebooktwittergoogle_pluslinkedin