version 0.8 beta

We are happy to announce the release of version 0.8 beta on Sourceforge. This release packages both the scale-out (distributed) and scale-up (single-server) versions of the bigdata RDF store. This release is capable of loading 1B triples in well under one hour on a 15 node cluster and has been used to load up to 13B triples on the same cluster. It also captures substantial improvements in the scale-out architecture. While query performance has not been optimized recently, it is nevertheless quite reasonable.

We

facebooktwittergoogle_pluslinkedin

6 thoughts on “version 0.8 beta

  1. florent

    Hi,
    This project seems more and more interesting each time i look at it.

    I can't find any mailing list and your discussion forum seems empty so I'll ask my questions there, feel free to redirect me to the proper place if necessary.
    Is it possible to use it inside a Sesame http server? If yes, do you have some exemple of the templates for the server configuration on the sesame side?
    When is sesame 2.2.4 support planned? The last one is regarding your free text search. Do you have some other exemples of how it works? We are currently using the LuceneSail, do you know how to comparres the 2 implementations?

  2. Mike Personick

    Hey Florent,

    Thanks for your interest in bigdata.

    I'll look into the Sesame questions today, I suspect using bigdata through the http server is simply a matter of getting the XML configuration snippet right. I'll also look into the changes needed for 2.2.4. It may just be some trivial API change.

    As for the free text index, did you see the example in the Getting Started guide on the wiki? There is some code in bigdata-sails/src/samples that demonstrates how it all works. If you need more specific guidance feel free to contact me via email.

    Thanks,
    Mike

  3. florent

    Hi,

    As far as I remember, to get it working on via the http, you need the n3 configuration file, a configuration class using this file and a factory able to initialize your sail. It's actually pretty trivial. I don't know the changes for the 2.2.4 on the other side.
    By the way what kind of sail is it? a stackable sail or we have to use it directly as a repository?

    For the free text index, I looked but it seems kind of basics. A feature i like in the sesame sail is the possibility to get snippet and score in your query result. You don't have such thing?

    Another question, We are relying on the virtual property (directSubClass and directSubProperty) from the sesame inferencer, do you have an equivalent, or is it planned?

    (and I'm afraid this is currently only curiosity questions 🙂 I don't have much time to test things and we don't need to scale yet. But time will come so that's why i'm keeping the project on the radar since it seems to be one of the few which could clearly scale…)

  4. Mike Personick

    Florent,

    I have not tried using it as a stackable SAIL, I'll try that out as well.

    You are right, the text index right now is quite basic, no snippets or scores.

    I am a bit fuzzy on how the direct type hierarchy results are calculated in Sesame 2, I remember clearly in Sesame 1 having to do these calculations in the RdfsRepository implementation. There is a DirectTypeHierarchyInferencer in Sesame 2 that looks like it might work with bigdata. I'll snoop around.

    Thanks,
    Mike

  5. Bryan Thompson

    There is more information available from the search results. The issue is just how we are materializing the results as bindings for RDF Values. It would be easy enough to have an alternative magic predicate which materialized triples [y assignedScore x]. If you point to the specifics of the search results scheme that you would like to see we can look into exposing more information.

  6. florent

    Hi,

    Nice to see all the fast answers.

    For the direct type inferencer, it's a separate sail wrapper so it might work out of box if your sail support it.

    concerning the fulltext, I'm using the lucene sail: http://dev.nepomuk.semanticdesktop.org/wiki/LuceneSail (you can see the example)
    especially the score and snippet. I never needed to use the restriction on property although it's look like a nice idea. I don't say you should do exactly the same, I know the sparql WG is also working on it…

    ok, and now i definitely need to find some times to test and play.

    florent

Leave a Reply

Your email address will not be published. Required fields are marked *