That's still subject to discussion. As you see, we are pursuing two goals: contribute to the Shindig project and do some research. Our first comparison was between SQL, Cypher, Gremlin and neo4j native API regarding speed and maintainability of the code. We found out, that Cypher is a good compromise. For this next step we want to experiment with different network protocols and payload types. In our first comparison, we found out that the existing REST interface of neo4j is not performing very well. Neo technologies has enhanced it since then, but we envision comparing a Web socket connection with the existing http connection and a raw TCP socket connection regarding speed. For the payload we are looking at JSON (currently used in neo4j), BSON and an own object serialization. The last comparison will deal with the commands passed from Shindig to neo4j. The obvious choice is Cypher, since this is becoming the default language for neo4j, but we also want to experiment with kind of stored procedures (not currently supported in neo4j), where we only transfer the name of the query and the parameters. The implementation could then be Cypher or native api on the neo4j side. Any suggestions? Regards René P.S.: Another interesting thing would be to experiment with asynchronous, non-blocking database connections, but I guess that doesn't make sense, as long as the rest of Shindig uses blocking calls.
-----Ursprüngliche Nachricht----- Von: Ryan Baxter [mailto:rbaxte...@apache.org] Gesendet: Donnerstag, 18. Juli 2013 22:31 An: dev@shindig.apache.org Betreff: Re: Review Request: Alternative database backend based on graph database neo4j Great. So what technically will the code that is contributed to Shindig do? How does it talk to the neo4j backend? On Thu, Jul 18, 2013 at 9:25 AM, René Peinl <rene.pe...@hof-university.de> wrote: > Dear Ryan, > fair question. The main contribution is to allow usage of neo4j as an > alternative database backend which improves performance significantly > in some areas compared to ORM and MySQL (see > http://www.edbt.org/Proceedings/2013-Genova/papers/workshops/a29-holzs > chuher > .pdf). > In addition to that, we have also implemented some of the missing > features from Open Social 2.0 like adding friends to the friend > network and some additional features not mentioned in the Open Social > specification, but state-of-the-art in social networks like friend and > group recommendations based on friends in common and groups of friends. > The limitation is, that these do only work or are only tested with > neo4j backend. > Regards > René > > -----Ursprüngliche Nachricht----- > Von: Ryan Baxter [mailto:rbaxte...@apache.org] > Gesendet: Mittwoch, 17. Juli 2013 02:32 > An: dev@shindig.apache.org > Cc: Ate Douma; Peter Neubauer; Florian Holzschuher > Betreff: Re: Review Request: Alternative database backend based on > graph database neo4j > > The split proposal sounds like a good approach, but what exactly would > be contributed to Shindig? > > On Tue, Jul 16, 2013 at 9:46 AM, René Peinl > <rene.pe...@hof-university.de> > wrote: >> Dear Ate, dear Shindig developers, >> it's been some time since this discussion because we were a bit >> frustrated and the project on our side was frozen for some time. >> Fortunately, I came up with an idea of how to circumvent this problem >> and this time wanted to do the legal check BEFORE we start the coding. >> >> The current code has a direct compilation dependency on either neo4j >> directly or at least their REST/JSON wrapper, which is also GPLv3 > licensed. >> That seems to be the problem. >> We therefore propose to split our code into two parts. One part >> should become part of the Apache Shindig project and will be licensed APLv2. >> The other part will directly use neo4j and therefore be licensed >> under >> GPLv3 and published on sourceforge or somewhere. Between those two >> parts there will be only network communication, no compilation dependency. >> >> Could somebody from the core team please confirm, that >> a) this new proposal will have no licensing problem (my German >> lawyer, who is specialized in OSS licensing confirmed that, but the >> ASF may still see it >> differently) >> b) the community is still interested in the neo4j backend as an >> option for Shindig >> >> Otherwise, we would not invest any more time in this issue. >> Regards >> René >> >> -----Ursprüngliche Nachricht----- >> Von: Ate Douma [mailto:a...@douma.nu] >> Gesendet: Mittwoch, 27. März 2013 15:22 >> An: dev@shindig.apache.org >> Cc: Peter Neubauer; René Peinl; Florian Holzschuher >> Betreff: Re: Review Request: Alternative database backend based on >> graph database neo4j >> >> Sorry about top posting (and dropping legal-discuss@) but as we >> didn't get any feedback yet on the below question from >> legal-discuss@, I've dived into it myself a bit further. >> >> I already came to the conclusion (and discussed this internally with >> another ASF >> member) that the current proposed contribution which directly uses >> Neo4J >> GPL3 licensed APIs cannot be allowed, as that (source) dependency >> will enforce the >> GPL3 license upon this contribution as a whole, optional or not. >> >> The possible 'workaround' of using a 3rd party library like >> spring-data-neo4j >> (only) which is ALv2 licensed I am also very doubtful about, because >> that library itself has direct usage and dependency on the Neo4J >> APIs, so it really is only 'hiding' the same problem. >> If this is a correct assumption, and I created an specific JIRA >> question [1] for legal-discuss@ for it, then the spring data >> 'workaround' is also tainted and not to be allowed either. >> >> I've also brought this forward to someone at Apache Camel which >> intend to (but hasn't yet) release a Camel Neo4J component, and it >> looks like they will decide also that Neo4J cannot be supported after >> all, at the > ASF. >> >> Pending the resolution of [1], it now seems doubtful to me Apache >> Shindig can accept the neo4j support contribution. >> If this becomes the case, I see two possible solutions, in preferred >> order (besides dropping it): >> >> a) NeoTechnology provides an additional (probably API only) library >> of its own under an ALv2 compatible license, which can be used to >> create integrations like these. This would be the most practical and >> beneficial solution IMO, similar to for instance the MongoDB (AGPL3 >> licensed) mongodb-java-driver library which is >> ALv2 licensed, see [2]. >> >> b) Provide the Shindig Neo4J support in a separate project outside >> the ASF, maybe at Apache Extras [3]. Downside of this of course is >> that alignment with Apache Shindig itself will be more difficult and >> likely trailing the development @Shindig. >> >> I'll keep the list noted on any progress or conclusion of [1]. >> >> Regards, Ate >> >> [1] https://issues.apache.org/jira/browse/LEGAL-162 >> [2] https://github.com/mongodb/mongo-java-driver >> [3] http://community.apache.org/apache-extras/faq.html >> >> On 03/11/2013 11:09 AM, Ate Douma wrote: >>> Now replying again, but to the proper mail address, see inline below. >>> >>> Cheers, Ate >>> >>> On 03/11/2013 11:05 AM, Ate Douma wrote: >>>> Another forward of an incorrectly addressed email for >>>> legal-discuss@ >>>> >>>> On 03/11/2013 09:41 AM, Peter Neubauer wrote: >>>>> Also, >>>>> keep in mind that only the enterprise components of Neo4j are >>>>> AGPL, the community edition, which I believe is the most >>>>> interesting part here, is GPL. >>> >>> That is useful information indeed. Thanks for correcting me in this! >>> >>> So for this case at hand I assume we only need to consider the >>> possibilities to use the GPL3 licensed community edition of neo4j. >>> >>> The question then is if at the ASF we may reference and explicitly >>> use >>> GPL3 licensed APIs in our AL2.0 licensed code, in an optional >>> module, which also requires these GPL3 libraries at runtime. >>> Even if we don't distribute those 3rd party GPL3 libraries ourselves. >>> >>>>> >>>>> /peter >>>>> >>>>> >>>>> Cheers, >>>>> >>>>> /peter neubauer >>>>> >>>>> G: neubauer.peter >>>>> S: peter.neubauer >>>>> P: +46 704 106975 >>>>> L: http://www.linkedin.com/in/neubauer >>>>> T: @peterneubauer >>>>> >>>>> Graph database introduction book for the uninitiated - >>>>> http://graphdatabases.com Neo4j questions? Please use SO - >>>>> http://stackoverflow.com/search?q=neo4j >>>>> >>>>> >>>>> On Mon, Mar 11, 2013 at 9:22 AM, René Peinl >> <rene.pe...@hof-university.de>wrote: >>>>> >>>>>> Dear Apache legal advisors, dear Shindig developers, as you can >>>>>> see from the discussion below, we have a possible license >>>>>> conflict between AGPL and APL v2. >>>>>> We want to integrate code that uses neo4j, a graph database which >>>>>> is licensed under AGPL, into Shindig. From my perspective it is >>>>>> not necessary to include any neo4j binaries nor code, but I'm not >>>>>> sure how this affects compilability. Maybe we can only use the >>>>>> REST API then and don't offer to run neo4j in embedded mode. >>>>>> I'm not a lawyer nor a licensing specialist, so please advise on >>>>>> how to proceed. Maybe we can find a workaround that ensures we >>>>>> are conforming to the licensing terms and still get the new >>>>>> functionality >> into Shindig. >>>>>> One suggestion that seems a good one was to check how Apache >>>>>> Camel deals with this issue. >>>>>> Regards and many thanks for clarification in advance René >>>>>> >>>>>> -----Ursprüngliche Nachricht----- >>>>>> Von: Ate Douma [mailto:a...@douma.nu] >>>>>> Gesendet: Montag, 11. März 2013 08:49 >>>>>> An: dev@shindig.apache.org >>>>>> Cc: Peter Neubauer >>>>>> Betreff: Re: Review Request: Alternative database backend based >>>>>> on graph database neo4j >>>>>> >>>>>> On 03/10/2013 11:59 PM, Matt Franklin wrote: >>>>>>> On Sunday, March 10, 2013, wrote: >>>>>>> >>>>>>>> Thanks for the insight Ate. >>>>>>>> >>>>>>>> Rene, I think we should take Ate's suggestion and send an email >>>>>>>> to legal-discussion@ (please CC shindig-dev@). If they say it >>>>>>>> is OK than we continue the discussion about integrating the patch. >>>>>>> >>>>>> >>>>>> Although the answer from Peter Neubauer / neotechnology is >>>>>> accommodating on this matter and seems to indicate *they* might >>>>>> think this is not a problem, reading the AGPL [1] license tells >>>>>> me >> something differently. >>>>>> I definitely would like this contribution to be acceptable, but >>>>>> we must be very sure we're not opening a can of worms here. >>>>>> >>>>>>> >>>>>>> I agree that legal should be consulted if we intent to ship a >>>>>>> war or other archive with any neo4j (or other agpl) licensed >>>>>>> binaries >> included. >>>>>> I don't think we can do that anyway. AGPL is a variant of GPL, >>>>>> and we're not allowed, by ASF policy, to distribute any GPL artifact. >>>>>> >>>>>>> >>>>>>> As a first mitigation step, why don't we make this a separate >>>>>>> maven module and only ship the source and non-inclusive jar? It >>>>>>> should not be a problem to ship a jar and source that only >>>>>>> references the neo4j libs as runtime dependencies. >>>>>> That might be a possibility to investigate. As Chris noted in >>>>>> another email, it might be doable as Camel seemingly also has a >>>>>> neo4j component. >>>>>> >>>>>> But also note: it will also depend on the type of reference such >>>>>> an optional module uses. If it requires explicit Java imports and >>>>>> direct usage of the neo4j APIs, this might qualifies as what is >>>>>> called in the AGPL 'Corresponding Source'. >>>>>> Especially as neo4j and Shindig provide and expect 'Remote >>>>>> Network Interaction' >>>>>> for which the AGPL is especially created to enforce its license >>>>>> to downstream users. IMO this can lead to a conflict with the >>>>>> AS2.0 license, to possibly not even be allowed distribution under >>>>>> that license from within the ASF, or not even its sources be >>>>>> checked into svn... >>>>>> >>>>>> But IANAL so indeed this should be run through legal-discuss@ first. >>>>>> >>>>>> [1] http://www.gnu.org/licenses/agpl-3.0.html >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mar 9, 2013, at 7:56 AM, René Peinl >>>>>>>> <rene.pe...@hof-university.de> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Dear Ate, >>>>>>>>> thanks for your comments. I already thought about this and >>>>>>>>> asked the >>>>>>>> guys from neo technologies. Here is the answer from Peter Neubauer. >>>>>>>>> >>>>>>>>> in principle (IANAL) it is ok to have ALv2 licensed code >>>>>>>>> binding to GPL >>>>>>>> code. In runtime, the user will not be shielded from the GPL >>>>>>>> core, which means the runtime will have GPL characteristics >>>>>>>> when you plug in >>>>>> Neo4j. >>>>>>>> That is exactly the intent, and should be ok. The bindings-code >>>>>>>> is development-time Apache license, regarding contributions and >>>>>>>> copyright etc, so I think this should be ok. >>>>>>>>> >>>>>>>>> I'm not quite sure if that answers your question. I can >>>>>>>>> further >>>>>>>> investigate if necessary. >>>>>>>>> Regards >>>>>>>>> René >>>>>>>>> >>>>>>>>> -----Ursprüngliche Nachricht----- >>>>>>>>> Von: Ate Douma [mailto:a...@douma.nu] >>>>>>>>> Gesendet: Freitag, 8. März 2013 14:18 >>>>>>>>> An: dev@shindig.apache.org >>>>>>>>> Betreff: Re: Review Request: Alternative database backend >>>>>>>>> based on graph >>>>>>>> database neo4j >>>>>>>>> >>>>>>>>> Just from the peanut gallery, but neo4j is AGPL licensed. >>>>>>>>> Normally any database backend access which is abstracted away >>>>>>>>> behind >>>>>>>> 'plain' >>>>>>>>> JDBC interfaces are allright to use, commercial versions or >>>>>>>>> otherwise >>>>>>>> licensed, because the end-user would have the option to choose >>>>>>>> whatever >>>>>>>> (compatible) database they want to use. >>>>>>>>> >>>>>>>>> However with neo4j this seems different. Even with only >>>>>>>>> optional support >>>>>>>> for neo4j, the neo4j integration might require explicit neo4j >>>>>>>> (Java) APIs and dependencies? I haven't reviewed the code for >>>>>>>> this, but if it imports neo4j APIs then their AGPL license can >>>>>>>> be too invasive and then possibly not acceptable for uses >>>>>>>> within our >>>>>>>> AL2.0 licensed >>>>>> codebase. >>>>>>>>> Or even if that could be allowed, I would make sure to check >>>>>>>>> and ask >>>>>>>> (legal-discuss@ etc.) if it would be acceptable from ASF policy POV. >>>>>>>>> >>>>>>>>> Regards, Ate >>>>>>>>> >>>>>>>>> On 03/07/2013 07:46 PM, Henry Saputra wrote: >>>>>>>>>> This is good news. >>>>>>>>>> >>>>>>>>>> One immediate comment is about the package name. >>>>>>>>>> Would it be possible to put it under org.apache.shindig >>>>>>>>>> rather than the de.hofuniversity? >>>>>>>>>> >>>>>>>>>> This would make the contributions uniform like from other >>>>>>>>>> companies and organizations. >>>>>>>>>> >>>>>>>>>> - Henry >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2013/3/6 René Peinl <rene.pe...@hof-university.de> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ----------------------------------------------------------- >>>>>>>>>>> This is an automatically generated e-mail. To reply, visit: >>>>>>>>>>> https://reviews.apache.org/r/9773/ >>>>>>>>>>> ----------------------------------------------------------- >>>>>>>>>>> >>>>>>>>>>> Review request for shindig. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Description >>>>>>>>>>> ------- >>>>>>>>>>> >>>>>>>>>>> Review for Shindig-1911 >>>>>>>>>>> Alternative database backend based on graph database neo4j >>>>>>>>>>> Any comments welcome. We are committed to further improve this. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This addresses bug Shindig-1911. >>>>>>>>>>> https://issues.apache.org/jira/browse/Shindig-1911 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Diffs >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>> /trunk/java/neo4j-backend/pom.xml PRE-CREATION >>>>>>>>>>> >>>>>>>>>>> /trunk/java/neo4j-backend/src/main/java/de/hofuniversity/iis >>>>>>>>>>> y >>>>>>>>>>> s >>>>>>>>>>> /gra >>>>>>>>>>> phb >>>>>>>>>>> ackend/Constants.java >>>>>>>>>>> PRE-CREATION >>>>>>>>>>> >>>>>>>>>>> /trunk/java/neo4j-backend/src/main/java/de/hofuniversity/iis >>>>>>>>>>> y >>>>>>>>>>> s >>>>>>>>>>> /gra >>>>>>>>>>> phb >>>>>>>>>>> ackend/GraphAPIModule.java >>>>>>>>>>> PRE-CREATION >>>>>>>>>>> >>>>>>>>>>> /trunk/java/neo4j-backend/src/main/java/de/hofuniversity/iis >>>>>>>>>>> y >>>>>>>>>>> s >>>>>>>>>>> /gra >>>>>>>>>>> phb >>>>>>>>>>> ackend/GraphConfig.java >>>>>>>>>>> PRE-CREATION >>>>>>>>>>> >>>>>>>>>>> /trunk/java/neo4j-backend/src/main/java/de/hofuniversity/iis >>>>>>>>>>> y >>>>>>>>>>> s >>>>>>>>>>> /gra >>>>>>>>>>> phb >>>>>>>>>>> ackend/GuiceModule.java >>>>>>>>>>> PRE-CREATION >>>>>>>>>>> >>>>>>>>>>> /trunk/java/neo4j-backend/src/main/java/de/hofuniversity/iis >>>>>>>>>>> y >>>>>>>>>>> s >>>>>>>>>>> /gra >>>>>>>>>>> phb >>>>>>>>>>> acke >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >> >> >> >
