[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288285#comment-16288285 ] ASF GitHub Bot commented on TINKERPOP-1786: --- Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/721 ^ [solved](https://github.com/apache/tinkerpop/commit/a86097d0823397f0485566e8fd5def7a82fb32f3). The `execute()` command was just missing a `waitFor()`, which sometimes led to a missing or incomplete / corrupt zip file. > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Assignee: Marko A. Rodriguez >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] tinkerpop issue #721: TINKERPOP-1786 Recipe and missing manifest items for S...
Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/721 ^ [solved](https://github.com/apache/tinkerpop/commit/a86097d0823397f0485566e8fd5def7a82fb32f3). The `execute()` command was just missing a `waitFor()`, which sometimes led to a missing or incomplete / corrupt zip file. ---
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288234#comment-16288234 ] Marc de Lignie commented on TINKERPOP-1786: --- I mean Daniel, not Marko... > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Assignee: Marko A. Rodriguez >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288230#comment-16288230 ] Marc de Lignie commented on TINKERPOP-1786: --- Hi Marko, I saw your note, I will take a look, Marc > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Assignee: Marko A. Rodriguez >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [Discussed] Integrating SPARQL-Gremlin 0.2 Plugin with the TinkerPop codebase
yes - please post questions here. i don't think you need to know much about TinkerPop internal structure. I'd think that sparql-gremlin is expected to be included in the root of the TinkerPop source as a sub-module to the top-level pom. That just means some minor changes to your pom.xml to get it to build along with everything else. See other projects for examples: https://github.com/apache/tinkerpop/blob/f5687ee4497bfbaef4ae89233e4c29f07001ed2c/gremlin-core/pom.xml#L20-L24 You can drop all of this because it is already defined in the root pom.xml: https://github.com/LITMUS-Benchmark-Suite/sparql-to-gremlin/blob/master/pom.xml#L30-L70 Looking at the rest of your pom.xml now, I'm not sure I understand everything your section is doing and if it's all necessary: The root pom.xml should handle the most common build/deploy options and they will be thus inherited to your sub-module pom which is why, for example, the gremlin-core pom is pretty simple for the section: https://github.com/apache/tinkerpop/blob/f5687ee4497bfbaef4ae89233e4c29f07001ed2c/gremlin-core/pom.xml#L119-L151 If there's anything you're sure can be removed from the sparql-gremlin pom.xml section based on how the TinkerPop root pom.xml is setup, the please feel free to cleanup as much as possible there. As for the general project structure of sparql-gremlin, I don't fully understand how it is arranged. There's /Queries /doc /docs/images /output /src and all of that is repeated inside of the /bin directory. something seems amiss there. maybe once that's cleared up a bit I can think more clearly on what additional changes you might need. Another important thing to considerdocumentation. Right now, it's all in the README. I think we will want a new section to the Reference Documentation, probably appearing after Gremlin Variants: http://tinkerpop.apache.org/docs/current/reference/#gremlin-variants Perhaps that could be named "Query Languages" where sparql-gremlin would be the first sub-section. That would set up for some future where we also had sql-gremlin. And perhaps a section for cypher-gremlin which could point to Neo4j's work in this area. You can find reference docs for TinkerPop here: https://github.com/apache/tinkerpop/tree/f5687ee4497bfbaef4ae89233e4c29f07001ed2c/docs/src/reference The easiest way to generate docs is with docker via: docker/build.sh -d without that you need hadoop running with appropriate configurations: http://tinkerpop.apache.org/docs/current/dev/developer/#documentation-environment Well, hope that gives you a few things to work on and think about for your first round of changes in your fork. Looking forward to seeing how this PR shapes up! On Tue, Dec 12, 2017 at 3:40 AM, Harsh Thakkar wrote: > Hi Stephen, > > Very well then, we will start the migration from today. Also we will > submit the signed iclas' today. > > If we have some questions regarding building the code properly, can we > feel free to ask them here? I assume we might need some guidance on how to > getting things plugged in correctly. We both are not much aware of the > internal structure of TinkerPop, so that is why. > > Also, if there is any specific documentation to help us with this, please > lend a pointer. > > Many thanks! > > On 2017-12-11 21:33, Stephen Mallette wrote: > > As there hasn't been any other opinions, It seems we have a lazy > consensus > > to accept sparql-gremlin into TinkerPop's code base. Cool! > > > > Harsh, I think you and Dharmen should proceed with the steps I listed > > above. Once you have the code integrated and building properly in your > > fork, please reply back and point us to it and we can start with some > > coarse grained review of what you have. > > > > Thanks, > > > > Stephen > > > > On Fri, Dec 8, 2017 at 8:44 AM, hars...@gmail.com > wrote: > > > > > Hi Stephen, > > > > > > Thanks for the insight on the process of this integration. I will > reply to > > > your comments in the same manner. > > > > > > 1. Yes, I will do the fork and migrate the code to the Tinkerpop > > > repository, after cleaning the code a bit. We also need to prepare a > > > detailed doc (how-to) for the plugin. This can also be done in > parallel, > > > depending upon the urgency. > > > > > > 2. Yes, we both are contributing to the v0.2 of the sparql-gremlin > plugin. > > > We will both submit the ICLAs. > > > > > > Yes, we (both) will continue to provide support for the 0.2 plugin and > > > also extend it in the future (trying to cover SPARQL 1.1 specification, > > > also fix the OPTIONAL fix in the current version). > > > > > > Looking forward to hear more on this from the devs :) > > > > > > Cheers! > > > > > > On 2017-12-08 13:41, Stephen Mallette wrote: > > > > I agree with Marko's thoughts, both on this topic of including > > > > sparql-gremlin as well as the wider topic of what should be included > in > > > > TinkerPop code base more generally. Providing a path for rdf/sparql > folks > > > > to get into th
Re: [Discussed] Integrating SPARQL-Gremlin 0.2 Plugin with the TinkerPop codebase
Hi Stephen, Very well then, we will start the migration from today. Also we will submit the signed iclas' today. If we have some questions regarding building the code properly, can we feel free to ask them here? I assume we might need some guidance on how to getting things plugged in correctly. We both are not much aware of the internal structure of TinkerPop, so that is why. Also, if there is any specific documentation to help us with this, please lend a pointer. Many thanks! On 2017-12-11 21:33, Stephen Mallette wrote: > As there hasn't been any other opinions, It seems we have a lazy consensus > to accept sparql-gremlin into TinkerPop's code base. Cool! > > Harsh, I think you and Dharmen should proceed with the steps I listed > above. Once you have the code integrated and building properly in your > fork, please reply back and point us to it and we can start with some > coarse grained review of what you have. > > Thanks, > > Stephen > > On Fri, Dec 8, 2017 at 8:44 AM, hars...@gmail.com wrote: > > > Hi Stephen, > > > > Thanks for the insight on the process of this integration. I will reply to > > your comments in the same manner. > > > > 1. Yes, I will do the fork and migrate the code to the Tinkerpop > > repository, after cleaning the code a bit. We also need to prepare a > > detailed doc (how-to) for the plugin. This can also be done in parallel, > > depending upon the urgency. > > > > 2. Yes, we both are contributing to the v0.2 of the sparql-gremlin plugin. > > We will both submit the ICLAs. > > > > Yes, we (both) will continue to provide support for the 0.2 plugin and > > also extend it in the future (trying to cover SPARQL 1.1 specification, > > also fix the OPTIONAL fix in the current version). > > > > Looking forward to hear more on this from the devs :) > > > > Cheers! > > > > On 2017-12-08 13:41, Stephen Mallette wrote: > > > I agree with Marko's thoughts, both on this topic of including > > > sparql-gremlin as well as the wider topic of what should be included in > > > TinkerPop code base more generally. Providing a path for rdf/sparql folks > > > to get into the TinkerPop world seems like a smart direction. > > > > > > Now, assuming that we have consensus to include sparql-gremlin in the > > > TinkerPop code base, the process will look something like this: > > > > > > 1. I think that Harsh should fork the TinkerPop repository and migrate > > > sparql-gremlin into its structure. From there we will provide > > > feedback/review to get that fork into best shape possible prior to his > > > submitting a pull request. I think we can handle initial feedback through > > > the dev list in a separate thread. > > > > > > 2. In parallel to the above item, it appears as though there are two > > > contributors on sparql-gremlin: > > > > > > https://github.com/LITMUS-Benchmark-Suite/sparql-to- > > gremlin/graphs/contributors > > > > > > Both contributors, Harsh and Dharmen, should submit ICLAs: > > > > > > http://apache.org/licenses/icla.pdf > > > > > > and send them to secret...@apache.org. > > > > > > 3. Once ICLAs are confirmed by secretary, Harsh can submit a pull request > > > from his fork where it can under go final review. > > > > > > Does that sound sensible to everyone? > > > > > > btw, Harsh, it sounds as though you intend to continue development on > > > sparql-gremlin after it is part of the TinkerPop repository...does > > Dharmen > > > intend to do the same? > > > > > > On Fri, Dec 8, 2017 at 6:54 AM, Stephen Mallette > > > wrote: > > > > > > > linking marko's reply from the user list: > > > > > > > > https://groups.google.com/d/msg/gremlin-users/zK9jj7bWvrQ/nE1VvhmeAAAJ > > > > > > > > On Thu, Dec 7, 2017 at 1:52 PM, hars...@gmail.com > > > > wrote: > > > > > > > >> Hello, dear Gremlin people! > > > >> > > > >> Apologies for raising this topic a bit late. I planned to start this > > > >> thread quite earlier but wasnât able to due to some reasons. > > > >> > > > >> === short == > > > >> == > > > >> I seek your guidance and also help for polishing and integrating the > > > >> sparql-gremlin 0.2 (https://github.com/LITMUS-Ben > > > >> chmark-Suite/sparql-to-gremlin) plugin in the apache tinkerpop code > > > >> base, succeeding its predecessor developed by Daniel Kupitz ( > > > >> https://github.com/dkuppitz/sparql-gremlin). The new plugin offers > > > >> support for a wide range of SPARQL queries from the SPARQL 1.0 > > features. > > > >> > > > >> > > > >> long == > > > >> === > > > >> > > > >> I am a Ph.D. student at the University of Bonn and work at the > > > >> intersection of semantic web and graph databases. My thesis is > > focused on > > > >> bridging the gap between these two domains by enabling support for > > SPARQL > > > >> querying of Property Graph databases. Thus, working on the > > SPARQL-Gremlin > > > >> interoperability was a
