Re: Code Freeze 3.2.7/3.3.1
It took a bit of effort as the docs had some build problems around python, but I've published SNAPSHOT docs from both branches: http://tinkerpop.apache.org/docs/3.2.7-SNAPSHOT/ http://tinkerpop.apache.org/docs/3.3.1-SNAPSHOT/ We still have a problem doing doc generation on Dockernot sure what's up with that atm, but it has something to do with that spark/yarn recipe that we merged in a while back. It's strange because I was fairly certain I'd generated docs with docker for that PR specifically, but I must have messed up (built the wrong branch successfully perhaps) because it's not working now on either branch. Kuppitz describes the problem here: https://github.com/apache/tinkerpop/pull/721 Hopefully we'll get it resolved soon... On Fri, Dec 8, 2017 at 7:44 AM, Stephen Mallette wrote: > As discussed in a separate thread, I'm initializing code freeze today, a > day earlier that we normally do, on the tp32 and master branch. Starting > today, the focus will be on testing and documentation for those branches. > SNAPSHOTs were already deployed earlier and i don't think there have been > recent changes. I'm not aware of any open PRs that need to be merged so we > are in good shape to move ahead. > > Please use this thread to keep track of issues that pop up during code > freeze week. >
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286521#comment-16286521 ] ASF GitHub Bot commented on TINKERPOP-1786: --- Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/721 I know, this PR was closed long ago, but I just ran into this issue when building the docs using Docker: ``` * source: /usr/src/tinkerpop/docs/src/recipes/olap-spark-yarn.asciidoc target: /usr/src/tinkerpop/target/postprocess-asciidoc/recipes/olap-spark-yarn.asciidoc progress: [] 10% Last 10 lines of /usr/src/tinkerpop/target/postprocess-asciidoc/recipes/olap-spark-yarn.asciidoc: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy26.getApplicationReport(Unknown Source) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplicationReport(YarnClientImpl.java:430) at org.apache.spark.deploy.yarn.Client.getApplicationReport(Client.scala:265) at org.apache.spark.deploy.yarn.Client.monitorApplication(Client.scala:937) at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend$MonitorThread.run(YarnClientSchedulerBackend.scala:144) ERROR org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend - Yarn application has already exited with state FAILED! ``` The docs build fine w/o Docker, so I guess it's something easy, like a missing dependency. Any idea @vtslab? > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Assignee: Marko A. Rodriguez >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] tinkerpop issue #721: TINKERPOP-1786 Recipe and missing manifest items for S...
Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/721 I know, this PR was closed long ago, but I just ran into this issue when building the docs using Docker: ``` * source: /usr/src/tinkerpop/docs/src/recipes/olap-spark-yarn.asciidoc target: /usr/src/tinkerpop/target/postprocess-asciidoc/recipes/olap-spark-yarn.asciidoc progress: [] 10% Last 10 lines of /usr/src/tinkerpop/target/postprocess-asciidoc/recipes/olap-spark-yarn.asciidoc: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy26.getApplicationReport(Unknown Source) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplicationReport(YarnClientImpl.java:430) at org.apache.spark.deploy.yarn.Client.getApplicationReport(Client.scala:265) at org.apache.spark.deploy.yarn.Client.monitorApplication(Client.scala:937) at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend$MonitorThread.run(YarnClientSchedulerBackend.scala:144) ERROR org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend - Yarn application has already exited with state FAILED! ``` The docs build fine w/o Docker, so I guess it's something easy, like a missing dependency. Any idea @vtslab? ---
Re: [Discussed] Integrating SPARQL-Gremlin 0.2 Plugin with the TinkerPop codebase
As there hasn't been any other opinions, It seems we have a lazy consensus to accept sparql-gremlin into TinkerPop's code base. Cool! Harsh, I think you and Dharmen should proceed with the steps I listed above. Once you have the code integrated and building properly in your fork, please reply back and point us to it and we can start with some coarse grained review of what you have. Thanks, Stephen On Fri, Dec 8, 2017 at 8:44 AM, hars...@gmail.com wrote: > Hi Stephen, > > Thanks for the insight on the process of this integration. I will reply to > your comments in the same manner. > > 1. Yes, I will do the fork and migrate the code to the Tinkerpop > repository, after cleaning the code a bit. We also need to prepare a > detailed doc (how-to) for the plugin. This can also be done in parallel, > depending upon the urgency. > > 2. Yes, we both are contributing to the v0.2 of the sparql-gremlin plugin. > We will both submit the ICLAs. > > Yes, we (both) will continue to provide support for the 0.2 plugin and > also extend it in the future (trying to cover SPARQL 1.1 specification, > also fix the OPTIONAL fix in the current version). > > Looking forward to hear more on this from the devs :) > > Cheers! > > On 2017-12-08 13:41, Stephen Mallette wrote: > > I agree with Marko's thoughts, both on this topic of including > > sparql-gremlin as well as the wider topic of what should be included in > > TinkerPop code base more generally. Providing a path for rdf/sparql folks > > to get into the TinkerPop world seems like a smart direction. > > > > Now, assuming that we have consensus to include sparql-gremlin in the > > TinkerPop code base, the process will look something like this: > > > > 1. I think that Harsh should fork the TinkerPop repository and migrate > > sparql-gremlin into its structure. From there we will provide > > feedback/review to get that fork into best shape possible prior to his > > submitting a pull request. I think we can handle initial feedback through > > the dev list in a separate thread. > > > > 2. In parallel to the above item, it appears as though there are two > > contributors on sparql-gremlin: > > > > https://github.com/LITMUS-Benchmark-Suite/sparql-to- > gremlin/graphs/contributors > > > > Both contributors, Harsh and Dharmen, should submit ICLAs: > > > > http://apache.org/licenses/icla.pdf > > > > and send them to secret...@apache.org. > > > > 3. Once ICLAs are confirmed by secretary, Harsh can submit a pull request > > from his fork where it can under go final review. > > > > Does that sound sensible to everyone? > > > > btw, Harsh, it sounds as though you intend to continue development on > > sparql-gremlin after it is part of the TinkerPop repository...does > Dharmen > > intend to do the same? > > > > On Fri, Dec 8, 2017 at 6:54 AM, Stephen Mallette > > wrote: > > > > > linking marko's reply from the user list: > > > > > > https://groups.google.com/d/msg/gremlin-users/zK9jj7bWvrQ/nE1VvhmeAAAJ > > > > > > On Thu, Dec 7, 2017 at 1:52 PM, hars...@gmail.com > > > wrote: > > > > > >> Hello, dear Gremlin people! > > >> > > >> Apologies for raising this topic a bit late. I planned to start this > > >> thread quite earlier but wasn’t able to due to some reasons. > > >> > > >> === short == > > >> == > > >> I seek your guidance and also help for polishing and integrating the > > >> sparql-gremlin 0.2 (https://github.com/LITMUS-Ben > > >> chmark-Suite/sparql-to-gremlin) plugin in the apache tinkerpop code > > >> base, succeeding its predecessor developed by Daniel Kupitz ( > > >> https://github.com/dkuppitz/sparql-gremlin). The new plugin offers > > >> support for a wide range of SPARQL queries from the SPARQL 1.0 > features. > > >> > > >> > > >> long == > > >> === > > >> > > >> I am a Ph.D. student at the University of Bonn and work at the > > >> intersection of semantic web and graph databases. My thesis is > focused on > > >> bridging the gap between these two domains by enabling support for > SPARQL > > >> querying of Property Graph databases. Thus, working on the > SPARQL-Gremlin > > >> interoperability was an obvious idea given the wide popularity of > Gremlin > > >> amongst the Graph DB vendors. > > >> > > >> The sparql-gremlin 0.1 (link - https://github.com/dkuppitz/ > sparql-gremlin) > > >> plugin was developed by Daniel Kupitz, which we have extended to > support > > >> various features of the SPARQL 1.0 specification and have tested using > > >> various synthetic datasets (such as Northwind dataset and the Berlin > sprawl > > >> benchmark [BSBM] dataset) and a wide range of SPARQL queries. > > >> > > >> The extended version of the plugin (sparql-gremlin 0.2, link - > > >> https://github.com/LITMUS-Benchmark-Suite/sparql-to-gremlin) > supports a > > >> variety of query modifiers (group-by, order-by, counts, etc) and > complex > > >> query features su
[jira] [Commented] (TINKERPOP-1851) Gremlin long options for -e and -i are not working properly
[
https://issues.apache.org/jira/browse/TINKERPOP-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286362#comment-16286362
]
ASF GitHub Bot commented on TINKERPOP-1851:
---
Github user asfgit closed the pull request at:
https://github.com/apache/tinkerpop/pull/765
> Gremlin long options for -e and -i are not working properly
> ---
>
> Key: TINKERPOP-1851
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1851
> Project: TinkerPop
> Issue Type: Bug
> Components: console
>Affects Versions: 3.2.6
>Reporter: stephen mallette
>Assignee: stephen mallette
>
> The long forms of {{-e}} which is {{--execute}} and {{-i}} which is
> {{--interactive}} are not being processed properly. The console just seems to
> hang itself.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] tinkerpop pull request #765: TINKERPOP-1851 Minor fix to allow long forms of...
Github user asfgit closed the pull request at: https://github.com/apache/tinkerpop/pull/765 ---
[jira] [Commented] (TINKERPOP-1851) Gremlin long options for -e and -i are not working properly
[
https://issues.apache.org/jira/browse/TINKERPOP-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286049#comment-16286049
]
ASF GitHub Bot commented on TINKERPOP-1851:
---
Github user dkuppitz commented on the issue:
https://github.com/apache/tinkerpop/pull/765
VOTE: +1
> Gremlin long options for -e and -i are not working properly
> ---
>
> Key: TINKERPOP-1851
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1851
> Project: TinkerPop
> Issue Type: Bug
> Components: console
>Affects Versions: 3.2.6
>Reporter: stephen mallette
>Assignee: stephen mallette
>
> The long forms of {{-e}} which is {{--execute}} and {{-i}} which is
> {{--interactive}} are not being processed properly. The console just seems to
> hang itself.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] tinkerpop issue #765: TINKERPOP-1851 Minor fix to allow long forms of -e and...
Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/765 VOTE: +1 ---
[jira] [Commented] (TINKERPOP-1852) Recipe fails for highly meshed connected components
[ https://issues.apache.org/jira/browse/TINKERPOP-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285838#comment-16285838 ] Marc de Lignie commented on TINKERPOP-1852: --- One question I have: can the current recipe be updated without providing a solution for OLAP? I do not feel comfortable in just suggesting that it might work for SparkGraphComputer without extensive testing that it has any value over the OLTP case. Also I feel that a proper distributed algo should provide performance benefits (py parallellization) over an OLTP case for a graph that still fits in memory. Further, it should be shown that an OLAP algo for connected components has the same order of magnitude performance as the one in Apache Spark Graphx. That being said I would prefer updating the OLTP case and leaving the OLAP case as work in progress (while removing the old OLAP example). > Recipe fails for highly meshed connected components > --- > > Key: TINKERPOP-1852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1852 > Project: TinkerPop > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.0, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > > On the JanusGraph user list the connected-components recipe was shown to fail > for a completely meshed component with a size of 35. The successive thread > [https://groups.google.com/forum/#!topic/janusgraph-users/NmwZyag1w2M] shows > a better solution, both as a mixed groovy-gremlin and a pure gremlin query > (with the latter having slightly less well scaling behavior because of the > required path computations). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
