[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850383#comment-15850383
 ] 

ASF GitHub Bot commented on TINKERPOP-1566:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/534
  
sorry - i was good with whatever @mike-tr-adamson was down for. he 
originally brought in the sasl stuff for tinkerpop, so i was glad he could take 
a moment to review and comment on this. if he's good with what's here now, we 
can have committers start to review things.


> Kerberos authentication for gremlin-server
> --
>
> Key: TINKERPOP-1566
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1566
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Reporter: Marc de Lignie
>Priority: Minor
>  Labels: security
> Fix For: 3.3.0
>
>
> Gremlin server would benefit from an explicit Kerberos authentication plugin, 
> because preparing and maintaining such a plugin is nontrivial. Also, many 
> other Apache project provide kerberized services.
> In gremlin-console the standard Krb5LoginModule can be configured. 
> Gremlin-server already includes the pluggable Sasl framework that can host 
> the proposed Kerberos authentication plugin. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] tinkerpop issue #534: TINKERPOP-1566 Kerberos authentication for gremlin-ser...

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/534
  
sorry - i was good with whatever @mike-tr-adamson was down for. he 
originally brought in the sasl stuff for tinkerpop, so i was glad he could take 
a moment to review and comment on this. if he's good with what's here now, we 
can have committers start to review things.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850362#comment-15850362
 ] 

ASF GitHub Bot commented on TINKERPOP-1566:
---

Github user vtslab commented on the issue:

https://github.com/apache/tinkerpop/pull/534
  
I interpreted almosta week of silence as consensus on the gremlin-driver 
behavior, so I made the following changes:
- restored gremlin-driver's Handler (checkout from master)
- added a ToDo to gremlin-driver's Handler regarding receiving allowed 
mechanisms from gremlin-server
- added GSSException fail options to three tests that required so
- corrected the changelog


> Kerberos authentication for gremlin-server
> --
>
> Key: TINKERPOP-1566
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1566
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Reporter: Marc de Lignie
>Priority: Minor
>  Labels: security
> Fix For: 3.3.0
>
>
> Gremlin server would benefit from an explicit Kerberos authentication plugin, 
> because preparing and maintaining such a plugin is nontrivial. Also, many 
> other Apache project provide kerberized services.
> In gremlin-console the standard Krb5LoginModule can be configured. 
> Gremlin-server already includes the pluggable Sasl framework that can host 
> the proposed Kerberos authentication plugin. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] tinkerpop issue #534: TINKERPOP-1566 Kerberos authentication for gremlin-ser...

[vtslab]

Github user vtslab commented on the issue:

https://github.com/apache/tinkerpop/pull/534
  
I interpreted almosta week of silence as consensus on the gremlin-driver 
behavior, so I made the following changes:
- restored gremlin-driver's Handler (checkout from master)
- added a ToDo to gremlin-driver's Handler regarding receiving allowed 
mechanisms from gremlin-server
- added GSSException fail options to three tests that required so
- corrected the changelog


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1271) SparkContext should be restarted if Killed and using Persistent Context

[Marko A. Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850242#comment-15850242
 ] 

Marko A. Rodriguez commented on TINKERPOP-1271:
---

Can you try below please:

{code}
cd spark-gremlin/
mvn clean install -DskipIntegrationTests=false
{code}

That will run for about 10 minutes.

> SparkContext should be restarted if Killed and using Persistent Context
> ---
>
> Key: TINKERPOP-1271
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1271
> Project: TinkerPop
>  Issue Type: Bug
>  Components: hadoop
>Affects Versions: 3.2.0-incubating, 3.1.2-incubating
>Reporter: Russell Spitzer
>
> If the persisted Spark Context is killed by the user via the Spark UI or is 
> terminated for some other error the Gremlin Console/Server is left with a 
> stopped Spark Context. This could be caught and the spark context recreated. 
> Oddly enough if you simply wait the context will "reset" itself or possible 
> get GC'd out of the system and everything works again. 
> ##Repo
> {code}
> gremlin> g.V().count()
> WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  
> - HADOOP_GREMLIN_LIBS is not set -- proceeding regardless
> ==>6
> gremlin> ERROR org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend 
>  - Application has been killed. Reason: Master removed our application: KILLED
> ERROR org.apache.spark.scheduler.TaskSchedulerImpl  - Lost executor 0 on 
> 10.150.0.180: Remote RPC client disassociated. Likely due to containers 
> exceeding thresholds, or network issues. Check driver logs for WARN messages.
> // Driver has been killed here via the Master UI
> gremlin> graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
> ==>hadoopgraph[gryoinputformat->gryooutputformat]
> gremlin> g.V().count()
> WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  
> - HADOOP_GREMLIN_LIBS is not set -- proceeding regardless
> java.lang.IllegalStateException: Cannot call methods on a stopped 
> SparkContext.
> This stopped SparkContext was created at:
> org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
> org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:53)
> org.apache.tinkerpop.gremlin.spark.structure.io.SparkContextStorage.open(SparkContextStorage.java:60)
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:122)
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> The currently active SparkContext was created at:
> org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
> org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:53)
> org.apache.tinkerpop.gremlin.spark.structure.io.SparkContextStorage.open(SparkContextStorage.java:60)
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:122)
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}
> Full trace from TP
> {code}
>   at 
> org.apache.spark.SparkContext.org$apache$spark$SparkContext$$assertNotStopped(SparkContext.scala:106)
>   at 
> org.apache.spark.SparkContext$$anonfun$newAPIHadoopRDD$1.apply(SparkContext.scala:1130)
>   at 
> org.apache.spark.SparkContext$$anonfun$newAPIHadoopRDD$1.apply(SparkContext.scala:1129)
>   at 
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
>   at 
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
>   at org.apache.spark.SparkContext.withScope(SparkContext.scala:714)
>   at 
> org.apache.spark.SparkContext.newAPIHadoopRDD(SparkContext.scala:1129)
>   at 
> org.apache.spark.api.java.JavaSparkContext.newAPIHadoopRDD(JavaSparkContext.scala:507)
>   at 
> org.apache.tinkerpop.gremlin.spark.structure.io.InputFormatRDD.readGraphRDD(InputFormatRDD.java:42)
>   at 
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:195)
>   at 
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run

[jira] [Commented] (TINKERPOP-1271) SparkContext should be restarted if Killed and using Persistent Context

[Artem Aliev (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850209#comment-15850209
 ] 

Artem Aliev commented on TINKERPOP-1271:


"mvn install" tests passed
 I have test it manually on master with spark 2.0 and back ported SPARK-19362, 
to check stop works.


> SparkContext should be restarted if Killed and using Persistent Context
> ---
>
> Key: TINKERPOP-1271
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1271
> Project: TinkerPop
>  Issue Type: Bug
>  Components: hadoop
>Affects Versions: 3.2.0-incubating, 3.1.2-incubating
>Reporter: Russell Spitzer
>
> If the persisted Spark Context is killed by the user via the Spark UI or is 
> terminated for some other error the Gremlin Console/Server is left with a 
> stopped Spark Context. This could be caught and the spark context recreated. 
> Oddly enough if you simply wait the context will "reset" itself or possible 
> get GC'd out of the system and everything works again. 
> ##Repo
> {code}
> gremlin> g.V().count()
> WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  
> - HADOOP_GREMLIN_LIBS is not set -- proceeding regardless
> ==>6
> gremlin> ERROR org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend 
>  - Application has been killed. Reason: Master removed our application: KILLED
> ERROR org.apache.spark.scheduler.TaskSchedulerImpl  - Lost executor 0 on 
> 10.150.0.180: Remote RPC client disassociated. Likely due to containers 
> exceeding thresholds, or network issues. Check driver logs for WARN messages.
> // Driver has been killed here via the Master UI
> gremlin> graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
> ==>hadoopgraph[gryoinputformat->gryooutputformat]
> gremlin> g.V().count()
> WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  
> - HADOOP_GREMLIN_LIBS is not set -- proceeding regardless
> java.lang.IllegalStateException: Cannot call methods on a stopped 
> SparkContext.
> This stopped SparkContext was created at:
> org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
> org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:53)
> org.apache.tinkerpop.gremlin.spark.structure.io.SparkContextStorage.open(SparkContextStorage.java:60)
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:122)
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> The currently active SparkContext was created at:
> org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
> org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:53)
> org.apache.tinkerpop.gremlin.spark.structure.io.SparkContextStorage.open(SparkContextStorage.java:60)
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:122)
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}
> Full trace from TP
> {code}
>   at 
> org.apache.spark.SparkContext.org$apache$spark$SparkContext$$assertNotStopped(SparkContext.scala:106)
>   at 
> org.apache.spark.SparkContext$$anonfun$newAPIHadoopRDD$1.apply(SparkContext.scala:1130)
>   at 
> org.apache.spark.SparkContext$$anonfun$newAPIHadoopRDD$1.apply(SparkContext.scala:1129)
>   at 
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
>   at 
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
>   at org.apache.spark.SparkContext.withScope(SparkContext.scala:714)
>   at 
> org.apache.spark.SparkContext.newAPIHadoopRDD(SparkContext.scala:1129)
>   at 
> org.apache.spark.api.java.JavaSparkContext.newAPIHadoopRDD(JavaSparkContext.scala:507)
>   at 
> org.apache.tinkerpop.gremlin.spark.structure.io.InputFormatRDD.readGraphRDD(InputFormatRDD.java:42)
>   at 
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:195)
>   at 
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617

Re: Architectural Best Practice for Graph Database Systems

[Stephen Mallette]

I think TinkerPop has most of its documentation focused on TinkerPop itself
which is a pretty big topic and less on "how to implement applications with
TinkerPop". I think the community could benefit from documentation like
that. Is that what the content you have is about when you refer to
"Architectural
Best Practice for Graph Database Systems"?

Just a personal opinion, but I will say that TinkerPop does have a certain
style to its documentation, its web site, etc. I'd personally like to see
all documentation fit that style, so depending on what's been written, it
might need some tweaking to make it fit the model that's been established
here.

That's about all I can say at the moment without more information about
what you currently have that you'd like to contribute. Perhaps you could
provide some more information about it.

On Thu, Feb 2, 2017 at 11:54 AM, Phil Tetlow 
wrote:

> All,
>
> IBM  has been working on graph database for some time (culminating in
> JanusGraph, a new community project under The Linux Foundation ) and has
> amassed lots of best practice in this area. We have also looked at a number
> of technology stacks, but now are settling on Titan and Tinkerpop as our
> standard - thereby offering native integration as an integral feature of
> Janus.
>
> Of late we have noticed a growing interest in graph database and we are
> excited about the level of contribution through mailing lists like this.
> That said, we have also noticed a distinct lack of shared insight into how
> best to design and build systems using graph, and we would be interested to
> hear if those on this list agree? We have been referring to this as
> “Architectural Best Practice for Graph Database Systems” and we already
> have a number of assets we would like to share openly. That presents a
> challenge, however, as currently we see no natural home for such materials
> within the open source community. Nevertheless, when we thought about it,
> the clear consensus pointed to some branch under Tinkerpop, and for this
> reason we are reaching out  to understand  if and  how best to proceed?
>
> Those interested from IBM will be watching this list with keen interest ,
> but to get immediate feedback I would be grateful if you could copy  to
> philip.tet...@uk.ibm.com.
>
> Many thanks indeed and we look forward to hearing from you.
>
> Regards,
>
> *Dr Phil Tetlow **CEng*
> , *FIET*
> 
> CTO Data Ecosystems, Chief Architect Big Data & Information Management
> Products, Adjunt Professor of *Web Science*
> at *Southampton University*
> 
> (Specialist in *Sociotechnical Systems*
>  & Reflective
> Engineering)
> Executive IT Architect, *IBM Academy of Technology*
>  (Leadership Team) & *W3C*
>  Member
> IBM Analytics Division
> --
>  *Phone:* 44-7740-923328 | *Mobile:* 44-7740-923328 (262999)
> * E-mail:* *philip.tet...@uk.ibm.com* 
> * Find me on:* [image: LinkedIn:]
> *LinkedIn*
>  [image: Twitter:]
> *Twitter*   *
> More on: **Web Science * *| **Power
> Laws* | *Mements*
> * My Books :* [image: LinkedIn:]*The Web's Awake*
> 
>  [image:
> Twitter:]*Understanding Information and Computation*
> 
>  *Personal
> Web Site*
> [image: IBM]
>
> 1175 Century Way
> Leeds, LS15 8ZB
> United Kingdom
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: Code Freeze 3.2.4

[Jason Plurad]

I've published the 3.2.4 SNAPSHOT out to Sonatype. Also note, that I've
added my key to the KEYS file, so you those of you that want to validate
the distribution will need to import it again.

https://lists.apache.org/thread.html/909821a85693eaee31b4bf89f9881b
d619ca2ff5db677f79c1bbbec8@%3Cdev.tinkerpop.apache.org%3E


On Thu, Feb 2, 2017 at 9:51 AM, Stephen Mallette 
wrote:

> With the vote from kuppitz 548 is done. I just merged it.
>
> On Thu, Feb 2, 2017 at 7:57 AM, Stephen Mallette 
> wrote:
>
> > We still need one more vote to get https://github.com/apache/
> > tinkerpop/pull/548 - any committers free to handle that so we can get
> > that merged?
> >
> > On Mon, Jan 30, 2017 at 11:08 AM, Jason Plurad 
> wrote:
> >
> >> We are now under code freeze for 3.2.4.
> >>
> >> There are still 2 outstanding pull requests that should be included for
> >> 3.2.4.
> >> https://github.com/apache/tinkerpop/pull/548
> >> https://github.com/apache/tinkerpop/pull/551
> >>
> >> If there are others currently in the queue that are critical for
> release,
> >> please let us know. As mentioned last week, it has been several months
> >> since we've had a maintenance release and in particular, there is a
> Groovy
> >> bump for a security fix that we need to make available.
> >>
> >> I'll start getting SNAPSHOT artifact and docs published. I'll update
> this
> >> thread again when they are available.
> >>
> >> -- Jason
> >>
> >
> >
>

Architectural Best Practice for Graph Database Systems

[Phil Tetlow]

All, 

IBM  has been working on graph database for some time (culminating in 
JanusGraph, a new community project under The Linux Foundation ) and has 
amassed lots of best practice in this area. We have also looked at a 
number of technology stacks, but now are settling on Titan and Tinkerpop 
as our standard - thereby offering native integration as an integral 
feature of Janus.

Of late we have noticed a growing interest in graph database and we are 
excited about the level of contribution through mailing lists like this. 
That said, we have also noticed a distinct lack of shared insight into how 
best to design and build systems using graph, and we would be interested 
to hear if those on this list agree? We have been referring to this as 
“Architectural Best Practice for Graph Database Systems” and we already 
have a number of assets we would like to share openly. That presents a 
challenge, however, as currently we see no natural home for such materials 
within the open source community. Nevertheless, when we thought about it, 
the clear consensus pointed to some branch under Tinkerpop, and for this 
reason we are reaching out  to understand  if and  how best to proceed?

Those interested from IBM will be watching this list with keen interest , 
but to get immediate feedback I would be grateful if you could copy  to 
philip.tet...@uk.ibm.com. 

Many thanks indeed and we look forward to hearing from you.

Regards,

Dr Phil Tetlow CEng, FIET
CTO Data Ecosystems, Chief Architect Big Data & Information Management 
Products, Adjunt Professor of Web Scienceat Southampton University
(Specialist in Sociotechnical Systems & Reflective Engineering)
Executive IT Architect, IBM Academy of Technology (Leadership Team) & W3C 
Member
IBM Analytics Division


 Phone: 44-7740-923328 | Mobile: 44-7740-923328 (262999)
 E-mail: philip.tet...@uk.ibm.com
 Find me on: LinkedIn Twitter   More on: Web Science | Power Laws| Mements
 My Books : The Web's Awake Understanding Information and Computation 
Personal Web Site


1175 Century Way
Leeds, LS15 8ZB
United Kingdom



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[jira] [Commented] (TINKERPOP-1621) Remove deprecated GremlnPlugin and related infrastructure

[stephen mallette (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850036#comment-15850036
 ] 

stephen mallette commented on TINKERPOP-1621:
-

Implementing this as part of TINKERPOP-1612

> Remove deprecated GremlnPlugin and related infrastructure
> -
>
> Key: TINKERPOP-1621
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1621
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: groovy
>Affects Versions: 3.2.3
>Reporter: stephen mallette
>  Labels: breaking, deprecation
>
> {{GremlinPlugin}} in gremlin-groovy and related 
> infrastructure/implementations was deprecated in 3.2.4 and replaced by a 
> system in gremlin-core. With the removal of gremlin-groovy-test in 
> TINKERPOP-1612 a fair bit of related code had to be removed which makes the 
> old {{GremlinPlugin}} code even less useful and essentially dead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (TINKERPOP-1622) Remove deprecated G functions in gremlin-groovy

[stephen mallette (JIRA)]

stephen mallette created TINKERPOP-1622:
---

 Summary: Remove deprecated G functions in gremlin-groovy
 Key: TINKERPOP-1622
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1622
 Project: TinkerPop
  Issue Type: Improvement
  Components: groovy
Affects Versions: 3.2.3
Reporter: stephen mallette
Priority: Minor


All classes in {{org.apache.tinkerpop.gremlin.groovy.function}} were deprecated 
long ago in 3.1.0. The can be safely removed at this point.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (TINKERPOP-1622) Remove deprecated G functions in gremlin-groovy

[stephen mallette (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850035#comment-15850035
 ] 

stephen mallette commented on TINKERPOP-1622:
-

Implementing this as part of TINKERPOP-1612

> Remove deprecated G functions in gremlin-groovy
> ---
>
> Key: TINKERPOP-1622
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1622
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: groovy
>Affects Versions: 3.2.3
>Reporter: stephen mallette
>Priority: Minor
>  Labels: breaking, deprecated
>
> All classes in {{org.apache.tinkerpop.gremlin.groovy.function}} were 
> deprecated long ago in 3.1.0. The can be safely removed at this point.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Code Freeze 3.2.4

[Stephen Mallette]

With the vote from kuppitz 548 is done. I just merged it.

On Thu, Feb 2, 2017 at 7:57 AM, Stephen Mallette 
wrote:

> We still need one more vote to get https://github.com/apache/
> tinkerpop/pull/548 - any committers free to handle that so we can get
> that merged?
>
> On Mon, Jan 30, 2017 at 11:08 AM, Jason Plurad  wrote:
>
>> We are now under code freeze for 3.2.4.
>>
>> There are still 2 outstanding pull requests that should be included for
>> 3.2.4.
>> https://github.com/apache/tinkerpop/pull/548
>> https://github.com/apache/tinkerpop/pull/551
>>
>> If there are others currently in the queue that are critical for release,
>> please let us know. As mentioned last week, it has been several months
>> since we've had a maintenance release and in particular, there is a Groovy
>> bump for a security fix that we need to make available.
>>
>> I'll start getting SNAPSHOT artifact and docs published. I'll update this
>> thread again when they are available.
>>
>> -- Jason
>>
>
>

[jira] [Commented] (TINKERPOP-1589) Re-Introduce CloseableIterator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850003#comment-15850003
 ] 

ASF GitHub Bot commented on TINKERPOP-1589:
---

Github user asfgit closed the pull request at:

https://github.com/apache/tinkerpop/pull/548


> Re-Introduce CloseableIterator
> --
>
> Key: TINKERPOP-1589
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1589
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: structure
>Affects Versions: 3.2.3
>Reporter: stephen mallette
>Assignee: stephen mallette
> Fix For: 3.2.4
>
>
> In Blueprints, we used to have {{CloseableIterable}}, which allowed users to 
> release resources that might have been held by the underlying graph database. 
> We didn't port that interface to TinkerPop 3.x for some reason. Graphs like 
> OrientDB require a {{close()}} method for iterators returned from methods 
> like {{Graph.vertices()}} and {{Graph.edges()}}. In addition, allowing 
> {{close()}} from those iterators would make the {{close()}} on {{Traversal}} 
> (non-remote) more useful and meaningful (right now it is an empty method by 
> default). 
> In Blueprints, {{Graph.getVertices()}} and {{Graph.getEdges()}} returned a 
> {{Iterable}} and the Graph could choose to return {{CloseableIterator}}. 
> Users who wanted to write provider agnostic code would then have to do:
> {code}
> if (iterable instanceof CloseableIterable))
>   ((CloseableIterable) iterable).close();
> {code}
> I'm not sure why we didn't simply return {{CloseableIterable}} from those 
> methods. Any reason we shouldn't do that now? Perhaps the approach is to 
> introduce {{CloseableIterator}} to 3.2.4, but keep the return type of 
> {{edges/vertices()}} as {{Iterator}} and then for 3.3.0 change the return 
> type to {{CloseableIterator}}. In this way, the feature can be available in 
> next release of 3.2.4 without any API changes and users can do the 
> {{instanceof}} check above if they need to. Then for 3.3.0 when we allow API 
> changes we can just make it more convenient with a {{CloseableIterator}} 
> return type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] tinkerpop pull request #548: TINKERPOP-1589 Re-introduced CloseableIterator

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/tinkerpop/pull/548


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator

[dkuppitz]

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/548
  
VOTE: +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1589) Re-Introduce CloseableIterator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849966#comment-15849966
 ] 

ASF GitHub Bot commented on TINKERPOP-1589:
---

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/548
  
VOTE: +1


> Re-Introduce CloseableIterator
> --
>
> Key: TINKERPOP-1589
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1589
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: structure
>Affects Versions: 3.2.3
>Reporter: stephen mallette
>Assignee: stephen mallette
> Fix For: 3.2.4
>
>
> In Blueprints, we used to have {{CloseableIterable}}, which allowed users to 
> release resources that might have been held by the underlying graph database. 
> We didn't port that interface to TinkerPop 3.x for some reason. Graphs like 
> OrientDB require a {{close()}} method for iterators returned from methods 
> like {{Graph.vertices()}} and {{Graph.edges()}}. In addition, allowing 
> {{close()}} from those iterators would make the {{close()}} on {{Traversal}} 
> (non-remote) more useful and meaningful (right now it is an empty method by 
> default). 
> In Blueprints, {{Graph.getVertices()}} and {{Graph.getEdges()}} returned a 
> {{Iterable}} and the Graph could choose to return {{CloseableIterator}}. 
> Users who wanted to write provider agnostic code would then have to do:
> {code}
> if (iterable instanceof CloseableIterable))
>   ((CloseableIterable) iterable).close();
> {code}
> I'm not sure why we didn't simply return {{CloseableIterable}} from those 
> methods. Any reason we shouldn't do that now? Perhaps the approach is to 
> introduce {{CloseableIterator}} to 3.2.4, but keep the return type of 
> {{edges/vertices()}} as {{Iterator}} and then for 3.3.0 change the return 
> type to {{CloseableIterator}}. In this way, the feature can be available in 
> next release of 3.2.4 without any API changes and users can do the 
> {{instanceof}} check above if they need to. Then for 3.3.0 when we allow API 
> changes we can just make it more convenient with a {{CloseableIterator}} 
> return type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (TINKERPOP-1621) Remove deprecated GremlnPlugin and related infrastructure

[stephen mallette (JIRA)]

stephen mallette created TINKERPOP-1621:
---

 Summary: Remove deprecated GremlnPlugin and related infrastructure
 Key: TINKERPOP-1621
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1621
 Project: TinkerPop
  Issue Type: Improvement
  Components: groovy
Affects Versions: 3.2.3
Reporter: stephen mallette


{{GremlinPlugin}} in gremlin-groovy and related infrastructure/implementations 
was deprecated in 3.2.4 and replaced by a system in gremlin-core. With the 
removal of gremlin-groovy-test in TINKERPOP-1612 a fair bit of related code had 
to be removed which makes the old {{GremlinPlugin}} code even less useful and 
essentially dead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Code Freeze 3.2.4

[Stephen Mallette]

We still need one more vote to get
https://github.com/apache/tinkerpop/pull/548 - any committers free to
handle that so we can get that merged?

On Mon, Jan 30, 2017 at 11:08 AM, Jason Plurad  wrote:

> We are now under code freeze for 3.2.4.
>
> There are still 2 outstanding pull requests that should be included for
> 3.2.4.
> https://github.com/apache/tinkerpop/pull/548
> https://github.com/apache/tinkerpop/pull/551
>
> If there are others currently in the queue that are critical for release,
> please let us know. As mentioned last week, it has been several months
> since we've had a maintenance release and in particular, there is a Groovy
> bump for a security fix that we need to make available.
>
> I'll start getting SNAPSHOT artifact and docs published. I'll update this
> thread again when they are available.
>
> -- Jason
>

[jira] [Commented] (TINKERPOP-1271) SparkContext should be restarted if Killed and using Persistent Context

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849869#comment-15849869
 ] 

ASF GitHub Bot commented on TINKERPOP-1271:
---

GitHub user artem-aliev opened a pull request:

https://github.com/apache/tinkerpop/pull/555

TINKERPOP-1271: Refactor SparkContext creation and handling of sc.stop()

org.apache.tinkerpop.gremlin.spark.structure.Spark is a SparkContext 
holder for SparkGraphComputer.
It was refactored to detect external stop calls and recreate 
SparkContext in that case.
Context creation process was reordered to make all configuration 
options take effect.
Spark.create() methods return created context now.
The external stop also requires SPARK-18751 fix, that was integrated 
into Spark 2.1.
By the way the refactoring and configuration loading part gives effect 
on all versions.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/artem-aliev/tinkerpop TINKERPOP-1271

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/555.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #555


commit 6bea8a562a7d2d2a940a5cb7db3f2a4ce09f3dac
Author: artemaliev 
Date:   2017-02-02T12:15:04Z

TINKERPOP-1271: Refactor SparkContext creation and handling of external 
sc.stop()
org.apache.tinkerpop.gremlin.spark.structure.Spark is a SparkContext 
holder for SparkGraphComputer.
It was refactored do detect external stop calls and recreate 
SparkContext in that case.
Context creation process was reordered to make all configuration 
options to take effect.
Spark.create() methods return created context now.
The external stop also requires SPARK-18751 fix, that was integrated 
into Spark 2.1.
By the way the refactoring and configuration loading part gives effect 
on all versions.




> SparkContext should be restarted if Killed and using Persistent Context
> ---
>
> Key: TINKERPOP-1271
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1271
> Project: TinkerPop
>  Issue Type: Bug
>  Components: hadoop
>Affects Versions: 3.2.0-incubating, 3.1.2-incubating
>Reporter: Russell Spitzer
>
> If the persisted Spark Context is killed by the user via the Spark UI or is 
> terminated for some other error the Gremlin Console/Server is left with a 
> stopped Spark Context. This could be caught and the spark context recreated. 
> Oddly enough if you simply wait the context will "reset" itself or possible 
> get GC'd out of the system and everything works again. 
> ##Repo
> {code}
> gremlin> g.V().count()
> WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  
> - HADOOP_GREMLIN_LIBS is not set -- proceeding regardless
> ==>6
> gremlin> ERROR org.apache.spark.scheduler.cluster.SparkDeploySchedulerBackend 
>  - Application has been killed. Reason: Master removed our application: KILLED
> ERROR org.apache.spark.scheduler.TaskSchedulerImpl  - Lost executor 0 on 
> 10.150.0.180: Remote RPC client disassociated. Likely due to containers 
> exceeding thresholds, or network issues. Check driver logs for WARN messages.
> // Driver has been killed here via the Master UI
> gremlin> graph = GraphFactory.open('conf/hadoop/hadoop-gryo.properties')
> ==>hadoopgraph[gryoinputformat->gryooutputformat]
> gremlin> g.V().count()
> WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  
> - HADOOP_GREMLIN_LIBS is not set -- proceeding regardless
> java.lang.IllegalStateException: Cannot call methods on a stopped 
> SparkContext.
> This stopped SparkContext was created at:
> org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
> org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:53)
> org.apache.tinkerpop.gremlin.spark.structure.io.SparkContextStorage.open(SparkContextStorage.java:60)
> org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer.lambda$submitWithExecutor$1(SparkGraphComputer.java:122)
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> The currently active SparkContext was created at:
> org.apache.spark.SparkContext.getOrCreate(SparkContext.scala)
> org.apache.tinkerpop.gremlin.spark.structure.Spark.create(Spark.java:53)
> org.apache.tinkerpop.gremlin.spark.structure.io.SparkContextStorage.open(SparkContextStorage.jav

[GitHub] tinkerpop pull request #555: TINKERPOP-1271: Refactor SparkContext creation ...

[artem-aliev]

GitHub user artem-aliev opened a pull request:

https://github.com/apache/tinkerpop/pull/555

TINKERPOP-1271: Refactor SparkContext creation and handling of sc.stop()

org.apache.tinkerpop.gremlin.spark.structure.Spark is a SparkContext 
holder for SparkGraphComputer.
It was refactored to detect external stop calls and recreate 
SparkContext in that case.
Context creation process was reordered to make all configuration 
options take effect.
Spark.create() methods return created context now.
The external stop also requires SPARK-18751 fix, that was integrated 
into Spark 2.1.
By the way the refactoring and configuration loading part gives effect 
on all versions.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/artem-aliev/tinkerpop TINKERPOP-1271

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/555.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #555


commit 6bea8a562a7d2d2a940a5cb7db3f2a4ce09f3dac
Author: artemaliev 
Date:   2017-02-02T12:15:04Z

TINKERPOP-1271: Refactor SparkContext creation and handling of external 
sc.stop()
org.apache.tinkerpop.gremlin.spark.structure.Spark is a SparkContext 
holder for SparkGraphComputer.
It was refactored do detect external stop calls and recreate 
SparkContext in that case.
Context creation process was reordered to make all configuration 
options to take effect.
Spark.create() methods return created context now.
The external stop also requires SPARK-18751 fix, that was integrated 
into Spark 2.1.
By the way the refactoring and configuration loading part gives effect 
on all versions.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1589) Re-Introduce CloseableIterator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849797#comment-15849797
 ] 

ASF GitHub Bot commented on TINKERPOP-1589:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/548
  
All tests pass with `docker/build.sh -t -n -i`

VOTE +1


> Re-Introduce CloseableIterator
> --
>
> Key: TINKERPOP-1589
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1589
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: structure
>Affects Versions: 3.2.3
>Reporter: stephen mallette
>Assignee: stephen mallette
> Fix For: 3.2.4
>
>
> In Blueprints, we used to have {{CloseableIterable}}, which allowed users to 
> release resources that might have been held by the underlying graph database. 
> We didn't port that interface to TinkerPop 3.x for some reason. Graphs like 
> OrientDB require a {{close()}} method for iterators returned from methods 
> like {{Graph.vertices()}} and {{Graph.edges()}}. In addition, allowing 
> {{close()}} from those iterators would make the {{close()}} on {{Traversal}} 
> (non-remote) more useful and meaningful (right now it is an empty method by 
> default). 
> In Blueprints, {{Graph.getVertices()}} and {{Graph.getEdges()}} returned a 
> {{Iterable}} and the Graph could choose to return {{CloseableIterator}}. 
> Users who wanted to write provider agnostic code would then have to do:
> {code}
> if (iterable instanceof CloseableIterable))
>   ((CloseableIterable) iterable).close();
> {code}
> I'm not sure why we didn't simply return {{CloseableIterable}} from those 
> methods. Any reason we shouldn't do that now? Perhaps the approach is to 
> introduce {{CloseableIterator}} to 3.2.4, but keep the return type of 
> {{edges/vertices()}} as {{Iterator}} and then for 3.3.0 change the return 
> type to {{CloseableIterator}}. In this way, the feature can be available in 
> next release of 3.2.4 without any API changes and users can do the 
> {{instanceof}} check above if they need to. Then for 3.3.0 when we allow API 
> changes we can just make it more convenient with a {{CloseableIterator}} 
> return type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] tinkerpop issue #548: TINKERPOP-1589 Re-introduced CloseableIterator

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/548
  
All tests pass with `docker/build.sh -t -n -i`

VOTE +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---