[jira] [Created] (TINKERPOP-2173) Incorrect reset of log level in integration test
Divij Vaidya created TINKERPOP-2173: --- Summary: Incorrect reset of log level in integration test Key: TINKERPOP-2173 URL: https://issues.apache.org/jira/browse/TINKERPOP-2173 Project: TinkerPop Issue Type: Improvement Components: server Affects Versions: 3.4.0 Reporter: Divij Vaidya This issue is regarding GremlinServerIntegrateTest.java and other similar tests. In the @After method, the intention is to reset the log level which was temporarily changed in the @Before method for the duration of the test. However, instead of resetting to the original state, the code ends up setting the same log level again. Proposed fix: Do not mutate the previousLogLevel field in the @After methods [1][2] [1][https://github.com/apache/tinkerpop/blob/master/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java#L149] [2][https://github.com/apache/tinkerpop/blob/master/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java#L124] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TINKERPOP-2076) TinkerPop does not build with current (v11) Java version
[
https://issues.apache.org/jira/browse/TINKERPOP-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787203#comment-16787203
]
Robert Dale commented on TINKERPOP-2076:
Did you try the workaround in https://issues.apache.org/jira/browse/SPARK-24421
?
> TinkerPop does not build with current (v11) Java version
>
>
> Key: TINKERPOP-2076
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2076
> Project: TinkerPop
> Issue Type: Improvement
> Components: build-release
>Affects Versions: 3.3.4
> Environment: $ java --version
> java 11 2018-09-25
> Java(TM) SE Runtime Environment 18.9 (build 11+28)
> Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11+28, mixed mode)
>Reporter: Steve Strassmann
>Assignee: stephen mallette
>Priority: Major
>
> I cannot build TinkerPop with a current (v11) Java version.
> On the gremlin-users Google group, Robert Dale suggests using Java 8, but
> that is deprecated. Recommended: support current Java versions. Stephen
> Mallette says "we need to start worrying about such things."
> Oracle [says Java 8 is
> deprecated|https://www.oracle.com/technetwork/java/javase/overview/index.html]:
> {quote}{color:#d04437}[End of Public Updates for Oracle JDK
> 8|https://www.oracle.com/technetwork/java/javase/eol-135779.html]{color}
> Oracle will not post further updates of Java SE 8 to its public download
> sites for commercial use after January 2019.
> {quote}
>
> See issue posted in Gremlin-users Google Group:
>
> [https://groups.google.com/forum/?utm_medium=email_source=footer#!topic/gremlin-users/Kgnq4BkrZXQ]
> {{unable to build Tinkerpop from master with mvn clean install. }}
>
> The error appears to be
> An API incompatibility was encountered while executing
> org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce:
> java.lang.ExceptionInInitializerError: null
> {code:java}
> $ git clone https://github.com/apache/tinkerpop.git
> $ git checkout master
> $ mvn --version
> Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe;
> 2018-06-17T14:33:14-04:00)
> Maven home: /opt/maven
> Java version: 11, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk-11.jdk/Contents/Home
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"
> {code}
>
>
> {code:java}
> $ mvn clean install
> [INFO] Scanning for projects...
> [WARNING] The project org.apache.tinkerpop:tinkerpop:pom:3.4.0-SNAPSHOT uses
> prerequisites which is only intended for maven-plugin projects but not for
> non maven-plugin projects. For such purposes you should use the
> maven-enforcer-plugin. See
> https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html
> [INFO]
>
> [INFO] Reactor Build Order:
> [INFO]
> [INFO] Apache TinkerPop
> [pom]
> [INFO] Apache TinkerPop :: Gremlin Shaded
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Core
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Test
> [jar]
> [INFO] Apache TinkerPop :: TinkerGraph Gremlin
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Groovy
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Driver
> [jar]
> [INFO] Apache TinkerPop :: Neo4j Gremlin
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Server
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Javascript
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Python
> [jar]
> [INFO] Apache TinkerPop :: Gremlin.Net
> [pom]
> [INFO] Apache TinkerPop :: Gremlin.Net - Source
> [pom]
> [INFO] Apache TinkerPop :: Gremlin.Net - Tests
> [pom]
> [INFO] Apache TinkerPop :: Hadoop Gremlin
> [jar]
> [INFO] Apache TinkerPop :: Spark Gremlin
> [jar]
> [INFO] Apache TinkerPop :: SPARQL Gremlin
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Console
> [jar]
> [INFO] Apache TinkerPop :: Gremlin Archetype
> [pom]
> [INFO] Apache TinkerPop :: Archetype - TinkerGraph
> [jar]
> [INFO] Apache TinkerPop :: Archetype - Server
> [jar]
> [INFO] Apache TinkerPop :: Archetype - DSL
> [jar]
> [INFO] Apache TinkerPop
Re: [DISCUSS] Java 11
I think I'm blockaded on Java 11 at this point. I've explained the situation in the last comment on the JIRA here: https://issues.apache.org/jira/browse/TINKERPOP-2076?focusedCommentId=16787167=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16787167 but the short story is that Spark isn't working right on Java 11 partly because Hadoop isn't working right on Java 11. The problem is compounded by the fact that even when Spark is released Java 11 support will likely only land on Spark 3.0 and given that TP 3.3.x and 3.4.x are bound to 2.x and likely staying there we probably wouldn't try to get Java 11 going until TP 3.5.0. And that of course stinks because then we have different Java version requirements between branches - bo. I believe that I should rebase my branch to master at this time and just keep it up to date because I did good work on that and solved a lot of upgrade problems. If anyone has a different sense of what's going on with Spark and Java 11, please let us know. Also, if you think we should go with a different plan for Java 11 support other than what I've laid out, then let's please discuss. On Tue, Mar 5, 2019 at 1:44 PM Stephen Mallette wrote: > I've been working on getting TinkerPop building on Java 11. > > https://issues.apache.org/jira/browse/TINKERPOP-2076 > > I started the work on the tp33 branch and have hit the first problem I > where I had some doubts about how to proceed. gremlin-groovy wont' build > with the current version of gmaven-plus which needs groovy 2.5.3 minimum. > we had long ago decided against a version bump for groovy on tp33 and > decided to let it live on 2.4.x. So, if we want tp33 to build on Java 11 we > would need to change that thinking which would mean backporting several > commits from master to tp33 around groovy upgrades. Specifically we would > need: > > > https://github.com/apache/tinkerpop/commit/80242be387e6d3b4daa0a7b045d7a4f463123321 > (2.5.2) > > https://github.com/apache/tinkerpop/commit/759d1a724eef3f76b48508ba8c49dcb992eff28f > (2.5.3) > > From there it's an easy jump to what master is currently on at 2.5.6. As I > finish writing this email I'm not sure I really have a doubt anymoreI > think I need to move ahead with this change because without it we'll have > to switch Java versions every time we switch branches and that's not nice. > Based on that, I guess I'll just move forward upgrading Groovy to 2.5.6 on > tp33. Hope that makes sense to everyone. > > >
[jira] [Commented] (TINKERPOP-2076) TinkerPop does not build with current (v11) Java version
[
https://issues.apache.org/jira/browse/TINKERPOP-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787167#comment-16787167
]
stephen mallette commented on TINKERPOP-2076:
-
I fixed a few problems with Gremlin Server and OLAP integration tests. Somehow
Giraph tests passedthankfully. Spark however seems to be presenting as a
problem. Getting:
{code}
java.lang.NoSuchMethodError:
sun.nio.ch.DirectBuffer.cleaner()Lsun/misc/Cleaner;
at
org.apache.spark.storage.StorageUtils$.cleanDirectBuffer(StorageUtils.scala:293)
at
org.apache.spark.storage.StorageUtils$.dispose(StorageUtils.scala:288)
at
org.apache.spark.util.io.ChunkedByteBuffer$$anonfun$dispose$1.apply(ChunkedByteBuffer.scala:144)
at
org.apache.spark.util.io.ChunkedByteBuffer$$anonfun$dispose$1.apply(ChunkedByteBuffer.scala:144)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at
org.apache.spark.util.io.ChunkedByteBuffer.dispose(ChunkedByteBuffer.scala:144)
at
org.apache.spark.storage.ByteBufferBlockData.dispose(BlockManager.scala:98)
at
org.apache.spark.storage.BlockManager.releaseLockAndDispose(BlockManager.scala:1478)
at
org.apache.spark.storage.BlockManager$$anonfun$2.apply$mcV$sp(BlockManager.scala:535)
at
org.apache.spark.util.CompletionIterator$$anon$1.completion(CompletionIterator.scala:46)
at
org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:35)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1799)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:1158)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:1158)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2062)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2062)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:108)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
{code}
Which is unfortunate because it's only one test that's failing and that's:
{{GryoSerializerIntegrateTest.shouldHaveAllRegisteredGryoSerializerClasses}}.
And with that, we're stuck given:
https://issues.apache.org/jira/browse/SPARK-24417
I spoke with someone who is following what is happening in Spark with Java 11
and it sounds like park of the hangup is Hadoop/Java 11, so that's gotta get
settled first. Then at some point spark will consume that. The consumption
point will likely be Spark 3.0 not 2.x which is what each of our branches are
on now. And the time estimate is 6+ months before that spark version drops. Who
knows what Spark 3.0 upgrade will look like for TP and we have quite the big
question mark on exactly when we will be able to build on Java 11. I suspect
that we won't see it on TinkerPop 3.3.x or 3.4.x. I imagine it will be on 3.5.0
whenever we decide to start working that. I also imagine that we'll be stuck
with something I didn't want which is a {{master}} branch that builds with Java
11 and older maintenance branches building on Java 8 indefinitely. Sorta just
adds friction to development. bo
> TinkerPop does not build with current (v11) Java version
>
>
> Key: TINKERPOP-2076
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2076
> Project: TinkerPop
> Issue Type: Improvement
> Components: build-release
>Affects Versions: 3.3.4
> Environment: $ java --version
> java 11 2018-09-25
> Java(TM) SE Runtime Environment 18.9 (build 11+28)
> Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11+28, mixed mode)
>Reporter: Steve Strassmann
>Assignee: stephen mallette
>Priority: Major
>
> I cannot build TinkerPop with a current (v11) Java version.
> On the gremlin-users Google group, Robert Dale suggests using Java 8, but
> that is deprecated. Recommended: support current Java versions. Stephen
> Mallette says "we need to start worrying about such things."
> Oracle [says Java 8 is
> deprecated|https://www.oracle.com/technetwork/java/javase/overview/index.html]:
> {quote}{color:#d04437}[End of Public Updates for Oracle JDK
> 8|https://www.oracle.com/technetwork/java/javase/eol-135779.html]{color}
> Oracle
[GitHub] [tinkerpop] FlorianHockmann commented on issue #1077: TINKERPOP-2135/2148 Fix removal of closed connections and add round-robin scheduling
Just added tests for the collection type which required to add `InternalsVisibleTo` as I didn't want to make the collection `public`, but that was probably a good idea in general as we can now also test other internal classes in the future more easily. My VOTE is now again +1 (pending a successful Travis build, the tests worked locally) and the PR is ready for review again. The Cassandra C# driver was really helpful here. The copy-on-write collection and the round-robin scheduling work now basically the same way as there. [ Full content available at: https://github.com/apache/tinkerpop/pull/1077 ] This message was relayed via gitbox.apache.org for dev@tinkerpop.apache.org
[jira] [Commented] (TINKERPOP-2148) "no connection available!" is being thrown despite lots of free connections
[
https://issues.apache.org/jira/browse/TINKERPOP-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786921#comment-16786921
]
Florian Hockmann commented on TINKERPOP-2148:
-
OK, no problem. I now added a change for this [in the PR for
TINKERPOP-2135|https://github.com/apache/tinkerpop/pull/1077] as it was already
necessary to change the collection type for that. That PR adds a custom
copy-on-write collection that uses an array as the underlying collection type.
This made it possible to add round-robin scheduling instead of the current
approach of searching for the least used connection which easily produces the
exception you described here (if the {{MaxInProcessPerConnection}} limit is set
to a small value). So, the approach is pretty similar to what you suggested
here, just without modifying the collection all the time via enqueue/dequeue.
> "no connection available!" is being thrown despite lots of free connections
> ---
>
> Key: TINKERPOP-2148
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2148
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.4.0
> Environment: Windows 10, C#, .NET Core, CosmosDB Graph
>Reporter: Zaoshi
>Assignee: Florian Hockmann
>Priority: Minor
>
> I am submitting multiple graph queries in parallel but in some cases it
> starts throwing this exception:
> {{Gremlin.Net.Driver.Exceptions.NoConnectionAvailableException : no
> connection available!}}
> {{ at Gremlin.Net.Driver.ConnectionPool.GetAvailableConnectionAsync()}}
> {{ at Gremlin.Net.Driver.GremlinClient.SubmitAsync[T](RequestMessage
> requestMessage)}}
> When GremlinClient is initialized with \{PoolSize = 1,
> MaxInProcessPerConnection = 8} everything works fine, however, with
> \{PoolSize = 8, MaxInProcessPerConnection = 1} it starts throwing those
> exceptions even though this connection pool should have same throughput. I
> have tried to increase pool to \{PoolSize = 128, MaxInProcessPerConnection =
> 1} but GremlinClient started to fail even more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (TINKERPOP-2148) "no connection available!" is being thrown despite lots of free connections
[
https://issues.apache.org/jira/browse/TINKERPOP-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Florian Hockmann reassigned TINKERPOP-2148:
---
Assignee: Florian Hockmann
> "no connection available!" is being thrown despite lots of free connections
> ---
>
> Key: TINKERPOP-2148
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2148
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.4.0
> Environment: Windows 10, C#, .NET Core, CosmosDB Graph
>Reporter: Zaoshi
>Assignee: Florian Hockmann
>Priority: Minor
>
> I am submitting multiple graph queries in parallel but in some cases it
> starts throwing this exception:
> {{Gremlin.Net.Driver.Exceptions.NoConnectionAvailableException : no
> connection available!}}
> {{ at Gremlin.Net.Driver.ConnectionPool.GetAvailableConnectionAsync()}}
> {{ at Gremlin.Net.Driver.GremlinClient.SubmitAsync[T](RequestMessage
> requestMessage)}}
> When GremlinClient is initialized with \{PoolSize = 1,
> MaxInProcessPerConnection = 8} everything works fine, however, with
> \{PoolSize = 8, MaxInProcessPerConnection = 1} it starts throwing those
> exceptions even though this connection pool should have same throughput. I
> have tried to increase pool to \{PoolSize = 128, MaxInProcessPerConnection =
> 1} but GremlinClient started to fail even more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
