[jira] [Closed] (TINKERPOP-2852) Update Maven plugin for docker-images building for M1 compatibility
[ https://issues.apache.org/jira/browse/TINKERPOP-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Xia closed TINKERPOP-2852. --- Fix Version/s: 3.7.0 3.6.3 3.5.6 Assignee: Yang Xia Resolution: Fixed > Update Maven plugin for docker-images building for M1 compatibility > --- > > Key: TINKERPOP-2852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2852 > Project: TinkerPop > Issue Type: Improvement > Components: build-release >Affects Versions: 3.5.4 >Reporter: Yang Xia >Assignee: Yang Xia >Priority: Critical > Fix For: 3.7.0, 3.6.3, 3.5.6 > > > The current Maven plug in we use for the `docker-image` building profile is > no longer maintained > ([https://github.com/spotify/dockerfile-maven)|https://github.com/spotify/dockerfile-maven).], > and it is also not compatible with M1 Macs. > We should consider swapping the plug in for an actively maintained one that > is M1 compatible, for example > [https://github.com/fabric8io/docker-maven-plugin/issues/1257.|https://github.com/fabric8io/docker-maven-plugin/issues/1257] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2852) Update Maven plugin for docker-images building for M1 compatibility
[ https://issues.apache.org/jira/browse/TINKERPOP-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716486#comment-17716486 ] ASF GitHub Bot commented on TINKERPOP-2852: --- xiazcy merged PR #2043: URL: https://github.com/apache/tinkerpop/pull/2043 > Update Maven plugin for docker-images building for M1 compatibility > --- > > Key: TINKERPOP-2852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2852 > Project: TinkerPop > Issue Type: Improvement > Components: build-release >Affects Versions: 3.5.4 >Reporter: Yang Xia >Priority: Critical > > The current Maven plug in we use for the `docker-image` building profile is > no longer maintained > ([https://github.com/spotify/dockerfile-maven)|https://github.com/spotify/dockerfile-maven).], > and it is also not compatible with M1 Macs. > We should consider swapping the plug in for an actively maintained one that > is M1 compatible, for example > [https://github.com/fabric8io/docker-maven-plugin/issues/1257.|https://github.com/fabric8io/docker-maven-plugin/issues/1257] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2852) Update Maven plugin for docker-images building for M1 compatibility
[ https://issues.apache.org/jira/browse/TINKERPOP-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716485#comment-17716485 ] ASF GitHub Bot commented on TINKERPOP-2852: --- xiazcy merged PR #2007: URL: https://github.com/apache/tinkerpop/pull/2007 > Update Maven plugin for docker-images building for M1 compatibility > --- > > Key: TINKERPOP-2852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2852 > Project: TinkerPop > Issue Type: Improvement > Components: build-release >Affects Versions: 3.5.4 >Reporter: Yang Xia >Priority: Critical > > The current Maven plug in we use for the `docker-image` building profile is > no longer maintained > ([https://github.com/spotify/dockerfile-maven)|https://github.com/spotify/dockerfile-maven).], > and it is also not compatible with M1 Macs. > We should consider swapping the plug in for an actively maintained one that > is M1 compatible, for example > [https://github.com/fabric8io/docker-maven-plugin/issues/1257.|https://github.com/fabric8io/docker-maven-plugin/issues/1257] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2852) Update Maven plugin for docker-images building for M1 compatibility
[ https://issues.apache.org/jira/browse/TINKERPOP-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716484#comment-17716484 ] ASF GitHub Bot commented on TINKERPOP-2852: --- xiazcy commented on PR #2007: URL: https://github.com/apache/tinkerpop/pull/2007#issuecomment-1522633044 VOTE +1 > Update Maven plugin for docker-images building for M1 compatibility > --- > > Key: TINKERPOP-2852 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2852 > Project: TinkerPop > Issue Type: Improvement > Components: build-release >Affects Versions: 3.5.4 >Reporter: Yang Xia >Priority: Critical > > The current Maven plug in we use for the `docker-image` building profile is > no longer maintained > ([https://github.com/spotify/dockerfile-maven)|https://github.com/spotify/dockerfile-maven).], > and it is also not compatible with M1 Macs. > We should consider swapping the plug in for an actively maintained one that > is M1 compatible, for example > [https://github.com/fabric8io/docker-maven-plugin/issues/1257.|https://github.com/fabric8io/docker-maven-plugin/issues/1257] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2901) Incorrect result caused by has(key, predicate)
[
https://issues.apache.org/jira/browse/TINKERPOP-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716454#comment-17716454
]
ASF GitHub Bot commented on TINKERPOP-2901:
---
xiazcy commented on PR #2017:
URL: https://github.com/apache/tinkerpop/pull/2017#issuecomment-1522431983
I think where to reduce the error may need a deeper thought/discussion. I'm
good with this change only fixing the error leak and explain how current
reduction works in documentation.
VOTE +1 pending doc updates.
> Incorrect result caused by has(key, predicate)
> --
>
> Key: TINKERPOP-2901
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2901
> Project: TinkerPop
> Issue Type: Bug
> Components: process
>Affects Versions: 3.5.5
>Reporter: Miracy Cavendish
>Priority: Critical
>
> Hi! I execute the following gremlin statements:
>
> {code:java}
> (1) g.V().where(__.values("PersonalId").is(within(4,8,7,10,9,3))).count()
> => 6
> (2)
> g.V().where(__.values("PersonalId").is(within(4,8,7,10,9,3))).has("prop4",
> gt(1)).count()
> => 0
> (3)
> g.V().where(__.values("PersonalId").is(within(4,8,7,10,9,3))).not(__.has("prop4",
> gt(1))).count()
> => 4
> # Query Partition => (1) should equals (2) + (3)
> (4)
> g.V().where(__.values("PersonalId").is(within(4,8,7,10,9,3))).values("prop4")
> => {-3.5550649939662193e+18, 1.1868212582852746e+18, NaN}
> (5) g.inject(-3.5550649939662193e+18, 1.1868212582852746e+18,
> Double.NaN).is(gt(1)).count()
> => 1{code}
> I believe there might be an issue with the has(key, predicate) method based
> on the results that I have obtained.
> However, I am having difficulty further isolating the bug. Would you be able
> to help me reproduce the issue and confirm whether there is indeed a bug, or
> if I may have misunderstood the has(key, predicate) method? Thank you.
> The graph is created as follows:
> {code:java}
> g.addV("Vlabel1").property("prop4",
> -2.2940462564994032e+17).property("prop30",
> "1VjYYDjUBPCI5pKd").property("prop2",
> 2.885236496196127e+18).property("prop27",
> -1.283089519858595e+18).property("prop28",
> -177934796716677957).property("prop11",
> Double.POSITIVE_INFINITY).property("prop7", Double.NaN).property("prop13",
> 1582080133).property("prop12", "ZfcpYP42sDhMoli").property("prop26",
> -369822694828473072).property("prop15", false).property("prop24",
> -2035481923).property("prop3", -8.101739153023826e+17).property("prop18",
> -8647655841519618606).property("prop17",
> Double.NEGATIVE_INFINITY).property("prop21", "9WXEq").property("prop5",
> 7015971082047412708).property("prop23", false).property("prop16",
> -2545300721734184124).property("prop8", false).property("prop9",
> Double.NEGATIVE_INFINITY).property("prop19", -1148359097).property("prop10",
> 707520215).property("prop29", 8230830081810437976).property("prop20",
> 3.8750319078997024e+17).property("prop14",
> 4460313502400034603).property("prop22", Double.NaN).property("PersonalId",
> 1)g.addV("Vlabel2").property("prop13", 844274909).property("prop14",
> -1075402369199667167).property("PersonalId",
> 2)g.addV("Vlabel3").property("prop7",
> 1.882109465594965e+18).property("prop15", false).property("prop4",
> -3.5550649939662193e+18).property("prop9",
> Double.POSITIVE_INFINITY).property("prop17",
> Double.NEGATIVE_INFINITY).property("prop24", 1755404150).property("prop21",
> "i4zl5FELCSxw977").property("prop13", 837413089).property("prop27",
> Double.NaN).property("prop12", "qihyjdv4ac").property("prop2",
> Double.NEGATIVE_INFINITY).property("prop22",
> Double.POSITIVE_INFINITY).property("prop11",
> -1.2786380272506816e+18).property("prop20",
> 1.69612491533634e+18).property("prop30",
> "IxJp6GCr5FRvbvpm6ua").property("prop29",
> -7960090835127371110).property("prop25", true).property("prop18",
> 1839549868490228748).property("prop16",
> -5686358023541700304).property("prop5",
> 5956272628376797503).property("prop19", -1735914445).property("prop23",
> true).property("prop1", Double.POSITIVE_INFINITY).property("prop3",
> -2.871901232555182e+16).property("prop26",
> 7437263094806337639).property("prop8", false).property("PersonalId",
> 3)g.addV("Vlabel4").property("prop3",
> 1.5543717370352646e+17).property("prop18",
> -7149746309261129994).property("prop24", -320785419).property("prop2",
> -1.3058283314253888e+18).property("prop1",
> -1.0673961612545854e+17).property("prop10", -601759324).property("prop16",
> -2436094159597901640).property("prop8", false).property("prop30",
> "gtrb5gw").property("prop26", 3672383367334424219).property("prop28",
> 478907755955172).property("prop12", "w6ursX930r2Y7r").property("prop27",
> 6.736691901333204e+17).property("prop6",
> -5129032340661124781).property("prop9",
> Double.POSITIVE_INFINITY).property("prop15",
[jira] [Closed] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Valentyn Kahamlyk closed TINKERPOP-2820.
Resolution: Fixed
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
> Fix For: 3.7.0, 3.6.3, 3.5.6
>
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Valentyn Kahamlyk updated TINKERPOP-2820:
-
Fix Version/s: 3.7.0
3.6.3
3.5.6
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
> Fix For: 3.7.0, 3.6.3, 3.5.6
>
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716429#comment-17716429
]
ASF GitHub Bot commented on TINKERPOP-2820:
---
vkagamlyk merged PR #2042:
URL: https://github.com/apache/tinkerpop/pull/2042
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716428#comment-17716428
]
ASF GitHub Bot commented on TINKERPOP-2820:
---
vkagamlyk commented on PR #2042:
URL: https://github.com/apache/tinkerpop/pull/2042#issuecomment-1522378147
VOTE +1
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716417#comment-17716417
]
ASF GitHub Bot commented on TINKERPOP-2820:
---
kenhuuu commented on code in PR #2042:
URL: https://github.com/apache/tinkerpop/pull/2042#discussion_r1176909962
##
gremlin-python/src/main/python/gremlin_python/process/graph_traversal.py:
##
@@ -1092,6 +1092,7 @@ def __verify_transaction_state(self, state,
error_message):
def __close_session(self, session):
self._session_based_connection.close()
Review Comment:
Would it make sense to put these two lines in the newly added
remove_session()?
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716416#comment-17716416
]
ASF GitHub Bot commented on TINKERPOP-2820:
---
kenhuuu commented on PR #2042:
URL: https://github.com/apache/tinkerpop/pull/2042#issuecomment-1522342732
VOTE +1
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716395#comment-17716395
]
ASF GitHub Bot commented on TINKERPOP-2820:
---
kenhuuu commented on code in PR #2042:
URL: https://github.com/apache/tinkerpop/pull/2042#discussion_r1176909962
##
gremlin-python/src/main/python/gremlin_python/process/graph_traversal.py:
##
@@ -1092,6 +1092,7 @@ def __verify_transaction_state(self, state,
error_message):
def __close_session(self, session):
self._session_based_connection.close()
Review Comment:
Would it make sense to put these two lines in the newly added
remove_session()?
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2820) gremlin-python _close_session race condition/FD leak
[
https://issues.apache.org/jira/browse/TINKERPOP-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716361#comment-17716361
]
ASF GitHub Bot commented on TINKERPOP-2820:
---
xiazcy commented on PR #2042:
URL: https://github.com/apache/tinkerpop/pull/2042#issuecomment-1522104447
LGTM VOTE + 1
> gremlin-python _close_session race condition/FD leak
>
>
> Key: TINKERPOP-2820
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2820
> Project: TinkerPop
> Issue Type: Bug
> Components: python
>Affects Versions: 3.6.1
>Reporter: Alex Hamilton
>Assignee: Valentyn Kahamlyk
>Priority: Critical
>
> There is a race condition in gremlin-python when closing session-based
> connections that results in leaking file descriptors for event loops -
> eventually leading to an `OSError [Errno 24] too many open files` error after
> enough transactions occur.
> The problem stems from a race condition when closing session based
> connections that causes the event loop opened for the session's connection to
> be left open.
> The problem is completely contained in these two methods from
> `gremlin_python.driver.client.py`
> ```py
> def close(self):
> # prevent the Client from being closed more than once. it raises errors
> if new jobby jobs
> # get submitted to the executor when it is shutdown
> if self._closed:
> return
> if self._session_enabled:
> self._close_session() # 1. (see below)
> log.info("Closing Client with url '%s'", self._url)
> while not self._pool.empty(): # 3. (see below)
> conn = self._pool.get(True)
> conn.close()
> self._executor.shutdown()
> self._closed = True
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> return conn.write(message).result() # 2. (see below)
> ```
> 1. `_close_session()` called
> 2. `.result()` waits for the _write_ to finish, but does *not* wait for the
> _read_ to finish. `conn` does not get put back into `self._pool` until AFTER
> the read finishes (`gremlin_python.driver.connection.Connection._receive()`).
> However, this method returns early and goes to 3.
> 3. this while loop is not entered to close out the connections. This leaves
> the conn's event loop running, never to be closed.
> I was able to solve this by modifying `_close_session` as follows:
> ```py
> def _close_session(self):
> message = request.RequestMessage(
> processor='session', op='close',
> args=\{'session': str(self._session)})
> conn = self._pool.get(True)
> try:
> write_result_set = conn.write(message).result()
> return write_result_set.all().result() # wait for _receive() to finish
> except protocol.GremlinServerError:
> pass
> ```
> I'm not sure if this is the correct solution, but wanted to point out the bug.
> In the meantime however, I wrote a context manager to handle this cleanup for
> me
> ```py
> @contextlib.contextmanager
> def transaction():
> tx = g.tx()
> gtx = tx.begin()
> try:
> yield tx, gtx
> tx.commit()
> except Exception as e:
> tx.rollback()
> finally:
> while not tx._session_based_connection._client._pool.empty():
> conn = tx._session_based_connection._client._pool.get(True)
> conn.close()
> logger.info("Closed abandoned session connection")
> with transaction() as (tx, gtx):
> foo = gtx.some_traversal().to_list()
> # do something with foo
> gtx.some_other_traversal().iterate()
> ```
> Cheers
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (TINKERPOP-2933) LazyBarrierStrategy affects the output order of query result
Lei Tang created TINKERPOP-2933:
---
Summary: LazyBarrierStrategy affects the output order of query
result
Key: TINKERPOP-2933
URL: https://issues.apache.org/jira/browse/TINKERPOP-2933
Project: TinkerPop
Issue Type: Bug
Reporter: Lei Tang
I find that when I remove `LazyBarrierStrategy`, sometimes the output order of
query result can be changed. I don't know whether it is a correct behavior,
but it sometimes can affect the query result, e.g., a `range()` or `limit()` in
a query.
For example, I can get the consistent result with the following queries.
{code:java}
gremlin> :>
g.withoutStrategies(LazyBarrierStrategy).V().out('el0','el1').has('vp1',
lt(0.30755568))
==>v[98]
==>v[14]
==>v[94]
gremlin> :> g.V().out('el0','el1').has('vp1', lt(0.30755568))
==>v[98]
==>v[14]
==>v[94]
{code}
However, I retrieved a different order of results with the following two
queries. I think they should also execute the same query results, as well as
the same order.
{code:java}
gremlin> :> g.V().out('el0','el1').not(__.has('vp1', lt(0.30755568)))
==>v[78]
==>v[35]
==>v[103]
==>v[103]
==>v[80]
==>v[80]
==>v[50]
gremlin> :>
g.withoutStrategies(LazyBarrierStrategy).V().out('el0','el1').not(__.has('vp1',
lt(0.30755568)))
==>v[78]
==>v[35]
==>v[103]
==>v[80]
==>v[103]
==>v[80]
==>v[50]
{code}
Thus, if I have some other operations after it, I will get different query
results.
{code:java}
gremlin> :>
g.withoutStrategies(LazyBarrierStrategy).V().out('el0','el1').not(__.has('vp1',
lt(0.30755568))).tail(3)
==>v[103]
==>v[80]
==>v[50]
gremlin> :> g.V().out('el0','el1').not(__.has('vp1', lt(0.30755568))).tail(3)
==>v[80]
==>v[80]
==>v[50]
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (TINKERPOP-2852) Update Maven plugin for docker-images building for M1 compatibility
[
https://issues.apache.org/jira/browse/TINKERPOP-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716133#comment-17716133
]
ASF GitHub Bot commented on TINKERPOP-2852:
---
FlorianHockmann commented on code in PR #2007:
URL: https://github.com/apache/tinkerpop/pull/2007#discussion_r1176113134
##
docs/src/upgrade/release-3.5.x.asciidoc:
##
@@ -30,24 +30,8 @@ complete list of all the modifications that are part of this
release.
=== Upgrading for Users
-=== Upgrading for Providers
-
- Local steps should handle non-Iterables
-
-`index` steps and local steps `count`, `dedup`, `max`, `mean`, `min`, `order`,
`range`, `sample`, `sum`, `tail` should
-work correctly with not only `Iterable` input, but also with arrays and single
values.
-
-Examples of queries that should be supported:
-
-[source,groovy]
-
-g.inject(1).max(local)
-
-
-[source,java]
-
-g.inject(new Integer[] {1,2},3).max(Scope.local).toList()
-
+Gremlin server docker image now supports both AMD64 and ARM64.
Multi-architecture image can now be found at
Review Comment:
Could you please add a heading for this?
> Update Maven plugin for docker-images building for M1 compatibility
> ---
>
> Key: TINKERPOP-2852
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2852
> Project: TinkerPop
> Issue Type: Improvement
> Components: build-release
>Affects Versions: 3.5.4
>Reporter: Yang Xia
>Priority: Critical
>
> The current Maven plug in we use for the `docker-image` building profile is
> no longer maintained
> ([https://github.com/spotify/dockerfile-maven)|https://github.com/spotify/dockerfile-maven).],
> and it is also not compatible with M1 Macs.
> We should consider swapping the plug in for an actively maintained one that
> is M1 compatible, for example
> [https://github.com/fabric8io/docker-maven-plugin/issues/1257.|https://github.com/fabric8io/docker-maven-plugin/issues/1257]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
