A little more detail about the failure mode:
I added a single node to the database, related it to itself
three ways, added it to the index.
committed the transaction - returned results from the commit
looked correct
Tried a get_properties() on the object returned from the
transaction commit
Just before the get_properties() failed, I printed that
object (Nigel's Neo4j node object), and got this result:
{"a":52,"c":3.14159,"b":2,"name":"Drone121"}
[I also printed the node _id value -- which was 0]
I updated the test in source control to add debug statements, and
run the test with debug on. If you run the Dockerfile, or just
wget using the URLs I supplied before, you'll have the newest
version of the test. Keep in mind that docker caches results, so
if you've cached the results of the Dockerfile, you'll have to
tweak it somehow (change the echo or something) to make sure it
grabs the latest source.
As a reminder, this works just fine with 2.0.x (where x >= 1)
versions of Neo4j. It only fails with 2.1.x versions of Neo4j.
The output with the current debug turned on looks like this:
Python 2.7.5
('COMMITTING THIS THING:', "{\n\tdb:
Resource(u'http://localhost:7474/db/data/'),\n\tclasses: {<class
'__main__.Drone'>: True},\n\tuniqueindexmap: {'Drone':
True},\n\tclasskeymap: {'Drone': {'index': 'Drone', 'vattr':
'name', 'key': 'Drone'}}\n\tbatchindex: None,\n\t clients:
\n[<__main__.Drone object at 0x27211d0>,\n\t newrels: \n[{'to':
<__main__.Drone object at 0x27211d0>, 'props': None, 'from':
<__main__.Drone object at 0x27211d0>, 'type': 'ISA'}, {'to':
<__main__.Drone object at 0x27211d0>, 'props': None, 'from':
<__main__.Drone object at 0x27211d0>, 'type': 'WASA'}, {'to':
<__main__.Drone object at 0x27211d0>, 'props': None, 'from':
<__main__.Drone object at 0x27211d0>, 'type': 'WILLBEA'},\n\t
deletions: \n\n\tweaknoderefs: {}\n\tstats: {'totaltime':
datetime.timedelta(0), 'index': 0, 'nodecreate': 0, 'separate': 0,
'relate': 0, 'lastcommit': None, 'nodedelete': 0, 'attrupdate':
0}\n\tbatch: None\n}")
Performing batch.create(0:
({"a":52,"c":3.14159,"b":2,"name":"Drone121"})) - for new node
Performing batch.create(0-[:ISA]->0): node relationships
ADDING rel 0-[:ISA]->0
Performing batch.create(0-[:WASA]->0): node relationships
ADDING rel 0-[:WASA]->0
Performing batch.create(0-[:WILLBEA]->0): node relationships
ADDING rel 0-[:WILLBEA]->0
add_to_index_or_fail: node 0; index Index(Node,
u'http://localhost:7474/db/data/index/node/Drone')("Drone","Drone121")
('Batch Updates constructed: Committing THIS THING:', "{\n\tdb:
Resource(u'http://localhost:7474/db/data/'),\n\tclasses: {<class
'__main__.Drone'>: True},\n\tuniqueindexmap: {'Drone':
True},\n\tclasskeymap: {'Drone': {'index': 'Drone', 'vattr':
'name', 'key': 'Drone'}}\n\tbatchindex: 4,\n\t clients:
\n[<__main__.Drone object at 0x27211d0>,\n\t newrels: \n[{'from':
<__main__.Drone object at 0x27211d0>, 'seqno': 1, 'abstract':
rel(0, 'ISA', 0), 'to': <__main__.Drone object at 0x27211d0>,
'props': None, 'type': 'ISA'}, {'from': <__main__.Drone object at
0x27211d0>, 'seqno': 2, 'abstract': rel(0, 'WASA', 0), 'to':
<__main__.Drone object at 0x27211d0>, 'props': None, 'type':
'WASA'}, {'from': <__main__.Drone object at 0x27211d0>, 'seqno': 3,
'abstract': rel(0, 'WILLBEA', 0), 'to': <__main__.Drone object at
0x27211d0>, 'props': None, 'type': 'WILLBEA'},\n\t deletions:
\n\n\tweaknoderefs: {}\n\tstats: {'totaltime':
datetime.timedelta(0), 'index': 1, 'nodecreate': 1, 'separate': 0,
'relate': 3, 'lastcommit': None, 'nodedelete': 0, 'attrupdate':
0}\n\tbatch: <py2neo.neo4j.WriteBatch object at 0x2721450>\n}")
SUBMITRESULT: (0 {"a":52,"c":3.14159,"b":2,"name":"Drone121"})
LOOKING at new node with batch index 0
NEW NODE looks like (0 {"a":52,"c":3.14159,"b":2,"name":"Drone121"})
SUBJ (our copy) looks like <__main__.Drone object at 0x27211d0>
Traceback (most recent call last):
File "store.py", line 1099, in <module>
testme()
File "store.py", line 1014, in testme
store.commit() # The updates have been captured...
File "store.py", line 923, in commit
% (newnode._id, str(newnode.get_properties())))
File
"/opt/rh/python27/root/usr/lib/python2.7/site-packages/py2neo/neo4j.py",
line 1398, in get_properties
self._properties = assembled(self._properties_resource._get())
or {}
File
"/opt/rh/python27/root/usr/lib/python2.7/site-packages/py2neo/neo4j.py",
line 367, in _get
raise ClientError(e)
py2neo.exceptions.ClientError: Cannot find node with id [0] in
database.
On 07/17/2014 03:47 PM, Nigel Small wrote:
Cool, thanks. I should get a chance to look into it over the next
couple of weeks.
Cheers
Nigel
On 17 July 2014 22:41, Alan Robertson <al...@unix.sh
<mailto:al...@unix.sh>> wrote:
And for good measure, if you use the Dockerfile quoted below
(and attached), this is a clean-room reproduction of the
problem in CentOS6 (with Python 2.7).
###############################################
FROM centos:latest
# Would prefer headless, but not available in CentOS6
RUN yum -y install wget java-1.7.0-openjdk scl-utils
redhat-lsb-core
RUN wget -qO- http://dev.centos.org/centos/6/SCL/scl.repo >>
/etc/yum.repos.d/centos.scl.repo
RUN yum -y install python27-python python27-python-setuptools
### Neo4j RPM
RUN wget http://debian.neo4j.org/neotechnology.gpg.key -O
/tmp/neo4j.key && rpm --import /tmp/neo4j.key && rm -f
/tmp/neo4j.key
RUN echo '[neo4j]' > /etc/yum.repos.d/neo4j.repo && echo
'name=Neo4j Yum Repo' >> /etc/yum.repos.d/neo4j.repo && echo
'baseurl=http://yum.neo4j.org' >> /etc/yum.repos.d/neo4j.repo
&& echo 'enabled=1' >> /etc/yum.repos.d/neo4j.repo && echo
'gpgcheck=1' >> /etc/yum.repos.d/neo4j.repo && yum -y install
neo4j
RUN scl enable python27 'easy_install pip'
RUN scl enable python27 'pip install py2neo'
RUN echo "Pulling Assimilation project source"
RUN mkdir /tmp/neotest && cd /tmp/neotest && wget -q
http://hg.linux-ha.org/assimilation/raw-file/tip/cma/store.py
&& wget -q
http://hg.linux-ha.org/assimilation/raw-file/tip/cma/assimevent.py
RUN NEO=neo4j; cd /tmp/neotest && /etc/init.d/${NEO} start &&
sleep 15 && scl enable python27 '/usr/bin/env python
--version; python store.py'
##################################################
Do this:
make a directory
cd into the directory you just made
copy the above text into a file in that directory - named
Dockerfile
docker.io <http://docker.io> build .
This is a completely clean-room demonstration of the problem
in CentOS6.
On 07/17/2014 08:16 AM, Alan Robertson wrote:
Hi Nigel,
Many thanks for your quick reply!
I can reproduce it with what I think is a smallish environment.
If you download my code from here:
http://hg.linux-ha.org/assimilation/archive/tip.tar.gz
Untar it, and go into the cma directory. Run the python
script *store.py*
It will fail. Each time you run it, it will give a higher
node in the graph that it cannot find.
If it doesn't fail, then we can try the docker thing to
reproduce the environment more exactly.
To try the docker version, you just need to install docker
and say "docker build ." (or docker.io <http://docker.io>)
with my Dockerfile in your directory. It will do the rest -
grab the OS, libraries, things we need, build my code and run
my tests. It just takes longer. For this environment, it
helps to understand docker when you want to debug it.
It's pretty awesome. But hopefully you won't need to do that.
I also have a more complex set of tests that only require
python that will likely fail if there's a Neo4j/Py2neo issue
that affects me. Maybe I should create a separate test
environment for that - that you could easily run.
On 07/15/2014 04:42 PM, Nigel Small wrote:
Familiar with the concept but have never actually used it.
On 15 July 2014 23:41, Alan Robertson <al...@unix.sh
<mailto:al...@unix.sh>> wrote:
Yes. There are lots of ways to do that. After all,
it's all open source :-D.
Are you familiar with docker?
On 07/15/2014 04:19 PM, Nigel Small wrote:
Can you share some code that I can run to recreate the
issue?
Cheers
Nigel
On 15 July 2014 23:09, Alan Robertson <al...@unix.sh
<mailto:al...@unix.sh>> wrote:
Neo4j package: neo4j-2.1.2-1.noarch
# pip list | grep py2neo
py2neo (1.6.4)
The context is this:
I just did a commit of a transaction which added
nodes to the database, and am now trying to
retrieve these same nodes by the ids for them which
were returned by the transaction.
tests.cma_test TestCMABasic.test_several_startups
... error: tests.cma_test
TestCMABasic.test_several_startups
Traceback (most recent call last):
File "./tests/cma_test.py", line 506, in
test_several_startups
CMAinit(io, cleanoutdb=True, debug=DEBUG)
File "./cmainit.py", line 75, in __init__
CMAdb.store.commit()
File "./store.py", line 912, in commit
for attr in newnode.get_properties():
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 1398, in get_properties
self._properties =
assembled(self._properties_resource._get()) or {}
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 367, in _get
raise ClientError(e)
ClientError: Cannot find node with id [0] in database.
ERROR in 2.49s
tests.cma_test TestCMABasic.test_startup ... error:
tests.cma_test TestCMABasic.test_startup
Traceback (most recent call last):
File "./tests/cma_test.py", line 433, in test_startup
CMAinit(io, cleanoutdb=True, debug=DEBUG)
File "./cmainit.py", line 75, in __init__
CMAdb.store.commit()
File "./store.py", line 912, in commit
for attr in newnode.get_properties():
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 1398, in get_properties
self._properties =
assembled(self._properties_resource._get()) or {}
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 367, in _get
raise ClientError(e)
ClientError: Cannot find node with id [1] in database.
--
Alan Robertson <al...@unix.sh>
<mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of
friendship... Let me claim from you at all times
your undisguised opinions." - William Wilberforce
--
Alan Robertson<al...@unix.sh> <mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim
from you at all times your undisguised opinions." - William Wilberforce
--
Alan Robertson<al...@unix.sh> <mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim from
you at all times your undisguised opinions." - William Wilberforce
--
You received this message because you are subscribed to the
Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
--
Alan Robertson<al...@unix.sh> <mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim from
you at all times your undisguised opinions." - William Wilberforce
--
You received this message because you are subscribed to the
Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
--
Alan Robertson<al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim from you
at all times your undisguised opinions." - William Wilberforce
--
You received this message because you are subscribed to the Google
Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
