Since the result said "cannot find node" the question is: Was that a
correct result? So, I used neo4j-shell to look at the database after
the failure - and it looked like the updates were not done. There is no
node zero (as indicated by the REST return), nor (for good measure) a
node 1 or 2. So the result was correct for the current database. The
database was wrong, but that result was correct (given the incorrect
database).
We appear to have gotten a successful return from the REST update
transaction, but in spite of that, the node I created wasn't there.
I will look further into what we got back from the REST transaction --
but right now, *this is looking like a Neo4j 2.1 bug*. I use Nigel's
latest version quite successfully with 2.0.1. I know you guys are at
OSCON. If you read this and can tell me what you want to see to
diagnose it, that would be awesome.
On 07/19/2014 12:09 PM, Alan Robertson wrote:
A little more detail about the failure mode:
I added a single node to the database, related it to itself three
ways, added it to the index.
committed the transaction - returned results from the commit
looked correct
Tried a get_properties() on the object returned from the
transaction commit
Just before the get_properties() failed, I printed that object
(Nigel's Neo4j node object), and got this result:
{"a":52,"c":3.14159,"b":2,"name":"Drone121"}
[I also printed the node _id value -- which was 0]
I updated the test in source control to add debug statements, and run
the test with debug on. If you run the Dockerfile, or just wget using
the URLs I supplied before, you'll have the newest version of the
test. Keep in mind that docker caches results, so if you've cached
the results of the Dockerfile, you'll have to tweak it somehow (change
the echo or something) to make sure it grabs the latest source.
As a reminder, this works just fine with 2.0.x (where x >= 1) versions
of Neo4j. It only fails with 2.1.x versions of Neo4j.
The output with the current debug turned on looks like this:
Python 2.7.5
('COMMITTING THIS THING:', "{\n\tdb:
Resource(u'http://localhost:7474/db/data/'),\n\tclasses: {<class
'__main__.Drone'>: True},\n\tuniqueindexmap: {'Drone':
True},\n\tclasskeymap: {'Drone': {'index': 'Drone', 'vattr': 'name',
'key': 'Drone'}}\n\tbatchindex: None,\n\t clients: \n[<__main__.Drone
object at 0x27211d0>,\n\t newrels: \n[{'to': <__main__.Drone object at
0x27211d0>, 'props': None, 'from': <__main__.Drone object at
0x27211d0>, 'type': 'ISA'}, {'to': <__main__.Drone object at
0x27211d0>, 'props': None, 'from': <__main__.Drone object at
0x27211d0>, 'type': 'WASA'}, {'to': <__main__.Drone object at
0x27211d0>, 'props': None, 'from': <__main__.Drone object at
0x27211d0>, 'type': 'WILLBEA'},\n\t deletions: \n\n\tweaknoderefs:
{}\n\tstats: {'totaltime': datetime.timedelta(0), 'index': 0,
'nodecreate': 0, 'separate': 0, 'relate': 0, 'lastcommit': None,
'nodedelete': 0, 'attrupdate': 0}\n\tbatch: None\n}")
Performing batch.create(0:
({"a":52,"c":3.14159,"b":2,"name":"Drone121"})) - for new node
Performing batch.create(0-[:ISA]->0): node relationships
ADDING rel 0-[:ISA]->0
Performing batch.create(0-[:WASA]->0): node relationships
ADDING rel 0-[:WASA]->0
Performing batch.create(0-[:WILLBEA]->0): node relationships
ADDING rel 0-[:WILLBEA]->0
add_to_index_or_fail: node 0; index Index(Node,
u'http://localhost:7474/db/data/index/node/Drone')("Drone","Drone121")
('Batch Updates constructed: Committing THIS THING:', "{\n\tdb:
Resource(u'http://localhost:7474/db/data/'),\n\tclasses: {<class
'__main__.Drone'>: True},\n\tuniqueindexmap: {'Drone':
True},\n\tclasskeymap: {'Drone': {'index': 'Drone', 'vattr': 'name',
'key': 'Drone'}}\n\tbatchindex: 4,\n\t clients: \n[<__main__.Drone
object at 0x27211d0>,\n\t newrels: \n[{'from': <__main__.Drone object
at 0x27211d0>, 'seqno': 1, 'abstract': rel(0, 'ISA', 0), 'to':
<__main__.Drone object at 0x27211d0>, 'props': None, 'type': 'ISA'},
{'from': <__main__.Drone object at 0x27211d0>, 'seqno': 2, 'abstract':
rel(0, 'WASA', 0), 'to': <__main__.Drone object at 0x27211d0>,
'props': None, 'type': 'WASA'}, {'from': <__main__.Drone object at
0x27211d0>, 'seqno': 3, 'abstract': rel(0, 'WILLBEA', 0), 'to':
<__main__.Drone object at 0x27211d0>, 'props': None, 'type':
'WILLBEA'},\n\t deletions: \n\n\tweaknoderefs: {}\n\tstats:
{'totaltime': datetime.timedelta(0), 'index': 1, 'nodecreate': 1,
'separate': 0, 'relate': 3, 'lastcommit': None, 'nodedelete': 0,
'attrupdate': 0}\n\tbatch: <py2neo.neo4j.WriteBatch object at
0x2721450>\n}")
SUBMITRESULT: (0 {"a":52,"c":3.14159,"b":2,"name":"Drone121"})
LOOKING at new node with batch index 0
NEW NODE looks like (0 {"a":52,"c":3.14159,"b":2,"name":"Drone121"})
SUBJ (our copy) looks like <__main__.Drone object at 0x27211d0>
Traceback (most recent call last):
File "store.py", line 1099, in <module>
testme()
File "store.py", line 1014, in testme
store.commit() # The updates have been captured...
File "store.py", line 923, in commit
% (newnode._id, str(newnode.get_properties())))
File
"/opt/rh/python27/root/usr/lib/python2.7/site-packages/py2neo/neo4j.py",
line 1398, in get_properties
self._properties = assembled(self._properties_resource._get()) or {}
File
"/opt/rh/python27/root/usr/lib/python2.7/site-packages/py2neo/neo4j.py",
line 367, in _get
raise ClientError(e)
py2neo.exceptions.ClientError: Cannot find node with id [0] in database.
On 07/17/2014 03:47 PM, Nigel Small wrote:
Cool, thanks. I should get a chance to look into it over the next
couple of weeks.
Cheers
Nigel
On 17 July 2014 22:41, Alan Robertson <al...@unix.sh
<mailto:al...@unix.sh>> wrote:
And for good measure, if you use the Dockerfile quoted below (and
attached), this is a clean-room reproduction of the problem in
CentOS6 (with Python 2.7).
###############################################
FROM centos:latest
# Would prefer headless, but not available in CentOS6
RUN yum -y install wget java-1.7.0-openjdk scl-utils redhat-lsb-core
RUN wget -qO- http://dev.centos.org/centos/6/SCL/scl.repo >>
/etc/yum.repos.d/centos.scl.repo
RUN yum -y install python27-python python27-python-setuptools
### Neo4j RPM
RUN wget http://debian.neo4j.org/neotechnology.gpg.key -O
/tmp/neo4j.key && rpm --import /tmp/neo4j.key && rm -f
/tmp/neo4j.key
RUN echo '[neo4j]' > /etc/yum.repos.d/neo4j.repo && echo
'name=Neo4j Yum Repo' >> /etc/yum.repos.d/neo4j.repo && echo
'baseurl=http://yum.neo4j.org' >> /etc/yum.repos.d/neo4j.repo &&
echo 'enabled=1' >> /etc/yum.repos.d/neo4j.repo && echo
'gpgcheck=1' >> /etc/yum.repos.d/neo4j.repo && yum -y install neo4j
RUN scl enable python27 'easy_install pip'
RUN scl enable python27 'pip install py2neo'
RUN echo "Pulling Assimilation project source"
RUN mkdir /tmp/neotest && cd /tmp/neotest && wget -q
http://hg.linux-ha.org/assimilation/raw-file/tip/cma/store.py &&
wget -q
http://hg.linux-ha.org/assimilation/raw-file/tip/cma/assimevent.py
RUN NEO=neo4j; cd /tmp/neotest && /etc/init.d/${NEO} start &&
sleep 15 && scl enable python27 '/usr/bin/env python --version;
python store.py'
##################################################
Do this:
make a directory
cd into the directory you just made
copy the above text into a file in that directory - named
Dockerfile
docker.io <http://docker.io> build .
This is a completely clean-room demonstration of the problem in
CentOS6.
On 07/17/2014 08:16 AM, Alan Robertson wrote:
Hi Nigel,
Many thanks for your quick reply!
I can reproduce it with what I think is a smallish environment.
If you download my code from here:
http://hg.linux-ha.org/assimilation/archive/tip.tar.gz
Untar it, and go into the cma directory. Run the python script
*store.py*
It will fail. Each time you run it, it will give a higher node
in the graph that it cannot find.
If it doesn't fail, then we can try the docker thing to
reproduce the environment more exactly.
To try the docker version, you just need to install docker and
say "docker build ." (or docker.io <http://docker.io>) with my
Dockerfile in your directory. It will do the rest - grab the
OS, libraries, things we need, build my code and run my tests.
It just takes longer. For this environment, it helps to
understand docker when you want to debug it.
It's pretty awesome. But hopefully you won't need to do that.
I also have a more complex set of tests that only require python
that will likely fail if there's a Neo4j/Py2neo issue that
affects me. Maybe I should create a separate test environment
for that - that you could easily run.
On 07/15/2014 04:42 PM, Nigel Small wrote:
Familiar with the concept but have never actually used it.
On 15 July 2014 23:41, Alan Robertson <al...@unix.sh
<mailto:al...@unix.sh>> wrote:
Yes. There are lots of ways to do that. After all, it's
all open source :-D.
Are you familiar with docker?
On 07/15/2014 04:19 PM, Nigel Small wrote:
Can you share some code that I can run to recreate the issue?
Cheers
Nigel
On 15 July 2014 23:09, Alan Robertson <al...@unix.sh
<mailto:al...@unix.sh>> wrote:
Neo4j package: neo4j-2.1.2-1.noarch
# pip list | grep py2neo
py2neo (1.6.4)
The context is this:
I just did a commit of a transaction which added nodes
to the database, and am now trying to retrieve these
same nodes by the ids for them which were returned by
the transaction.
tests.cma_test TestCMABasic.test_several_startups ...
error: tests.cma_test TestCMABasic.test_several_startups
Traceback (most recent call last):
File "./tests/cma_test.py", line 506, in
test_several_startups
CMAinit(io, cleanoutdb=True, debug=DEBUG)
File "./cmainit.py", line 75, in __init__
CMAdb.store.commit()
File "./store.py", line 912, in commit
for attr in newnode.get_properties():
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 1398, in get_properties
self._properties =
assembled(self._properties_resource._get()) or {}
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 367, in _get
raise ClientError(e)
ClientError: Cannot find node with id [0] in database.
ERROR in 2.49s
tests.cma_test TestCMABasic.test_startup ... error:
tests.cma_test TestCMABasic.test_startup
Traceback (most recent call last):
File "./tests/cma_test.py", line 433, in test_startup
CMAinit(io, cleanoutdb=True, debug=DEBUG)
File "./cmainit.py", line 75, in __init__
CMAdb.store.commit()
File "./store.py", line 912, in commit
for attr in newnode.get_properties():
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 1398, in get_properties
self._properties =
assembled(self._properties_resource._get()) or {}
File
"/opt/rh/python27/root/usr/lib64/python2.7/site-packages/py2neo/neo4j.py",
line 367, in _get
raise ClientError(e)
ClientError: Cannot find node with id [1] in database.
--
Alan Robertson <al...@unix.sh>
<mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of
friendship... Let me claim from you at all times your
undisguised opinions." - William Wilberforce
--
Alan Robertson<al...@unix.sh> <mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim
from you at all times your undisguised opinions." - William Wilberforce
--
Alan Robertson<al...@unix.sh> <mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim from
you at all times your undisguised opinions." - William Wilberforce
--
You received this message because you are subscribed to the
Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
--
Alan Robertson<al...@unix.sh> <mailto:al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim from
you at all times your undisguised opinions." - William Wilberforce
--
You received this message because you are subscribed to the
Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
--
Alan Robertson<al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim from you
at all times your undisguised opinions." - William Wilberforce
--
You received this message because you are subscribed to the Google
Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to neo4j+unsubscr...@googlegroups.com
<mailto:neo4j+unsubscr...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
--
Alan Robertson <al...@unix.sh> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim from you
at all times your undisguised opinions." - William Wilberforce
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to neo4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
