Indeed the implicit id can cause problems on any use case regarding
synchronization between neo4j and some other datasource (or replication
between two neo4j instances)

For example if i import data from an xml file, and then rerun the import, i
would expect the data in my neo4j instance to be overwritten/updated...

It would be really sweet it it would be possible to define this id before
the node is created, as a String for example. But i guess you guys use long
for a good reason.

-atle

On Tue, Jan 19, 2010 at 1:00 PM, Rick Bullotta <
rick.bullo...@burningskysoftware.com> wrote:

> There is really no "natural" way to express complex graphs (something more
> than hierarchal) in something like XML or JSON, but it can be done as long
> as each entity has a unique identification of some kind (e.g. GraphML's
> IDs).
>
> Barring any reason not to, it would seem that GraphML would be the most
> logical place to start.  It seems to recognize many of the potential
> complications(e.g. the parse "hints") and is extensible.
>
> One primary disconnect point is the dependency on "ids".  Neo nodes and
> relationships have an implicit ID (the long value representing the node or
> relationship), but may or may not have an explicit ID.  Thus, as mentioned
> previously, the identity may not be the same on import as it was an export,
> unless an explicit ID is provided for each node/relationship.  Our current
> graph model is a mix of both.  Some nodes have ID's (typically a name),
> others do not.
>
> Also, GraphML would need to be extended to support the concept of
> relationship types, but this seems to be fairly straightforward using
> custom
> attributes, elements and/or xlink.
>
>
>
> -----Original Message-----
> From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org]
> On
> Behalf Of Craig Taverner
> Sent: Tuesday, January 19, 2010 4:07 AM
> To: Neo user discussions
> Subject: Re: [Neo] Import/export
>
> I was wondering if the neo-shell or the neo4j.rb in IRB would solve this
> requirement (easily creating or loading some initial graph). I have not
> played much with the shell, but know that it has commands for making nodes
> and relationships. But I think it is best for interactive work, and I think
> it is not ideal for scripting. On the other hand the Ruby API provides a
> command-line/scripting DSL for generating a graph and since it is also very
> easy to read data from a file, it is easy to read the file and create the
> graph in a language not entirely unlike your original 'javascript-like'
> example.
>
> While on that note, I said javascript-like, but if we focus on the 'state
> transfer' example you gave, we're talking about JSON, and I think I might
> get a few votes for suggesting that as a nicer alternative to XML for a
> generic data structure.
>
> So, if you use JSON, the Java API and a JSON library would suffice to build
> the graph. If you are willing to deviate a little from the syntax, you
> could
> have the format in Ruby and directly executable in neo4j.rb (which I think
> is even cooler :-)
>
> I also personally think both XML and JSON represent implicit tree
> structures, and so any XML or JSON dataset can be loaded as a tree graph
> with generic code (and no need for hashes of node ids or any caches).
> However, things get slightly tricky when we need to translate XML/JSON
> closed graph contructs into the graph, but even that seems achievable (with
> hashes/caches ;-)
>
> On Tue, Jan 19, 2010 at 8:55 AM, David Montag <da...@montag.se> wrote:
>
> > Hi,
> >
> > Having read the replies and thought about it more, I think my initial
> > e-mail
> > had a slightly wrong focus. The technical details that have surfaced so
> far
> > are interesting, and would definitely be relevant, should an
> implementation
> > be attempted.
> >
> > However.
> >
> > What I personally would like to know is, do you think there's a need for
> > initial data sets in the first place? Because that is the problem that I
> > initially set out to solve. Then I kind of got ahead of myself and
> started
> > thinking about the hows, and not the whats and whys. Simply zipping up a
> > couple of pre-populated stores with different graphs would actually solve
> > the problem. Maybe not in the most elegant and/or maintainable way, but
> > still. Export/import is a much broader feature.
> >
> > Opinions? Don't get me wrong, I'm not trying to kill the tech discussion.
> > I'm just trying to solve the actual problem that I ran into. And if you
> > think export/import would be useful too, great! I'd be happy to continue
> > that discussion as well.
> >
> > Also, let me make it clear that this (i.e. initial data sets) isn't
> > something I'm doing as a project for myself. I would expect it to be a
> > community effort, benefiting everyone. So I actually *want* to know if
> you
> > like the ideas or not, in addition to solutions. With the awesomeness
> that
> > is the Neo4j community, it shouldn't be a problem. :)
> >
> > -David
> >
> > On Tue, Jan 19, 2010 at 2:32 AM, Rick Bullotta <
> > rick.bullo...@burningskysoftware.com> wrote:
> >
> > > Actually, I think there's one other key "gotcha" to be aware of.
> > >
> > > Rewiring relationships when importing should not assume anything about
> > the
> > > nodeID's.  While the nodeID's are a useful "unique identifier" in the
> > > export
> > > process, on import, you'd want to create a HashMap or similar structure
> > > that
> > > you populate with the "old" and "new" node ID's as you create them in
> the
> > > first pass through (nodes/properties), then use the "old" nodeIDs
> > > referenced
> > > in the exported relationships as your lookup to get the "new" nodeIDs.
> > >
> > > Could be kinda memory intensive for really large graphs (since you'd
> have
> > > to
> > > keep a HashMap entry of Long/Long for each node), but probably
> > manageable.
> > > In the worst case you could keep the translation table on disk and
> chunk
> > it
> > > in as needed.
> > >
> > > -----Original Message-----
> > > From: user-boun...@lists.neo4j.org [mailto:
> user-boun...@lists.neo4j.org]
> > > On
> > > Behalf Of Rob Challen
> > > Sent: Monday, January 18, 2010 6:25 PM
> > > To: Neo user discussions
> > > Subject: Re: [Neo] Import/export
> > >
> > > Rdf seems a good candidate to me.
> > >
> > > Having said that it might just be pretty easy to write out the graph
> > > in a spreadsheet (nodes and properties in one tab and relationship
> > > triples and properties in another) and import that, as long as you
> > > aren't fussed about maintaining data types.
> > >
> > > Rob.
> > >
> > > On 18/01/2010, Peter Neubauer <neubauer.pe...@gmail.com> wrote:
> > > > Hi David,
> > > > one thing would be to provide example node spaces, maybe even as
> > > > Amazon EC2 AMIs, or downloadable nodespaces.
> > > >
> > > > Regrading XML format, I think GraphML is the most standard thing
> > > > there, Gremlin already has a GraphML importer that can be used to
> > > > import data into Neo4j,
> > > >
> > >
> >
> http://wiki.github.com/tinkerpop/gremlin/graphml-reader-and-writer-library
> > > > . Probably not hard to write directly onto Neo4j.
> > > >
> > > > Anyone knowing about a good other binary format?
> > > >
> > > > WDYT?
> > > >
> > > > Cheers,
> > > >
> > > > /peter neubauer
> > > >
> > > > COO and Sales, Neo Technology
> > > >
> > > > GTalk:      neubauer.peter
> > > > Skype       peter.neubauer
> > > > Phone       +46 704 106975
> > > > LinkedIn   http://www.linkedin.com/in/neubauer
> > > > Twitter      http://twitter.com/peterneubauer
> > > >
> > > > http://www.neo4j.org                - Your high performance graph
> > > database.
> > > > http://gremlin.tinkerpop.com    - PageRank in 2 lines of code.
> > > >
> > > >
> > > >
> > > > On Mon, Jan 18, 2010 at 8:37 PM, David Montag <da...@montag.se>
> wrote:
> > > >> Hi,
> > > >>
> > > >> This weekend I was toying around with Neo4j. I wanted to do some
> > > indexing
> > > >> experiments. Unfortunately I found myself without a graph to work
> > with.
> > > >> Sure, I could write some code to generate a graph for me, but it'd
> be
> > a
> > > >> one-time-thing. I wanted to get going *now*. That got me thinking
> > about
> > > >> import/export functionality.
> > > >>
> > > >> I think a command-line import tool would be useful, accompanied by
> > (and
> > > >> built on) a Java API. Both of them would be tied to a certain
> > > >> representation
> > > >> format. The export can be represented in different ways, where two
> > > >> possible
> > > >> ways are:
> > > >> - State transfer: (node{id:1, name:foo}, node{id:2},
> > rel{start:1,end:2,
> > > >> type=bar}, ...)
> > > >> - Operation transfer: (id1 = create node, id2 = create node, create
> > rel
> > > >> id1->id2 type bar, ...)
> > > >>
> > > >> I guess the state transfer feels like the more straightforward one.
> > The
> > > >> diff-style nature of the operation transfer might be useful in other
> > > >> cases.
> > > >>
> > > >> When I first thought of this, the target user was somebody who
> wanted
> > to
> > > >> get
> > > >> started with a graph, and didn't want to write code to do an import
> > > >> "manually". Maybe the import/export can extend to other use cases,
> but
> > > >> this
> > > >> was the primary one. A possible workflow could be db exported to
> file,
> > > >> file
> > > >> published, file downloaded, file imported into db.
> > > >>
> > > >> In the end, it would be great if new users could download sample
> data
> > > sets
> > > >> and import them into a Neo4j instance without writing a single line
> of
> > > >> code.
> > > >> Which also gets me thinking about a command-line tool to create an
> > empty
> > > >> Neo4j instance to import into. The actual implementations of the
> tools
> > > are
> > > >> trivial. It's the discussion that leads to the implementation that's
> > > >> important.
> > > >>
> > > >> Does this sound like anything that would interest people? If so,
> > > (digging
> > > >> into details) what kind of representation do you guys think would be
> > > best?
> > > >> I
> > > >> was thinking XML, but a binary format might be better for
> performance
> > > >> (size/primitives ratio). Maybe both? Because I do like the idea of a
> > > >> human-readable (and editable) format. If you don't think it would be
> > > >> useful
> > > >> I would love to hear why.
> > > >>
> > > >> This is just a brain dump of my thoughts. Surely others have thought
> > of
> > > >> this
> > > >> as well. I'm just getting the discussion started. WDYT?
> > > >>
> > > >> -David
> > > >> _______________________________________________
> > > >> Neo mailing list
> > > >> User@lists.neo4j.org
> > > >> https://lists.neo4j.org/mailman/listinfo/user
> > > >>
> > > > _______________________________________________
> > > > Neo mailing list
> > > > User@lists.neo4j.org
> > > > https://lists.neo4j.org/mailman/listinfo/user
> > > >
> > >
> > > --
> > > Sent from my mobile device
> > > _______________________________________________
> > > Neo mailing list
> > > User@lists.neo4j.org
> > > https://lists.neo4j.org/mailman/listinfo/user
> > >
> > > _______________________________________________
> > > Neo mailing list
> > > User@lists.neo4j.org
> > > https://lists.neo4j.org/mailman/listinfo/user
> > >
> > _______________________________________________
> > Neo mailing list
> > User@lists.neo4j.org
> > https://lists.neo4j.org/mailman/listinfo/user
> >
> _______________________________________________
> Neo mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
> _______________________________________________
> Neo mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
_______________________________________________
Neo mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user

Reply via email to