Re: [Neo4j] Suggestions on how to order relationships?
how about "weight"? On Thu, Jan 19, 2012 at 6:23 PM, Aseem Kishore wrote: > Hi there, > > Take a Facebook-like example, where users can "like" different > movies/music/celebs/etc. Suppose we want to let users drag-and-drop these > movies/etc. on their profile pages, to let them e.g. show their favorite > movies first. > > How would you guys recommend achieving that in Neo4j? If possible, I'd like > to avoid creating a node for every relationship (redundancy/overhead as we > do this more and more). > > I can't think of anything better than putting properties on the > relationship. Maybe index numbers (e.g. rel D might have "index: 0", rel B > might have "index: 1", etc.), but that's essentially reordering an array, > which sucks. The other option that thus came to mind was to mimic a linked > list: have an "after" property that contains the ID of the relationship > this one comes after (and/or maybe a "before" property instead or in > addition). > > Just wondering if there are better ideas! Thanks. =) > > Aseem > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register > and consider posting at https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user -- Best wishes, Linan Wang Architect, Programmer, PhD ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Using in-memory DB for unit tests
Why call tx.failure in close db?
On Wednesday, November 30, 2011, Jon Walton wrote:
> Greetings - I have started my first project using neo4j, and am having
some
> trouble with unit tests. I must be missing something, but when I
retrieve
> a node using an incoming direction, I am unable to get properties from
that
> node.The first test works fine, the second one fails with a
> NotFoundException on the property "asn". (although it does find the
> end-node with no trouble) Can anyone point out what I am doing wrong?
>
> Thanks,
>
> Jon
>
> /**
> * $Id$
> */
> package com.mcafee.tsw.webgraph;
>
> import org.junit.After;
> import org.junit.Before;
> import org.junit.Test;
> import org.neo4j.graphdb.Direction;
> import org.neo4j.graphdb.GraphDatabaseService;
> import org.neo4j.graphdb.Node;
> import org.neo4j.graphdb.RelationshipType;
> import org.neo4j.graphdb.Transaction;
> import org.neo4j.test.ImpermanentGraphDatabase;
>
> /**
> */
> public class TestMe {
>
>private GraphDatabaseService svc;
>private Transaction tx;
>
>/**
> * @throws java.lang.Exception
> */
>@Before
>public void initDB() throws Exception {
>svc = new ImpermanentGraphDatabase();
>tx = svc.beginTx();
>}
>
>/**
> * @throws java.lang.Exception
> */
>@After
>public void closeDB() throws Exception {
>tx.failure();
>tx.finish();
>svc.shutdown();
>svc = null;
>}
>
>// This one works.
>@Test
>public void testOutgoing() {
>
>Node network = svc.createNode();
>Node as = svc.createNode();
>
>as.setProperty("asn", "123");
>network.createRelationshipTo(as, IPAddressRelationshipType.AS);
>
>
>
System.out.println(network.getSingleRelationship(IPAddressRelationshipType.AS,
> Direction.OUTGOING)
>.getEndNode().getProperty("asn"));
>
>}
>
>// This one results in a not found exception on the 'asn' property.
>@Test
>public void testIncoming() {
>
>Node network = svc.createNode();
>Node as = svc.createNode();
>as.setProperty("asn", "123");
>
>as.createRelationshipTo(network, IPAddressRelationshipType.AS);
>
>
>
System.out.println(network.getSingleRelationship(IPAddressRelationshipType.AS,
> Direction.INCOMING)
>.getEndNode().getProperty("asn"));
>
>}
>
> }
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best wishes,
Linan Wang
Architect, Programmer, PhD
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to boost performance?
ry key. Please find the DDL for the two
>>> >> tables.
>>> >>>>>
>>> >>>>> CREATE TABLE `pci`.`cells` (
>>> >>>>> `id` varchar(32) collate utf8_bin NOT NULL,
>>> >>>>> `x_pos` double default NULL,
>>> >>>>> `y_pos` double default NULL,
>>> >>>>> `pci` smallint(6) default '0',
>>> >>>>> PRIMARY KEY (`id`)
>>> >>>>> )
>>> >>>>>
>>> >>>>> CREATE TABLE `pci`.`relations` (
>>> >>>>> `id` int(11) NOT NULL auto_increment,
>>> >>>>> `source` varchar(32) collate utf8_bin default NULL,
>>> >>>>> `target` varchar(32) collate utf8_bin default NULL,
>>> >>>>> PRIMARY KEY (`id`),
>>> >>>>> KEY `src_idx` (`source`),
>>> >>>>> KEY `src_target` (`target`)
>>> >>>>> )
>>> >>>>>
>>> >>>>> So as you can see, a simple secondary table contains the
>>> relationship
>>> >>>> with
>>> >>>>> source and targets pointing to the cells table.
>>> >>>>>
>>> >>>>> I've loaded this exact same DB into a neoserver running on the same
>>> >>>>> machine: A Blade with 26 cpus (6 cores each) and 16gb RAM.
>>> >>>>>
>>> >>>>> One of the requirements we have is to find all associations of my
>>> >>>>> associations. Something that in neo I did like this:
>>> >>>>>
>>> >>>>> START n = node(3)
>>> >>>>> MATCH n-->()-->(x)
>>> >>>>> return x
>>> >>>>>
>>> >>>>> For this specific node it returns 6475 nodes.
>>> >>>>>
>>> >>>>> I have tested this before using Hibernate in two modes: without a L2
>>> >>>> cache,
>>> >>>>> and with an L2 Cache (Ehcache standalone no replication).
>>> >>>>> Here's a snippet of the code that loads it, so you can understand
>>> >> what's
>>> >>>>> going under the hood:
>>> >>>>>
>>> >>>>>
>>> >>>>> @Override
>>> >>>>> public List loadCellWithRealtions(String... ids) {
>>> >>>>> Session session = (Session) em.getDelegate();
>>> >>>>> Criteria c = session.createCriteria(Cell.class)
>>> >>>>> .setFetchMode("incomingRelations", FetchMode.SELECT)
>>> >>>>> .setFetchMode("outgoingRelations", FetchMode.SELECT)
>>> >>>>> .add(Restrictions.in("id", Arrays.asList(ids)));
>>> >>>>> List results = c.list();
>>> >>>>> for(Cell cell : results){
>>> >>>>> Hibernate.initialize(cell.getIncomingRelations());
>>> >>>>> Hibernate.initialize(cell.getOutgoingRelations());
>>> >>>>> }
>>> >>>>> return results;
>>> >>>>> }
>>> >>>>>
>>> >>>>> @Override
>>> >>>>> public List loadCellWithNeighbourRelations(String... ids) {
>>> >>>>> List cells = loadCellWithRealtions(ids);
>>> >>>>> for(Cell c : cells){
>>> >>>>> for(Relation r : c.getIncomingRelations()){
>>> >>>>> Hibernate.initialize(r.getSource().getIncomingRelations());
>>> >>>>> Hibernate.initialize(r.getSource().getOutgoingRelations());
>>> >>>>> }
>>> >>>>> for(Relation r : c.getOutgoingRelations()){
>>> >>>>> Hibernate.initialize(r.getTarget().getIncomingRelations());
>>> >>>>> Hibernate.initialize(r.getTarget().getOutgoingRelations());
>>> >>>>> }
>>> >>>>> }
>>> >>>>> return cells;
>>> >>>>> }
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> So the first method executes one query and 2 subselects to find a
>>> cell
>>> >>>> and
>>> >>>>> all relations, the second method, iterate over each relation and do
>>> the
>>> >>>>> same. So I pretty much will have something like 3+r*3 selects on db,
>>> >>>> where
>>> >>>>> r is the number of relations right.
>>> >>>>>
>>> >>>>> Ok, to be a bit fair with the tests, I've ran this for the same
>>> node 10
>>> >>>>> times (get a chance to warm the caches), exclude the longest and
>>> >> smallest
>>> >>>>> result, and then took a mean of it. Here's the results:
>>> >>>>>
>>> >>>>> EhCache: 70ms
>>> >>>>> Plain Hibernate: 550ms
>>> >>>>>
>>> >>>>> I still don't have a version of neo4j code running integrated in the
>>> >> app
>>> >>>>> server, but the idea is to use REST API. Running the query on the
>>> REST
>>> >>>> API
>>> >>>>> took over 2 seconds on average, but due the large size of the
>>> response,
>>> >>>>> network lagging was the issue. So I ran the same query 10 times
>>> using
>>> >> the
>>> >>>>> web console, and the average time for neo was 300ms
>>> >>>>>
>>> >>>>> Before asking anything I do know that we will have more complex
>>> queries
>>> >>>>> where neo will shine, but I need to improve those results in order
>>> to
>>> >>>> sell
>>> >>>>> it here :), with those numbers, ppl will just say that having a
>>> cache
>>> >> and
>>> >>>>> using Relational model would suffice.
>>> >>>>>
>>> >>>>> Anything I could do to improve this?
>>> >>>>>
>>> >>>>> Regards
>>> >>>>> ___
>>> >>>>> Neo4j mailing list
>>> >>>>> User@lists.neo4j.org
>>> >>>>> https://lists.neo4j.org/mailman/listinfo/user
>>> >>>>
>>> >>>> ___
>>> >>>> Neo4j mailing list
>>> >>>> User@lists.neo4j.org
>>> >>>> https://lists.neo4j.org/mailman/listinfo/user
>>> >>>>
>>> >>> ___
>>> >>> Neo4j mailing list
>>> >>> User@lists.neo4j.org
>>> >>> https://lists.neo4j.org/mailman/listinfo/user
>>> >>
>>> >> ___
>>> >> Neo4j mailing list
>>> >> User@lists.neo4j.org
>>> >> https://lists.neo4j.org/mailman/listinfo/user
>>> >>
>>> > ___
>>> > Neo4j mailing list
>>> > User@lists.neo4j.org
>>> > https://lists.neo4j.org/mailman/listinfo/user
>>>
>>> ___
>>> Neo4j mailing list
>>> User@lists.neo4j.org
>>> https://lists.neo4j.org/mailman/listinfo/user
>>>
>>
>>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best wishes,
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j upcoming features importance poll
the top feature i want: node insertion with checking of uniq external id: get_or_create On Sat, Nov 19, 2011 at 11:39 AM, Pablo Pareja wrote: > Hi all, > > I was thinking it'd be cool to create a sort of a poll in order to know > which features (that are missing right now...) are the most important ones > for the community. I just did a quick google search for free online poll > creation platforms and found doodle site, (btw do you know a better site to > do this?). > The address for the poll is: > http://www.doodle.com/wg8k77vwq6b654bv > So far I just added three features that came to my mind while I was > creating it, so please say which features you're missing and I'll add them > so that we can all vote for them or not. > What do you think about all this? > Cheers, > > Pablo > > -- > Pablo Pareja Tobes > > My site http://about.me/pablopareja > LinkedIn http://www.linkedin.com/in/pabloparejatobes > Twitter http://www.twitter.com/pablopareja > > Creator of Bio4j --> http://www.bio4j.com > > http://www.ohnosequences.com > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best wishes, Linan Wang Architect, Programmer, PhD ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Activity Streams and Twitter Sample App
one more solution. set a sampling_ratio, say 10:
Person1.outgoing(:follows).outgoing(:tweeted).depth(2).filter("position.length()==
2;") .prune("position.returnedNodesCount() > 100 * sampling_ratio")
then do a sort based on timestamp.
the goal is not to get the perfect result but *good enough* ones
depends on your experience.
On Thu, Nov 3, 2011 at 4:48 PM, maxdemarzi wrote:
> Came up with another possibility:
>
> G) Store the Latest 100 tweeted relationship ids with dates as a property of
> the User Node, and a custom Breadth First Traversal that evaluates the list
> of every follower's latest 100 tweets before deciding which relationships to
> follow.
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Activity-Streams-and-Twitter-Sample-App-tp3477669p3477693.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best wishes,
Linan Wang
Architect, Programmer, PhD
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] some questions
to clear the full db, stop the server and delete the files in data folder :) On Fri, Oct 28, 2011 at 3:57 PM, Marko Rodriguez wrote: > Hi, > >> 2. Assume I have a big graph in the REST server, what is the best way to >> remove the whole graph? Of course deleting each node and its relationships >> and its index is not convenient. > > > Via Gremlin Plugin, you can call g.clear(). > > HTH, > Marko. > > http://markorodriguez.com > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best wishes, Linan Wang Architect, Programmer, PhD ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST, Transactions and Uniqueness
peter,
a quick implementation is submitted. pull request sent. my naming
ability is probably not the best part. see if the codes make sense.
thanks.
On Wed, Sep 28, 2011 at 10:10 PM, Peter Neubauer
wrote:
> Linan,
> It's high prio in the backlog for 1.6, so this WILL be taken care of. I
> believe there is also an issue on this on Github? If not, please raise one
> to track progress for the community. Thanks!
>
> /peter
>
> Sent from my phone.
> On Sep 28, 2011 10:02 PM, "Linan Wang" wrote:
>> Peter,
>> I feel uniqueness has been a recurring theme in neo4j applications,
>> especially when it's used heavily on algorithms traversing existing
>> data. it would be great if it's supported in kernel level:
>>
>> interface NodeUniquenessConstraint
>> {
>> public Node getNode();
>> public void setupNode(Node newNode);
>> }
>>
>> public Node getOrCreateNode(NodeUniquenessConstraint constraint)
>> {
>> // Preparation, acquires lock, etc.
>> Node n = constraint.getNode();
>> if(n == null)
>> {
>> n = createNode();
>> constraint.setupNode(n);
>> }
>> ...
>> return n;
>> }
>>
>> or, there is something similar already implemented?
>>
>> On Tue, Sep 27, 2011 at 9:38 PM, Peter Neubauer
>> wrote:
>>> Guys,
>>> Maps are now supported as parameters, look at the Gremlin plugin för
>>> reference in the docs. Will add that for parameters to the cypher plugin
>>> too.
>>>
>>> Thanks for chipping in!
>>>
>>> /peter
>>>
>>> Sent from my phone.
>>> On Sep 27, 2011 8:12 PM, "Tony Wooster" wrote:
>>>> Hi Linan,
>>>>
>>>> That's essentially what I implemented, but the logic just became that
>>>> much more tortured when going over REST. Like I said, less of a Java
>>>> programmer. The implementation I came up with on the REST side (hacky
>>>> though it may be) was this:
>>>>
>>>> @Description( "An extension to help maintain unique relationships" )
>>>> public class IndexTester extends ServerPlugin
>>>> {
>>>> @Name( "error_if_in_node_index" )
>>>> @Description( "Will return a 4xx error if a key/value pair is found
> in
>>> "+
>>>> "a given index. Also errors if the index doesn't
>>> exist.")
>>>> @PluginTarget( GraphDatabaseService.class )
>>>> public Boolean errorIfInNodeIndex(
>>>> @Source GraphDatabaseService graphDb,
>>>> @Description( "Name of the index to earch." )
>>>> @Parameter( name = "indexName" ) String indexName,
>>>> @Description( "Name of key to search." )
>>>> @Parameter( name = "key" ) String key,
>>>> @Description( "Value to search for." )
>>>> @Parameter( name = "value" ) String value )
>>>> throws BadInputException
>>>> {
>>>>
>>>> if ( !graphDb.index().existsForNodes( indexName ) )
>>>> throw new BadInputException("Index doesn't exist", new
>>>> NotFoundException());
>>>>
>>>> Index index = graphDb.index().forNodes( indexName );
>>>>
>>>> if (index.get(key, value).size() > 0)
>>>> throw new BadInputException("Key/value pair found in
> index");
>>>> return null;
>>>> }
>>>> }
>>>>
>>>> I'm still not entirely certain that this is the appropriate way to go;
>>>> probably a better solution would be a more general "add node with
>>>> unique, indexed fields" command that's slightly more functional than
>>>> this batch-operation-quirks based hack. As an aside -- does anyone
>>>> know when/if lists of maps for parameters will be implemented for REST
>>>> plugins?
>>>>
>>>> Thanks for the response!
>>>>
>>>> -T
>>>>
>>>> On Thu, Sep 22, 2011 at 4:57 PM, Linan Wang wrote:
>>>>>
>>>>> Hi,
>>>>> i had the issue few days ago and thanks to McKinley I got a workable
>>>>> solution. i think the best way to do is through unmanaged extension.
>>>
Re: [Neo4j] REST, Transactions and Uniqueness
Peter,
I feel uniqueness has been a recurring theme in neo4j applications,
especially when it's used heavily on algorithms traversing existing
data. it would be great if it's supported in kernel level:
interface NodeUniquenessConstraint
{
public Node getNode();
public void setupNode(Node newNode);
}
public Node getOrCreateNode(NodeUniquenessConstraint constraint)
{
// Preparation, acquires lock, etc.
Node n = constraint.getNode();
if(n == null)
{
n = createNode();
constraint.setupNode(n);
}
...
return n;
}
or, there is something similar already implemented?
On Tue, Sep 27, 2011 at 9:38 PM, Peter Neubauer
wrote:
> Guys,
> Maps are now supported as parameters, look at the Gremlin plugin för
> reference in the docs. Will add that for parameters to the cypher plugin
> too.
>
> Thanks for chipping in!
>
> /peter
>
> Sent from my phone.
> On Sep 27, 2011 8:12 PM, "Tony Wooster" wrote:
>> Hi Linan,
>>
>> That's essentially what I implemented, but the logic just became that
>> much more tortured when going over REST. Like I said, less of a Java
>> programmer. The implementation I came up with on the REST side (hacky
>> though it may be) was this:
>>
>> @Description( "An extension to help maintain unique relationships" )
>> public class IndexTester extends ServerPlugin
>> {
>> @Name( "error_if_in_node_index" )
>> @Description( "Will return a 4xx error if a key/value pair is found in
> "+
>> "a given index. Also errors if the index doesn't
> exist.")
>> @PluginTarget( GraphDatabaseService.class )
>> public Boolean errorIfInNodeIndex(
>> @Source GraphDatabaseService graphDb,
>> @Description( "Name of the index to earch." )
>> @Parameter( name = "indexName" ) String indexName,
>> @Description( "Name of key to search." )
>> @Parameter( name = "key" ) String key,
>> @Description( "Value to search for." )
>> @Parameter( name = "value" ) String value )
>> throws BadInputException
>> {
>>
>> if ( !graphDb.index().existsForNodes( indexName ) )
>> throw new BadInputException("Index doesn't exist", new
>> NotFoundException());
>>
>> Index index = graphDb.index().forNodes( indexName );
>>
>> if (index.get(key, value).size() > 0)
>> throw new BadInputException("Key/value pair found in index");
>> return null;
>> }
>> }
>>
>> I'm still not entirely certain that this is the appropriate way to go;
>> probably a better solution would be a more general "add node with
>> unique, indexed fields" command that's slightly more functional than
>> this batch-operation-quirks based hack. As an aside -- does anyone
>> know when/if lists of maps for parameters will be implemented for REST
>> plugins?
>>
>> Thanks for the response!
>>
>> -T
>>
>> On Thu, Sep 22, 2011 at 4:57 PM, Linan Wang wrote:
>>>
>>> Hi,
>>> i had the issue few days ago and thanks to McKinley I got a workable
>>> solution. i think the best way to do is through unmanaged extension.
>>> the overhead of multiple REST calls could make the matter more
>>> complex.
>>>
>>> here is part of my ObjectFactory class. in my situation it's an
>>> external id needs to be uniq. feel free to correct my codes :) the
>>> performance is not ideal though. on my imac I got around 50 insertions
>>> per sec. the bottle neck is not memory.
>>>
>>> public T get(long externalId)
>>> {
>>> // try asynchronized read first;
>>> Index idx = getDefaultNodeIndex();
>>>
>>> IndexHits h = idx.get(IDX_KEY_EXTERNAL_ID,
> externalId);
>>> Node n = h.getSingle();
>>> h.close();
>>>
>>> if(n != null)
>>> return wrap(n);
>>>
>>> // if not found, try synchronized version;
>>> return null;
>>> }
>>>
>>> public T getOrCreate(long externalId)
>>> {
>>>
>>> T ret = get( externalId );
>>> if(ret != null)
>>>
Re: [Neo4j] REST, Transactions and Uniqueness
> 2. Attempt to delete __WRITE_LOCK__ from UserRef node (will 4xx if property
> is non-existent)
> 3. POST to extension, to check if username is in users index (4xx if it
> exists)
> 3a. Transaction will fail at this point if user is in index
> 4. Add a User node
> 5. Relate it back to the UserRef node
> 5. Add it to the index
> 6. Release lock on UserRef supernode by re-adding __WRITE_LOCK__ property
> 7. Finish HTTP request, done
>
> This seems to work, but, again, are there any blind-spots that I'm unaware
> of? How about if this goes to a HA cluster?
>
> Finally, somewhat related, are some concerns with the batch API
> back-reference capability. This appears to manifest itself as a blind
> string-replace of '{[id]}' in provided fields. This _seems_ like it could
> have some security/annoying bug concerns relating to user-provided data
> (local portion of email address includes the substring '{1}', for example,
> which is valid per the email spec). I currently don't see any way around
> this except to restrict user input. Any thoughts?
>
> Anyway, thanks for any comments and responses!
>
> -Tony Wooster
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/REST-Transactions-and-Uniqueness-tp3360054p3360054.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Error in performance docs - Stack Size
k/m/g all work. just tried on 1.6. On Thu, Sep 22, 2011 at 2:22 AM, Romiko Derbynew wrote: > Sorry, clicked wrong button: > http://wiki.neo4j.org/content/Configuration_Settings > > The stack size is set by specifying the -Xss???m parameter to hotspot, where > ??? is the stack size in megabytes. > > As far as I am aware, according to this document: > http://www.caucho.com/resin-3.0/performance/jvm-tuning.xtp#stack-size > The stack size in in kilobytes, which kind a makes sense. > -Xss determines the size of the stack: -Xss1024k > > Please confirm? > > From: Romiko Derbynew > Sent: Thursday, 22 September 2011 11:19 AM > To: 'user@lists.neo4j.org' > Subject: Error in performance docs - Stack Size > > Hi Guys, > > I think there is an error in the performance docs. I am currently tuning the > stack size as we use multiple threads, however, I see this in the docs: > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Creating a graph database with BatchInserter and getting the node degree of every node
Hi stephan,
I mis-calculated the size of relationshipstore.db. i thought it was
around 8G instead of 85G. the only option left i think is to build
index. something like this:
idx = db.index().forNode("knows");
idx.add(thisguy, "knows", thatguy.getId());
idx.add(thatguy, "known_by", thisguy.getId());
the benefit is that when querying, the return size is pre-calculated
so it would save some iteration time.
the problem is the index files size, should around 85G.
On Wed, Sep 21, 2011 at 11:44 AM, st3ven wrote:
> Hi Linan,
>
> I just tried it with the outgoing relationships, but unfortunately that
> didn't speed things up.
>
> The size of my db is around 140GB and so it is not possible for me to dumb
> the full directory into a ramfs.
> My files on the hard disk have the following size:
> neostore.nodestore.db = 31MB
> neostore.relationshipstore.db = 85GB
> neostore.propertystore.db = 65GB
> neostore.propertystore.db.strings = 180MB
> Is there maybe a chance of reducing the size of my database?
>
> Cheers,
> Stephan
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Creating-a-graph-database-with-BatchInserter-and-getting-the-node-degree-of-every-node-tp3351599p3355074.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Creating a graph database with BatchInserter and getting the node degree of every node
Stephan, what's the size of your db? if it's under 10G, how about just dump the full directory into to a ramfs. leave 1G to jvm and it'll do heavy io on the ramfs. i think it's a simple solution and could yield interesting result. please let me know the result if you tried. thanks On Tue, Sep 20, 2011 at 5:41 PM, Peter Neubauer wrote: > Steven, > the index is built into the DB, so you can use something like > http://docs.neo4j.org/chunked/snapshot/tutorials-java-embedded-index.html > to index all your nodes into Lucene (in one index, the node as key, > the number of relationships as numeric value when creating them). When > reading, you would simply request all keys from the index and iterate > over them. I am not terribly sure how much fast it is, but given that > you are just loading up documents, Lucene should be reasonably fast. > > Let us know if that works out! > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://startupbootcamp.org/ - Öresund - Innovation happens HERE. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > > > > On Tue, Sep 20, 2011 at 6:01 PM, st3ven wrote: >> Hello Peter, >> >> it's a pity that neo4j doesn't support full graph-scans. >> >> Is there maybe a possibility to cache more relationships to speed things up >> a little bit. >> I recognized that only the iteration over the relationships is taking hours. >> The time to get all relationships of one node is quite fast. >> >> I think I could try your second solution: >> - Store the relationships as a property in an Index (e.g. Lucene) and >> as the index for all entries. Thus, you are using an index for what it >> is good at - global operations over all documents. >> >> But I didn't understood it correctly. Do you mean an Index which stores the >> ID of a relationship and creating such an Index for every node? >> Could you maybe give me a code example for that? >> That would be very kind of you. >> >> The first solution is not really realizable, because I don't know the number >> of relationships of every node. >> I would have to count the relationships before the insertion and that would >> make my database useless for the node degree query. >> >> Thank you very much for your help! >> >> Cheers, >> Stephan >> >> -- >> View this message in context: >> http://neo4j-community-discussions.438527.n3.nabble.com/Creating-a-graph-database-with-BatchInserter-and-getting-the-node-degree-of-every-node-tp3351599p3352509.html >> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Creating a graph database with BatchInserter and getting the node degree of every node
ber of edges:" + > count); > } > } > Is there maybe a possibilty to cache more relationships or do you have any > idea hot to speedup the iteration progress. > > Thanks for your help again! > > Cheers, > Stephan > > -- > View this message in context: > http://neo4j-community-discussions.438527.n3.nabble.com/Creating-a-graph-database-with-BatchInserter-and-getting-the-node-degree-of-every-node-tp3351599p3352415.html > Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Creating a graph database with BatchInserter and getting the node degree of every node
hi stephan
i'm wondering if any difference if you could specify the relationship
when counting degrees:
RelationshipType knows = DynamicRelationshipType.withName("KNOWS");
Iterable rels = node.getRelationship(knows);
count = com.google.common.collect.Iterables.size(rels);
besides, do you know where is the bottle neck is, the node iteration
or relationship retrieval?
On Tue, Sep 20, 2011 at 1:38 PM, st3ven wrote:
> Hi,
>
> I already tried these java parameters, but that didn't really speedup the
> process and i already turned atime off.
> As Java parameters I am using right now -d64 -server -Xms7G -Xmx14G
> -XX:+UseParallelGC -XX:+UseNUMA
> What I've also noticed is, that reading from the database is really slow on
> my hard disk.
> It just reads 1mb/s and sometimes 8mb/s, but that is really slow. My hard
> disk can normally read and copy files much faster.
> Also very strange is, that the workload of the hard disk is around 99% with
> reading 1mb/s.
>
> My OS is Ubuntu Linux x64 and my file system is ext4.
>
> On the neo4j Wiki I found some performance guides, but these didn't really
> help.
> Do you know what I can do else?
>
>
> Perfomance Guides:
> http://wiki.neo4j.org/content/Linux_Performance_Guide
> http://wiki.neo4j.org/content/Linux_Performance_Guide
> http://wiki.neo4j.org/content/Configuration_Settings
> http://wiki.neo4j.org/content/Configuration_Settings
>
> I also added a configurtion file, but it seems that my Java program doesn't
> use all of the Ram.
>
> Thanks for your help!
>
> Cheers,
> Stephan
>
>
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Creating-a-graph-database-with-BatchInserter-and-getting-the-node-degree-of-every-node-tp3351599p3351881.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Creating a graph database with BatchInserter and getting the node degree of every node
hi Stephan,
have you set the -Xms, -XX:+UseNUMA, and -XX:+UseConcMarkSweepGC? they
could speedup the process significantly.
also, if you like, the jrockit is fast and free now. give it a try.
btw, which file system you are using? have you turned off atime?
On Tue, Sep 20, 2011 at 12:00 PM, st3ven wrote:
> Peter,
>
> the import of the data into the graph database is not the main problem for
> me.
> The lookup of nodes from the index is fast enough for me.
> To create the database it took me nearly half a day.
>
> My main problem here is getting the node degree of every node.
> As I already said I am using this code to get the node degree of every node:
>
> for (Node node : db.getAllNodes()) {
> counter = 0;
>
> if (node.getId() > 0) {
> for (Relationship rel :
> node.getRelationships()) {
> counter++;
> }
>
> System.out.println(node.getProperty("name").toString() + ": "
> + counter);
> }
>
> }
>
> After 3 days I only got the node degree of 8 nodes and I want to
> optimize my traversal here, cause this is very slow.
> What can I do to make this faster or do I have to change my code for getting
> the node degree?
> I only posted my import code because I thought I could maybe optimize there
> something for this traversal.
>
> Thank you very much for your help!
>
> Cheers,
> Stephan
>
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Creating-a-graph-database-with-BatchInserter-and-getting-the-node-degree-of-every-node-tp3351599p3351664.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> _______
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] lock or not?
Got it. i really appreciate your help. On Wed, Sep 14, 2011 at 11:14 PM, McKinley wrote: > I mean that if you are not running the REST server or high availability then > you can assume that even if you only put the read lock in the Java > thread/object world, the database will not change. No other process exists > that could change it. You do not need to bother with a read lock in the > database. > > Cheers, > > McKinley > > On Wed, Sep 14, 2011 at 3:07 PM, Linan Wang wrote: > >> McKinley >> Thank you very much for the detailed explanation. however, I don't get >> the part about "only one JVM will access database". >> neo4j doesn't support multiple JVMs has write access to the same db, right? >> >> On Wed, Sep 14, 2011 at 9:14 PM, McKinley wrote: >> > If a second thread reads that there is no node with external_id 123 in >> > between the time that a first thread finds no node and elects to create >> it, >> > you will get 2 nodes with external_id 123. So yes, you need to introduce >> a >> > lock and synchronize. >> > _______ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] lock or not?
McKinley Thank you very much for the detailed explanation. however, I don't get the part about "only one JVM will access database". neo4j doesn't support multiple JVMs has write access to the same db, right? On Wed, Sep 14, 2011 at 9:14 PM, McKinley wrote: > If a second thread reads that there is no node with external_id 123 in > between the time that a first thread finds no node and elects to create it, > you will get 2 nodes with external_id 123. So yes, you need to introduce a > lock and synchronize. > > You can create read locks in the graph database but you probably will not > need to do that. In your case you are reading from an index and I am not > sure what locks are available for indexes. If you are running an embedded > database and you can be sure that only your one JVM will access the database > then you can just lock in the Java thread/object space and elect that other > threads must wait to read if the node exists until the first thread has > created it. If you need to communicate concurrency via the database, you > will have to do all the same locking in the Java space and add a read lock > on some control node in Neo4j. > > If you want to create a read lock on a node you can do the following: > > lockManager = graphDb.getConfig().getLockManager(); > lockManager.getReadLock(someNode); > ... > lockManager.releaseReadLock(someNode, null); > > Threading and synchronization in Java will perform great. Depending on your > model, most of the time your node will exist and two or more threads will > just queue up and get the same reference. That waiting is inefficient by > nature, but if your business rules require it then you have to do it. > > Cheers, > > McKinley > > On Wed, Sep 14, 2011 at 12:45 PM, Linan Wang wrote: > >> hi all, >> I have implemented a function >> UserFactory.get_or_create_with_external_id(long external_id). the >> function basically search the index, see if a node already created and >> has the property "external_id" equals to external_id. if there is such >> a node, return, if not, create one, assign property and index it. >> this function will be called inside of transactions, from unmanaged >> extensions. so it's multi-threaded scenario. >> my question is if i need to mark the function synchronized. my guess >> is true, and i feel it'll hurt performance badly. if my guess is >> wrong, why? thanks >> >> -- >> Best regards >> >> Linan Wang >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] lock or not?
hi all, I have implemented a function UserFactory.get_or_create_with_external_id(long external_id). the function basically search the index, see if a node already created and has the property "external_id" equals to external_id. if there is such a node, return, if not, create one, assign property and index it. this function will be called inside of transactions, from unmanaged extensions. so it's multi-threaded scenario. my question is if i need to mark the function synchronized. my guess is true, and i feel it'll hurt performance badly. if my guess is wrong, why? thanks -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Unmanaged Extensions
Yes, it works! thank you very much! On Mon, Sep 12, 2011 at 8:22 PM, Peter Neubauer wrote: > Thanks Linan for the find! > > I corrected the spelling of the function. Also, JAXRS is not related > to Server Plugin lifecycle - it is a raw JAXRS registration into the > webserver. > > For your test jar, I got things working when checking in the export > dialog to include directory information, see the attached picture. > Then, I can set > > org.neo4j.server.thirdparty_jaxrs_classes=org.neo4j.examples=/hello > > in neo4j-server/properties > > and get with curl http://localhost:7474/hello/0 > > Hello World, nodeId=0 > > > Does that work for you? > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://startupbootcamp.org/ - Öresund - Innovation happens HERE. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > > > > On Mon, Sep 12, 2011 at 8:12 PM, Linan Wang wrote: >> what i've found: >> PluginInitializer >> (https://github.com/neo4j/community/blob/master/server/src/main/java/org/neo4j/server/modules/PluginInitializer.java) >> 1, probably has a spell mistake: intitializePackages, maybe >> initializePackages? >> 2, i don't understand how pluginlifecycle is related to thirdparty jaxrs >> >> On Mon, Sep 12, 2011 at 6:58 PM, Linan Wang wrote: >>> yes, i did it. i package it into jar file just to show that i've >>> modified the file. >>> will it related to the 64 jvm shipped in osx lion? i'm reading the >>> source codes... >>> >>> On Mon, Sep 12, 2011 at 6:55 PM, Peter Neubauer >>> wrote: >>>> Linan, >>>> you need to change the neo4j-server.properties in the Neo4j Server >>>> conf/ directory where you want to deploy your extension in order to >>>> get this picked up, not packaging it with the extension jar. Could you >>>> please try that? See also >>>> http://docs.neo4j.org/chunked/snapshot/server-unmanaged-extensions.html >>>> for details. >>>> >>>> Cheers, >>>> >>>> /peter neubauer >>>> >>>> GTalk: neubauer.peter >>>> Skype peter.neubauer >>>> Phone +46 704 106975 >>>> LinkedIn http://www.linkedin.com/in/neubauer >>>> Twitter http://twitter.com/peterneubauer >>>> >>>> http://www.neo4j.org - Your high performance graph database. >>>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >>>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >>>> >>>> >>>> >>>> On Mon, Sep 12, 2011 at 5:01 PM, Linan Wang wrote: >>>>> sure. it's here: https://github.com/wangii/test2/blob/master/test2.jar >>>>> >>>>> On Mon, Sep 12, 2011 at 3:56 PM, Peter Neubauer >>>>> wrote: >>>>>> Thanks Linan, >>>>>> Could you also put up the jar you deploy so I can test it, since you >>>>>> seem to >>>>>> rely on Eclipse building which I can't recreate? >>>>>> >>>>>> /peter >>>>>> >>>>>> Sent from my phone. >>>>>> On Sep 12, 2011 4:14 PM, "Linan Wang" wrote: >>>>>>> Peter, >>>>>>> thanks. it's here: https://github.com/wangii/test2 >>>>>>> must be something in the doc i didn't notice, even though i've read 5 >>>>>> times! >>>>>>> >>>>>>> On Mon, Sep 12, 2011 at 3:06 PM, Peter Neubauer >>>>>>> wrote: >>>>>>>> Linan, >>>>>>>> got a small project to build this for so I can test it out? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> >>>>>>>> /peter neubauer >>>>>>>> >>>>>>>> GTalk: neubauer.peter >>>>>>>> Skype peter.neubauer >>>>>>>> Phone +46 704 106975 >>>>>>>> LinkedIn http://www.linkedin.com/in/neubauer >>>>>>>> Twitter http://twitter.com/peterneu
Re: [Neo4j] Unmanaged Extensions
what i've found: PluginInitializer (https://github.com/neo4j/community/blob/master/server/src/main/java/org/neo4j/server/modules/PluginInitializer.java) 1, probably has a spell mistake: intitializePackages, maybe initializePackages? 2, i don't understand how pluginlifecycle is related to thirdparty jaxrs On Mon, Sep 12, 2011 at 6:58 PM, Linan Wang wrote: > yes, i did it. i package it into jar file just to show that i've > modified the file. > will it related to the 64 jvm shipped in osx lion? i'm reading the > source codes... > > On Mon, Sep 12, 2011 at 6:55 PM, Peter Neubauer > wrote: >> Linan, >> you need to change the neo4j-server.properties in the Neo4j Server >> conf/ directory where you want to deploy your extension in order to >> get this picked up, not packaging it with the extension jar. Could you >> please try that? See also >> http://docs.neo4j.org/chunked/snapshot/server-unmanaged-extensions.html >> for details. >> >> Cheers, >> >> /peter neubauer >> >> GTalk: neubauer.peter >> Skype peter.neubauer >> Phone +46 704 106975 >> LinkedIn http://www.linkedin.com/in/neubauer >> Twitter http://twitter.com/peterneubauer >> >> http://www.neo4j.org - Your high performance graph database. >> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >> >> >> >> On Mon, Sep 12, 2011 at 5:01 PM, Linan Wang wrote: >>> sure. it's here: https://github.com/wangii/test2/blob/master/test2.jar >>> >>> On Mon, Sep 12, 2011 at 3:56 PM, Peter Neubauer >>> wrote: >>>> Thanks Linan, >>>> Could you also put up the jar you deploy so I can test it, since you seem >>>> to >>>> rely on Eclipse building which I can't recreate? >>>> >>>> /peter >>>> >>>> Sent from my phone. >>>> On Sep 12, 2011 4:14 PM, "Linan Wang" wrote: >>>>> Peter, >>>>> thanks. it's here: https://github.com/wangii/test2 >>>>> must be something in the doc i didn't notice, even though i've read 5 >>>> times! >>>>> >>>>> On Mon, Sep 12, 2011 at 3:06 PM, Peter Neubauer >>>>> wrote: >>>>>> Linan, >>>>>> got a small project to build this for so I can test it out? >>>>>> >>>>>> Cheers, >>>>>> >>>>>> /peter neubauer >>>>>> >>>>>> GTalk: neubauer.peter >>>>>> Skype peter.neubauer >>>>>> Phone +46 704 106975 >>>>>> LinkedIn http://www.linkedin.com/in/neubauer >>>>>> Twitter http://twitter.com/peterneubauer >>>>>> >>>>>> http://www.neo4j.org - Your high performance graph >>>> database. >>>>>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >>>>>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Sep 12, 2011 at 4:04 PM, wangii wrote: >>>>>>> Got the same ResourceConfig problem: The ResourceConfig instance does >>>> not >>>>>>> contain any root resource. >>>>>>> tried to put jar file into /lib, /plugins, modified >>>> neo4j-server.properties >>>>>>> file. >>>>>>> anything I missed here? thanks >>>>>>> >>>>>>> -- >>>>>>> View this message in context: >>>> http://neo4j-community-discussions.438527.n3.nabble.com/Unmanaged-Extensions-tp2660066p3329694.html >>>>>>> Sent from the Neo4j Community Discussions mailing list archive at >>>> Nabble.com. >>>>>>> _______ >>>>>>> Neo4j mailing list >>>>>>> User@lists.neo4j.org >>>>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>>>> >>>>>> ___ >>>>>> Neo4j mailing list >>>>>> User@lists.neo4j.org >>>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards >>>>> >>>>> Linan Wang >>>>> ___ >>>>> Neo4j mailing list >>>>> User@lists.neo4j.org >>>>> https://lists.neo4j.org/mailman/listinfo/user >>>> ___ >>>> Neo4j mailing list >>>> User@lists.neo4j.org >>>> https://lists.neo4j.org/mailman/listinfo/user >>>> >>> >>> >>> >>> -- >>> Best regards >>> >>> Linan Wang >>> ___ >>> Neo4j mailing list >>> User@lists.neo4j.org >>> https://lists.neo4j.org/mailman/listinfo/user >>> >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > > > > -- > Best regards > > Linan Wang > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Unmanaged Extensions
yes, i did it. i package it into jar file just to show that i've modified the file. will it related to the 64 jvm shipped in osx lion? i'm reading the source codes... On Mon, Sep 12, 2011 at 6:55 PM, Peter Neubauer wrote: > Linan, > you need to change the neo4j-server.properties in the Neo4j Server > conf/ directory where you want to deploy your extension in order to > get this picked up, not packaging it with the extension jar. Could you > please try that? See also > http://docs.neo4j.org/chunked/snapshot/server-unmanaged-extensions.html > for details. > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://startupbootcamp.org/ - Öresund - Innovation happens HERE. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > > > > On Mon, Sep 12, 2011 at 5:01 PM, Linan Wang wrote: >> sure. it's here: https://github.com/wangii/test2/blob/master/test2.jar >> >> On Mon, Sep 12, 2011 at 3:56 PM, Peter Neubauer >> wrote: >>> Thanks Linan, >>> Could you also put up the jar you deploy so I can test it, since you seem to >>> rely on Eclipse building which I can't recreate? >>> >>> /peter >>> >>> Sent from my phone. >>> On Sep 12, 2011 4:14 PM, "Linan Wang" wrote: >>>> Peter, >>>> thanks. it's here: https://github.com/wangii/test2 >>>> must be something in the doc i didn't notice, even though i've read 5 >>> times! >>>> >>>> On Mon, Sep 12, 2011 at 3:06 PM, Peter Neubauer >>>> wrote: >>>>> Linan, >>>>> got a small project to build this for so I can test it out? >>>>> >>>>> Cheers, >>>>> >>>>> /peter neubauer >>>>> >>>>> GTalk: neubauer.peter >>>>> Skype peter.neubauer >>>>> Phone +46 704 106975 >>>>> LinkedIn http://www.linkedin.com/in/neubauer >>>>> Twitter http://twitter.com/peterneubauer >>>>> >>>>> http://www.neo4j.org - Your high performance graph >>> database. >>>>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >>>>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >>>>> >>>>> >>>>> >>>>> On Mon, Sep 12, 2011 at 4:04 PM, wangii wrote: >>>>>> Got the same ResourceConfig problem: The ResourceConfig instance does >>> not >>>>>> contain any root resource. >>>>>> tried to put jar file into /lib, /plugins, modified >>> neo4j-server.properties >>>>>> file. >>>>>> anything I missed here? thanks >>>>>> >>>>>> -- >>>>>> View this message in context: >>> http://neo4j-community-discussions.438527.n3.nabble.com/Unmanaged-Extensions-tp2660066p3329694.html >>>>>> Sent from the Neo4j Community Discussions mailing list archive at >>> Nabble.com. >>>>>> ___ >>>>>> Neo4j mailing list >>>>>> User@lists.neo4j.org >>>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>>> >>>>> ___ >>>>> Neo4j mailing list >>>>> User@lists.neo4j.org >>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards >>>> >>>> Linan Wang >>>> ___ >>>> Neo4j mailing list >>>> User@lists.neo4j.org >>>> https://lists.neo4j.org/mailman/listinfo/user >>> ___ >>> Neo4j mailing list >>> User@lists.neo4j.org >>> https://lists.neo4j.org/mailman/listinfo/user >>> >> >> >> >> -- >> Best regards >> >> Linan Wang >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Unmanaged Extensions
sure. it's here: https://github.com/wangii/test2/blob/master/test2.jar On Mon, Sep 12, 2011 at 3:56 PM, Peter Neubauer wrote: > Thanks Linan, > Could you also put up the jar you deploy so I can test it, since you seem to > rely on Eclipse building which I can't recreate? > > /peter > > Sent from my phone. > On Sep 12, 2011 4:14 PM, "Linan Wang" wrote: >> Peter, >> thanks. it's here: https://github.com/wangii/test2 >> must be something in the doc i didn't notice, even though i've read 5 > times! >> >> On Mon, Sep 12, 2011 at 3:06 PM, Peter Neubauer >> wrote: >>> Linan, >>> got a small project to build this for so I can test it out? >>> >>> Cheers, >>> >>> /peter neubauer >>> >>> GTalk: neubauer.peter >>> Skype peter.neubauer >>> Phone +46 704 106975 >>> LinkedIn http://www.linkedin.com/in/neubauer >>> Twitter http://twitter.com/peterneubauer >>> >>> http://www.neo4j.org - Your high performance graph > database. >>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >>> >>> >>> >>> On Mon, Sep 12, 2011 at 4:04 PM, wangii wrote: >>>> Got the same ResourceConfig problem: The ResourceConfig instance does > not >>>> contain any root resource. >>>> tried to put jar file into /lib, /plugins, modified > neo4j-server.properties >>>> file. >>>> anything I missed here? thanks >>>> >>>> -- >>>> View this message in context: > http://neo4j-community-discussions.438527.n3.nabble.com/Unmanaged-Extensions-tp2660066p3329694.html >>>> Sent from the Neo4j Community Discussions mailing list archive at > Nabble.com. >>>> ___ >>>> Neo4j mailing list >>>> User@lists.neo4j.org >>>> https://lists.neo4j.org/mailman/listinfo/user >>>> >>> _______ >>> Neo4j mailing list >>> User@lists.neo4j.org >>> https://lists.neo4j.org/mailman/listinfo/user >>> >> >> >> >> -- >> Best regards >> >> Linan Wang >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Unmanaged Extensions
Peter, thanks. it's here: https://github.com/wangii/test2 must be something in the doc i didn't notice, even though i've read 5 times! On Mon, Sep 12, 2011 at 3:06 PM, Peter Neubauer wrote: > Linan, > got a small project to build this for so I can test it out? > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://startupbootcamp.org/ - Öresund - Innovation happens HERE. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > > > > On Mon, Sep 12, 2011 at 4:04 PM, wangii wrote: >> Got the same ResourceConfig problem: The ResourceConfig instance does not >> contain any root resource. >> tried to put jar file into /lib, /plugins, modified neo4j-server.properties >> file. >> anything I missed here? thanks >> >> -- >> View this message in context: >> http://neo4j-community-discussions.438527.n3.nabble.com/Unmanaged-Extensions-tp2660066p3329694.html >> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] performance guide
Andreas, thanks. can't wait for your result. one correction on my list: Retrieve a certain types of relationships of a node should be faster than retrieve all. i mis-read the source code in NodeImpl.java, in which "private volatile RelIdArray[] relationships" is an array of array. the updated list: 1, Read id of a node/relationship. 2, Read property value of a node/relationship. 3, Set property value of a node/relationship. 4, Retrieve a node/relationship by id. 5, Create a new node/relationship. 6, Retrieve a node by external id from node index. 7, Retrieve a certain subset of relationships of a node from RelationshipIndex. 8, Retrieve all relationships of a node. 9, Retrieve a certain types of relationships of a node. 10, Add a node to index. 11, Add a relationship to index. could anyone confirm that to iterate relationships of a node, Node.getRelationships() is faster than using RelationshipIndex.get(KEY, VAL, node, null)? thanks On Sat, Sep 10, 2011 at 1:46 AM, Andreas Kollegger wrote: > Hi Linan, > > These are good questions. I am currently following a similar approach to > characterizing the performance of basic graph operations, and then also > simple graph algorithms on various "shapes" of graphs. My initial > investigation is on small graphs on my laptop, but this weekend I'm testing > large graphs on Amazon instances (large, as in > 500 million nodes). > > I'll post the summary of my findings from this weekend, then continue to > elaborate on the operations and different server configurations. > > Best, > Andreas > > On Sep 9, 2011, at 4:29 PM, Linan Wang wrote: > >> hi, >> I once read a blog about how long it take for computation operations >> such as read from L1 cache, RAM access, send packet from usa to eu, >> etc. having a rough idea of the relative cost of major computation >> operations, we have better control. for example, in my project i also >> use property to store a cityid apart from create a relationship >> between user to city, because if reading an int property is 10x faster >> than getSingleRelationship + relationship.getEndNode + cityNode.getId. >> >> I'm wondering if could having similar list for neo4j operations. I >> understand some operations could be faster than another depending on >> different conditions. but let's only focus on basic read/write >> operations. >> >> below is my guess, from fastest to slowest, feel free to correct me >> and add more. >> >> 1, Read id of a node/relationship. >> 2, Read property value of a node/relationship. >> 3, Set property value of a node/relationship. >> 4, Retrieve a node/relationship by id. >> 5, Create a new node/relationship. >> 6, Retrieve a node by external id from node index. >> 7, Retrieve all relationships of a node. >> 8, Retrieve a certain subset of relationships of a node from >> RelationshipIndex. >> 9, Retrieve a certain types of relationships of a node. >> 10, Add a node to index. >> 11, Add a relationship to index. >> >> -- >> Best regards >> >> Linan Wang >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] performance guide
hi, I once read a blog about how long it take for computation operations such as read from L1 cache, RAM access, send packet from usa to eu, etc. having a rough idea of the relative cost of major computation operations, we have better control. for example, in my project i also use property to store a cityid apart from create a relationship between user to city, because if reading an int property is 10x faster than getSingleRelationship + relationship.getEndNode + cityNode.getId. I'm wondering if could having similar list for neo4j operations. I understand some operations could be faster than another depending on different conditions. but let's only focus on basic read/write operations. below is my guess, from fastest to slowest, feel free to correct me and add more. 1, Read id of a node/relationship. 2, Read property value of a node/relationship. 3, Set property value of a node/relationship. 4, Retrieve a node/relationship by id. 5, Create a new node/relationship. 6, Retrieve a node by external id from node index. 7, Retrieve all relationships of a node. 8, Retrieve a certain subset of relationships of a node from RelationshipIndex. 9, Retrieve a certain types of relationships of a node. 10, Add a node to index. 11, Add a relationship to index. -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] chain TraversalDescription?
Actually i'm fine with nesting multiple traversers to solve the
problem if deep inside gremlin, evaluator and fluentPipeline are just
nice constructs over the same solution. is it correct?
thanks
On Fri, Sep 9, 2011 at 7:03 PM, Linan Wang wrote:
> Thanks! I really appreciate. i'll have a deeper look at the
> FluentPipeline and Gremlin options later.
>
> Regarding using Evaluator in
> https://github.com/neo4j/community/blob/master/embedded-examples/src/test/java/org/neo4j/examples/orderedpath/OrderedPathTest.java
> my question:
> When the Evaluator.evaluate is called, the last node in the path is
> already retrieved, right? using the example, at depth 1, all my dogs
> and books i wrote have been retrieved, what the evaluator does is to
> decide not to continue from the, and exclude it in the return. am i
> right? any way to prevent from retrieving them at all? imagine i've
> written 10,000 books and have 1m dogs, visiting these nodes would be
> waste of time.
>
> On Fri, Sep 9, 2011 at 4:12 PM, Peter Neubauer
> wrote:
>> That's right!
>>
>> Looking all forward to that!
>>
>> Cheers,
>>
>> /peter neubauer
>>
>> GTalk: neubauer.peter
>> Skype peter.neubauer
>> Phone +46 704 106975
>> LinkedIn http://www.linkedin.com/in/neubauer
>> Twitter http://twitter.com/peterneubauer
>>
>> http://www.neo4j.org - Your high performance graph database.
>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>>
>>
>>
>> On Fri, Sep 9, 2011 at 5:10 PM, Marko Rodriguez wrote:
>>> Hey,
>>>
>>>> Hi Linan,
>>>> http://docs.neo4j.org/chunked/snapshot/tutorials-java-embedded-traversal.html#_new_traversal_framework
>>>> has an example of an ordered path, does that help? The code for it is
>>>> at
>>>> https://github.com/neo4j/community/blob/master/embedded-examples/src/test/java/org/neo4j/examples/orderedpath/OrderedPathTest.java
>>>>
>>>> Also, Gremlin, as Marko states, is applicable here.
>>>
>>>
>>> One more point to this. The next release of the Pipes comes with
>>> FluentPipeline which will allow you to, in native Java, do this:
>>>
>>> Pipe pipe = new
>>> FluentPipeline(graph.getVertex(1)).out("read").in("wrote").uniqueObject().out("pet").property("name");
>>> while(pipe.hasNext()) {
>>> String petName = pipe.next();
>>> }
>>>
>>> In this way, you can effect the same "Gremlin-esque" behavior, but in pure
>>> Java.
>>>
>>> This is planned for a September 21st released and, if all goes well, will
>>> be in the next release of Neo4j.
>>>
>>> Take care,
>>> Marko.
>>>
>>> http://markorodriguez.com
>>> ___
>>> Neo4j mailing list
>>> User@lists.neo4j.org
>>> https://lists.neo4j.org/mailman/listinfo/user
>>>
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
>
>
>
> --
> Best regards
>
> Linan Wang
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] chain TraversalDescription?
Thanks! I really appreciate. i'll have a deeper look at the
FluentPipeline and Gremlin options later.
Regarding using Evaluator in
https://github.com/neo4j/community/blob/master/embedded-examples/src/test/java/org/neo4j/examples/orderedpath/OrderedPathTest.java
my question:
When the Evaluator.evaluate is called, the last node in the path is
already retrieved, right? using the example, at depth 1, all my dogs
and books i wrote have been retrieved, what the evaluator does is to
decide not to continue from the, and exclude it in the return. am i
right? any way to prevent from retrieving them at all? imagine i've
written 10,000 books and have 1m dogs, visiting these nodes would be
waste of time.
On Fri, Sep 9, 2011 at 4:12 PM, Peter Neubauer
wrote:
> That's right!
>
> Looking all forward to that!
>
> Cheers,
>
> /peter neubauer
>
> GTalk: neubauer.peter
> Skype peter.neubauer
> Phone +46 704 106975
> LinkedIn http://www.linkedin.com/in/neubauer
> Twitter http://twitter.com/peterneubauer
>
> http://www.neo4j.org - Your high performance graph database.
> http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
>
>
>
> On Fri, Sep 9, 2011 at 5:10 PM, Marko Rodriguez wrote:
>> Hey,
>>
>>> Hi Linan,
>>> http://docs.neo4j.org/chunked/snapshot/tutorials-java-embedded-traversal.html#_new_traversal_framework
>>> has an example of an ordered path, does that help? The code for it is
>>> at
>>> https://github.com/neo4j/community/blob/master/embedded-examples/src/test/java/org/neo4j/examples/orderedpath/OrderedPathTest.java
>>>
>>> Also, Gremlin, as Marko states, is applicable here.
>>
>>
>> One more point to this. The next release of the Pipes comes with
>> FluentPipeline which will allow you to, in native Java, do this:
>>
>> Pipe pipe = new
>> FluentPipeline(graph.getVertex(1)).out("read").in("wrote").uniqueObject().out("pet").property("name");
>> while(pipe.hasNext()) {
>> String petName = pipe.next();
>> }
>>
>> In this way, you can effect the same "Gremlin-esque" behavior, but in pure
>> Java.
>>
>> This is planned for a September 21st released and, if all goes well, will be
>> in the next release of Neo4j.
>>
>> Take care,
>> Marko.
>>
>> http://markorodriguez.com
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] except CypherParser have any other methods for paging query ?
have you tried RelationshipIndex?
RelationshipIndex idx = db.index().forRelationship("...");
idx.add(rel, "IS_A_FAN", 1);
...
IndexHits hits = idx.get("IS_A_FAN", 1, null, theGuy);
...
On Mon, Sep 5, 2011 at 7:16 AM, iamyuanlong wrote:
> hi,everybody
> I use CypherParser to query one user's fans.
> some code like this.
>
>
>
> But when a user have 4 fans . when i query his fans . it cost my a
> long time to get the result.
> when i use the webbench -c 100 -t 60 http://...
> just have 287 successed 0 failed.
> Does anyone can tell me have any other ways to paging query one's fans
> in Neo4j?
>
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-except-CypherParser-have-any-other-methods-for-paging-query-tp3310029p3310029.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] chain TraversalDescription?
Hi, say people read books, books are written by authors, authors has dogs, then how to reach the dogs of authors of books i've read? when i'm also a writer, also have dogs, the TraversalDescription seems inefficient since it'll also traverse my own books and dogs first, before my customised StopEvaluator stops the expansion process. my feeling is RelationshipExpander is the way to go but stucked at the problem to get the depth info. my goal is to have the designated RelationshipExpander to tell traverser that at depth 1 only looking for OUTGOING read relationship, depth 2 for INCOMING wrote relationship and depth 3 for OUTGOING has relationship. any comment is appreciated! thanks. -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] regarding supernode
how about i put the codes onto github and you guys can comment and
modify it. then include the project as a sample?
On Fri, Sep 9, 2011 at 8:58 AM, Peter Neubauer wrote:
> Linan,
> I think your example would be great for doing a JUnit test showing
> this. Niels, could you do that, plz? In that case, I can add a
> graph and exaplanations to it.
>
> /peter
>
> On Thu, Sep 8, 2011 at 11:25 PM, Linan Wang wrote:
>> On Wed, Sep 7, 2011 at 5:21 PM, Linan Wang wrote:
>>> hi,
>>> I don't quite understand RelationshipIndex and RelationshipExpander.
>>> say I have a supernode city (beijing), it has 10 m users links to
>>> through relationship LIVES_IN. so how should I index? should be
>>> something like:
>>> RelationshipIndex idx = db.index().forRelationships("CITY_LIVES_IN");
>>> idx.add(rel, "LIVES_IN", "Beijing");
>>> if so, what's the advantage over this?
>>> Index idx = db.index().forNodes("CITY_LIVES_IN");
>>> idx.add(user, "LIVES_IN", "beijing");
>>> (I read source code of LuceneIndex.java, found out that the
>>> implementation of the add method is shared between Index and
>>> RelationshipIndex.)
>> ok, i answer my own question:
>> RelationshipIndex has the function query which takes startNode and
>> endNode as extra parameters.
>> so if traverse only depth 1, it could be faster than using Traverser.
>> am i right here? (please say yes!)
>> then the question is how to take advantage of it for more than 1?
>>
>>>
>>> about RelationshipExpander. i don't see how RelationshipIndex could
>>> help combining with RelationshipExpander, when use
>>> GraphAlgoFactory.shortestPath(RelationshipExpander expander, int
>>> maxDepth)?
>>>
>>> thanks for help!
>>>
>>> --
>>> Best regards
>>>
>>> Linan Wang
>>>
>>
>>
>>
>> --
>> Best regards
>>
>> Linan Wang
>> ___
>> Neo4j mailing list
>> User@lists.neo4j.org
>> https://lists.neo4j.org/mailman/listinfo/user
>>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Blocking access to the Neo4j web admin interface
done. https://github.com/neo4j/community/pull/16 On Thu, Sep 8, 2011 at 11:21 PM, Michael Hunger wrote: > Linan, > > your diff didn't make it could you just issue an pull request for that. > > And Peter should get you sign a CLA btw. > > Cheers > > Michael > > Am 08.09.2011 um 18:33 schrieb Linan Wang: > >> tested the idea, it doesn't work. so i made simple changes to the >> server code and diff is attached. >> to change the binding ip of the webserver: add following line to >> conf/neo4j-server.properties: >> org.neo4j.server.webserver.address=127.0.0.1 >> >> On Thu, Sep 8, 2011 at 2:55 PM, Peter Neubauer >> wrote: >>> That sounds good. Could you try it and report back? Would love to add >>> it to the manual and as a setting. >>> >>> Cheers, >>> >>> /peter neubauer >>> >>> GTalk: neubauer.peter >>> Skype peter.neubauer >>> Phone +46 704 106975 >>> LinkedIn http://www.linkedin.com/in/neubauer >>> Twitter http://twitter.com/peterneubauer >>> >>> http://www.neo4j.org - Your high performance graph database. >>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >>> >>> >>> >>> On Thu, Sep 8, 2011 at 3:34 PM, Linan Wang wrote: >>>> since neo4j just uses jetty, i think the simple solution would be add >>>> option in neo4j shell script: >>>> -Djetty.host=127.0.0.1 to make it only listen to local request. then >>>> use ssh tunnel to expose service to designated machines. >>>> >>>> On Thu, Sep 8, 2011 at 2:18 PM, Peter Neubauer >>>> wrote: >>>>> Hi there, >>>>> you can block access to it by blocking the access to the URL >>>>> (localhost:.../webadmin) and even /db/manage. That requires probably >>>>> to set up apache and mod_proxy in front of the Neo4j server, but I >>>>> think that is a good idea in production scenarios anyway. >>>>> >>>>> http://docs.neo4j.org/chunked/snapshot/operations-security.html >>>>> >>>>> Cheers, >>>>> >>>>> /peter neubauer >>>>> >>>>> GTalk: neubauer.peter >>>>> Skype peter.neubauer >>>>> Phone +46 704 106975 >>>>> LinkedIn http://www.linkedin.com/in/neubauer >>>>> Twitter http://twitter.com/peterneubauer >>>>> >>>>> http://www.neo4j.org - Your high performance graph database. >>>>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >>>>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >>>>> >>>>> >>>>> >>>>> On Thu, Sep 8, 2011 at 3:08 PM, carze wrote: >>>>>> I'm making use of the Neo4j REST API to power a website and was >>>>>> wondering if >>>>>> there was any way to block access to the web admin interface. Currently >>>>>> the >>>>>> DB is in read-only mode but the web admin panel is accessibly by anyone >>>>>> who >>>>>> can stumble upon the URL. >>>>>> >>>>>> -- >>>>>> View this message in context: >>>>>> http://neo4j-community-discussions.438527.n3.nabble.com/Blocking-access-to-the-Neo4j-web-admin-interface-tp3319626p3319626.html >>>>>> Sent from the Neo4j Community Discussions mailing list archive at >>>>>> Nabble.com. >>>>>> ___ >>>>>> Neo4j mailing list >>>>>> User@lists.neo4j.org >>>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>>> >>>>> ___ >>>>> Neo4j mailing list >>>>> User@lists.neo4j.org >>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards >>>> >>>> Linan Wang >>>> ___ >>>> Neo4j mailing list >>>> User@lists.neo4j.org >>>> https://lists.neo4j.org/mailman/listinfo/user >>>> >>> ___ >>> Neo4j mailing list >>> User@lists.neo4j.org >>> https://lists.neo4j.org/mailman/listinfo/user >>> >> >> >> >> -- >> Best regards >> >> Linan Wang >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] regarding supernode
On Wed, Sep 7, 2011 at 5:21 PM, Linan Wang wrote:
> hi,
> I don't quite understand RelationshipIndex and RelationshipExpander.
> say I have a supernode city (beijing), it has 10 m users links to
> through relationship LIVES_IN. so how should I index? should be
> something like:
> RelationshipIndex idx = db.index().forRelationships("CITY_LIVES_IN");
> idx.add(rel, "LIVES_IN", "Beijing");
> if so, what's the advantage over this?
> Index idx = db.index().forNodes("CITY_LIVES_IN");
> idx.add(user, "LIVES_IN", "beijing");
> (I read source code of LuceneIndex.java, found out that the
> implementation of the add method is shared between Index and
> RelationshipIndex.)
ok, i answer my own question:
RelationshipIndex has the function query which takes startNode and
endNode as extra parameters.
so if traverse only depth 1, it could be faster than using Traverser.
am i right here? (please say yes!)
then the question is how to take advantage of it for more than 1?
>
> about RelationshipExpander. i don't see how RelationshipIndex could
> help combining with RelationshipExpander, when use
> GraphAlgoFactory.shortestPath(RelationshipExpander expander, int
> maxDepth)?
>
> thanks for help!
>
> --
> Best regards
>
> Linan Wang
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Blocking access to the Neo4j web admin interface
tested the idea, it doesn't work. so i made simple changes to the server code and diff is attached. to change the binding ip of the webserver: add following line to conf/neo4j-server.properties: org.neo4j.server.webserver.address=127.0.0.1 On Thu, Sep 8, 2011 at 2:55 PM, Peter Neubauer wrote: > That sounds good. Could you try it and report back? Would love to add > it to the manual and as a setting. > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://startupbootcamp.org/ - Öresund - Innovation happens HERE. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > > > > On Thu, Sep 8, 2011 at 3:34 PM, Linan Wang wrote: >> since neo4j just uses jetty, i think the simple solution would be add >> option in neo4j shell script: >> -Djetty.host=127.0.0.1 to make it only listen to local request. then >> use ssh tunnel to expose service to designated machines. >> >> On Thu, Sep 8, 2011 at 2:18 PM, Peter Neubauer >> wrote: >>> Hi there, >>> you can block access to it by blocking the access to the URL >>> (localhost:.../webadmin) and even /db/manage. That requires probably >>> to set up apache and mod_proxy in front of the Neo4j server, but I >>> think that is a good idea in production scenarios anyway. >>> >>> http://docs.neo4j.org/chunked/snapshot/operations-security.html >>> >>> Cheers, >>> >>> /peter neubauer >>> >>> GTalk: neubauer.peter >>> Skype peter.neubauer >>> Phone +46 704 106975 >>> LinkedIn http://www.linkedin.com/in/neubauer >>> Twitter http://twitter.com/peterneubauer >>> >>> http://www.neo4j.org - Your high performance graph database. >>> http://startupbootcamp.org/ - Öresund - Innovation happens HERE. >>> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. >>> >>> >>> >>> On Thu, Sep 8, 2011 at 3:08 PM, carze wrote: >>>> I'm making use of the Neo4j REST API to power a website and was wondering >>>> if >>>> there was any way to block access to the web admin interface. Currently the >>>> DB is in read-only mode but the web admin panel is accessibly by anyone who >>>> can stumble upon the URL. >>>> >>>> -- >>>> View this message in context: >>>> http://neo4j-community-discussions.438527.n3.nabble.com/Blocking-access-to-the-Neo4j-web-admin-interface-tp3319626p3319626.html >>>> Sent from the Neo4j Community Discussions mailing list archive at >>>> Nabble.com. >>>> ___ >>>> Neo4j mailing list >>>> User@lists.neo4j.org >>>> https://lists.neo4j.org/mailman/listinfo/user >>>> >>> ___ >>> Neo4j mailing list >>> User@lists.neo4j.org >>> https://lists.neo4j.org/mailman/listinfo/user >>> >> >> >> >> -- >> Best regards >> >> Linan Wang >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Blocking access to the Neo4j web admin interface
since neo4j just uses jetty, i think the simple solution would be add option in neo4j shell script: -Djetty.host=127.0.0.1 to make it only listen to local request. then use ssh tunnel to expose service to designated machines. On Thu, Sep 8, 2011 at 2:18 PM, Peter Neubauer wrote: > Hi there, > you can block access to it by blocking the access to the URL > (localhost:.../webadmin) and even /db/manage. That requires probably > to set up apache and mod_proxy in front of the Neo4j server, but I > think that is a good idea in production scenarios anyway. > > http://docs.neo4j.org/chunked/snapshot/operations-security.html > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://startupbootcamp.org/ - Öresund - Innovation happens HERE. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > > > > On Thu, Sep 8, 2011 at 3:08 PM, carze wrote: >> I'm making use of the Neo4j REST API to power a website and was wondering if >> there was any way to block access to the web admin interface. Currently the >> DB is in read-only mode but the web admin panel is accessibly by anyone who >> can stumble upon the URL. >> >> -- >> View this message in context: >> http://neo4j-community-discussions.438527.n3.nabble.com/Blocking-access-to-the-Neo4j-web-admin-interface-tp3319626p3319626.html >> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] regarding supernode
hi,
I don't quite understand RelationshipIndex and RelationshipExpander.
say I have a supernode city (beijing), it has 10 m users links to
through relationship LIVES_IN. so how should I index? should be
something like:
RelationshipIndex idx = db.index().forRelationships("CITY_LIVES_IN");
idx.add(rel, "LIVES_IN", "Beijing");
if so, what's the advantage over this?
Index idx = db.index().forNodes("CITY_LIVES_IN");
idx.add(user, "LIVES_IN", "beijing");
(I read source code of LuceneIndex.java, found out that the
implementation of the add method is shared between Index and
RelationshipIndex.)
about RelationshipExpander. i don't see how RelationshipIndex could
help combining with RelationshipExpander, when use
GraphAlgoFactory.shortestPath(RelationshipExpander expander, int
maxDepth)?
thanks for help!
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index entry removal after relationship deletion
as far as i know, no. you need to do it yourself. hint: http://docs.neo4j.org/chunked/1.4.1/indexing-update.html On Wed, Sep 7, 2011 at 1:25 AM, Nuo Yan wrote: > Hey guys, > > When a relationship gets deleted, will it be removed from the index > automatically? My test shows yes, but I want to be absolutely certain. > > For example, a relationship :foo exists between node A and B and it's > indexed on the foo_rels index with key node_a's id and value node_b's id. > When I delete this relationship, do I also have to manually remove it from > the index (i.e. a DELETE to /index/relationship/foo_rels/15/23/100 where 15 > is the key, 23 is the value, 100 is the relationship id)? > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 10 questions
thank you very much for the detailed replied. i'm wondering if neo4j had considered the option to use redis or other key-value store for storing properties and focus upon pure graph, similar to the role Lucene plays. i assume the major problem is the performance problem due to address lookup process inside redis value reading and communication overhead. but the gain is significant in scalability. just some random thoughts. On Fri, Sep 2, 2011 at 9:50 PM, Peter Neubauer wrote: > Linan, > see inline ... > > On Fri, Sep 2, 2011 at 9:01 PM, Linan Wang wrote: > >> is it >> https://github.com/peterneubauer/graph-collections/wiki/Indexed-relationships >> ? >> seems not included in the current stable ver. >> > No, this is work-in-progress, we are quite strict when it come sto including > things into the official release, since we need to test and document it > better. So, feel free to test it, a lot of code in there is stable, and > expect to fork and contribute if you find things to fix! > > >> >> 4, what's the best practice to do bulk insertion when running (not >> >> seed initial data)? i read post says that too many insertions within a >> >> transaction may lead to memory problem? what's the proper mount of >> >> insertion within a transaction? >> >> >> > Yes, transaction data is kept in memory before calling commit and >> flushing >> > to disk, so overly large TX might result in memory problems. OTOH small >> TX >> > incur higher IO load. >> i'll probably do it with smaller batches (~1k operations per batch) >> from an external queue. does it sounds reasonable? >> > >> > Yes. From experience, there seems to be a lot of cases where transactions > hold between 1K and 10K operations and give a good performance vs. RAM vs. > persistence balance, if you can afford it for your data. > > >> > >> >> 5, is there a suggested max length for string/array property? would it >> >> be better to put into sql? >> >> >> > Well, the String store block size is adjustable (and we are working on >> even >> > better layouts there), but for big strings like documents, a fiel system >> or >> > Key/Value store might be better, and just keeping the reference to the >> > location makes more sense. >> ok, i'll use redis for strings. >> >> Probably a sensible choice. There might even be an Neo4j index coming out > for Redis, making it transactional with the graph like Lucene. > > >> > >> > 6, say a facebook user may "likes" thousands of things, and these >> >> things are sparsly connected. in this case, things should be modeled >> > >> > as nodes or array property? >> >> >> > Nodes. Sparse connections are one of the places where Neo4j shines - a >> > fairly balanced graph where supernodes are seldom. >> > >> could you give a bottom number qualifies "supernode"? say 1k >> connections within a graph of 1m nodes? >> >> with the current store layout, probably 1K is a good number. We are working > with store changes that require less reads, but don't explicitly take care > of supernodes. The is in plan, in which case this number will change upwards > with good performance :) > >> > >> >> 7, where can i find an example to use domain models with serverplugin? >> >> i want to put my data in a standalone server and just use the >> >> serverplugin, unmanaged extension. should i just put the domain models >> >> into the same serverplugin jar? >> >> >> > Yes, I would do that. However, if you are not expecting to return Nodes, >> > Relationships or Properties, an unmanaged extension will give you the >> full >> > API of REST services. One extension that way is for instance the >> scripting >> > extension, see https://github.com/neo4j/script-extension >> thanks. seems i really should look into github instead of neo4j.org ;) >> > Well, it's hard to list everything, we are right now trying to put as much > as possible into the manual which can be generated, tested and curated, and > serve as a reference. > > >> > >> > Sorry for the delay, hope this helps. Let us know if you have more >> > questions! >> many thanks! i understand documentation is probably not your top >> priority at this point, but since we are all programmers, we can read >> codes. i feel samples on wiki and downloads are not updated to use the >> most recent release. >> > I think what you are seeing is the Wiki getting outdated over time. We are > in the process of moving the Wiki content into docs.neo4j.org, that one is > MUCH better up to date - all code you see in there is generated from running > tests. For real. So I think this will change soon to a point where we can > delete most of the Wiki pages. Sorry for the inconvenience! > > /peter > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 10 questions
yes, it's a smart question! On Fri, Sep 2, 2011 at 4:06 PM, Rick Otten wrote: > Should one make an effort to keep "node 0" from becoming a 'supernode'? > > > -Original Message- > From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On > Behalf Of Steven Kalemkiewicz > Sent: Friday, September 02, 2011 10:58 AM > To: Neo4j user discussions > Subject: Re: [Neo4j] 10 questions > > What would you consider the lower-bound to be to classify a node as a > supernode? I saw that you referred to a city node with 100K relationships... > > -Steve > > On Fri, Sep 2, 2011 at 10:33 AM, Peter Neubauer < > peter.neuba...@neotechnology.com> wrote: > >> > 1, what's the general rule for choosing properties or relationship? >> > say a User lives in a City, which just contains a simple int id >> > value. to find users live in a city, i can do a simple traversal, of >> > all user nodes, or find the city node first, then collect all the >> > users. seems to me both ways work and share same level of performance. >> > (am i right here?) >> > >> Generally, if a number of properties really is denoting the same >> concept (like a city) and you don't want to duplicate the data, and be >> able to traverse or query it, I would introduce nodes. However, if the >> node woudl turn into a supernode (like a city node with 100K >> relationships), then consider introducing an in-graph indexing >> structure, or an out-of-graph external index like Lucene in order to >> look up relationships or nodes when you need them, since that will be >> cheaper. >> >> 6, say a facebook user may "likes" thousands of things, and these >> > things are sparsly connected. in this case, things should be modeled >> > as nodes or array property? >> > >> Nodes. Sparse connections are one of the places where Neo4j shines - a >> fairly balanced graph where supernodes are seldom. >> >> > _______ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 10 questions
On Fri, Sep 2, 2011 at 3:33 PM, Peter Neubauer wrote: > Hi Linan, > trying fast stabs at answers inline before heading home :) > > On Thu, Sep 1, 2011 at 3:29 AM, Linan Wang wrote: > >> hi, >> got some questions not found simple answers from the documents. i bet >> some of them are pretty primitive, bear with me please. >> >> 1, what's the general rule for choosing properties or relationship? >> say a User lives in a City, which just contains a simple int id >> value. to find users live in a city, i can do a simple traversal, of >> all user nodes, or find the city node first, then collect all the >> users. seems to me both ways work and share same level of performance. >> (am i right here?) >> > Generally, if a number of properties really is denoting the same concept > (like a city) and you don't want to duplicate the data, and be able to > traverse or query it, I would introduce nodes. However, if the node woudl > turn into a supernode (like a city node with 100K relationships), then > consider introducing an in-graph indexing structure, or an out-of-graph > external index like Lucene in order to look up relationships or nodes when > you need them, since that will be cheaper. > is it https://github.com/peterneubauer/graph-collections/wiki/Indexed-relationships ? seems not included in the current stable ver. > >> 2, does index operation add/remove/modify threadsafe, don't need >> lock/transaction? >> > Yes, but the index framework is transactional as well as the graph. You need > TX for any modifying operation, but not for reads. > > >> 3, does it simple property writing operations also need to be wrapped >> inside transaction? if so, in the imdb exmaple >> tutor/domain/MovieImpl.java underlyingNode.setProperty is used neither >> within transaction, nor put into a save method, do all setProperty >> works inside a transaction? >> > See Anders reply and above. Got the two. thanks! > > >> 4, what's the best practice to do bulk insertion when running (not >> seed initial data)? i read post says that too many insertions within a >> transaction may lead to memory problem? what's the proper mount of >> insertion within a transaction? >> > Yes, transaction data is kept in memory before calling commit and flushing > to disk, so overly large TX might result in memory problems. OTOH small TX > incur higher IO load. i'll probably do it with smaller batches (~1k operations per batch) from an external queue. does it sounds reasonable? > > >> 5, is there a suggested max length for string/array property? would it >> be better to put into sql? >> > Well, the String store block size is adjustable (and we are working on even > better layouts there), but for big strings like documents, a fiel system or > Key/Value store might be better, and just keeping the reference to the > location makes more sense. ok, i'll use redis for strings. > > 6, say a facebook user may "likes" thousands of things, and these >> things are sparsly connected. in this case, things should be modeled > > as nodes or array property? >> > Nodes. Sparse connections are one of the places where Neo4j shines - a > fairly balanced graph where supernodes are seldom. > could you give a bottom number qualifies "supernode"? say 1k connections within a graph of 1m nodes? > >> 7, where can i find an example to use domain models with serverplugin? >> i want to put my data in a standalone server and just use the >> serverplugin, unmanaged extension. should i just put the domain models >> into the same serverplugin jar? >> > Yes, I would do that. However, if you are not expecting to return Nodes, > Relationships or Properties, an unmanaged extension will give you the full > API of REST services. One extension that way is for instance the scripting > extension, see https://github.com/neo4j/script-extension thanks. seems i really should look into github instead of neo4j.org ;) > > 8, the warning in the documentation about unmanaged extension is >> scary. what i can see is that people may use bad ways, instead of >> Iterator/IteratorWrappers. any comment on this? >> > Yeah. It's just a warning, no sudden death. With that approach, you are > inventing your own API and can do whatever you want, for good and bad. > > >> 9, i'm not sure if it's trival: find out users who are only 2 >> relationships a way (use twitter example: my followees' followers), >> live in same city, group by age and gender. also retrieve all their >> followees. i want to do the traversal in java, where can i find an >> ex
Re: [Neo4j] 10 questions
great. i thought transaction only applies to nodes operations. seems
it also including indexing. it's handy!
other 9 questions? :)
On Fri, Sep 2, 2011 at 12:55 PM, Anders Nawroth
wrote:
> Hi!
>
>>> Seems like the node and index modifications belong in the same
>>> transaction, to make sure any modifications to nodes are always
>>> reflected in the indexes as well. Otherwise they could get out of sync
>>> if your application crashes after the commit of the first,
>>> node-modifying transaction.
>> it seems confusing. if indexing actions wrapped inside a transaction
>> and the transaction fails, will indexing action automatically get
>> rollback? think this example:
>
> Yes, it will be rolled back - that's the point of performing multiple
> operations in the same transaction.
>
> /anders
>
>>
>> class User{
>> public void dosomething(){
>> //node actions
>> //index actions
>> }
>> }
>>
>> class Ext extends ServerPlugin{
>> public action(){
>> // get an array of users;
>> Transaction tx = graphDb.beginTx();
>> try
>> {
>> ... // operations that work with the graph
>> for(User u:users){
>> u.dosomething();
>> }
>> tx.success();
>> }
>> finally
>> {
>> tx.finish();
>> }
>> }
>>
>> }
>>
>>>
>>> /anders
>>>
>>>
>>>>>
>>>>>
>>>>> /anders
>>>>>
>>>>>> 4, what's the best practice to do bulk insertion when running (not
>>>>>> seed initial data)? i read post says that too many insertions within a
>>>>>> transaction may lead to memory problem? what's the proper mount of
>>>>>> insertion within a transaction?
>>>>>> 5, is there a suggested max length for string/array property? would it
>>>>>> be better to put into sql?
>>>>>> 6, say a facebook user may "likes" thousands of things, and these
>>>>>> things are sparsly connected. in this case, things should be modeled
>>>>>> as nodes or array property?
>>>>>> 7, where can i find an example to use domain models with serverplugin?
>>>>>> i want to put my data in a standalone server and just use the
>>>>>> serverplugin, unmanaged extension. should i just put the domain models
>>>>>> into the same serverplugin jar?
>>>>>> 8, the warning in the documentation about unmanaged extension is
>>>>>> scary. what i can see is that people may use bad ways, instead of
>>>>>> Iterator/IteratorWrappers. any comment on this?
>>>>>> 9, i'm not sure if it's trival: find out users who are only 2
>>>>>> relationships a way (use twitter example: my followees' followers),
>>>>>> live in same city, group by age and gender. also retrieve all their
>>>>>> followees. i want to do the traversal in java, where can i find an
>>>>>> examples?
>>>>>> 10, i've had horrible experience in turning jvm options. have neo4j
>>>>>> been running on Zing JVM, hp nonstop jvm? are they better options?
>>>>>>
>>>>>> thanks in advance
>>>>>>
>>>>>>
>>>>>> Best regards
>>>>>>
>>>>>> Linan Wang
>>>>>> ___
>>>>>> Neo4j mailing list
>>>>>> User@lists.neo4j.org
>>>>>> https://lists.neo4j.org/mailman/listinfo/user
>>>>> ___
>>>>> Neo4j mailing list
>>>>> User@lists.neo4j.org
>>>>> https://lists.neo4j.org/mailman/listinfo/user
>>>>>
>>>>
>>>>
>>>>
>>> ___
>>> Neo4j mailing list
>>> User@lists.neo4j.org
>>> https://lists.neo4j.org/mailman/listinfo/user
>>>
>>
>>
>>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 10 questions
hi
On Fri, Sep 2, 2011 at 10:18 AM, Anders Nawroth
wrote:
> Hi!
>
>>> All modifying operations need to be performed inside a transaction. In
>>> most cases it makes sense to perform multiple operations in a single
>>> transaction. For example in a web application it may be a good fit to
>>> wrap the handling of one request in a transaction. So if a method
>>> doesn't start a transaction, it just means that it's handled at a higher
>>> level in the application.
>> got this part. then do you suggest to leave indexing action outside of
>> node operation? say i have a domain model and it handles indexing
>> actions along with underlying nodes modification, then in an web app,
>> should I wrap only nodes operations inside transaction and do indexing
>> only after it success?
>
> Seems like the node and index modifications belong in the same
> transaction, to make sure any modifications to nodes are always
> reflected in the indexes as well. Otherwise they could get out of sync
> if your application crashes after the commit of the first,
> node-modifying transaction.
it seems confusing. if indexing actions wrapped inside a transaction
and the transaction fails, will indexing action automatically get
rollback? think this example:
class User{
public void dosomething(){
//node actions
//index actions
}
}
class Ext extends ServerPlugin{
public action(){
// get an array of users;
Transaction tx = graphDb.beginTx();
try
{
... // operations that work with the graph
for(User u:users){
u.dosomething();
}
tx.success();
}
finally
{
tx.finish();
}
}
}
>
> /anders
>
>
>>>
>>>
>>> /anders
>>>
>>>> 4, what's the best practice to do bulk insertion when running (not
>>>> seed initial data)? i read post says that too many insertions within a
>>>> transaction may lead to memory problem? what's the proper mount of
>>>> insertion within a transaction?
>>>> 5, is there a suggested max length for string/array property? would it
>>>> be better to put into sql?
>>>> 6, say a facebook user may "likes" thousands of things, and these
>>>> things are sparsly connected. in this case, things should be modeled
>>>> as nodes or array property?
>>>> 7, where can i find an example to use domain models with serverplugin?
>>>> i want to put my data in a standalone server and just use the
>>>> serverplugin, unmanaged extension. should i just put the domain models
>>>> into the same serverplugin jar?
>>>> 8, the warning in the documentation about unmanaged extension is
>>>> scary. what i can see is that people may use bad ways, instead of
>>>> Iterator/IteratorWrappers. any comment on this?
>>>> 9, i'm not sure if it's trival: find out users who are only 2
>>>> relationships a way (use twitter example: my followees' followers),
>>>> live in same city, group by age and gender. also retrieve all their
>>>> followees. i want to do the traversal in java, where can i find an
>>>> examples?
>>>> 10, i've had horrible experience in turning jvm options. have neo4j
>>>> been running on Zing JVM, hp nonstop jvm? are they better options?
>>>>
>>>> thanks in advance
>>>>
>>>>
>>>> Best regards
>>>>
>>>> Linan Wang
>>>> ___
>>>> Neo4j mailing list
>>>> User@lists.neo4j.org
>>>> https://lists.neo4j.org/mailman/listinfo/user
>>> ___
>>> Neo4j mailing list
>>> User@lists.neo4j.org
>>> https://lists.neo4j.org/mailman/listinfo/user
>>>
>>
>>
>>
> ___
> Neo4j mailing list
> User@lists.neo4j.org
> https://lists.neo4j.org/mailman/listinfo/user
>
--
Best regards
Linan Wang
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 10 questions
Hi anders, thanks for the clarification. On Fri, Sep 2, 2011 at 8:32 AM, Anders Nawroth wrote: > Hi! > > 2011-09-01 03:29, Linan Wang: >> 2, does index operation add/remove/modify threadsafe, don't need >> lock/transaction? >> 3, does it simple property writing operations also need to be wrapped >> inside transaction? if so, in the imdb exmaple >> tutor/domain/MovieImpl.java underlyingNode.setProperty is used neither >> within transaction, nor put into a save method, do all setProperty >> works inside a transaction? > > All modifying operations need to be performed inside a transaction. In > most cases it makes sense to perform multiple operations in a single > transaction. For example in a web application it may be a good fit to > wrap the handling of one request in a transaction. So if a method > doesn't start a transaction, it just means that it's handled at a higher > level in the application. got this part. then do you suggest to leave indexing action outside of node operation? say i have a domain model and it handles indexing actions along with underlying nodes modification, then in an web app, should I wrap only nodes operations inside transaction and do indexing only after it success? > > > /anders > >> 4, what's the best practice to do bulk insertion when running (not >> seed initial data)? i read post says that too many insertions within a >> transaction may lead to memory problem? what's the proper mount of >> insertion within a transaction? >> 5, is there a suggested max length for string/array property? would it >> be better to put into sql? >> 6, say a facebook user may "likes" thousands of things, and these >> things are sparsly connected. in this case, things should be modeled >> as nodes or array property? >> 7, where can i find an example to use domain models with serverplugin? >> i want to put my data in a standalone server and just use the >> serverplugin, unmanaged extension. should i just put the domain models >> into the same serverplugin jar? >> 8, the warning in the documentation about unmanaged extension is >> scary. what i can see is that people may use bad ways, instead of >> Iterator/IteratorWrappers. any comment on this? >> 9, i'm not sure if it's trival: find out users who are only 2 >> relationships a way (use twitter example: my followees' followers), >> live in same city, group by age and gender. also retrieve all their >> followees. i want to do the traversal in java, where can i find an >> examples? >> 10, i've had horrible experience in turning jvm options. have neo4j >> been running on Zing JVM, hp nonstop jvm? are they better options? >> >> thanks in advance >> >> >> Best regards >> >> Linan Wang >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 10 questions
Thanks! The Matrix example I was referring to http://docs.neo4j.org/chunked/stable/indexing-add.html Sent from my iPad On 1 Sep 2011, at 21:21, Peter Neubauer wrote: > Hi there, > would love to answer but had no time today, will try tomorrow. Also, very > valid points about the docs, will try to put as much of the answers into the > docs as possible. What Matrix example are you looking at? > > Cheers, > > /peter neubauer > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Your high performance graph database. > http://startupbootcamp.org/- Öresund - Innovation happens HERE. > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. > > > On Thu, Sep 1, 2011 at 10:10 PM, wangii wrote: > >> anyone show some love ;) >> seriously, the product is great but not the documentation. e.g. about the >> general rule for choosing property or relationship: in the matrix example, >> the year of a movie should be modelled as relationship since every year >> lots >> of movies are produced. >> >> -- >> View this message in context: >> http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-10-questions-tp3300093p3302418.html >> Sent from the Neo4j Community Discussions mailing list archive at >> Nabble.com. >> ___ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] 10 questions
hi, got some questions not found simple answers from the documents. i bet some of them are pretty primitive, bear with me please. 1, what's the general rule for choosing properties or relationship? say a User lives in a City, which just contains a simple int id value. to find users live in a city, i can do a simple traversal, of all user nodes, or find the city node first, then collect all the users. seems to me both ways work and share same level of performance. (am i right here?) 2, does index operation add/remove/modify threadsafe, don't need lock/transaction? 3, does it simple property writing operations also need to be wrapped inside transaction? if so, in the imdb exmaple tutor/domain/MovieImpl.java underlyingNode.setProperty is used neither within transaction, nor put into a save method, do all setProperty works inside a transaction? 4, what's the best practice to do bulk insertion when running (not seed initial data)? i read post says that too many insertions within a transaction may lead to memory problem? what's the proper mount of insertion within a transaction? 5, is there a suggested max length for string/array property? would it be better to put into sql? 6, say a facebook user may "likes" thousands of things, and these things are sparsly connected. in this case, things should be modeled as nodes or array property? 7, where can i find an example to use domain models with serverplugin? i want to put my data in a standalone server and just use the serverplugin, unmanaged extension. should i just put the domain models into the same serverplugin jar? 8, the warning in the documentation about unmanaged extension is scary. what i can see is that people may use bad ways, instead of Iterator/IteratorWrappers. any comment on this? 9, i'm not sure if it's trival: find out users who are only 2 relationships a way (use twitter example: my followees' followers), live in same city, group by age and gender. also retrieve all their followees. i want to do the traversal in java, where can i find an examples? 10, i've had horrible experience in turning jvm options. have neo4j been running on Zing JVM, hp nonstop jvm? are they better options? thanks in advance Best regards Linan Wang ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
