Thank you, I was just now able to reproduce this!
2014-03-17 11:28 GMT+01:00 Sotiris Beis <sot.b...@gmail.com>:
> Hi Mattias,
>
> here is some more details:
>
> - The function to create the graph
> public void createGraphForMassiveLoad(String dbPath) {
> System.out.println("Creating Neo4j Graph Database for massive load . . .
> .");
> Map<String, String> config = new HashMap<String, String>();
> config.put("cache_type", "none");
> config.put("use_memory_mapped_buffers", "true");
> config.put("neostore.nodestore.db.mapped_memory", "200M");
> config.put("neostore.relationshipstore.db.mapped_memory", "1000M");
> config.put("neostore.propertystore.db.mapped_memory", "250M");
> config.put("neostore.propertystore.db.strings.mapped_memory", "250M");
> inserter = BatchInserters.inserter(dbPath, config);
> indexProvider = new LuceneBatchInserterIndexProvider(inserter);
> nodes = indexProvider.nodeIndex("nodes", MapUtil.stringMap("type",
> "exact"));
> }
>
> - The code to load the data
> public void createGraph(String datasetDir) {
> System.out.println("Loading data in massive mode in Neo4j database . . .
> .");
>
> inserter.createDeferredSchemaIndex(Neo4jGraphDatabase.NODE_LABEL).on("nodeId").create();
> try {
> BufferedReader reader = new BufferedReader(new InputStreamReader(new
> FileInputStream(datasetDir)));
> String line;
> int lineCounter = 1;
> // Map<String, Object> properties;
> // IndexHits<Long> cache;
> long srcNode, dstNode;
> while((line = reader.readLine()) != null) {
> if(lineCounter > 4) {
> String[] parts = line.split("\t");
> srcNode = getOrCreate(parts[0]);
> dstNode = getOrCreate(parts[1]);
> inserter.createRelationship(srcNode, dstNode,
> Neo4jGraphDatabase.RelTypes.SIMILAR, null);
> }
> lineCounter++;
> }
> reader.close();
> }
> catch (IOException e) {
> e.printStackTrace();
> }
> nodes.flush();
> }
> private long getOrCreate(String value) {
> Long id = cache.get(Long.valueOf(value));
> if(id == null) {
> Map<String, Object> properties = MapUtil.map("nodeId", value);
> id = inserter.createNode(properties, Neo4jGraphDatabase.NODE_LABEL);
> cache.put(Long.valueOf(value), id);
> nodes.add(id, properties);
> }
> return id;
> }
>
> - The dataset I made the tests:
> http://snap.stanford.edu/data/email-Enron.html
>
> Unfortunately the database is not available right now, but it's pretty
> easy to reproduce with the above code.
>
> Thanks,
> Sotiris
>
>
> Τη Κυριακή, 16 Μαρτίου 2014 4:48:16 μ.μ. UTC+2, ο χρήστης Mattias Persson
> έγραψε:
>>
>> Yup, it's probably a regression introduced in the recent store format
>> changes. I'd love to track this and be able to reproduce it. How were the
>> relationships added to the node? Distribution of types/directions and also
>> in which order they were added. Could you provide detailed information
>> about that, or provide the database zipped up to me directly (
>> mat...@neotechnology.com) ?
>>
>> Thanks in advance
>>
>>
>> 2014-03-13 12:26 GMT+01:00 Sotiris Beis <sot....@gmail.com>:
>>
>>> Ok, can you suggest me another temporary solution?
>>>
>>>
>>> On 03/13/2014 01:24 PM, Michael Hunger wrote:
>>>
>>> Thanks for the feedback. Could be related to the changes in the store
>>> format for heavily connected nodes.
>>>
>>> We'll investigate.
>>>
>>> Cheers,
>>>
>>> Michael
>>>
>>> ----
>>> (michael <http://twitter.com/mesirii>)-[:SUPPORTS]->(*YOU*)-[:USE]->(
>>> Neo4j <http://neo4j.org>)
>>> Learn Online <http://neo4j.org/learn/online_course>,
>>> Offline<http://www.neo4j.org/events> or
>>> Read a Book <http://graphdatabases.com> (in Deutsch<http://bit.ly/das-buch>
>>> )
>>> We're trading T-shirts for cool Graph Models <http://bit.ly/graphgist>
>>>
>>>
>>>
>>> Am 13.03.2014 um 12:22 schrieb Sotiris Beis <sot....@gmail.com>:
>>>
>>> Which version of Neo4j are you using?
>>>
>>> I use neo4j-2.1.0-M01
>>>
>>> You use a Set which elminates duplicates. You probably have duplicate
>>> neighbourId's that are only 100 distinct ones.
>>>
>>> That was my first thought, but I cheched it. There are no dublicates.
>>> Why do you think the result is different when I use this test line
>>>
>>> System.out.println(IteratorUtil.count(n.getRelationships(Direction.
>>> OUTGOING)));
>>>
>>> before the iteration of the relationships?
>>>
>>> n 03/13/2014 01:00 PM, Michael Hunger wrote:
>>>
>>> Which version of Neo4j are you using?
>>>
>>> You use a Set which elminates duplicates. You probably have duplicate
>>> neighbourId's that are only 100 distinct ones.
>>>
>>> And you close the transaction twice. It is an auto-closable resource
>>> so you can remove your manual tx.close() line.
>>>
>>> Cheers,
>>>
>>> Michael
>>>
>>> ----
>>> (michael <http://twitter.com/mesirii>)-[:SUPPORTS]->(*YOU*)-[:USE]->(
>>> Neo4j <http://neo4j.org/>)
>>> Learn Online <http://neo4j.org/learn/online_course>,
>>> Offline<http://www.neo4j.org/events> or
>>> Read a Book <http://graphdatabases.com/> (in Deutsch<http://bit.ly/das-buch>
>>> )
>>> We're trading T-shirts for cool Graph Models <http://bit.ly/graphgist>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Am 13.03.2014 um 11:21 schrieb Sotiris Beis <sot....@gmail.com>:
>>>
>>> I have the following case study. I want to get the neighbors of the
>>> certain node. My function is this one:
>>> public Set<Integer> getNeighborsIds(int nodeId) {
>>> Set<Integer> neighbours = new HashSet<Integer>();
>>> try (Transaction tx = neo4jGraph.beginTx()) {
>>> Node n = nodeIndex.get("nodeId", nodeId).getSingle();
>>> for(Relationship relationship : n.getRelationships(Direction.OUTGOING))
>>> {
>>> Node neighbour = relationship.getEndNode();
>>> String neighbourId = (String)neighbour.getProperty("nodeId");
>>> neighbours.add(Integer.valueOf(neighbourId));
>>> }
>>> tx.success();
>>> tx.close();
>>> }
>>> return neighbours;
>>> }
>>>
>>> I know that this node has 255 neighbors but this function return only
>>> 100 nodes. When I put this line
>>> System.out.println(IteratorUtil.count(n.getRelationships(Direction.
>>> OUTGOING)));
>>>
>>>
>>> 4 and 5 line the function returns 255 neighbors. The most strange is
>>> that the above function prints out the value 100. Is this a bug or can
>>> anyone explain this to me?
>>>
>>> Thanks,
>>> Sotiris
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Neo4j" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to neo4j+un...@googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Neo4j" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/
>>> topic/neo4j/stHamJpQSBk/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> neo4j+un...@googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Neo4j" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to neo4j+un...@googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Neo4j" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/
>>> topic/neo4j/stHamJpQSBk/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> neo4j+un...@googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Neo4j" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to neo4j+un...@googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Mattias Persson, [mat...@neotechnology.com]
>> Hacker, Neo Technology
>> www.neotechnology.com
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to neo4j+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to neo4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
