Sorry to reply to this old thread, but I'm actually facing the situation you're describing at the end: having possibly many rels to traverse. As you mentioned, the fix in 2.1 won't fix this and there is the workaround you describe above or even, as Mark Needham and Cédric Fauvet told me, the possibility of fanning out such dense relationships with Max de Marzi unmanaged extension: https://github.com/maxdemarzi/dense.
Do you know if this issue will still be tackled in 2.2? Would you mind sharing some insights (like the approach currently considered by the engineering team)? This may be a very naive outlook on the matter, but wouldn't it be possible to actually implement the fanning out (as done with the unmanaged extension of Max de Marzi) at the storage level and introduce the notion of paging into core APIs? This notion of page, as in the unmanaged extension, is currently taking the shape of a relationship depth variable. With no specified page (or ranges of page), this would remain backwards-compatible: all relationships would be traversed, else only the ones in the specified page. With guaranteed backwards-compatibility, people who currently don't need to worry about this still wouldn't need to. People in need of this wouldn't a workaround anymore. Again, I'm aware this has huge impacts anyway and I may miss many others. But I'm really curious about the approaches that are considered. Sorry if that does not make sense at all. Le mardi 7 janvier 2014 15:07:55 UTC+1, Michael Hunger a écrit : > > Jason, > > we want to add a better internal structure for these dense nodes in Neo4j > 2.1 then it will be handled internally. > > These workarounds are only until then. > > Auto-indexes or legacy-indexes of relationships would allow you to quickly > access your type1 relationships even if all the other relationships have > not yet been loaded from disk. > You would kind of skip the graph navigation for this node then and instead > look up the type1 relationships from the index and follow them to their > end-nodes. > > But whenever you would navigate across your domain node you'd be hit again > by the large number of rels. So the separation out to a "dense-meta-node" > (I just made that up). is the better general solution. > > You can even evolve that solution one step further by adding a tree of > type2 relationships starting at the "dense-meta-node" which is filled > incrementally and whose subtrees can be accessed by an additional selection > criteria (like a property). > > > Am 07.01.2014 um 14:54 schrieb Jason W <ja...@genebygene.com <javascript:> > >: > > Thanks Michael - I understood your workaround and see that it would work. > > However, I wanted to confirm that neo4j does not have native ability to > filter quickly by relationship label? I just did some more testing using > the "profile" statement, and it looks like it is doing some native > filtering by relationship label. Also, can auto indexes or legacy indexes > be useful here? > > > On Tuesday, January 7, 2014 1:49:16 AM UTC-6, Michael Hunger wrote: >> >> I meant something like this: >> >> >> Am 07.01.2014 um 07:11 schrieb Jason W <ja...@genebygene.com>: >> >> Michael, >> What do you mean by filter on the relationship type? I was under the >> impression that all relationships for a node are in the same linked list >> and there's no fast way to filter for the relationship type without >> traversing through the whole linked list. >> >> >> >> On Monday, January 6, 2014 5:43:13 AM UTC-6, Michael Hunger wrote: >>> >>> Jason, >>> >>> The release is afaik planned either for some time in Q2 2014. >>> >>> actually the workarounds are not so difficult. >>> >>> The easiest one is to add a separate node which holds the 1M >>> relationships. So your actual node has just 1+100, the 100 that you >>> mentioned of type 2 and the one to the separate node. >>> >>> So you can ignore the 1M relationships most (or all of the time) from >>> this direction and only check from the other side. >>> >>> Wouldn't you filter on the relationship-type if you are only looking for >>> active / inactive ? >>> >>> Cheers >>> >>> Michael >>> >>> Am 05.01.2014 um 23:38 schrieb Jason W <ja...@genebygene.com>: >>> >>> Michael, >>> This is a major performance bottleneck for me as I have nodes that may >>> have 1M relationships of type 1, but only a hundred relationships of type >>> 2. Traversal performance for type 2 will be brought down due to the type 1 >>> relationships. I've read through the archives and it seems like this was >>> being worked on over a year ago, but was never released. This is the main >>> issue preventing me from purchasing an enterprise license, as I cannot >>> commit to production use with this limitation. When is the planned release >>> data on your roadmap? >>> >>> My concrete use case is measuring cellular interactions in different >>> people. For example, one person may have 1 million active cells, but only >>> several hundred inactive cells. I need to be able to quickly query out the >>> active and inactive cells with some filtering based on properties. Some >>> type of index on the relationship that can help me quickly filter down to >>> the cells I'm interested in. I also need to query out which active/inactive >>> cells are shared between groups of people. >>> >>> >>> >>> On Sunday, January 5, 2014 7:09:34 AM UTC-6, Michael Hunger wrote: >>>> >>>> This will be addressed in Neo4j 2.1 or 2.2 depending on the roadmap >>>> >>>> What is your concrete use-case? >>>> >>>> There are some ways to mitigate it manually for now depending on your >>>> use-cases >>>> >>>> Eg introduce a hierarchy or a separate node for the large >>>> relationship-set >>>> >>>> Or index certain rels that are important to access quickly in a legacy >>>> relationsship index >>>> >>>> >>>> >>>> >>>> >>>> Sent from mobile device >>>> >>>> Am 05.01.2014 um 13:42 schrieb Stefan Armbruster <ml...@ >>>> armbruster-it.de>: >>>> >>>> Your understanding here is correct. >>>> In your example: Given a node with 10E7 relationships and you want to >>>> know if it is connected to node x. If you know that x has significant >>>> lesser relationships you can reverse the traversal direction and check if >>>> x >>>> is connected to a. This will be cheaper as only the smaller linked list of >>>> node x needs to be iterated. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Neo4j" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to neo4j+un...@googlegroups.com. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> >>> >>> >>> This message contains confidential information and is intended only for >>> the individual named. If you are not the named addressee you should not >>> disseminate, distribute or copy this e-mail. Please notify the sender >>> immediately by e-mail if you have received this e-mail by mistake and >>> delete this e-mail from your system. If you are not the intended recipient >>> you are notified that disclosing, copying, distributing or taking any >>> action in reliance on the contents of this information is strictly >>> prohibited. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Neo4j" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to neo4j+un...@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> >> >> >> This message contains confidential information and is intended only for >> the individual named. If you are not the named addressee you should not >> disseminate, distribute or copy this e-mail. Please notify the sender >> immediately by e-mail if you have received this e-mail by mistake and >> delete this e-mail from your system. If you are not the intended recipient >> you are notified that disclosing, copying, distributing or taking any >> action in reliance on the contents of this information is strictly >> prohibited. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to neo4j+un...@googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > > This message contains confidential information and is intended only for > the individual named. If you are not the named addressee you should not > disseminate, distribute or copy this e-mail. Please notify the sender > immediately by e-mail if you have received this e-mail by mistake and > delete this e-mail from your system. If you are not the intended recipient > you are notified that disclosing, copying, distributing or taking any > action in reliance on the contents of this information is strictly > prohibited. > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to neo4j+un...@googlegroups.com <javascript:>. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
