Hi,
Note that I have fixed the nested group() OLAP NPE that is raised in 3.2.0. The
work is currently in the PR below awaiting VOTEs for merge.
https://github.com/apache/incubator-tinkerpop/pull/318
The following two new test cases exposed your NPE problem as well as a
serialization problem for "over-the-wire"-based OLAP engines.
https://github.com/apache/incubator-tinkerpop/pull/318/files#diff-fdac84746ff6c6d5e230b8a3c791e2ccR508
My original solution to this problem yielded poor performance (5x worse than
master/), but after some "ah ha!"-moments, I was able to arrive at the same
performance of master/.
https://github.com/apache/incubator-tinkerpop/pull/318#issuecomment-221280324
HTH,
Marko.
http://markorodriguez.com
On May 20, 2016, at 7:19 AM, Marko Rodriguez <[email protected]> wrote:
> Hi Jen,
>
> Note that that exception is no longer a thing in 3.2.0. You can do
> mid-traversal barriers in 3.2.0. Next, as Daniel pointed out earlier in the
> thread, your particular query in 3.2.0 gives an NPE :) ! However, I know why
> and will have it fixed for 3.2.1
>
> https://issues.apache.org/jira/browse/TINKERPOP-1307
> ** I was dumb about a transient in a serialization.
>
> Take care,
> Marko.
>
> http://markorodriguez.com
>
> On May 20, 2016, at 5:09 AM, Daniel Kuppitz <[email protected]> wrote:
>
>> Hi Jen,
>>
>> would it be sufficient to filter right at the beginning based on the number
>> of out-edges?
>>
>> g.V().hasLabel('song').filter(outE('followedBy').count().is(gt(10))).match(
>> ...
>>
>> Cheers,
>> Daniel
>>
>>
>>
>>
>> On Fri, May 20, 2016 at 10:48 AM, Jen <[email protected]> wrote:
>> When I try Daniel's query in 3.1.1 OLAP Gremlin (Spark) I get the classic
>> error:
>> Global traversals on GraphComputer may not contain mid-traversal barriers:
>> GroupStep([SelectOneStep(songName)],[GroupStep([SelectOneStep(followedName)],[SelectOneStep(sungBy),
>> FoldStep])])
>>
>> Any idea if there's a way to do this in OLAP gremlin? I can't really use
>> OLTP for my query.
>>
>> Jen
>>
>> On Thursday, May 19, 2016 at 9:41:04 PM UTC+1, Daniel Kuppitz wrote:
>> The nested grouping throws exceptions in 3.2.1 (I believe it were
>> NoSuchElementExceptions).
>>
>> On Thu, May 19, 2016 at 10:22 PM, Marko Rodriguez <[email protected]> wrote:
>> @Daniel: Why doesn't it work in OLAP?
>>
>> Marko.
>>
>> http://markorodriguez.com
>>
>> On May 19, 2016, at 1:42 PM, Daniel Kuppitz <[email protected]> wrote:
>>
>>> Hi Jen,
>>>
>>> you need to unfold() the outer map in order to filter its entries. This
>>> query worked for me in 3.2.1-SNAPSHOT:
>>>
>>> g.V().hasLabel('song').match(
>>> __.as('songV').values('name').as('songName'),
>>> __.as('songV').out('followedBy').as('followedV'),
>>> __.as('followedV').values('name').as('followedName'),
>>> __.as('followedV').out('sungBy').as('sungByV'),
>>> __.as('sungByV').values('name').as('sungBy'),
>>> __.as('followedV').out('writtenBy').as('writtenByV'),
>>> __.as('writtenByV').values('name').as('writtenBy')
>>> ).select('songName','followedName','sungBy','writtenBy').
>>> group().by(select('songName')).
>>> by(group().by(select('followedName')).
>>> by(select('sungBy').fold())).unfold().
>>> filter(select(values).count(local).is(gt(10)))
>>>
>>> ... only in OLTP though.
>>>
>>> Cheers,
>>> Daniel
>>>
>>>
>>> On Thu, May 19, 2016 at 6:49 PM, Jen <[email protected]> wrote:
>>> Hi,
>>>
>>> I have a question about filtering a nested group statement in OLAP (Spark)
>>> Gremlin with Tinkerpop 3.1.1, related to another recent question of mine:
>>> https://groups.google.com/d/msg/gremlin-users/of6cV5gGAoE/Hr9Szu-OAgAJ
>>>
>>> I have created a nested group statement, like this example with the
>>> Grateful Dead data:
>>> resMap = g.V().hasLabel('song').match(
>>> __.as('songV').values('name').as('songName'),
>>> __.as('songV').out('followedBy').as('followedV'),
>>> __.as('followedV').values('name').as('followedName'),
>>> __.as('followedV').out('sungBy').as('sungByV'),
>>> __.as('sungByV').values('name').as('sungBy'),
>>> __.as('followedV').out('writtenBy').as('writtenByV'),
>>> __.as('writtenByV').values('name').as('writtenBy')).
>>> select('songName','followedName','sungBy','writtenBy').
>>> group().by(select('songName')).by(group().by(select('followedName')).by(select('sungBy').fold())).next()
>>> Which returns results like:
>>> ==>WE BID YOU GOODNIGHT={FOOLISH HEART=[Garcia], SHAKEDOWN STREET=[Garcia],
>>> BEAT IT ON DOWN THE LINE=[Weir], HELL IN A BUCKET=[Weir_Mydland], PROMISED
>>> LAND=[Weir], MEXICALI BLUES=[Weir], TOUCH OF GREY=[Garcia], JACK
>>> STRAW=[Weir], MORNING DEW=[Garcia], COLD RAIN AND SNOW=[Garcia], BIG
>>> RAILROAD BLUES=[Garcia], FEEL LIKE A STRANGER=[Weir]}
>>> ==>I WANT YOU={BALLAD OF A THIN MAN=[Weir]}
>>> ==>STIR IT UP={NEW MINGLEWOOD BLUES=[Weir], DRUMS=[Grateful_Dead]}
>>> ==>SALT LAKE CITY={FRIEND OF THE DEVIL=[Garcia_Dawson]}
>>> ==>WILLIE AND THE HAND JIVE={CANDYMAN=[Garcia], IKO IKO=[Garcia], THE
>>> WHEEL=[Garcia_Kreutzmann], ROW JIMMY=[Garcia], GOING DOWN THE ROAD FEELING
>>> BAD=[Garcia], DRUMS=[Grateful_Dead]}
>>> ==>I NEED A MIRACLE={FOOLISH HEART=[Garcia], CUMBERLAND
>>> BLUES=[Garcia_Lesh], DARK STAR=[Garcia], BLACK PETER=[Garcia], CHINA CAT
>>> SUNFLOWER=[Garcia], COMES A TIME=[Garcia], TOUCH OF GREY=[Garcia], TERRAPIN
>>> STATION=[Garcia], MORNING DEW=[Garcia], DRUMS=[Grateful_Dead], WHARF
>>> RAT=[Garcia], GOOD LOVING=[Pigpen_Weir_Mydland], EYES OF THE
>>> WORLD=[Garcia], WANG DANG DOODLE=[Weir], MAGGIES FARM=[All], HERE COMES
>>> SUNSHINE=[Garcia], BROWN EYED WOMEN=[Garcia], SHAKEDOWN STREET=[Garcia],
>>> MIGHT AS WELL=[Garcia], PLAYING IN THE BAND=[Weir_Hart], DEATH DONT HAVE NO
>>> MERCY=[Garcia], NEW MINGLEWOOD BLUES=[Weir], HEY JUDE=[Pigpen_Mydland],
>>> SCARLET BEGONIAS=[Garcia], AROUND AND AROUND=[Weir], STANDER ON THE
>>> MOUNTAIN=[Hornsby, Hornsby, Hornsby, Hornsby], STANDING ON THE
>>> MOON=[Garcia], THROWING STONES=[Weir], CASEY JONES=[Garcia], SUNSHINE
>>> DAYDREAM=[Weir], UNCLE JOHNS BAND=[Garcia], HES GONE=[Garcia], SHE BELONGS
>>> TO ME=[Weir_Garcia], ITS ALL OVER NOW=[Pigpen_Weir], GOING DOWN THE ROAD
>>> FEELING BAD=[Garcia], THAT WOULD BE SOMETHING=[Garcia], STAGGER
>>> LEE=[Garcia], STELLA BLUE=[Garcia], BERTHA=[Garcia], CRAZY
>>> FINGERS=[Garcia], ATTICS OF MY LIFE=[Garcia], THE
>>> WHEEL=[Garcia_Kreutzmann], SO MANY ROADS=[Garcia], CHINA DOLL=[Garcia],
>>> JAM=[Grateful_Dead], COLD RAIN AND SNOW=[Garcia], LAZY LIGHTNING=[Weir],
>>> SHIP OF FOOLS=[Garcia]}
>>>
>>> Now I would like to add a filter in the group statement returning only
>>> results where there are more than 10 songs in a "followedBy" group for a
>>> single result row. For example, these results would be filtered out:
>>> ==>HARD TO HANDLE={TELL MAMA=[Etta_James]}
>>> ==>WHY DONT WE DO IT IN THE ROAD={CHINA CAT SUNFLOWER=[Garcia], AROUND AND
>>> AROUND=[Weir], DRUMS=[Grateful_Dead], STELLA BLUE=[Garcia], BERTHA=[Garcia]}
>>> but these results would not:
>>> ==>SUNSHINE DAYDREAM={SAMSON AND DELILAH=[Weir], HELL IN A
>>> BUCKET=[Weir_Mydland], ONE MORE SATURDAY NIGHT=[Weir], TOUCH OF
>>> GREY=[Garcia], BIRDSONG=[Garcia], WHARF RAT=[Garcia], GOOD
>>> LOVING=[Pigpen_Weir_Mydland], BROKEDOWN PALACE=[Garcia], FEEL LIKE A
>>> STRANGER=[Weir], SHAKEDOWN STREET=[Garcia], IKO IKO=[Garcia], DONT EASE ME
>>> IN=[Garcia], AROUND AND AROUND=[Weir], FOREVER YOUNG=[Neil_Young], COLD
>>> RAIN AND SNOW=[Garcia], TURN ON YOUR LOVE LIGHT=[Pigpen_Weir], BOX OF
>>> RAIN=[Lesh], SHIP OF FOOLS=[Garcia]}
>>> ==>GOING DOWN THE ROAD FEELING BAD={BLACK PETER=[Garcia], HELL IN A
>>> BUCKET=[Weir_Mydland], JOHNNY B GOODE=[Weir], NOT FADE AWAY=[Weir],
>>> PROMISED LAND=[Weir], MORNING DEW=[Garcia], BIRDSONG=[Garcia],
>>> DRUMS=[Grateful_Dead], WHARF RAT=[Garcia], GOOD
>>> LOVING=[Pigpen_Weir_Mydland], EYES OF THE WORLD=[Garcia], PLAYING IN THE
>>> BAND=[Weir_Hart], NEW MINGLEWOOD BLUES=[Weir], JACK STRAW=[Weir], BABY WHAT
>>> YOU WANT ME TO DO=[Pigpen_Mydland], AROUND AND AROUND=[Weir], STANDING ON
>>> THE MOON=[Garcia], TURN ON YOUR LOVE LIGHT=[Pigpen_Weir], THROWING
>>> STONES=[Weir], CASEY JONES=[Garcia], SUNSHINE DAYDREAM=[Weir], UNCLE JOHNS
>>> BAND=[Garcia], SUGAR MAGNOLIA=[Weir], ONE MORE SATURDAY NIGHT=[Weir],
>>> ALABAMA GETAWAY=[Garcia], STELLA BLUE=[Garcia], ESTIMATED PROPHET=[Weir], I
>>> NEED A MIRACLE=[Weir], BLUES FOR ALLAH=[Garcia], ATTICS OF MY
>>> LIFE=[Garcia], BROKEDOWN PALACE=[Garcia], FEEL LIKE A STRANGER=[Weir], HARD
>>> TO HANDLE=[Pigpen], THE WHEEL=[Garcia_Kreutzmann], ALL ALONG THE
>>> WATCHTOWER=[Weir], HEY BO DIDDLEY=[Garcia]}
>>>
>>> I've tried a few things but nothing has worked so far. Any help would be
>>> appreciated.
>>>
>>> Jen
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gremlin-users/d05a2dfc-0848-46da-8517-d5ded85291f5%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gremlin-users/CA%2Bf9seUwjfCX3J%2B1oCfwp4Vdkh-iwrWGubTzG_GNYkkHCve-9g%40mail.gmail.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/1441D2B0-3A77-4A4A-B112-5D02C456AFEB%40gmail.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/6ca0819a-c480-4d2f-9e9e-db20006e072a%40googlegroups.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/CA%2Bf9seUa_bspdvbOhOzkT%3D2SHk-XMPk%3DjRdMNoOhhNMpOCdNCg%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
