Hi, Alexander,
Very glad that you figured it out! Thanks!
-Yi
On Tue, Nov 17, 2015 at 1:41 PM, Alexander Filipchik <afilipc...@gmail.com>
wrote:
> Just want to update you on this one. After some time spent in debugging I
> found that the actual problem was a piece of our code that was calling
> next() on a range iterator twice :(.
> After removing the duplicate call everything works as expected.
>
> Thank you!
>
> Alex
>
> On Mon, Nov 16, 2015 at 10:45 PM, Yi Pan <nickpa...@gmail.com> wrote:
>
> > Hi, Alexander,
> >
> > Sorry to reply late on this one. I embedded my questions and comments
> > in-between the lines:
> >
> > On Sun, Nov 15, 2015 at 7:07 PM, Alexander Filipchik <
> afilipc...@gmail.com
> > >
> > wrote:
> >
> > >
> > > nodeIterator = store.range(
> > > String.join(".", nodeId, String.valueOf(Character.MIN_VALUE)),
> > > String.join(".", nodeId, String.valueOf(Character.MAX_VALUE)));
> > >
> > >
> > Theoretically, what you want is a prefix scan, the start key should be
> > nodeId + '.' and end key should be nodeId + '.' + maxId, in which maxId
> > should have each character = Character.MAX_VALUE with total length that
> is
> > equal or greater than the max possible nodeId.
> >
> > I restreamed RockDB changelog topic and I can see all this edges stored
> > > there, but query still returnes only 4.3M nodes.
> > >
> >
> > Could you help to clarify what you did here to "see all these edges" and
> to
> > "query still returns only 4.3M nodes"?
> >
> >
> > > 1) Have anyone seen such a behaviour before?
> > >
> >
> > Not I am aware of.
> >
> >
> > > 2) What is the best way to debug it on a remote machine? Any particular
> > > logs to look for? Any RockDb config params that should be enabled?
> > >
> >
> > You can try to add Jmx debug port option to task.opts. With Samza 0.10
> > (latest from trunk), the JMX server port is reported from the AppMaster's
> > web API. As for the state store config, you can try to disable the
> > CachedStore to prevent any potential issues w/ cache management.
> >
> >
> > > 3) Is it a good idea to store a graph in such a format?
> > >
> >
> > As long as you can partition the data based on nodeId, it should be fine.
> >
> >
> > >
> > > Thank you,
> > > Alex
> > >
> >
> > Please let us know if you find any issues with your use case.
> >
> > -Yi
> >
>
