> When you use Queryable State you are actually querying multiple
> underlying stores, i.e., one per partition.
>
> Huh? I was only querying one partition. In my example, I have a user's
> posts. Upon creation, they are routed to a particular partition using a
> partitioner that hashes the post's user ID. The posts are then indexed on
> that partition by prefixed keys using the method described above. When
> querying, I am only querying the one partition that has all of the user's
> posts. As far as I know, I am not querying across multiple partitions.
> Furthermore, I did not even think this was possible, given the fact that
> Interactive Queries require you to manually forward requests that should go
> to other partitions.
>
>
Each KafkaStreams instance is potentially responsible for multiple
partitions, so when you use Queryable State on a particular instance you
are querying all partitions for that store on the given instance.
>
>
>
>
>
> On Thu, Mar 16, 2017 at 2:11 PM, Damian Guy <damian....@gmail.com> wrote:
>
> > I think what you are seeing is that the order is not guaranteed across
> > partitions. When you use Queryable State you are actually querying
> multiple
> > underlying stores, i.e., one per partition. The implementation iterates
> > over one store/partition at a time, so the ordering will appear random.
> > This could be improved
> >
> > The tombstone records appearing in the results seems like a bug.
> >
> > Thanks,
> > Damian
> >
> > On Thu, 16 Mar 2017 at 17:37 Matthias J. Sax <matth...@confluent.io>
> > wrote:
> >
> > > Can you check if the problem exist for 0.10.2, too? (0.10.2 is
> > > compatible to 0.10.1 broker -- so you can upgrade your Streams code
> > > independently from the brokers).
> > >
> > > About the range: I did double check this, and I guess my last answer
> was
> > > not correct, and range() should return ordered data, but I got a follow
> > > up question: what the key type and serializer you use? Internally, data
> > > is stored in serialized form and ordered according to
> > > `LexicographicByteArrayComparator` -- thus, if the serialized bytes
> > > don't reflect the order of the deserialized data, it returned range
> > > shows up unordered to you.
> > >
> > >
> > > -Matthias
> > >
> > >
> > >
> > >
> > > On 3/16/17 10:14 AM, Dmitry Minkovsky wrote:
> > > > Hi Matthias. Thank you for your response.
> > > >
> > > > Yes, I was able to reproduce the null issue reliably. I can't open a
> > JIRA
> > > > at this time, but I can say I was using 0.10.1.0 and it was trivial
> to
> > > > reproduce. Just send records and the tombstones to a table topic.
> Then
> > > scan
> > > > the range. You'll see the tombstones.
> > > >
> > > > Indeed, ranges are returned with no specific order. I'm not sure what
> > you
> > > > mean that default stores are hash-based, but this ordering thing is a
> > > shame
> > > > because it kind of kills the ability to use KS as a full fledged DB
> > that
> > > > lets you index things like HBase (composite keys for lists of items).
> > Is
> > > > that how RocksDB works? Just returns range scans in random order? I
> > don't
> > > > know C++ so the documentation is a bit opaque to me. But what's the
> > point
> > > > of scanning a range if the data comes in some random order? That
> being
> > > the
> > > > case, the number of possible use-case scenarios seem to become
> > > > significantly limited.
> > > >
> > > >
> > > > Thank you!
> > > > Dmitry
> > > >
> > > > On Tue, Mar 14, 2017 at 1:12 PM, Matthias J. Sax <
> > matth...@confluent.io>
> > > > wrote:
> > > >
> > > >>> However,
> > > >>>> for keys that have been tombstoned, it does return null for me.
> > > >>
> > > >> Sound like a bug. Can you reliable reproduce this? Would you mind
> > > >> opening a JIRA?
> > > >>
> > > >> Can you check if this happens for both cases: caching enabled and
> > > >> disabled? Or only for once case?
> > > >>
> > > >>
> > > >>> "No ordering guarantees are provided."
> > > >>
> > > >> That is correct. Internally, default stores are hash-based -- thus,
> we
> > > >> don't give a sorted list/iterator back. You could replace RocksDB
> > with a
> > > >> custom store though.
> > > >>
> > > >>
> > > >> -Matthias
> > > >>
> > > >>
> > > >> On 3/13/17 3:56 PM, Dmitry Minkovsky wrote:
> > > >>> I am using interactive streams to query tables:
> > > >>>
> > > >>> ReadOnlyKeyValueStore<Messages.ByUserAndDate,
> > > >>> Messages.UserLetter> store
> > > >>> = streams.store("view-user-drafts",
> > > >>> QueryableStoreTypes.keyValueStore());
> > > >>>
> > > >>> Documentation says that #range() should not return null values.
> > > However,
> > > >>> for keys that have been tombstoned, it does return null for me.
> > > >>>
> > > >>> Also, I noticed only just now that "No ordering guarantees are
> > > >> provided." I
> > > >>> haven't done enough testing or looked at the code carefully enough
> > yet
> > > >> and
> > > >>> wonder if someone who knows could confirm: is this true? Is this
> > common
> > > >> to
> > > >>> all store implementations? I was hoping to use interactive streams
> > like
> > > >>> HBase to scan ranges. It appears this is not possible.
> > > >>>
> > > >>> Thank you,
> > > >>> Dmitry
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> > >
> >
>
