[DISCUSS] new idea: reverse reading a topic

[Alexandre DUVAL]

Hi,

I'm wondering if it is possible to introduce a new feature on Pulsar 
which will enable users to read topic from a defined MessageId to 
previous messages until the begin of the topic.

I tried to use Pulsar SQL but it requires so much RAM even for little 
queries (due to Presto design).

Currently, every read in Pulsar are expected to be going forward. So it 
might be a bit tricky to prevent every weird behavior by introducing the 
feature.

I'm currently tried to make an MVP/POC by introducting a readReverse 
field in the CommandSubscribe that is used by ReaderAPI and currently 
looking for to create a getFirstMessageId() on ManagedLedger 
(https://github.com/CleverCloud/pulsar/pull/3). I also removed 
startPosition < endPosition sanity checks in BookKeeper locally 
(https://github.com/CleverCloud/bookkeeper/pull/2).

We definitely prefer a readPrevious(), hasPreviousMessageAvailable() in 
the ReaderAPI.

I'm not familiar with these internals such as NonDurableCursor, 
RangeEntryCache, ManagedCursor so it's a bit tricky.

So I wondering someone to help/guide me or even directly handle the 
subject (or the discuss).

Regards,

Kannar