Aka-shi commented on issue #6302:
URL: 
https://github.com/apache/incubator-pinot/issues/6302#issuecomment-778922524


   @mcvsubbu 
   >Would you want all partitions of the stream to pause , or just some 
specific partitions??
   
   For all partitions. The consumption for the table itself should be stopped. 
I was thinking something like this. 
   1. User pauses stream -> pinot server commits the current consuming 
segments(for all partitions) along with offsets. 
   2. User resumes stream -> Pinot starts consumption from all partitions from 
previously committed offsets. 
   
   >Would you want all partitions/tables in a given server to pause?
   
   I was expecting the pause option to be at a table level. Because of this 
#6555 . If the pause and reset APIs are available at a table level, then the 
user can pause the current stream(which would commit current segments and pause 
the stream), reset the offsets, and resume consumption  from earliest/latest 
offsets as per config. 
   
   >After a pause, if the server is restarted, how do you desire the server 
should come back up? Should it consume up to the exact same paused place again? 
Should it simply not consume after the last completed/committed point? Should 
it forget that it was paused?
   
   If a table is paused and server restarts after it, then considering the 
previous consuming segments were already committed when the table is paused, I 
would expect the server to not consume after the restart too. My understanding 
is when we pause the stream, we are changing the state of the table and it 
should not start consuming until the user resumes the stream himself. 
   
   >Further if there are multiple replicas, do you want them all to pause at 
the same place after a pause command?
   
   Yes. That's what I would expect when I pause the stream. No replica of any 
partition consumer should be active is what I feel. 
   
   PS: Just putting it out here. When we pause the table, the table should 
still be available for querying the already consumed data. If not, this would 
be more or less like the enable/disable API. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to