[GitHub] srkukarni commented on issue #1766: Implementing PushSource on top of Source

[GitBox]

srkukarni commented on issue #1766:  Implementing PushSource on top of Source
URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388279890
 
 
   Not really. Infact this is closer to Storm's KafkaSpout which is a pull model


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] srkukarni commented on issue #1766: Implementing PushSource on top of Source

[GitBox]

srkukarni commented on issue #1766:  Implementing PushSource on top of Source
URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388278263
 
 
   So the main interface is the Source<> interface which has a read() blocking 
call. Since kafka is also a pull interface, kafkasource can just basically 
extend source<> directly and implement read() as kafkaclient.poll(). The 
mechanics of internal buffering is for efficiency reasons, but it would be 
something like this
   read() {
   if bufferedQueue.empty  bufferedQueue = kafkaclient.poll(some reasonable 
number); 
   pop and return back;
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] srkukarni commented on issue #1766: Implementing PushSource on top of Source

[GitBox]

srkukarni commented on issue #1766:  Implementing PushSource on top of Source
URL: https://github.com/apache/incubator-pulsar/pull/1766#issuecomment-388275286
 
 
   Since this is a smallish change, would rather do it here itself? It actually 
simpifies the code of kafka source


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services