Github user tdas commented on the issue: https://github.com/apache/spark/pull/20767 The idea is good. But how do you propose exposing that information? Periodic print out in the log? From a different angle, I would rather not do feature creep in this PR that is intended to be backported to 2.3. On Mar 15, 2018 7:31 PM, "tedyu" <notificati...@github.com> wrote: > *@tedyu* commented on this pull request. > ------------------------------ > > In external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/ > KafkaDataConsumer.scala > <https://github.com/apache/spark/pull/20767#discussion_r174984237>: > > > CachedKafkaDataConsumer(newInternalConsumer) > > - } else if (existingInternalConsumer.inuse) { > + } else if (existingInternalConsumer.inUse) { > // If consumer is already cached but is currently in use, then return a new consumer > NonCachedKafkaDataConsumer(newInternalConsumer) > > Maybe keep an internal counter for how many times the non cached consumer > is created. > This would give us information on how effective the cache is > > â > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/20767#pullrequestreview-104439601>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAoerMcXNmKmobW4ws25hx3OvcER-1Ptks5teyPogaJpZM4SiC1I> > . >
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org