I think this is the right JIRA to fix this issue ( https://issues.apache.org/jira/browse/SPARK-7111). It should be in Spark 1.4.
On Thu, Dec 10, 2015 at 12:32 AM, Cody Koeninger <c...@koeninger.org> wrote: > Looks like probably > > https://issues.apache.org/jira/browse/SPARK-8701 > > so 1.5.0 > > On Wed, Dec 9, 2015 at 10:25 AM, Dan Dutrow <dan.dut...@gmail.com> wrote: > >> I'm on spark version 1.4.1. I couldn't find documentation that said it >> was fixed, so I thought maybe it was still an open issue. Any idea what the >> fix version is? >> >> On Wed, Dec 9, 2015 at 11:10 AM Cody Koeninger <c...@koeninger.org> >> wrote: >> >>> Which version of spark are you on? I thought that was added to the >>> spark UI in recent versions. >>> >>> DIrect api doesn't have any inherent interaction with zookeeper. If you >>> need number of messages per batch and aren't on a recent enough version of >>> spark to see them in the ui, you can get them programatically from the >>> offset ranges. See the definition of count() in recent versions of >>> KafkaRDD for an example. >>> >>> On Wed, Dec 9, 2015 at 9:39 AM, Dan Dutrow <dan.dut...@gmail.com> wrote: >>> >>>> Is there documentation for how to update the metrics (#messages per >>>> batch) in the Spark Streaming tab when using the Direct API? Does the >>>> Streaming tab get its information from Zookeeper or something else >>>> internally? >>>> -- >>>> Dan ✆ >>>> >>> >>> -- >> Dan ✆ >> > >
