I'm on spark version 1.4.1. I couldn't find documentation that said it was fixed, so I thought maybe it was still an open issue. Any idea what the fix version is?
On Wed, Dec 9, 2015 at 11:10 AM Cody Koeninger <c...@koeninger.org> wrote: > Which version of spark are you on? I thought that was added to the spark > UI in recent versions. > > DIrect api doesn't have any inherent interaction with zookeeper. If you > need number of messages per batch and aren't on a recent enough version of > spark to see them in the ui, you can get them programatically from the > offset ranges. See the definition of count() in recent versions of > KafkaRDD for an example. > > On Wed, Dec 9, 2015 at 9:39 AM, Dan Dutrow <dan.dut...@gmail.com> wrote: > >> Is there documentation for how to update the metrics (#messages per >> batch) in the Spark Streaming tab when using the Direct API? Does the >> Streaming tab get its information from Zookeeper or something else >> internally? >> -- >> Dan ✆ >> > > -- Dan ✆
