You may be able to mix StreamingListener & SparkListener to get meaningful information about your task. however you need to connect a lot of pieces to make sense of the flow..
Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Mon, Jun 30, 2014 at 9:58 PM, Chen Song <chen.song...@gmail.com> wrote: > I am new to spark streaming and wondering if spark streaming tracks > counters (e.g., how many rows in each consumer, how many rows routed to an > individual reduce task, etc.) in any form so I can get an idea of how data > is skewed? I checked spark job page but don't seem to find any. > > > > -- > Chen Song > >
