I am trying to deal with the following scenario: A bunch of minor fragments are doing things in parallel. Each of them could skip some records. Since the downstream minor fragment needs to know the sum of skipped-record-counts (in order to just display or see if the number exceeds the threshold) in the upstreams, each upstream minor fragment needs to pass this scalar with RecordBatch.
Since this seems impacting the protocol of RecordBatch, I am looking for some advice here. Thanks.
