Inside a CompositeTransformation --
> is it Ok to spawn threads / use CountDownLatch to perform multiple
ParDo on the same data item at the same item
> or all the calls to individual ParDo inherently parallelized
For example, I need to execute generateAndStoreGraphData() ,
enerateAndStoreTimeSerieseData1() , generateAndStoreTimeSerieseData1()*
-- in parallel .*
static class MultiOps extends PTransform<PCollection<T>, PCollection<T>> {
public PCollection<T> apply(PCollection<T> dataSet) {
dataSet.apply(ParDo.of( generateAndStoreGraphData());
dataSet.apply(ParDo.of( generateAndStoreTimeSerieseData1());
dataSet.apply(ParDo.of( generateAndStoreTimeSerieseData1());
return results;
}
}
=====================
Thanks
Kaniska