Hi, Counting always requires a job to be executed. Not sure if this is what you want to do, but if you want to prevent to get an empty result due to an empty cross input, you can use a mapPartition() with parallelism 1 to emit a special record, in case the MapPartitionFunction didn't see any data.
Best, Fabian Am Mi., 7. Nov. 2018 um 14:02 Uhr schrieb bastien dine < bastien.d...@gmail.com>: > Hello, > > I would like to a way to count a dataset to check if it is empty or not.. > But .count() throw an execution and I do not want to do separe job > execution plan, as hthis will trigger multiple reading.. > I would like to have something like.. > > Source -> map -> count -> if 0 -> do someting > if not -> do something > > > More concrete i would like to check if one of my dataset is empty before > doing a cross operation.. > > Thanks, > Bastien > > >
