Hi Folks, I know this is not the optimal way to use beam :-) But assume I only use the spark runner.
I have a spark library (very complex) that emits a spark dataframe (or RDD). I also have an existing complex beam pipeline that can do post processing on the data inside the dataframe. However, the beam part needs a pcollection to start with. The question is, how can I convert a spark RDD into a pcollection? Thanks -Yushu
