Re: Questions related to writing data to S3

On 23 Apr 2017, at 19:49, Richard Hanson > wrote: I have a streaming job which writes data to S3. I know there are saveAs functions helping write data to S3. But it bundles all elements then writes out to S3. use Hadoop 2.8.x binaries and

Questions related to writing data to S3

I have a streaming job which writes data to S3. I know there are saveAs functions helping write data to S3. But it bundles all elements then writes out to S3. So my first question - Is there any way to let saveAs functions write data in batch or single elements instead of whole bundle?