Beam works with S3 out of the box, you can provide s3://... paths to anything that works with files. Don't remember if it's already available in 2.4 or only starting 2.5 - does it currently not work for you?
On Tue, May 29, 2018, 2:26 PM S. Sahayaraj <[email protected]> wrote: > Hello All, > > The data source for our Beam pipleline is in S3 bucket, Is > there any built-in I/O Connector available with Java samples? If so, can > you please guide me how to integrate with them?. > > > > I am using Bean SDK for Java version 2.4.0 and Spark > runner in clustered deployment. > > > > <dependency> > > <groupId>org.apache.beam</groupId> > > <artifactId>beam-*sdks*-java-core</artifactId> > > <version>2.4.0</version> > > </dependency> > > > > Cheers, > > S. Sahayaraj >
