Re: How to Use samza-hdfs

2016-12-20 Thread Rui Tang
've tried out the HDFS producer, too. In my experience, you won't be able > to see changes written into HDFS in realtime. The content of the files > become visible only after they get closed. > > You can probably play with the "producer.hdfs.write.batch.size.bytes"

Re: How to Use samza-hdfs

2016-12-20 Thread Hai Lu
ot;producer.hdfs.write.batch.size.bytes" >> config to force rolling over to new files so you can see the results of the >> previous one. >> >> Thanks, >> Hai >> >> On Mon, Dec 19, 2016 at 11:29 PM, Rui Tang wrote: >> >> I'm using samza

Re: How to Use samza-hdfs

2016-12-20 Thread Rui Tang
realtime. The content of the files > become visible only after they get closed. > > You can probably play with the "producer.hdfs.write.batch.size.bytes" > config to force rolling over to new files so you can see the results of the > previous one. > > Thanks, > Hai >

Re: How to Use samza-hdfs

2016-12-20 Thread Rui Tang
r they get closed. > > You can probably play with the "producer.hdfs.write.batch.size.bytes" > config to force rolling over to new files so you can see the results of the > previous one. > > Thanks, > Hai > > On Mon, Dec 19, 2016 at 11:29 PM, Rui Tang wrote: > &

Re: How to Use samza-hdfs

2016-12-20 Thread Hai Lu
o force rolling over to new files so you can see the results of the previous one. Thanks, Hai On Mon, Dec 19, 2016 at 11:29 PM, Rui Tang wrote: > I'm using samza-hdfs to write Kafka streams to HDFS, but I can't make it > work. > > Here is my samza job's

How to Use samza-hdfs

2016-12-19 Thread Rui Tang
I'm using samza-hdfs to write Kafka streams to HDFS, but I can't make it work. Here is my samza job's properties file: # Job job.factory.class=org.apache.samza.job.yarn.YarnJobFactory job.name=kafka2hdfs # YARN yarn.package.path=file://${basedir}/target/${project.artifactId

Re: Samza hdfs

2015-11-13 Thread Rick Mangi
: >> >>> Hi Aram >>> >>> May I know why you're moving off from Spark streaming to Samza? >>> >>> On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan < >>> aram.mkrtch...@picsart.com.invalid> wrote: >>> >>>> H

Re: Samza hdfs

2015-11-13 Thread Aram Mkrtchyan
; >> On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan < >> aram.mkrtch...@picsart.com.invalid> wrote: >> >>> Hi Yi, >>> Many thanks! :) >>> It's wonderful ! We really need samza-hdfs module. >>> Thanks to samza being very simple we have been

Re: Samza hdfs

2015-11-11 Thread Navina Ramesh
wrote: > Hi Aram > > May I know why you're moving off from Spark streaming to Samza? > > On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan < > aram.mkrtch...@picsart.com.invalid> wrote: > > > Hi Yi, > > Many thanks! :) > > It's wonderful

Re: Samza hdfs

2015-11-11 Thread Aram Mkrtchyan
ay I know why you're moving off from Spark streaming to Samza? > > On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan < > aram.mkrtch...@picsart.com.invalid> wrote: > > > Hi Yi, > > Many thanks! :) > > It's wonderful ! We really need samza-hdfs module. >

Re: Samza hdfs

2015-11-10 Thread Bae, Jae Hyeon
Hi Aram May I know why you're moving off from Spark streaming to Samza? On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan < aram.mkrtch...@picsart.com.invalid> wrote: > Hi Yi, > Many thanks! :) > It's wonderful ! We really need samza-hdfs module. > Thanks to samza be

Re: Samza hdfs

2015-11-10 Thread Aram Mkrtchyan
Hi Yi, Many thanks! :) It's wonderful ! We really need samza-hdfs module. Thanks to samza being very simple we have been able to create cassandra producer for samza and we can share it if you think it's will be useful for others. Thanks. On Wed, Nov 11, 2015 at 9:13 AM, Yi Pan wr

Re: Samza hdfs

2015-11-10 Thread Yi Pan
move from spark streaming to Samza and we want to use > samza as our main stream processor. > We would like to know when you are planning to release 0.10.0 and will > samza-hdfs be included in that release. > > Thanks. >

Samza hdfs

2015-11-10 Thread Aram Mkrtchyan
Hi guys, We are considering to move from spark streaming to Samza and we want to use samza as our main stream processor. We would like to know when you are planning to release 0.10.0 and will samza-hdfs be included in that release. Thanks.