've tried out the HDFS producer, too. In my experience, you won't be able
> to see changes written into HDFS in realtime. The content of the files
> become visible only after they get closed.
>
> You can probably play with the "producer.hdfs.write.batch.size.bytes"
ot;producer.hdfs.write.batch.size.bytes"
>> config to force rolling over to new files so you can see the results of the
>> previous one.
>>
>> Thanks,
>> Hai
>>
>> On Mon, Dec 19, 2016 at 11:29 PM, Rui Tang wrote:
>>
>> I'm using samza
realtime. The content of the files
> become visible only after they get closed.
>
> You can probably play with the "producer.hdfs.write.batch.size.bytes"
> config to force rolling over to new files so you can see the results of the
> previous one.
>
> Thanks,
> Hai
>
r they get closed.
>
> You can probably play with the "producer.hdfs.write.batch.size.bytes"
> config to force rolling over to new files so you can see the results of the
> previous one.
>
> Thanks,
> Hai
>
> On Mon, Dec 19, 2016 at 11:29 PM, Rui Tang wrote:
>
&
o force rolling over to new files so you can see the results of the
previous one.
Thanks,
Hai
On Mon, Dec 19, 2016 at 11:29 PM, Rui Tang wrote:
> I'm using samza-hdfs to write Kafka streams to HDFS, but I can't make it
> work.
>
> Here is my samza job's
I'm using samza-hdfs to write Kafka streams to HDFS, but I can't make it
work.
Here is my samza job's properties file:
# Job
job.factory.class=org.apache.samza.job.yarn.YarnJobFactory
job.name=kafka2hdfs
# YARN
yarn.package.path=file://${basedir}/target/${project.artifactId
:
>>
>>> Hi Aram
>>>
>>> May I know why you're moving off from Spark streaming to Samza?
>>>
>>> On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan <
>>> aram.mkrtch...@picsart.com.invalid> wrote:
>>>
>>>> H
;
>> On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan <
>> aram.mkrtch...@picsart.com.invalid> wrote:
>>
>>> Hi Yi,
>>> Many thanks! :)
>>> It's wonderful ! We really need samza-hdfs module.
>>> Thanks to samza being very simple we have been
wrote:
> Hi Aram
>
> May I know why you're moving off from Spark streaming to Samza?
>
> On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan <
> aram.mkrtch...@picsart.com.invalid> wrote:
>
> > Hi Yi,
> > Many thanks! :)
> > It's wonderful
ay I know why you're moving off from Spark streaming to Samza?
>
> On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan <
> aram.mkrtch...@picsart.com.invalid> wrote:
>
> > Hi Yi,
> > Many thanks! :)
> > It's wonderful ! We really need samza-hdfs module.
>
Hi Aram
May I know why you're moving off from Spark streaming to Samza?
On Tue, Nov 10, 2015 at 11:04 PM, Aram Mkrtchyan <
aram.mkrtch...@picsart.com.invalid> wrote:
> Hi Yi,
> Many thanks! :)
> It's wonderful ! We really need samza-hdfs module.
> Thanks to samza be
Hi Yi,
Many thanks! :)
It's wonderful ! We really need samza-hdfs module.
Thanks to samza being very simple we have been able to create cassandra
producer for samza and we can share it if you think it's will be useful for
others.
Thanks.
On Wed, Nov 11, 2015 at 9:13 AM, Yi Pan wr
move from spark streaming to Samza and we want to use
> samza as our main stream processor.
> We would like to know when you are planning to release 0.10.0 and will
> samza-hdfs be included in that release.
>
> Thanks.
>
Hi guys,
We are considering to move from spark streaming to Samza and we want to use
samza as our main stream processor.
We would like to know when you are planning to release 0.10.0 and will
samza-hdfs be included in that release.
Thanks.
14 matches
Mail list logo