Re: kafka producer failed

2015-07-24 Thread Yi Pan
Hi, Selina, Your question is not clear. {quote} When the messages was send to Kafka by KafkaProducer, It always failed when the message more than 3000 - 4000 messages. {quote} What's failing? The error stack shows errors on the consumer side and you were referring to failures to produce to

Review Request 36769: SAMZA-742 Add a Contribution section in README of samza and hello-samza

2015-07-24 Thread Aleksandar Pejakovic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36769/ --- Review request for samza. Repository: samza Description --- Added

Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

2015-07-24 Thread Stuart Davidson
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36768/ --- Review request for samza. Repository: samza-hello-samza Description ---

Review Request 36770: SAMZA-742 Add a Contribution section in README of samza and hello-samza

2015-07-24 Thread Aleksandar Pejakovic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36770/ --- Review request for samza. Repository: samza-hello-samza Description ---

Review Request 36767: SAMZA-423: Integrate Lucene into Samza

2015-07-24 Thread Robert Zuljevic
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36767/ --- Review request for samza. Repository: samza Description --- Initial

Re: Samza: can not produce new data to kafka

2015-07-24 Thread Shadi Noghabi
Selina, You should probably check a few things 1. Your log files to see if you have any errors. Also, does you job fail or continues running? 2. Does this line logger.info(key=+key+: message=+message); write any logs? 3. This might not be the only reason, but you are sending messages of type

Re: kafka producer failed

2015-07-24 Thread Job-Selina Wu
Hi, Yi: I am wondering if the problem can be fixed by the parameter max.message.size at kafka.producer.ProducerConfig for the topic size? My Http Server send message to Kafka. The last message shown on console is message=timestamp=06-20-2015 id=678 ip=22.231.113.68 browser=Safari

Re: kafka producer failed

2015-07-24 Thread Job-Selina Wu
Hi, All: Do you think it could be caused by memory, virtual memory size? Sincerely, Selina On Fri, Jul 24, 2015 at 1:54 PM, Job-Selina Wu swucaree...@gmail.com wrote: Hi, Navina: Thanks for your reply: the files are listed below: Your help is highly appreciated. Sincerely,

Re: Can I get an example of using the ElasticSearch producer?

2015-07-24 Thread Job-Selina Wu
Dear All: I like to have an example of using the ElasticSearch producer also. Thanks Selina swucaree...@gmail.com On Fri, Jul 24, 2015 at 1:03 PM, Woessner, Leo leo.woess...@pearson.com wrote: Can I get an example of using the ElasticSearch producer? leo.woess...@pearson.com

Re: Can I get an example of using the ElasticSearch producer?

2015-07-24 Thread Yan Fang
Hi guys, Thank you for being interested in this new producer. The producer is only in the master branch, so if you are using the 0.9.1 version, you wont get this support. If by any chance you are using the latest version, 1) here are some configuration

Re: kafka producer failed

2015-07-24 Thread Job-Selina Wu
Hi, Navina: Thanks for your reply: the files are listed below: Your help is highly appreciated. Sincerely, Selina The producer.properties for Kafka: # Producer Basics # # list

Re: Samza: can not produce new data to kafka

2015-07-24 Thread Yan Fang
{quote} I did not set auto.create.topics.enable anywhere {quote} Fine. Then its default to true. No worries. {quote} My job is listed as below. However I am wondering how can I know if my method public void* process*(IncomingMessageEnvelope envelope, MessageCollector collector, TaskCoordinator

Re: Samza: can not produce new data to kafka

2015-07-24 Thread Job-Selina Wu
Hi, Yan: Thanks for reply my email in detail. All the files at Yarn logs shown below. No Exception under samza-Demo/deploy/yarn/logs. I guess the StreamTask did not called ... Partial stdout file

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

2015-07-24 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36768/#review92951 --- bin/grid-elastic (line 1)

Re: Review Request 36770: SAMZA-742 Add a Contribution section in README of samza and hello-samza

2015-07-24 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36770/#review92936 --- Ship it! Ship It! - Navina Ramesh On July 24, 2015, 8:57 a.m.,

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

2015-07-24 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36545/#review92938 --- Thanks for picking this up! It feels good to look at a refactored

Re: Samza: can not produce new data to kafka

2015-07-24 Thread Yan Fang
The code and the property seem good to me. collector.send(new OutgoingMessageEnvelope(OUTPUT_STREAM, message));should work. So I am curious if you accidentally disabled auto.create.topics.enable ...Can you also try to send msgs from cmd line to demo-duplicate to see if it gets anything. Let me

Re: Samza: can not produce new data to kafka

2015-07-24 Thread Job-Selina Wu
Hi, Shadi: Thans a lot for your reply. 1. There is no error log at Kafka and Samza 2. this line logger.info(key=+key+: message=+message); write log correctly as below: [image: Inline image 1] This are my last two message with right count 3. I tried both way below, none of them

Can I get an example of using the ElasticSearch producer?

2015-07-24 Thread Woessner, Leo
Can I get an example of using the ElasticSearch producer? leo.woess...@pearson.com

Re: Security on YARN

2015-07-24 Thread Chen Song
Can someone give some context on this? I can volunteer myself and try working on this. Chen On Thu, Jul 2, 2015 at 4:29 AM, Qi Fu q...@talend.com wrote: Hi Yi Yan, Many thanks for your information. I have created a jira for this: https://issues.apache.org/jira/browse/SAMZA-727 I'm willing

Re: Security on YARN

2015-07-24 Thread Yan Fang
Hi Chen Song, If you can work on this issue, it will be great. 1. the related ticket is https://issues.apache.org/jira/browse/SAMZA-727 2. most of the change will happen in Yarn AM and Yarn client parts. The code sits in the samza-yarn package

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

2015-07-24 Thread Yan Fang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36163/#review93019 ---

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

2015-07-24 Thread József Márton Jung
On July 23, 2015, 10:35 p.m., Yan Fang wrote: samza-core/src/main/java/org/apache/samza/checkpoint/CheckpointManager.java, line 46 https://reviews.apache.org/r/36545/diff/1/?file=1013337#file1013337line46 To be consistent, lets go with TaskName, not the String. Corrected. On

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

2015-07-24 Thread József Márton Jung
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36545/ --- (Updated July 24, 2015, 12:27 p.m.) Review request for samza. Changes