Re: Security on YARN

Hi Chen Song, If you can work on this issue, it will be great. 1. the related ticket is https://issues.apache.org/jira/browse/SAMZA-727 2. most of the change will happen in Yarn AM and Yarn client parts. The code sits in the samza-yarn package

Re: Security on YARN

Can someone give some context on this? I can volunteer myself and try working on this. Chen On Thu, Jul 2, 2015 at 4:29 AM, Qi Fu wrote: > Hi Yi & Yan, > > Many thanks for your information. I have created a jira for this: > https://issues.apache.org/jira/browse/SAMZA-727 > I'm willing to test i

Re: Review Request 36163: SAMZA-690: changelog topic creation should not be in the container code

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36163/#review93019 --- samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinat

Re: Samza and sliding window

Yi, Thanks a lot for all the help. I am able to get the sliding window working. Here is what I have now to get it to work: for (KeyValueIterator i = store.all(); i.hasNext();) { Entry next= (Entry) i.next(); String key = (String) next.getKey(); String value = (String) s

Re: kafka producer failed

what are the log messages from the kafka brokers? these look like client messages indicating a broker problem. On Fri, Jul 24, 2015 at 1:18 PM, Job-Selina Wu wrote: > Hi, Yi: > > I am wondering if the problem can be fixed by the parameter " > max.message.size" at kafka.producer.ProducerCo

Re: kafka producer failed

Hi, Selina, I assume that you were referring to "max.message.bytes" in Kafka producer config? There is no "max.message.size" config. If you were referring to the "max.message.bytes", it has nothing to do w/ the number of messages in a topic, it is the limit on a single message size in bytes in Kaf

Re: kafka producer failed

Hi, All: Do you think it could be caused by memory, virtual memory size? Sincerely, Selina On Fri, Jul 24, 2015 at 1:54 PM, Job-Selina Wu wrote: > Hi, Navina: > > Thanks for your reply: the files are listed below: > > Your help is highly appreciated. > > Sincerely, > Selina > >

Re: Samza: can not produce new data to kafka

Hi, Yan: Thanks for reply my email in detail. All the files at Yarn logs shown below. No Exception under samza-Demo/deploy/yarn/logs. I guess the StreamTask did not called ... Partial stdout file (samza-Demo/deploy/yarn/logs/userlogs/application_1437767867729_0001/container_1437767867

Re: kafka producer failed

Hi, Navina: Thanks for your reply: the files are listed below: Your help is highly appreciated. Sincerely, Selina The producer.properties for Kafka: # Producer Basics # # list

Re: Samza: can not produce new data to kafka

{quote} I did not set auto.create.topics.enable anywhere {quote} Fine. Then its default to true. No worries. {quote} My job is listed as below. However I am wondering how can I know if my method "public void* process*(IncomingMessageEnvelope envelope, MessageCollector collector, TaskCoordinator

Re: Samza: can not produce new data to kafka

Hi, Yan and Shadi: I made a mistake. Actually, there is no log at /tmp/kafka-logs created by " logger.info("key="+key+": message="+message); ". The log I provided actually is log for input topic "http-demo" at /tmp/kafka-logs/http-demo-0 My job is listed as below. However I am wonderin

Re: kafka producer failed

Hi Selina, Looks like the attachment got filtered. Can you past the config in the email or at pastebin? Thanks! Navina On Fri, Jul 24, 2015 at 1:18 PM, Job-Selina Wu wrote: > Hi, Yi: > > I am wondering if the problem can be fixed by the parameter " > max.message.size" at kafka.producer.P

Re: Can I get an example of using the ElasticSearch producer?

Hi guys, Thank you for being interested in this new producer. The producer is only in the master branch, so if you are using the 0.9.1 version, you wont get this support. If by any chance you are using the latest version, 1) here are some configuration http://samza.apache.org/learn/documentation

Re: Can I get an example of using the ElasticSearch producer?

Dear All: I like to have an example of using the ElasticSearch producer also. Thanks Selina swucaree...@gmail.com On Fri, Jul 24, 2015 at 1:03 PM, Woessner, Leo wrote: > Can I get an example of using the ElasticSearch producer? > > leo.woess...@pearson.com >

Re: kafka producer failed

Hi, Yi: I am wondering if the problem can be fixed by the parameter " max.message.size" at kafka.producer.ProducerConfig for the topic size? My Http Server send message to Kafka. The last message shown on console is "message=timestamp=06-20-2015 id=678 ip=22.231.113.68 browser=Safari

Can I get an example of using the ElasticSearch producer?

Can I get an example of using the ElasticSearch producer? leo.woess...@pearson.com

Re: Samza: can not produce new data to kafka

The code and the property seem good to me. collector.send(new OutgoingMessageEnvelope(OUTPUT_STREAM, message));should work. So I am curious if you accidentally disabled auto.create.topics.enable ...Can you also try to send msgs from cmd line to "demo-duplicate" to see if it gets anything. Let me

Re: Samza: can not produce new data to kafka

Hi, Shadi: Thans a lot for your reply. 1. There is no error log at Kafka and Samza 2. this line " logger.info("key="+key+": message="+message); " write log correctly as below: [image: Inline image 1] This are my last two message with right count 3. I tried both way below, none of them

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36768/#review92953 --- It will also be nice to have a tutorial added to the website on how

Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36768/#review92951 --- bin/grid-elastic (line 1)

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36545/#review92938 --- Thanks for picking this up! It feels good to look at a refactored co

Re: Review Request 36770: SAMZA-742 Add a Contribution section in README of samza and hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36770/#review92936 --- Ship it! Ship It! - Navina Ramesh On July 24, 2015, 8:57 a.m., A

Re: Review Request 36769: SAMZA-742 Add a Contribution section in README of samza and hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36769/#review92935 --- Ship it! Looks good to me! README.md (line 84)

Re: Samza: can not produce new data to kafka

Selina, You should probably check a few things 1. Your log files to see if you have any errors. Also, does you job fail or continues running? 2. Does this line " logger.info("key="+key+": message="+message); " write any logs? 3. This might not be the only reason, but you are sending messages of t

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36545/ --- (Updated July 24, 2015, 12:27 p.m.) Review request for samza. Changes ---

Re: Review Request 36545: SAMZA-682 Refactor Coordinator stream messages

> On July 23, 2015, 10:35 p.m., Yan Fang wrote: > > samza-core/src/main/java/org/apache/samza/checkpoint/CheckpointManager.java, > > line 46 > > > > > > To be consistent, lets go with TaskName, not the String. Corre

Review Request 36770: SAMZA-742 Add a Contribution section in README of samza and hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36770/ --- Review request for samza. Repository: samza-hello-samza Description ---

Review Request 36769: SAMZA-742 Add a Contribution section in README of samza and hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36769/ --- Review request for samza. Repository: samza Description --- Added Contri

Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to samza-hello-samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36768/ --- Review request for samza. Repository: samza-hello-samza Description ---

Review Request 36767: SAMZA-423: Integrate Lucene into Samza

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36767/ --- Review request for samza. Repository: samza Description --- Initial comm