Re: Review Request 34585: SAMZA-681: create a unit test harness to easily test samza tasks

2016-03-14 Thread George Li
/InMemoryTaskContext.java (line 50) <https://reviews.apache.org/r/34585/#comment185737> Can store implement KeyValueStore? Maybe we can use InMemoryKeyValueStore here? - George Li On May 22, 2015, 5:23 a.m., Luis De Pombo

Re: Deploying samza job to a remote yarn resource manager

2015-06-23 Thread George Li
your case, it should point to hadoop home: $HOME/hadoop-2.6.0/ Please let me know if it works. Cheers! Navina On 6/22/15, 2:11 PM, "George Li" wrote: >Hi, > >According to > >https://samza.apache.org/learn/documentation/0.9/jobs/yarn-jobs.html > >, to deploy sam

Deploying samza job to a remote yarn resource manager

2015-06-22 Thread George Li
Hi, According to https://samza.apache.org/learn/documentation/0.9/jobs/yarn-jobs.html , to deploy samza jobs to a remote yarn resource manager, I need to set the HADOOPYARNHOME environment variable. What should be the value of this variable if my yarn-site.xml is at $HOME/hadoop-2.6.0/etc/had

Re: Samza job throughput much lower than Kafka throughput

2015-05-22 Thread George Li
"container-name":"samza-container-0", "source":"samza-container-0", "job-name":"container-performance", "samza-version":"0.9.0", "version":"0.0.1" } }{ "metrics":{

Re: Samza job throughput much lower than Kafka throughput

2015-05-21 Thread George Li
t;> broker. >> >> Also when you say it is "slower than a program reading directly from >> Kafka." which consumer did your program use to read data from Kafka? >> >> Guozhang >> >> >> On Wed, May 20, 2015 at 5:01 PM, George Li wrote: &g

Re: Samza job throughput much lower than Kafka throughput

2015-05-21 Thread George Li
read data from Kafka? Guozhang On Wed, May 20, 2015 at 5:01 PM, George Li wrote: > Hi Yi, > > Thanks for the reply. Below is my job config and code. > > When we run this job inside our dev docker container, which has zookeeper, > broker, and yarn installed locally, its throug

Re: Samza job throughput much lower than Kafka throughput

2015-05-20 Thread George Li
05/2015 05:03 PM Subject:Re: Samza job throughput much lower than Kafka throughput Hi, George, Could you share w/ us the code and configuration of your sample test job? Thanks! -Yi On Wed, May 20, 2015 at 1:19 PM, George Li wrote: > Hi, > > We are evaluating Samza's perfor

Samza job throughput much lower than Kafka throughput

2015-05-20 Thread George Li
Hi, We are evaluating Samza's performance, and our sample job with TestPerformanceTask is much slower than a program reading directly from Kafka. Scenario: * Cluster: 1 master node for Zookeeper and yarn. 3 Kafka broker nodes 3 yarn worker nodes * Kafka: Topic has only 1 partition. Average me