Re: How to create partitions from input

2014-11-05 Thread Chris Riccomini
Hey Renato, It seems to me that your consumer is not reading any messages. This seems concerning: 2014-11-05 22:28:08 OrderFeed [ERROR] Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. R

Re: How to create partitions from input

2014-11-05 Thread Renato Marroquín Mogrovejo
Hi Chris, I found out it was a silly mistake but as soon as I saw the logs, it was easy to take it down. Now my consumer gets this and does not do any progress. I guess the database records are not getting into the partition, could this be because I am creating my partition every time the register

Re: How to create partitions from input

2014-11-05 Thread Chris Riccomini
Hey Renato, Here's your problem: 2014-11-05 15:10:14 SamzaContainer$ [INFO] Failed to create a consumer for order, so skipping. 2014-11-05 15:10:14 SamzaContainer$ [INFO] Got system consumers: Set() Samza is unable to instantiate your SystemConsumer. Turn the log-level to debug, so you can see

Re: How to create partitions from input

2014-11-05 Thread Chris Riccomini
Hey Renato, It looks like your config is good. I'm wondering if your consumer is failing to get created. I'm looking for this line in the logs: "Failed to create a consumer for order, so skipping" If the consumer creation fails, Samza will continue on, and assume that you won't use the consum

Re: How to create partitions from input

2014-11-05 Thread Renato Marroquín Mogrovejo
Hi Chris, This is what my config file looks like: # Job job.factory.class=org.apache.samza.job.yarn.YarnJobFactory job.name=order-feed # YARN yarn.package.path=file://${basedir}/target/${project.artifactId}-${pom.version}-dist.tar.gz # Task task.class=samza.examples.order.task.OrderFeedStreamTask

Re: How to create partitions from input

2014-11-05 Thread Chris Riccomini
Hey Renato, Could you send the config for your job? The stack trace is indicating that you are using a system "task.inputs" that isn't defined in the configuration. For example, if you have: task.inputs=kafka.my-topic But, you haven't defined a "kafka" system (systems.kafka.*) then you will s

Fwd: How to create partitions from input

2014-11-05 Thread Renato Marroquín Mogrovejo
Hi Yang, I tried setting the task.input to have the same key as the system's name, but I keep on getting error while trying to run it: 2014-11-05 09:42:07 OffsetManager [INFO] Successfully loaded starting offsets: Map(SystemStreamPartition [partition=Partition [partition=0], system=order, stream=