RE: Relaunching init

2015-10-08 Thread Jordi Blasi Uribarri
way for satisfying your use-case. Let me know if this solution will work for you. Thanks! Navina On Thu, Oct 8, 2015 at 2:45 AM, Jordi Blasi Uribarri wrote: > Hi, > > Is there a way of forcing the init of a job being rerun without > killing the job and restarting it? > >

Relaunching init

2015-10-08 Thread Jordi Blasi Uribarri
–HUP like in BIND servers (ok, I had a time as sysadmin ☺ ) or maybe a way of programmatically calling a running job to reload. Thanks , Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads

RE: Versioning question

2015-09-30 Thread Jordi Blasi Uribarri
, Jordi Blasi Uribarri wrote: > I was going through files I have in the system and I found some > strange things that make me wonder about correct versioning. I am not > sure but It may be the origin of the problems I see. > > When I installed Samza I ran the following: > git cl

Versioning question

2015-09-29 Thread Jordi Blasi Uribarri
.loadClass(ClassLoader.java:358) Do I have a versioning issue? Which library does this class belong to? Thanks Jordi ________ Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]

RE: process killing

2015-09-29 Thread Jordi Blasi Uribarri
t Yarn leaks out some orphaned processes but really have no chance to reliably reproduce it. If you see this repeatedly, it would be nice to capture the reproduce sequence and provide it to Yarn community for help. Please keep us cc'ed as well. Thanks! -Yi On Mon, Sep 28, 2015 at 2:33 AM, J

RE: container is running beyond virtual memory limits

2015-09-28 Thread Jordi Blasi Uribarri
t but it does not change. Some things that I am describing do not make sense for me, so I am lost on what to do or where to look. Thanks for your help, Jordi -Mensaje original- De: Jordi Blasi Uribarri [mailto:jbl...@nextel.es] Enviado el: lunes, 28 de septiembre

RE: process killing

2015-09-28 Thread Jordi Blasi Uribarri
.. interesting. What did you see in the application master's logs? I saw that the remaining processes running are SamzaAppMasters. On Tue, Sep 22, 2015 at 1:05 AM, Jordi Blasi Uribarri wrote: > Hi, > > I have two machines running yarn and samza. They are samza 0.9.1 and > h

RE: container is running beyond virtual memory limits

2015-09-28 Thread Jordi Blasi Uribarri
-table.html This allows you to add additional JVM opts when launching the containers. -Yi On Mon, Sep 28, 2015 at 1:48 AM, Jordi Blasi Uribarri wrote: > The three tasks have a similar options file, like this one. > > task.class=flow.OperationJob > job.name=flow.OperationJob > jo

RE: limits configuration

2015-09-28 Thread Jordi Blasi Uribarri
curiosity, did you just change the yarn-site.xml for RM or for NMs as well? On Fri, Sep 25, 2015 at 3:23 AM, Jordi Blasi Uribarri wrote: > Hi, > > I think that this is more a Yarn question than a Samza one, but I am > really stuck on this. I am having problems understanding some

RE: container is running beyond virtual memory limits

2015-09-28 Thread Jordi Blasi Uribarri
size should be set to less than yarn.container.memory.mb. -Yi On Tue, Sep 22, 2015 at 4:32 AM, Jordi Blasi Uribarri wrote: > I am seeing that I can not get even a single job running. I have > recovered the original configuration of yarn-site.xml and > capacity-scheduler.xml and that

limits configuration

2015-09-25 Thread Jordi Blasi Uribarri
the logs. What is preventing the jobs from getting to RUNNING state? Thanks. Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]

RE: container is running beyond virtual memory limits

2015-09-22 Thread Jordi Blasi Uribarri
killing them. Is there any place where I can look to remove temporary files or something similar? Thanks jordi -Mensaje original- De: Jordi Blasi Uribarri [mailto:jbl...@nextel.es] Enviado el: martes, 22 de septiembre de 2015 10:06 Para: dev@samza.apache.org Asunto: container is

container is running beyond virtual memory limits

2015-09-22 Thread Jordi Blasi Uribarri
5-09-22 09:54:36,663 INFO [AsyncDispatcher event handler] launcher.ContainerLaunch (ContainerLaunch.java:cleanupContainer(370)) - Cleaning up container container_1442908447829_0001_01_01 ________ Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bil

RE: process killing

2015-09-22 Thread Jordi Blasi Uribarri
ng the process the hard way > is enough. > > - I am finding some memory consumption problems that I don’t know > if they are related with this. Maybe I will describe them in another > message. > > Thnaks, > > Jordi > > Jo

process killing

2015-09-22 Thread Jordi Blasi Uribarri
related with this. Maybe I will describe them in another message. Thnaks, Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]

RE: memory limits

2015-09-15 Thread Jordi Blasi Uribarri
...@gmail.com On Fri, Sep 11, 2015 at 4:21 AM, Jordi Blasi Uribarri wrote: > Hi, > > I am trying to implement an environment that requires multiple > combined samza jobs for different tasks. I see that there is a limit > to the number of jobs that can be running at the same time as they

memory limits

2015-09-11 Thread Jordi Blasi Uribarri
environment (as long as we are speaking of Big Data, we need big amounts of resources ☺ ) but my lab does not have so much ram. Is there a way to reduce this limit so I can test it properly? I am using Samza 0.9. Thanks in advance, Jordi Jordi Blasi Uribarri Área I

RE: Kill All Jobs

2015-08-14 Thread Jordi Blasi Uribarri
- Shekar > > On Thu, Aug 6, 2015 at 12:21 AM, Jordi Blasi Uribarri > > wrote: > > > Hi, > > > > As a little present (and I know this is not the way to get the code > > in > the > > project, but I am new to this sharing). I just made a simple scrip

RE: Missing a change log offset for SystemStreamPartition

2015-08-12 Thread Jordi Blasi Uribarri
is an on-going work on broadcast stream here: https://issues.apache.org/jira/browse/SAMZA-676. -Yi On Tue, Aug 11, 2015 at 1:11 PM, Jordi Blasi Uribarri wrote: > Hi, > > What I am trying to develop is (I think) an evolution of the > Stream-table join. For every message processed,

Re: Missing a change log offset for SystemStreamPartition

2015-08-11 Thread Jordi Blasi Uribarri
ported. And what does it exactly mean when you say "load the DB to be able to use it from the consuming job"? We may be of more help if we know your use case in more details. Thanks! -Yi On Tue, Aug 11, 2015 at 3:00 AM, Jordi Blasi Uribarri wrote: > InitableTask tas kwas missing.

RE: Missing a change log offset for SystemStreamPartition

2015-08-11 Thread Jordi Blasi Uribarri
mail.com > > On Mon, Aug 10, 2015 at 3:24 AM, Jordi Blasi Uribarri > > wrote: > > > Just for making it easier to reproduce the problem I just reduced > > the > code > > of the job to the minimum: > > > > package test; > > > > impor

RE: Missing a change log offset for SystemStreamPartition

2015-08-10 Thread Jordi Blasi Uribarri
Container.main(SamzaContainer.scala) The job fails even when there is no message sent to the input topic. Samza is version 0.9.1 and kafka 0.8.2. Thanks, Jordi -Mensaje original- De: Jordi Blasi Uribarri [mailto:jbl...@nextel.es] Enviado el: lunes, 10 de agosto de 2015 10:26 Para:

RE: Missing a change log offset for SystemStreamPartition

2015-08-10 Thread Jordi Blasi Uribarri
yanfang...@gmail.com On Thu, Aug 6, 2015 at 1:16 AM, Jordi Blasi Uribarri wrote: > I changed the job name and the store name. I was defining two > different stores and in case that was the problem, I also eliminated the > second one. > I am getting the same exceptio

RE: Missing a change log offset for SystemStreamPartition

2015-08-06 Thread Jordi Blasi Uribarri
ception is caused by the wrong offset format, I believe. Let me know how the new job goes. Thanks, Fang, Yan yanfang...@gmail.com On Wed, Aug 5, 2015 at 12:51 AM, Jordi Blasi Uribarri wrote: > Hi, > > I am trying to use the Keystore to manage some state information. > Basically this

Kill All Jobs

2015-08-06 Thread Jordi Blasi Uribarri
-F ' ' '{ print $1 }' | while read linea do $(dirname $0)/kill-yarn-job.sh $linea done Hope it helps. Bye Jordi ____ Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]

Missing a change log offset for SystemStreamPartition

2015-08-05 Thread Jordi Blasi Uribarri
ainer.SamzaContainer$.safeMain(SamzaContainer.scala:108) at org.apache.samza.container.SamzaContainer$.main(SamzaContainer.scala:87) at org.apache.samza.container.SamzaContainer.main(SamzaContainer.scala) Is there something wrong? Thanks, Jordi ________ Jordi

RE: testThreadInterruptInOperationSleep on clean installation

2015-08-04 Thread Jordi Blasi Uribarri
ld issues. Thanks! Navina On Mon, Aug 3, 2015 at 11:53 PM, Jordi Blasi Uribarri wrote: > Hi, > > There was no gradle process and I have tried to run it increasing the > log level. It seems that the test that is failing is the one related to > Rocksdb. > > How can I run the

log4j configuration

2015-08-04 Thread Jordi Blasi Uribarri
h the code is being executed (I can see the messages going through) I see no trace written. I don’t have experience with log4j and maybe it is there where I have to look but, I am missing something? Thanks, Jordi ________ Jordi Blasi Uribarri Área

RE: testThreadInterruptInOperationSleep on clean installation

2015-08-03 Thread Jordi Blasi Uribarri
to run those two tests in the eclipse (or some other ways) without gradle ? I doubt both are related to the gradle. Thanks, Fang, Yan yanfang...@gmail.com On Sun, Aug 2, 2015 at 11:45 PM, Jordi Blasi Uribarri wrote: > Hi, > > I am trying to do a clean installation of Samza on a new

Dinamically asigning output topic

2015-08-03 Thread Jordi Blasi Uribarri
job properties file, so I need to know the topic name. Is there a way to free from this dependency? Thanks. Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]

testThreadInterruptInOperationSleep on clean installation

2015-08-02 Thread Jordi Blasi Uribarri
t I am missing? Thanks in advance. Bye Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]

Samza install quick guide

2015-03-31 Thread Jordi Blasi Uribarri
Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2014.png]

RE: java.lang.NoClassDefFoundError on Yarn job

2015-03-27 Thread Jordi Blasi Uribarri
attempt. Failing the application. What I am doing wrong? Thanks, Jordi -Mensaje original- De: Jordi Blasi Uribarri [mailto:jbl...@nextel.es] Enviado el: viernes, 27 de marzo de 2015 9:45 Para: dev@samza.apache.org Asunto: RE: java.lang.NoClassDefFoundError on Yarn job Solved

RE: java.lang.NoClassDefFoundError on Yarn job

2015-03-27 Thread Jordi Blasi Uribarri
Solved. My aplication was using 0.9.0 version of yarn. When downgraded to 0.8.0 it worked. Thanks, Jordi -Mensaje original- De: Jordi Blasi Uribarri [mailto:jbl...@nextel.es] Enviado el: viernes, 27 de marzo de 2015 9:05 Para: dev@samza.apache.org Asunto: RE

RE: java.lang.NoClassDefFoundError on Yarn job

2015-03-27 Thread Jordi Blasi Uribarri
ira/browse/SAMZA-456) which I just hit as well. You probably need to add a couple more jars to your YARN lib dir. Cheers, Roger On Thu, Mar 26, 2015 at 9:21 AM, Jordi Blasi Uribarri wrote: > Hi: > > I got samza running a job in local mode with the property:

java.lang.NoClassDefFoundError on Yarn job

2015-03-26 Thread Jordi Blasi Uribarri
auncher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 5 more I guess there is a problem with the job package but I am not sure how to solve it. Thanks, Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel

RE: cannot be cast to java.lang.String

2015-03-26 Thread Jordi Blasi Uribarri
step. Thanks a lot, Jordi -Mensaje original- De: Jordi Blasi Uribarri [mailto:jbl...@nextel.es] Enviado el: jueves, 26 de marzo de 2015 9:21 Para: dev@samza.apache.org Asunto: RE: cannot be cast to java.lang.String I am not sure of understanding what you want to say. Does this mean

RE: cannot be cast to java.lang.String

2015-03-26 Thread Jordi Blasi Uribarri
y we need to be explicit about the serialization format. On Wed, Mar 25, 2015 at 3:14 AM, Jordi Blasi Uribarri wrote: > I am using the Kafka command line producer, so I understand that I am > sending a String. > > bin/kafka-console-producer.sh --broker-list localhost:9092

RE: cannot be cast to java.lang.String

2015-03-25 Thread Jordi Blasi Uribarri
on, Mar 23, 2015 at 9:52 AM, Jordi Blasi Uribarri wrote: > Looks like that was one error. > > I have set the property like this: > systems.kafka.streams.syslog.samza.msg.serde=string > > But I am still getting the same error. Now I am seeing a different > thing in th

Re: Tags for Samza in stack overflow

2015-03-24 Thread Jordi Blasi Uribarri
I tried to do it in the question you (kindly) answered and found that same problem. I only have 150 reputation (I have some distance yet also :) ). I don't really know if there is a way to do it collectively or you should address Stackoverflow admins as project "owners" and ask for a tag. What

RE: cannot be cast to java.lang.String

2015-03-23 Thread Jordi Blasi Uribarri
fined a 'string' serializer in your config OR systems.kafka.streams.syslog.samza.msg.serde=json // Depending on the corresponding format of your input data On Mon, Mar 23, 2015 at 9:24 AM, Jordi Blasi Uribarri wrote: > Hi, > > As I understand it, I am setting "kafk

RE: cannot be cast to java.lang.String

2015-03-23 Thread Jordi Blasi Uribarri
e config params for the correct stream ? On Mon, Mar 23, 2015 at 3:31 AM, Jordi Blasi Uribarri wrote: > Hello, > > I have managed to get samza up and running an simple test job that > just sends the received message. This is the code: > > public class job1 implements StreamTas

cannot be cast to java.lang.String

2015-03-23 Thread Jordi Blasi Uribarri
serializer to String but I has the same effect. Thanks, Jordi ________ Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2014.png]

RE: SamzaException: no job factory class defined

2015-03-18 Thread Jordi Blasi Uribarri
8/jobs/configuration-table.html #An example job.factory.class=org.apache.samza.job.local.ThreadJobFactory Cheers, Roger On Wed, Mar 18, 2015 at 8:45 AM, Jordi Blasi Uribarri wrote: > Hello, > > I am trying to run my first job (publish what receives) in Samza and I > think that all

SamzaException: no job factory class defined

2015-03-18 Thread Jordi Blasi Uribarri
definition or I still have something missing in the system. Thanks in advance, Jordi Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2014.png]

RE: NoSuchMethodError

2015-03-17 Thread Jordi Blasi Uribarri
it to: org.apache.samza.config.factories.PropertiesConfigFactory Curious where you're getting that value from? We haven't had a "samza.*" prefix to packages since we open sourced Samza. What docs are you looking at? Cheers, Chris On Tue, Mar 17, 2015 at 5:45 AM, Jordi Blasi Uribar

Re: NoSuchMethodError

2015-03-17 Thread Jordi Blasi Uribarri
job, which has all of its >required components: > > http://maven.apache.org/plugins/maven-assembly-plugin/ > >If you're not using Maven, Gradle and SBT can both assemble .tgz files as >well. > >Cheers, >Chris > >On Mon, Mar 16, 2015 at 4:11 AM, Jordi Blasi Uribarr

NoSuchMethodError

2015-03-16 Thread Jordi Blasi Uribarri
your help, Jordi ________ Jordi Blasi Uribarri Área I+D+i jbl...@nextel.es Oficina Bilbao [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2014.png]