Re: Containers getting killed by application master
Hello Bhupesh,
The Kafka operator seems to be the one crashing. I am using the Kafka 0.9
operator from Malhar on a kafka broker cluster running CDH kafka 2.x.
Attaching the logs of this particular operator for reference.
Please note that there is an exception from the netty driver and I believe
this is not the root cause as I have observed this exception being thrown
from the cassandra across other stacks as well.
However the following lines in the log for the killed operator looks
suspicious:
2016-05-18 07:17:17,556 INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Marking
the coordinator 2147483576 dead.
2016-05-18 07:17:17,556 WARN
org.apache.apex.malhar.kafka.AbstractKafkaInputOperator: Exceptions in
committing offsets
eventdetails_ingestion-8=OffsetAndMetadata{offset=1375350, metadata=''} :
org.apache.kafka.common.errors.NotCoordinatorForGroupException: This is not
the correct coordinator for this group.
2016-05-18 07:21:23,611 INFO
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator: Offset
commit for group ced_Consumer failed due to NOT_COORDINATOR_FOR_GROUP, will
find new coordinator and retry
2016-05-18 07:21:23,611 INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Marking
the coordinator 2147483577 dead.
2016-05-18 07:21:23,611 WARN
org.apache.apex.malhar.kafka.AbstractKafkaInputOperator: Exceptions in
committing offsets
eventdetails_ingestion-8=OffsetAndMetadata{offset=1377033, metadata=''} :
org.apache.kafka.common.errors.NotCoordinatorForGroupException: This is not
the correct coordinator for this group.
2016-05-18 07:21:23,612 INFO
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator: Offset
commit for group ced_Consumer failed due to NOT_COORDINATOR_FOR_GROUP, will
find new coordinator and retry
2016-05-18 07:21:23,612 WARN
org.apache.apex.malhar.kafka.AbstractKafkaInputOperator: Exceptions in
committing offsets
eventdetails_ingestion-8=OffsetAndMetadata{offset=1377073, metadata=''} :
org.apache.kafka.common.errors.NotCoordinatorForGroupException: This is not
the correct coordinator for this group.
2016-05-18 07:22:17,950 INFO
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator: Offset
commit for group ced_Consumer failed due to NOT_COORDINATOR_FOR_GROUP, will
find new coordinator and retry
2016-05-18 07:22:17,950 INFO
org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Marking
the coordinator 2147483576 dead.
Regards,
Ananth
On Wed, May 18, 2016 at 5:29 PM, Bhupesh Chawda <bhup...@datatorrent.com>
wrote:
> Hi Ananth,
>
> Do the containers that are getting killed belong to any specific operator?
> Or are these getting killed randomly.
> I'll suggest to have a look at the operator / container logs.
> You can also check this using: yarn logs --applicationId
>
> ~Bhupesh
>
> On Wed, May 18, 2016 at 12:22 AM, Ananth Gundabattula <
> agundabatt...@gmail.com> wrote:
>
>> Thanks all for the inputs.
>>
>> @Yogi: I do not have any operators that are dynamically partitioned. I
>> have not implemented any definePartition() in any of my operators.
>>
>> @Bhupesh: I am not using the JSON parser operator from Malhar. I do use
>> jackson parser as an instance inside my operator that does some application
>> level logic. The stack trace seems to be coming from the Apex pubsub codec
>> handler.
>>
>> @Ashwin : The window ID seems to be moving forward.
>>
>> I would like to understand more as to what we mean by container failure ?
>> I am assuming that Apex automatically relaunches a container if it fails
>> for whatever reason. In fact I do see operators getting killed ( and on
>> clicking the details button , I see the message posted at the beginning of
>> this thread)
>>
>> One thing I want to note is that the operators are recreated
>> automatically when they fail and after a couple of days, even this recovery
>> process seems to be broken. i.e. new instances of the operators are not
>> created automatically after they are dead and the app runs in a lower
>> operators count mode ( and hence some data is not getting processed)
>>
>> I observed this behavior on non-HA enabled cluster. ( CDH 5.7 ) and
>> hence I do not suspect Yarn HA is causing this. I am currently ruling out
>> network issues as this would mean all operators need to exhibit some sort
>> of blips. ( Please correct me if I am wrong in this assumption)
>>
>> Regards,
>> Ananth
>>
>>
>>
>> On Wed, May 18, 2016 at 4:53 PM, Yogi Devendra <yogideven...@apache.org>
>> wrote:
>>
>>> There are some instances of "Heartbeat for unknown operator" in the log.
>>> So, looks like operators are sending the heartbeats. But, STRAM is not
>>> able to
Re: Containers getting killed by application master
Thanks all for the inputs. @Yogi: I do not have any operators that are dynamically partitioned. I have not implemented any definePartition() in any of my operators. @Bhupesh: I am not using the JSON parser operator from Malhar. I do use jackson parser as an instance inside my operator that does some application level logic. The stack trace seems to be coming from the Apex pubsub codec handler. @Ashwin : The window ID seems to be moving forward. I would like to understand more as to what we mean by container failure ? I am assuming that Apex automatically relaunches a container if it fails for whatever reason. In fact I do see operators getting killed ( and on clicking the details button , I see the message posted at the beginning of this thread) One thing I want to note is that the operators are recreated automatically when they fail and after a couple of days, even this recovery process seems to be broken. i.e. new instances of the operators are not created automatically after they are dead and the app runs in a lower operators count mode ( and hence some data is not getting processed) I observed this behavior on non-HA enabled cluster. ( CDH 5.7 ) and hence I do not suspect Yarn HA is causing this. I am currently ruling out network issues as this would mean all operators need to exhibit some sort of blips. ( Please correct me if I am wrong in this assumption) Regards, Ananth On Wed, May 18, 2016 at 4:53 PM, Yogi Devendra <yogideven...@apache.org> wrote: > There are some instances of "Heartbeat for unknown operator" in the log. > So, looks like operators are sending the heartbeats. But, STRAM is not > able to identify the operator. > > In the past, I observed similar behavior when I was trying to define the > dynamic partitioning for some operator. > > > ~ Yogi > > On 18 May 2016 at 12:12, Ashwin Chandra Putta <ashwinchand...@gmail.com> > wrote: > >> Ananth, >> >> The heartbeat timeout means that the operator is not sending back the >> window heartbeat information to the app master. It usually happens because >> of one of two reasons. >> >> 1. System failure - container died, network failure etc. >> 2. Windows not moving forward in the operator. Some business logic in the >> operator is blocking the windows. You can observe the window IDs on the UI >> for the given operator when it is running to quickly find out if this is >> the issue. >> >> Regards, >> Ashwin. >> On May 17, 2016 11:05 PM, "Ananth Gundabattula" <agundabatt...@gmail.com> >> wrote: >> >> Hello Sandeep, >> >> Thanks for the response. Please find attached the app master log. >> >> It looks like it got killed due to a heartbeat timeout. I will have to >> see why I am getting a heartbeat timeout. I also see a JSON parser >> exception in the logs in the log attached. Is it a harmless exception ? >> >> >> Regards, >> Ananth >> >> On Wed, May 18, 2016 at 2:45 PM, Sandeep Deshmukh < >> sand...@datatorrent.com> wrote: >> >>> Dear Ananth, >>> >>> Could you please check the STRAM logs for any details of these >>> containers. The first guess would be container going out of memory . >>> >>> Regards, >>> Sandeep >>> >>> On Wed, May 18, 2016 at 10:05 AM, Ananth Gundabattula < >>> agundabatt...@gmail.com> wrote: >>> >>>> Hello All, >>>> >>>> I was wondering what would be the case for a container to be killed by >>>> the application master ? >>>> >>>> I see the following in the UI when I click on details : >>>> >>>> " >>>> >>>> Container killed by the ApplicationMaster. >>>> Container killed on request. Exit code is 143 >>>> Container exited with a non-zero exit code 143 >>>> >>>> " >>>> >>>> I see zome exceptions in the dtgateway.log and am not sure if they are >>>> related. >>>> >>>> I am running Apex 3.3.0 on CDH 5.7 and HA enabled (HA for YARN as well as >>>> HDFS is enabled). >>>> >>>> >>>> >>>> >>>> >>> >> >
Re: YARN memory settings and the Apex memory model
Thanks Shubham. I shall bump up the memory a bit more. I was wondering how the operator memory relates to the YARN container memory settings ? Or it depends on the deployment models ? For example , if the deployment model is thread local, the YARN container needs to be ( considering above example ) configured for atleast memory of 2048 * number of operators + Buffer Server Size ? If the deployment model were not Thread local, it would make YARN container requirements for memory lower per container ? Regards, Ananth On Thu, May 12, 2016 at 7:19 PM, Shubham Pathak <shub...@datatorrent.com> wrote: > Hello Ananth, > > Looks like operator requires more memory. > You may add this property to have more memory allocated to the container. > > In properties.xml , for operator O in the application you may specify the > property : > > > dt.operator.*O*.attr.MEMORY_MB > 2048 > > > Thanks, > Shubham > > On Thu, May 12, 2016 at 1:35 PM, Ananth Gundabattula < > agundabatt...@gmail.com> wrote: > >> Hello All, >> >> I am seeing the following log from the web ui ocassionally when my >> operators are getting killed. Is there any way I can control the memory >> settings that are used to communicate with YARN when negotiating a >> container ? >> >> How does the typical yarn settings for a container heap and max memory >> relate to the Apex memory allocation model. >> >> The info messages I see in the web UI are as follows: >> >> Container [pid=14699,containerID=container_1462863487071_0015_01_12] is >> running beyond physical memory limits. Current usage: 1.5 GB of 1.5 GB >> physical memory used; 6.1 GB of 3.1 GB virtual memory used. Killing >> container. >> Dump of the process-tree for container_1462863487071_0015_01_12 : >> |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) >> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE >> |- 14817 14699 14699 14699 (java) 1584 1654 6426968064 393896 >> /usr/java/default/bin/java -Xmx4429185024 >> -Ddt.attr.APPLICATION_PATH=hdfs://dwh109.qaperf2.sac.int.threatmetrix.com:8020/user/dtadmin/datatorrent/apps/application_1462863487071_0015 >> >> -Djava.io.tmpdir=/data3/yarn/nm/usercache/root/appcache/application_1462863487071_0015/container_1462863487071_0015_01_12/tmp >> -Ddt.cid=container_1462863487071_0015_01_12 >> -Dhadoop.root.logger=INFO,RFA >> -Dhadoop.log.dir=/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12 >> -Ddt.loggers.level=com.datatorrent.*:INFO,org.apache.*:INFO >> com.datatorrent.stram.engine.StreamingContainer >> |- 14699 14697 14699 14699 (bash) 1 2 108646400 303 /bin/bash -c >> /usr/java/default/bin/java -Xmx4429185024 >> -Ddt.attr.APPLICATION_PATH=hdfs://dwh109.qaperf2.sac.int.threatmetrix.com:8020/user/dtadmin/datatorrent/apps/application_1462863487071_0015 >> >> -Djava.io.tmpdir=/data3/yarn/nm/usercache/root/appcache/application_1462863487071_0015/container_1462863487071_0015_01_12/tmp >> -Ddt.cid=container_1462863487071_0015_01_12 >> -Dhadoop.root.logger=INFO,RFA >> -Dhadoop.log.dir=/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12 >> -Ddt.loggers.level=com.datatorrent.*:INFO,org.apache.*:INFO >> com.datatorrent.stram.engine.StreamingContainer >> 1>/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12/stdout >> >> 2>/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12/stderr >> >> Container killed on request. Exit code is 143 >> Container exited with a non-zero exit code 143 >> >> >> Regards, >> >> Ananth >> >> >> >
YARN memory settings and the Apex memory model
Hello All, I am seeing the following log from the web ui ocassionally when my operators are getting killed. Is there any way I can control the memory settings that are used to communicate with YARN when negotiating a container ? How does the typical yarn settings for a container heap and max memory relate to the Apex memory allocation model. The info messages I see in the web UI are as follows: Container [pid=14699,containerID=container_1462863487071_0015_01_12] is running beyond physical memory limits. Current usage: 1.5 GB of 1.5 GB physical memory used; 6.1 GB of 3.1 GB virtual memory used. Killing container. Dump of the process-tree for container_1462863487071_0015_01_12 : |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE |- 14817 14699 14699 14699 (java) 1584 1654 6426968064 393896 /usr/java/default/bin/java -Xmx4429185024 -Ddt.attr.APPLICATION_PATH=hdfs://dwh109.qaperf2.sac.int.threatmetrix.com:8020/user/dtadmin/datatorrent/apps/application_1462863487071_0015 -Djava.io.tmpdir=/data3/yarn/nm/usercache/root/appcache/application_1462863487071_0015/container_1462863487071_0015_01_12/tmp -Ddt.cid=container_1462863487071_0015_01_12 -Dhadoop.root.logger=INFO,RFA -Dhadoop.log.dir=/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12 -Ddt.loggers.level=com.datatorrent.*:INFO,org.apache.*:INFO com.datatorrent.stram.engine.StreamingContainer |- 14699 14697 14699 14699 (bash) 1 2 108646400 303 /bin/bash -c /usr/java/default/bin/java -Xmx4429185024 -Ddt.attr.APPLICATION_PATH=hdfs://dwh109.qaperf2.sac.int.threatmetrix.com:8020/user/dtadmin/datatorrent/apps/application_1462863487071_0015 -Djava.io.tmpdir=/data3/yarn/nm/usercache/root/appcache/application_1462863487071_0015/container_1462863487071_0015_01_12/tmp -Ddt.cid=container_1462863487071_0015_01_12 -Dhadoop.root.logger=INFO,RFA -Dhadoop.log.dir=/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12 -Ddt.loggers.level=com.datatorrent.*:INFO,org.apache.*:INFO com.datatorrent.stram.engine.StreamingContainer 1>/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12/stdout 2>/data3/yarn/container-logs/application_1462863487071_0015/container_1462863487071_0015_01_12/stderr Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 Regards, Ananth
Re: Specify class loader policy
Hello Pramod, Yes. The description is correct about the problem. I was about to resolve this issue just yesterday by placing a different jar inside apex/lib that provides the same class that were causing the issue in the first place. The original javaee-api jar had additional classes that were interfering with the metrics and other capabilities of the engine. I do however suspect there is an issue with the classloader ( note that this is a guess only based on my observations) - This happens only for apps that have other JSR 303 based annotations ( Hibernate + Spring + JPA annotations in the code as was my case) - Apex engine uses the Apache BVal in spite of other validation jars being present in the application jars list - In this process, the Apex engine is trying to use its classloader to locate the jars that are expected to be part of the current class being loaded ( import statements) while being launched. - At this juncture, it fails with a class not found error/exception. In my case there was another jar from the Hibernate world which provided a subset of the classes that are available in javaee-api and by placing that jar ( hibernate-jpa-api jar) , I was able to see the metrics as well as launch the apps without issues. Thanks for your time on this. Regards, Ananth On Fri, May 6, 2016 at 11:43 PM, Pramod Immaneni <pra...@datatorrent.com> wrote: > Ananth, > > Thanks for the details. Just so that I have understood you correctly once > you drop the javaee-api-7.0 jar in the engine lib folder the app launches > successfully but that causes the monitoring service to not work or start > correctly. Would it be possible for you to share your app package. If it is > something you cannot share publicly in a list maybe you can send it to me > offline. > > Thanks > > On Wed, May 4, 2016 at 12:17 PM, Ananth Gundabattula < > agundabatt...@gmail.com> wrote: > >> Hello Pramod, >> >> Thanks for the reply. Apologies that I could not reply immediately. >> >> Here is our problem: >> >> We have an app that includes Spring + Hibernate + JPA along with other >> libraries that are part of the lib/ folder inside the "apa" package. The >> application runs perfectly fine in the "LocalMode" of a unit test. However >> when trying to deploy the app into the Apex server ( either using the >> command line dtcli or web ui) , there is an exception stating that the >> class javax.persistence.Persistence is not found. >> >> We did try including the javaee-api-7.0 jar in the maven dependencies of >> the app but to no avail. The only ugly hack was to copy this jar into the >> Apex engines lib folder. The apps start working functionally after this but >> we lost all capacity to view the "monitoring" of the app in the UI. >> >> This copying of the jar also resulted in the Apex engine not being able >> to restart if we want to restart it. We needed to move the jar out of the >> way , restart it , put this jar back in and then reinstall and rerun the >> apps. Of course this is all because of the ugly hack and we need to solve >> this properly. >> >> >> I believe the core issue is arising because the validation framework is >> pulling bval ( Apache Bean Validation ) for the JPA components that are >> present in the map. This is leading to the parent classloader somehow >> trying to look for classes in the lib folder of the Apex engine and >> ignoring the libs in the app lib folder. This is just a guess and am kind >> of stumped how to figure it out correctly. >> >> I did a dump of the class loaders and the jars they are loading at the >> launch time of the app. Please find attached a copy of the two main class >> loaders and their jars. I have also attached a copy of the stack trace of >> the error I am getting when I launch the app in Apex. >> >> >> >> >> Regards, >> Ananth >> >> On Wed, May 4, 2016 at 10:14 AM, Pramod Immaneni <pra...@datatorrent.com> >> wrote: >> >>> Hi Ananth, >>> >>> Are you facing a specific problem? We include the application jars first >>> in the classpath, followed by apex jars and then hadoop jars. >>> >>> Thanks >>> >>> On Tue, May 3, 2016 at 4:56 AM, Ananth Gundabattula < >>> agundabatt...@gmail.com> wrote: >>> >>>> Hello All, >>>> >>>> Is there a way we can specify the class loader policy for an Apex app ? >>>> Basically I am looking at something equivalent of the user first class >>>> loader policy available in the traditional hadoop world. ( >>>> mapreduce.job.user.classpath.first) >>>> >>>> Regards, >>>> Ananth >>>> >>> >>> >> >
Specify class loader policy
Hello All, Is there a way we can specify the class loader policy for an Apex app ? Basically I am looking at something equivalent of the user first class loader policy available in the traditional hadoop world. ( mapreduce.job.user.classpath.first) Regards, Ananth
Re: javax.persistence.Persistence classNotFoundError
Hello Siyuan, Please find attached the output of the unzip -t command on the apa package. The library jar is indeed present in there. What confuses me here is that the stack trace suggests that it is coming when the engine is validating the App ( and not from my application code). I am suspecting one of the following is causing this : - the bval ( JSR 303 ) jar validation path is somehow conflicting with the JPA validation framework and is causing this issue but am not sure. - CDH 5.7 might have changed restructured a few jars ?? ( I am not sure of this either as I am not sure how Apex uses the underlying Hadoop distribution jars ) Pasting the stack trace once more here for reference: An error occurred trying to launch the application. Server message: java.lang.NoClassDefFoundError: javax/persistence/Persistence at org.apache.bval.jsr303.resolver.JPATraversableResolver.isReachable(JPATraversableResolver.java:34) at org.apache.bval.jsr303.resolver.DefaultTraversableResolver.isReachable(DefaultTraversableResolver.java:60) at org.apache.bval.jsr303.resolver.CachingTraversableResolver.isReachable(CachingTraversableResolver.java:82) at org.apache.bval.jsr303.ConstraintValidation.isReachable(ConstraintValidation.java:241) at org.apache.bval.jsr303.ConstraintValidation.validate(ConstraintValidation.java:166) at org.apache.bval.jsr303.ConstraintValidation.validate(ConstraintValidation.java:141) at org.apache.bval.util.ValidationHelper.validateProperty(ValidationHelper.java:233) at org.apache.bval.util.ValidationHelper.validateBean(ValidationHelper.java:216) at org.apache.bval.jsr303.ClassValidator.validateBeanNet(ClassValidator.java:393) at org.apache.bval.jsr303.ClassValidator.validate(ClassValidator.java:149) at com.datatorrent.stram.plan.logical.LogicalPlan.validate(LogicalPlan.java:1672) at com.datatorrent.stram.StramClient.(StramClient.java:161) at com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:509) at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2050) at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3456) at com.datatorrent.stram.cli.DTCli.access$7100(DTCli.java:106) at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1895) at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449) Fatal error encountered Regards, Ananth On Tue, May 3, 2016 at 3:22 AM, hsy...@gmail.com <hsy...@gmail.com> wrote: > Hey Ananth, > > Can you unzip your apa file to see if it is in the lib folder? If it is > not there, it means the packaging is not correct. > > Regards, > Siyuan > > On Mon, May 2, 2016 at 4:39 AM, Ananth Gundabattula < > agundabatt...@gmail.com> wrote: > >> Hello All, >> >> I was able to get around the issue pasted above by manually copying the >> javaee-api-7.0 jar inside the lib folder of Apex engine. It may be noted >> that the code works perfectly fine in a unit test that launches all of the >> operators in the DAG. ( Works end to end ) . However when I launch it on >> CDH 5.7 I run into the exception pasted above. It may be noted that the >> exception is being raised when I launch the app and the Apex engine is >> trying to validate the newly deployed app. >> >> I am suspecting that this could be an issue with incompatibility with CDH >> 5.7 ( The Hadoop stack on which I am trying to run Apex 3.3 on ) >> >> Could anyone help me if they were able to run Apex on top of CDH 5.7. I >> got the same exception with both Apex 3.3 and 3.2. >> >> Regards, >> Ananth >> >> On Sat, Apr 30, 2016 at 11:34 AM, Ananth Gundabattula < >> agundabatt...@gmail.com> wrote: >> >>> Hello All, >>> >>> I am getting the following exception when I launch my Apex app. >>> >>> I tried including javaee-api- version 7.0 as well to get around this >>> error but does not seem to take effect. >>> >>> Any ideas why javax.persistence.Persistence is not getting loaded by the >>> APex engine classloader ? >>> >>> An error occurred trying to launch the application. Server message: >>>> java.lang.NoClassDefFoundError: javax/persistence/Persistence at >>>> org.apache.bval.jsr303.resolver.JPATraversableResolver.isReachable(JPATraversableResolver.java:34) >>>> at >>>> org.apache.bval.jsr303.resolver.DefaultTraversableResolver.isReachable(DefaultTraversableResolver.java:60) >>>> at >>>> org.apache.bval.jsr303.resolver.CachingTraversableResolver.isReachable(CachingTraversableResolver.java:82) >>>> at >>>> org.apache.bval.jsr303.ConstraintValidation.isReachable(ConstraintValidation.java:241) >>>> at >>>> org.apache.bval.jsr303.ConstraintValidation.validat
Re: javax.persistence.Persistence classNotFoundError
Hello All, I was able to get around the issue pasted above by manually copying the javaee-api-7.0 jar inside the lib folder of Apex engine. It may be noted that the code works perfectly fine in a unit test that launches all of the operators in the DAG. ( Works end to end ) . However when I launch it on CDH 5.7 I run into the exception pasted above. It may be noted that the exception is being raised when I launch the app and the Apex engine is trying to validate the newly deployed app. I am suspecting that this could be an issue with incompatibility with CDH 5.7 ( The Hadoop stack on which I am trying to run Apex 3.3 on ) Could anyone help me if they were able to run Apex on top of CDH 5.7. I got the same exception with both Apex 3.3 and 3.2. Regards, Ananth On Sat, Apr 30, 2016 at 11:34 AM, Ananth Gundabattula < agundabatt...@gmail.com> wrote: > Hello All, > > I am getting the following exception when I launch my Apex app. > > I tried including javaee-api- version 7.0 as well to get around this error > but does not seem to take effect. > > Any ideas why javax.persistence.Persistence is not getting loaded by the > APex engine classloader ? > > An error occurred trying to launch the application. Server message: >> java.lang.NoClassDefFoundError: javax/persistence/Persistence at >> org.apache.bval.jsr303.resolver.JPATraversableResolver.isReachable(JPATraversableResolver.java:34) >> at >> org.apache.bval.jsr303.resolver.DefaultTraversableResolver.isReachable(DefaultTraversableResolver.java:60) >> at >> org.apache.bval.jsr303.resolver.CachingTraversableResolver.isReachable(CachingTraversableResolver.java:82) >> at >> org.apache.bval.jsr303.ConstraintValidation.isReachable(ConstraintValidation.java:241) >> at >> org.apache.bval.jsr303.ConstraintValidation.validate(ConstraintValidation.java:166) >> at >> org.apache.bval.jsr303.ConstraintValidation.validate(ConstraintValidation.java:141) >> at >> org.apache.bval.util.ValidationHelper.validateProperty(ValidationHelper.java:233) >> at >> org.apache.bval.util.ValidationHelper.validateBean(ValidationHelper.java:216) >> at >> org.apache.bval.jsr303.ClassValidator.validateBeanNet(ClassValidator.java:393) >> at org.apache.bval.jsr303.ClassValidator.validate(ClassValidator.java:149) >> at >> com.datatorrent.stram.plan.logical.LogicalPlan.validate(LogicalPlan.java:1672) >> at com.datatorrent.stram.StramClient.(StramClient.java:161) at >> com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:509) >> at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2050) >> at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3456) at >> com.datatorrent.stram.cli.DTCli.access$7100(DTCli.java:106) at >> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1895) at >> com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449) Fatal error >> encountered > > > Regards, > Ananth >
WebService errors in DT 3.3/3.2
Hello All,
I am trying to launch my app on Apex/DT 3.3 version running on top of CDH
5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
deployments show the following behavior. (We have a requirement to run Apex
on CDH 5.7 )
I was wondering if you could advise on any workarounds for the following
issue:
I am not able to see any metrics in the web console when I open the
"Monitor" section. The application code seems to be working fine as I see
data flow happening fine as expected. However I see no metrics ( even the
top level container metrics) .
One more observed issue is that when I click on the container ID from the
monitor section, the UI crashes. ( In the sense the UI simply does not show
anything ). I am however able to get some preliminary UI ( barring the
metrics ) if I click on the container link from the monitor section if the
application is in the "ACCEPTED" state. The container link does not work
the moment the app moves from ACCEPTED to Running State.
The DtGateway shows the following logs :
2016-05-02 11:08:48,964 WARN com.datatorrent.stram.client.StramAgent:
> Caught exception when retrieving web service info for app
> application_1462007878015_0037
> org.codehaus.jettison.json.JSONException: A JSONObject text must begin
> with '{' at character 1 of
>
>
> Error 500
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>
> HTTP ERROR 500
> Problem accessing /ws. Reason:
>
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;Caused
> by:java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>
> Powered by Jetty://
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> at
> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
> at org.codehaus.jettison.json.JSONObject.(JSONObject.java:169)
> at org.codehaus.jettison.json.JSONObject.(JSONObject.java:266)
> at
> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
> at
>
javax.persistence.Persistence classNotFoundError
Hello All, I am getting the following exception when I launch my Apex app. I tried including javaee-api- version 7.0 as well to get around this error but does not seem to take effect. Any ideas why javax.persistence.Persistence is not getting loaded by the APex engine classloader ? An error occurred trying to launch the application. Server message: > java.lang.NoClassDefFoundError: javax/persistence/Persistence at > org.apache.bval.jsr303.resolver.JPATraversableResolver.isReachable(JPATraversableResolver.java:34) > at > org.apache.bval.jsr303.resolver.DefaultTraversableResolver.isReachable(DefaultTraversableResolver.java:60) > at > org.apache.bval.jsr303.resolver.CachingTraversableResolver.isReachable(CachingTraversableResolver.java:82) > at > org.apache.bval.jsr303.ConstraintValidation.isReachable(ConstraintValidation.java:241) > at > org.apache.bval.jsr303.ConstraintValidation.validate(ConstraintValidation.java:166) > at > org.apache.bval.jsr303.ConstraintValidation.validate(ConstraintValidation.java:141) > at > org.apache.bval.util.ValidationHelper.validateProperty(ValidationHelper.java:233) > at > org.apache.bval.util.ValidationHelper.validateBean(ValidationHelper.java:216) > at > org.apache.bval.jsr303.ClassValidator.validateBeanNet(ClassValidator.java:393) > at org.apache.bval.jsr303.ClassValidator.validate(ClassValidator.java:149) > at > com.datatorrent.stram.plan.logical.LogicalPlan.validate(LogicalPlan.java:1672) > at com.datatorrent.stram.StramClient.(StramClient.java:161) at > com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:509) > at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2050) > at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3456) at > com.datatorrent.stram.cli.DTCli.access$7100(DTCli.java:106) at > com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1895) at > com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449) Fatal error > encountered Regards, Ananth
Re: Best practises to specify the Application
Just wanted to update you all that upgrading the JDK on Apex server did the trick. Thanks a lot for your time David/Shubham/Sandeep/Ram. On Sat, Apr 30, 2016 at 6:51 AM, Ananth Gundabattula < agundabatt...@gmail.com> wrote: > > I guess the issue is related to the fact that the runtime jdk which Apex > server engine is running on a lower version than that of the compiled app > and not the archetype issue at all. I did attempt with the archetype as > well but came up with the same effect. The app is compiled using 1.8 > source + target but the Apex server is running on a machine with jdk 1.7 > installed. That was the root cause. By the way, the original pom project > artifact had the exact project structure as that was generated by the > archetype. I shall make sure I use the archetype from hence on. The pom > attached also explictly defines the jdk versions for both source and > target. > > The web UI while installing the apa package failed with a Fatal error and > on refreshing showed the entry for the apa package but did not show any > applications inside it. I guess we have a bug in the UI wherein an improper > installation of an app results in an entry being shown in the UI. I believe > it should not show the app entry at all if the installation fails. > > Regards, > Ananth > > On Sat, Apr 30, 2016 at 3:54 AM, David Yan <da...@datatorrent.com> wrote: > >> Hi Ananth, >> >> Please always use the archetype to generate your project. >> In the archetype, among many other things, it also enforces that the >> target java class files to be compatible with JDK 1.7. >> And it seems that your project is missing that as well and that's why you >> see the UnsupportedClassVersionError error. >> >> David >> >> On Fri, Apr 29, 2016 at 1:20 AM, Ananth Gundabattula < >> agundabatt...@gmail.com> wrote: >> >>> I had a look at the dtgateway.log and observed the following stacktrace >>> : I guess the root cause might be Apex running on a lower version of java >>> and the application code compiled with a higher version. I shall update >>> everyone after the upgrade to see if it fixes the issue. >>> >>> >>> Fatal error encountered >>>> >>>> 2016-04-29 08:04:45,181 ERROR >>>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception >>>> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError: >>>> com/tx/y/z/Operator : Unsupported major.minor version 52.0 >>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >>>> at >>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>>> at >>>> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297) >>>> at >>>> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010) >>>> at >>>> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755) >>>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449) >>>> >>> >>> >>> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula < >>> agundabatt...@gmail.com> wrote: >>> >>>> Thanks Shubham. >>>> >>>> Those properties were being set by the parent pom which are inherited >>>> by the current pom.xml ( Kind of wanted all apex apps to have same version >>>> structure etc) and hence added that property as part of parent pom. >>>> >>>> I am still stumped as to what is causing this. >>>> >>>> Regards, >>>> Ananth >>>> >>>> >>>> >>>> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak < >>>> shub...@datatorrent.com> wrote: >>>> >>>>> Hi Ana
Re: Best practises to specify the Application
I guess the issue is related to the fact that the runtime jdk which Apex server engine is running on a lower version than that of the compiled app and not the archetype issue at all. I did attempt with the archetype as well but came up with the same effect. The app is compiled using 1.8 source + target but the Apex server is running on a machine with jdk 1.7 installed. That was the root cause. By the way, the original pom project artifact had the exact project structure as that was generated by the archetype. I shall make sure I use the archetype from hence on. The pom attached also explictly defines the jdk versions for both source and target. The web UI while installing the apa package failed with a Fatal error and on refreshing showed the entry for the apa package but did not show any applications inside it. I guess we have a bug in the UI wherein an improper installation of an app results in an entry being shown in the UI. I believe it should not show the app entry at all if the installation fails. Regards, Ananth On Sat, Apr 30, 2016 at 3:54 AM, David Yan <da...@datatorrent.com> wrote: > Hi Ananth, > > Please always use the archetype to generate your project. > In the archetype, among many other things, it also enforces that the > target java class files to be compatible with JDK 1.7. > And it seems that your project is missing that as well and that's why you > see the UnsupportedClassVersionError error. > > David > > On Fri, Apr 29, 2016 at 1:20 AM, Ananth Gundabattula < > agundabatt...@gmail.com> wrote: > >> I had a look at the dtgateway.log and observed the following stacktrace : >> I guess the root cause might be Apex running on a lower version of java and >> the application code compiled with a higher version. I shall update >> everyone after the upgrade to see if it fixes the issue. >> >> >> Fatal error encountered >>> >>> 2016-04-29 08:04:45,181 ERROR >>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception >>> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError: >>> com/tx/y/z/Operator : Unsupported major.minor version 52.0 >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >>> at >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>> at >>> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297) >>> at >>> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010) >>> at >>> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755) >>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449) >>> >> >> >> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula < >> agundabatt...@gmail.com> wrote: >> >>> Thanks Shubham. >>> >>> Those properties were being set by the parent pom which are inherited by >>> the current pom.xml ( Kind of wanted all apex apps to have same version >>> structure etc) and hence added that property as part of parent pom. >>> >>> I am still stumped as to what is causing this. >>> >>> Regards, >>> Ananth >>> >>> >>> >>> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shub...@datatorrent.com >>> > wrote: >>> >>>> Hi Ananth, >>>> >>>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex >>>> -DarchetypeArtifactId=apex-app-archetype >>>> -DarchetypeVersion=3.3.0-incubating >>>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp >>>> -Dversion=1.0-SNAPSHOT >>>> and it works for me as well. >>>> >>>> I compared the pom.xml file and noticed values for following properties >>>> were missing. >>>> >>>> >>>> >
Re: Best practises to specify the Application
I had a look at the dtgateway.log and observed the following stacktrace : I
guess the root cause might be Apex running on a lower version of java and
the application code compiled with a higher version. I shall update
everyone after the upgrade to see if it fixes the issue.
Fatal error encountered
>
> 2016-04-29 08:04:45,181 ERROR
> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> com/tx/y/z/Operator : Unsupported major.minor version 52.0
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> at
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> at
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>
On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
agundabatt...@gmail.com> wrote:
> Thanks Shubham.
>
> Those properties were being set by the parent pom which are inherited by
> the current pom.xml ( Kind of wanted all apex apps to have same version
> structure etc) and hence added that property as part of parent pom.
>
> I am still stumped as to what is causing this.
>
> Regards,
> Ananth
>
>
>
> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shub...@datatorrent.com>
> wrote:
>
>> Hi Ananth,
>>
>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>> -Dversion=1.0-SNAPSHOT
>> and it works for me as well.
>>
>> I compared the pom.xml file and noticed values for following properties
>> were missing.
>>
>>
>>
>> ${apache.apex.apppackage.classpath}
>>
>> ${apache.apex.engine.version}
>>
>>
>>
>> Could you add the following to pom.xml and try again.
>>
>>
>>
>> 3.3.0-incubating
>>
>> lib/*.jar
>>
>>
>>
>> Thanks,
>> Shubham
>>
>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>> agundabatt...@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> I reattempted the packaging with the archetype based approach. I guess
>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>> pointing it out.
>>>
>>> The default package generated by the archetype command seems to be
>>> working fine when uploading it but the moment I add in my code and
>>> dependencies it goes back the issue I am seeing :
>>>
>>>
>>> My app package looks like app.zip ( I removed all of the
>>> application.class for brevity sake ) . Please note that I deleted all the
>>> jars under /lib folder and the contents of the /app folder in the apa
>>> pacakge.
>>>
>>> I have also attached a copy of my pom.xml file.
>>>
>>> Could you please advise if you something amiss right away in the
>>> structure below ? It may be noted that I retained the classes of the
>>> original simple project generated by the archetype and even they stopped
>>> showing with the launch button the moment I added in all of the application
>>> dependencies in maven.
>>>
>>> Regards,
>>> Ananth
>>>
>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> I just tried with 3.3.0-incubating with the following and it's working:
>>>>
>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>&
Re: Best practises to specify the Application
.RELEASE.jar OK testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK testing: lib/spring-expression-4.1.9.RELEASE.jar OK testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK testing: lib/spring-orm-4.1.9.RELEASE.jar OK testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK testing: lib/spring-tx-4.1.9.RELEASE.jar OK testing: lib/spring-web-4.1.9.RELEASE.jar OK testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK testing: lib/super-csv-2.3.1.jar OK testing: lib/super-csv-joda-2.3.1.jar OK testing: lib/xml-apis-1.0.b2.jar OK testing: lib/zookeeper-3.4.6.jar OK testing: conf/OK testing: conf/properties.xml OK No errors detected in compressed data of cassandra-events-1.2-SNAPSHOT.apa. Contents of the properties.xml file under the conf folder is here: dt.application.CED.class com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp An alias for the application dt.operator.*.attr.MEMORY_MB 512 dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER com.datatorrent.common.partitioner.StatelessPartitioner:10 dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT 20 Regards, Ananth On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com> wrote: > Hi Ananth, > > Can you do an "unzip -t" on your apa file and paste the output here? > Make sure the jar containing the application classes should be in the app > directory inside the apa. > > Can I know why the apex-app-archetype is not used in your apa project? > > David > > On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula < > agundabatt...@gmail.com> wrote: > >> Hello All, >> >> I am having the following issue with my application. I have a maven >> project that has a class called XYZ that extends StreamingApplication. >> >> My maven packaging is ensuring that the final "apa" extension package has >> the right folder structure if exploded on a disk. >> >> However I am not able to locate the Application name and the "launch" >> button next to it after uploading the "apa" package via the browser. >> >> I tried looking at the docs to see how to specify a class as an App that >> can be launched from a packaged application but could not find much. Could >> anyone of you please point me to the right way of associating a class as an >> application that will get enabled with a launch button in the web ui ? The >> class XYZ has the annotation called @ApplicationAnnotation on top of it. >> >> Regards, >> Ananth >> > >
Re: Unit tests not picking maven dependencies for LocalMode
Hello Chinmay,
Here is the pom snippet :
net.sf.opencsv
opencsv
${open-csv-version}
The dependency is being injected via this module dependency:
Apex Application (main module ) => Module B => Module C => opencsv (
dependency above).
I guess I need to also share with you the fact that a part of the
application logic framework is being booted up by using a spring context.
The Apex app in the operator tries to "inject a Spring Context" and pull
out a bean that uses the OpenCSV class inside it. The class that is
obtained as a bean is then used in the process method of the operator.
eventDetailsStatementGenerator = (EventDetailsStatementGenerator)
CassandraEventDetailsAppContext.getCassandraEventDetailsAppContext()
.getIOCContext().getBean(DWHAppBeanNames.EVENTDETAILS_STMT_GENERATOR_BEAN_NAME);
The above method is being called in the activate() method of the operator.
The operator uses a SpringContext booted up via a
AnnotationApplicationConfig class from Spring.
Regards,
Ananth
On Fri, Apr 22, 2016 at 5:11 PM, Chinmay Kolhatkar <chin...@apache.org>
wrote:
> Hi Ananth,
>
> You can have a full cycle of run in local mode for apex app. This seems to
> be a dependency related issue.
>
> Can you please share the code snippet of pom.xml where the missing maven
> dependency is added?
> Also, are you trying the load a class from that jar dependency using any
> class loader?
>
> Thanks,
> Chinmay.
>
>
> On Fri, Apr 22, 2016 at 12:24 PM, Ananth Gundabattula <
> agundabatt...@gmail.com> wrote:
>
>>
>> Hello All,
>>
>> Some of my unit tests are not able to execute a flow end to end because a
>> jar is not being picked up at runtime though it is part of the maven
>> dependencies etc …
>>
>> I believe in the actual scenario when deploying it to an Apex server, the
>> metadata file that is used in packaging the application artefacts specifies
>> an application path etc. However in my unit tests , this does not seem to
>> apply/take effect. Am I right in assuming that we cannot achieve a full
>> cycle of run using the local mode setup of the Apex app in a unit test ?
>>
>> Are there any recommendations to get around this issue ?
>>
>> Here is the code snippet that I am trying :
>>
>> @Test
>> public void testApplication() throws IOException, Exception {
>> try {
>> LocalMode lma = LocalMode.newInstance();
>> Configuration conf = new Configuration(false);
>>
>> conf.addResource(this.getClass().getResourceAsStream("/dag-test-props.xml"));
>> lma.prepareDAG(new CassandraEventDetailsStreamingApp(), conf);
>> LocalMode.Controller lc = lma.getController();
>> lc.run();
>> } catch (ConstraintViolationException e) {
>> Assert.fail("constraint violations: " + e.getConstraintViolations());
>> }
>> }
>>
>>
>>
>> Regards,
>> Ananth
>>
>
>
Re: A few questions about operator
Thanks all for your time and responses.
On Wed, Apr 20, 2016 at 9:08 AM, hsy...@gmail.com <hsy...@gmail.com> wrote:
> Hey Ananth,
>
> There is no OffsetManager in 0.9 kafka operator because the operator
> commit the offsets(all data that has been processed by the entire
> application) in kafka implicitly. The offset is supposed to be stored with
> consumer id, and here the consumer id is the application name. So if you
> don't want to start the whole application from last checkpoint but also you
> want the kafka input operator to continue consuming from where the
> application is left off, you can start a new application with same name.
>
> Regards,
> Siyuan
>
> On Tue, Apr 19, 2016 at 3:08 PM, Ashwin Chandra Putta <
> ashwinchand...@gmail.com> wrote:
>
>> Hey Ananth,
>>
>> Please find the answers inline.
>>
>> Regards,
>> Ashwin.
>>
>> On Tue, Apr 19, 2016 at 12:20 PM, Ananth Gundabattula <
>> agundabatt...@gmail.com> wrote:
>>
>>> Hello All,
>>>
>>> I was wondering if you could help me with the following questions as I
>>> was not able to locate the info from the docs:
>>>
>>>
>>>- When is the constructor of an operator called ? The docs say once
>>>in the lifetime of an operator but I was wondering whether the definition
>>>of "lifetime" spans across start/stop/crash ( because of a coding error )
>>>of an apex application ?
>>>
>>> [Ashwin] The operator has a life cycle. Constructor --> checkpointed
>> state applied --> setup() --> loop {beginWindow() --> loop {process() for
>> each input port} --> endWindow()} --> tearDown(). When an operator
>> recovers, the cycle is the same. The checkpointed state is the state from
>> last known checkpoint before the operator crashed.
>>
>>>
>>>- Regarding backpressure and the buffer server, how does the buffer
>>>server survive application crashes ? I mean considering a situation when
>>>the bufferserver itself dies for whatever reason, is it guaranteed a
>>>downstream operator will eventually catchup with an Upstream operator
>>> when
>>>the buffer server is brought back up?
>>>
>>> [Ashwin] The buffer server always lives with the upstream operator. In
>> fact, it lives within the same JVM as upstream operator. If an operator
>> fails, the upstream operator's buffer server will have the required data
>> state. If the upstream operator fails, its upstream operator's buffer
>> server has the data state and so on.
>>
>>>
>>>- Is there an equivalent of the OffsetManager for the 0.9 version of
>>>the Kafka operator ?
>>>- Am I correct in assuming that the moment we rename an application,
>>>the semantics of the Kafka operator will completely change and might end
>>> up
>>>reading from the "initialOffset" by the application code ? How is the
>>>semantics maintained for a definition of "application name" ? Does every
>>>deploy of the application code result in a new application or it is
>>> simply
>>>using the @ApplicationAnnotation(name="") instance to define this
>>> meaning ?
>>>
>>> [Ashwin] You can launch an application from its previous state by using
>> the -originalAppId parameter and provide the yarn application id from
>> its previous run's checkpointed state, it should apply to all operator
>> within the dag including the kafka input operator. You can also provide a
>> new name for the application using attribute dt.attr.APPLICATION_NAME.
>> eg: launch pi-demo-3.4.0-incubating-SNAPSHOT.apa -originalAppId
>> application_1459879799578_8727 -Ddt.attr.APPLICATION_NAME="pidemo v201"
>>
>>>
>>> Thanks for your time.
>>>
>>>
>>> Regards,
>>>
>>> Ananth
>>>
>>
>>
>>
>> --
>>
>> Regards,
>> Ashwin.
>>
>
>
A few questions about operator
Hello All, I was wondering if you could help me with the following questions as I was not able to locate the info from the docs: - When is the constructor of an operator called ? The docs say once in the lifetime of an operator but I was wondering whether the definition of "lifetime" spans across start/stop/crash ( because of a coding error ) of an apex application ? - Regarding backpressure and the buffer server, how does the buffer server survive application crashes ? I mean considering a situation when the bufferserver itself dies for whatever reason, is it guaranteed a downstream operator will eventually catchup with an Upstream operator when the buffer server is brought back up? - Is there an equivalent of the OffsetManager for the 0.9 version of the Kafka operator ? - Am I correct in assuming that the moment we rename an application, the semantics of the Kafka operator will completely change and might end up reading from the "initialOffset" by the application code ? How is the semantics maintained for a definition of "application name" ? Does every deploy of the application code result in a new application or it is simply using the @ApplicationAnnotation(name="") instance to define this meaning ? Thanks for your time. Regards, Ananth
Re: Apex engine and malhar maven versions for Kafka 0.9 version operator
Thanks for the clarification Siyuan.
On Sun, Apr 17, 2016 at 3:34 PM, hsy...@gmail.com <hsy...@gmail.com> wrote:
> Hey Ananth,
>
> We have semantic version control. For public library/Api, they should stay
> no change in major release, so most operator in 3.3.x should be able to
> work with 3.3.0 engine. The only exception is Api that marked
> @evolving(nonstable). Kafka 0.9 is one of them. But I don't see any feature
> of that operator couldn't work with 3.3.0 engine. And we will try best to
> not change the public interface for Kafka 0.9 operator, unless there is a
> big demand for some new feature from the community
>
> Regards,
> Siyuan
>
>
> On Saturday, April 16, 2016, Ananth Gundabattula <agundabatt...@gmail.com>
> wrote:
>
>> Hello Siyuan/Ram,
>>
>> Thanks for the inputs.
>>
>> I was able to resolve the problem by excluding the dependencies of
>> malhar-library and malhar-contrib from the dependency sections of the
>> apex-engine , apex-api .
>>
>> This made the 3.3.1-incubating version of the malhar kick into the
>> classpath and subsequently the malhar-kafka library with 3.3.1-incubating
>> version) .
>>
>> Thanks for your time.
>>
>> This brings in one more question : Is it an acceptable practice that I
>> package malhar-operator library in the lib path of my "apa" package ? In my
>> case I had version 3.3.1 in my packaging structure which I want in my
>> runtime as well ( due to 0.9 kafka operator ) but the apex engine itself is
>> a version behind ( 3.3.0 ) .. I believe malhar-versions and apex-version
>> will be on different pace with respect to evolution and versioning ?
>>
>>
>>
>> Regards,
>> Ananth
>>
>> On Sun, Apr 17, 2016 at 1:43 PM, hsy...@gmail.com <hsy...@gmail.com>
>> wrote:
>>
>>> Hi Ananth,
>>>
>>> Could you please try to add malhar-library in test scope? Thanks!
>>>
>>> Regards,
>>> Siyuan
>>>
>>>
>>> On Saturday, April 16, 2016, Ananth Gundabattula <
>>> agundabatt...@gmail.com> wrote:
>>>
>>>> Hello all,
>>>>
>>>> I would like to run a unit test code using the new Kafka Operator that
>>>> supports the 0.9 version protocol.
>>>>
>>>> In this process, I included the malhar-kafka library version (
>>>> 3.3.1-incubating ) and am using the apex-engine ( version 3.3.0 ) with
>>>> as test/provided.
>>>>
>>>> The compilation works fine but my unit tests fail to run properly with
>>>> " java.lang.ClassNotFoundException:
>>>> com.datatorrent.lib.util.KryoCloneUtils" exception.
>>>>
>>>>
>>>> What is the recommended way to run a unit test which uses Kafka 0.9
>>>> operator integrated with the Apex engine ? I am assuming that the
>>>> malhar-contrib library kafka operator is not 0.9 compliant ..
>>>>
>>>> The unit test code is like this :
>>>>
>>>> The class CassandraEventDetailsStreamingApp extends the
>>>> AbstractKafkaInputOperator in the below snippet of code.
>>>>
>>>> The exception arises in the method lma.getController();
>>>>
>>>> @Test
>>>> public void testApplication() throws IOException, Exception {
>>>> try {
>>>> LocalMode lma = LocalMode.newInstance();
>>>> Configuration conf = new Configuration(false);
>>>>
>>>> conf.addResource(this.getClass().getResourceAsStream("/dag-test-props.xml"));
>>>> lma.prepareDAG(new CassandraEventDetailsStreamingApp(), conf);
>>>> LocalMode.Controller lc = lma.getController();
>>>> lc.run();
>>>> } catch (ConstraintViolationException e) {
>>>> Assert.fail("constraint violations: " +
>>>> e.getConstraintViolations());
>>>> }
>>>> }
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>
>>
Re: Apex engine and malhar maven versions for Kafka 0.9 version operator
Hello Siyuan/Ram,
Thanks for the inputs.
I was able to resolve the problem by excluding the dependencies of
malhar-library and malhar-contrib from the dependency sections of the
apex-engine , apex-api .
This made the 3.3.1-incubating version of the malhar kick into the
classpath and subsequently the malhar-kafka library with 3.3.1-incubating
version) .
Thanks for your time.
This brings in one more question : Is it an acceptable practice that I
package malhar-operator library in the lib path of my "apa" package ? In my
case I had version 3.3.1 in my packaging structure which I want in my
runtime as well ( due to 0.9 kafka operator ) but the apex engine itself is
a version behind ( 3.3.0 ) .. I believe malhar-versions and apex-version
will be on different pace with respect to evolution and versioning ?
Regards,
Ananth
On Sun, Apr 17, 2016 at 1:43 PM, hsy...@gmail.com <hsy...@gmail.com> wrote:
> Hi Ananth,
>
> Could you please try to add malhar-library in test scope? Thanks!
>
> Regards,
> Siyuan
>
>
> On Saturday, April 16, 2016, Ananth Gundabattula <agundabatt...@gmail.com>
> wrote:
>
>> Hello all,
>>
>> I would like to run a unit test code using the new Kafka Operator that
>> supports the 0.9 version protocol.
>>
>> In this process, I included the malhar-kafka library version (
>> 3.3.1-incubating ) and am using the apex-engine ( version 3.3.0 ) with
>> as test/provided.
>>
>> The compilation works fine but my unit tests fail to run properly with "
>> java.lang.ClassNotFoundException: com.datatorrent.lib.util.KryoCloneUtils"
>> exception.
>>
>>
>> What is the recommended way to run a unit test which uses Kafka 0.9
>> operator integrated with the Apex engine ? I am assuming that the
>> malhar-contrib library kafka operator is not 0.9 compliant ..
>>
>> The unit test code is like this :
>>
>> The class CassandraEventDetailsStreamingApp extends the
>> AbstractKafkaInputOperator in the below snippet of code.
>>
>> The exception arises in the method lma.getController();
>>
>> @Test
>> public void testApplication() throws IOException, Exception {
>> try {
>> LocalMode lma = LocalMode.newInstance();
>> Configuration conf = new Configuration(false);
>>
>> conf.addResource(this.getClass().getResourceAsStream("/dag-test-props.xml"));
>> lma.prepareDAG(new CassandraEventDetailsStreamingApp(), conf);
>> LocalMode.Controller lc = lma.getController();
>> lc.run();
>> } catch (ConstraintViolationException e) {
>> Assert.fail("constraint violations: " + e.getConstraintViolations());
>> }
>> }
>>
>>
>>
>> Regards,
>> Ananth
>>
>
Apex engine and malhar maven versions for Kafka 0.9 version operator
Hello all,
I would like to run a unit test code using the new Kafka Operator that
supports the 0.9 version protocol.
In this process, I included the malhar-kafka library version (
3.3.1-incubating ) and am using the apex-engine ( version 3.3.0 ) with
as test/provided.
The compilation works fine but my unit tests fail to run properly with "
java.lang.ClassNotFoundException: com.datatorrent.lib.util.KryoCloneUtils"
exception.
What is the recommended way to run a unit test which uses Kafka 0.9
operator integrated with the Apex engine ? I am assuming that the
malhar-contrib library kafka operator is not 0.9 compliant ..
The unit test code is like this :
The class CassandraEventDetailsStreamingApp extends the
AbstractKafkaInputOperator in the below snippet of code.
The exception arises in the method lma.getController();
@Test
public void testApplication() throws IOException, Exception {
try {
LocalMode lma = LocalMode.newInstance();
Configuration conf = new Configuration(false);
conf.addResource(this.getClass().getResourceAsStream("/dag-test-props.xml"));
lma.prepareDAG(new CassandraEventDetailsStreamingApp(), conf);
LocalMode.Controller lc = lma.getController();
lc.run();
} catch (ConstraintViolationException e) {
Assert.fail("constraint violations: " + e.getConstraintViolations());
}
}
Regards,
Ananth
