Interesting. Which hbase / Phoenix releases are you using ? The following method has been removed from Put:
public Put setWriteToWAL(boolean write) { Please make sure the Phoenix release is compatible with your hbase version. Cheers On Fri, Jan 15, 2016 at 6:20 AM, Siddharth Ubale < siddharth.ub...@syncoms.com> wrote: > Hi, > > > > > > This is the log from the application : > > > > 16/01/15 19:23:19 INFO yarn.ApplicationMaster: Unregistering > ApplicationMaster with SUCCEEDED (diag message: Shutdown hook called before > final status was reported.) > > 16/01/15 19:23:19 INFO yarn.ApplicationMaster: Deleting staging directory > .sparkStaging/application_1452763526769_0011 > > 16/01/15 19:23:19 INFO remote.RemoteActorRefProvider$RemotingTerminator: > Shutting down remote daemon. > > 16/01/15 19:23:19 INFO remote.RemoteActorRefProvider$RemotingTerminator: > Remote daemon shut down; proceeding with flushing remote transports. > > 16/01/15 19:23:19 INFO remote.RemoteActorRefProvider$RemotingTerminator: > Remoting shut down. > > 16/01/15 19:23:19 INFO util.Utils: Shutdown hook called > > 16/01/15 19:23:19 INFO > client.HConnectionManager$HConnectionImplementation: Closing zookeeper > sessionid=0x1523f753f6f0061 > > 16/01/15 19:23:19 INFO zookeeper.ClientCnxn: EventThread shut down > > 16/01/15 19:23:19 INFO zookeeper.ZooKeeper: Session: 0x1523f753f6f0061 > closed > > 16/01/15 19:23:19 ERROR yarn.ApplicationMaster: User class threw > exception: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.client.Put.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Put; > > java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.client.Put.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Put; > > at > org.apache.phoenix.schema.PTableImpl$PRowImpl.newMutations(PTableImpl.java:639) > > at > org.apache.phoenix.schema.PTableImpl$PRowImpl.<init>(PTableImpl.java:632) > > at > org.apache.phoenix.schema.PTableImpl.newRow(PTableImpl.java:557) > > at > org.apache.phoenix.schema.PTableImpl.newRow(PTableImpl.java:573) > > at > org.apache.phoenix.execute.MutationState.addRowMutations(MutationState.java:185) > > at > org.apache.phoenix.execute.MutationState.access$200(MutationState.java:79) > > at > org.apache.phoenix.execute.MutationState$2.init(MutationState.java:258) > > at > org.apache.phoenix.execute.MutationState$2.<init>(MutationState.java:255) > > at > org.apache.phoenix.execute.MutationState.toMutations(MutationState.java:253) > > at > org.apache.phoenix.execute.MutationState.toMutations(MutationState.java:243) > > at > org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1840) > > at > org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:744) > > at > org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:186) > > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:303) > > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:295) > > at > org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:293) > > at > org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1236) > > at > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1891) > > at > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1860) > > at > org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77) > > at > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1860) > > at > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162) > > at > org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:131) > > at > org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133) > > at > java.sql.DriverManager.getConnection(DriverManager.java:664) > > at > java.sql.DriverManager.getConnection(DriverManager.java:270) > > at > spark.phoenix.PhoenixConnect.getConnection(PhoenixConnect.java:26) > > at > spark.stream.eventStream.startStream(eventStream.java:105) > > at > time.series.wo.agg.InputStreamSpark.main(InputStreamSpark.java:38) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:497) > > at > org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) > > Thanks, > > Siddharth > > > > > > *From:* Ted Yu [mailto:yuzhih...@gmail.com] > *Sent:* Friday, January 15, 2016 7:43 PM > *To:* Siddharth Ubale <siddharth.ub...@syncoms.com> > *Cc:* user@spark.apache.org > *Subject:* Re: Spark App -Yarn-Cluster-Mode ===> Hadoop_conf_**.zip file. > > > > bq. check application tracking > page:http://slave1:8088/proxy/application_1452763526769_0011/ > Then <http://slave1:8088/proxy/application_1452763526769_0011/Then>, ... > > > > Have you done the above to see what error was in each attempt ? > > > > Which Spark / hadoop release are you using ? > > > > Thanks > > > > On Fri, Jan 15, 2016 at 5:58 AM, Siddharth Ubale < > siddharth.ub...@syncoms.com> wrote: > > Hi, > > > > I am trying to run a Spark streaming application in yarn-cluster mode. > However I am facing an issue where the job ends asking for a particular > Hadoop_conf_**.zip file in hdfs location. > > Can any one guide with this? > > The application works fine in local mode only it stops abruptly for want > of memory. > > > > Below is the error stack trace: > > > > diagnostics: Application application_1452763526769_0011 failed 2 times due > to AM Container for appattempt_1452763526769_0011_000002 exited with > exitCode: -1000 > > For more detailed output, check application tracking page: > http://slave1:8088/proxy/application_1452763526769_0011/Then, click on > links to logs of each attempt. > > Diagnostics: File does not exist: > hdfs://slave1:9000/user/hduser/.sparkStaging/application_1452763526769_0011/__hadoop_conf__1057113228186399290.zip > > *java.io.FileNotFoundException: File does not exist: > hdfs://slave1:9000/user/hduser/.sparkStaging/application_1452763526769_0011/__hadoop_conf__1057113228186399290.zip* > > at > org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1122) > > at > org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114) > > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114) > > at > org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251) > > at > org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61) > > at > org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359) > > at > org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357) > > at java.security.AccessController.doPrivileged(Native > Method) > > at javax.security.auth.Subject.doAs(Subject.java:422) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > > at > org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356) > > at > org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60) > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > Failing this attempt. Failing the application. > > ApplicationMaster host: N/A > > ApplicationMaster RPC port: -1 > > queue: default > > start time: 1452866026622 > > final status: FAILED > > tracking URL: > http://slave1:8088/cluster/app/application_1452763526769_0011 > > user: hduser > > Exception in thread "main" org.apache.spark.SparkException: Application > application_1452763526769_0011 finished with failed status > > at > org.apache.spark.deploy.yarn.Client.run(Client.scala:841) > > at > org.apache.spark.deploy.yarn.Client$.main(Client.scala:867) > > at org.apache.spark.deploy.yarn.Client.main(Client.scala) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:497) > > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664) > > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169) > > at > org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192) > > at > org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111) > > at > org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > > 16/01/15 19:23:53 INFO Utils: Shutdown hook called > > 16/01/15 19:23:53 INFO Utils: Deleting directory > /tmp/spark-b6ebcb83-efff-432a-9a7a-b4764f482d81 > > java.lang.UNIXProcess$ProcessPipeOutputStream@7a0a6f73 1 > > > > > > > > Siddharth Ubale, > > *Synchronized Communications * > > *#43, Velankani Tech Park, Block No. II, * > > *3rd Floor, Electronic City Phase I,* > > *Bangalore – 560 100* > > *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>* > > *Web:* *www.syncoms.com* <http://www.syncoms.com/> > > *[image: LogoNEWmohLARGE]* > > *London*|*Bangalore*|*Orlando* > > > > *we innovate, plan, execute, and transform the business​* > > > > >