[GitHub] twill pull request #65: (TWILL-254) Update to use ContainerId.fromString
GitHub user cbaenziger opened a pull request: https://github.com/apache/twill/pull/65 (TWILL-254) Update to use ContainerId.fromString This moves away from the deprecated `ConverterUitls.toContainerId` and updates the build to use Hadoop 2.7.2 to avoid folks grabbing master and having a failure trying to run on clusters with code newer than November 18th, 2014 (release of Hadoop 2.6.0). You can merge this pull request into a Git repository by running: $ git pull https://github.com/cbaenziger/twill master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/twill/pull/65.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #65 commit 391553308ccc0539b72fac6bf13cd819ce3bb5e1 Author: Clay BaenzigerDate: 2018-02-03T03:35:37Z (TWILL-254) Update to use ContainerId.fromString ---
[jira] [Commented] (TWILL-254) Update to use ContainerId.fromString
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351654#comment-16351654 ] ASF GitHub Bot commented on TWILL-254: -- GitHub user cbaenziger opened a pull request: https://github.com/apache/twill/pull/65 (TWILL-254) Update to use ContainerId.fromString This moves away from the deprecated `ConverterUitls.toContainerId` and updates the build to use Hadoop 2.7.2 to avoid folks grabbing master and having a failure trying to run on clusters with code newer than November 18th, 2014 (release of Hadoop 2.6.0). You can merge this pull request into a Git repository by running: $ git pull https://github.com/cbaenziger/twill master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/twill/pull/65.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #65 commit 391553308ccc0539b72fac6bf13cd819ce3bb5e1 Author: Clay BaenzigerDate: 2018-02-03T03:35:37Z (TWILL-254) Update to use ContainerId.fromString > Update to use ContainerId.fromString > > > Key: TWILL-254 > URL: https://issues.apache.org/jira/browse/TWILL-254 > Project: Apache Twill > Issue Type: Bug > Components: yarn >Affects Versions: 0.12.0 >Reporter: Clay B. >Priority: Trivial > > Hello, trying to use 0.12 or master, I get the following on a Hortonworks > 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by > specifically building for Hadoop 2.6+ via {{mvn package > -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) > I have a patch to move us away from the deprecated (see YARN-1942) call in > the stack trace anyhow and provide Hadoop 2.6 as a supported profile. > {code:java} > Exception in thread "main" java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) > Caused by: java.lang.RuntimeException: > java.lang.reflect.InvocationTargetException > at com.google.common.base.Throwables.propagate(Throwables.java:160) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) > ... 5 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) > ... 7 more > Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: > container_e164_1516738752188_0024_01_01 > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) > at > org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) > at > org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) > at > org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) > ... 12 more > Caused by: java.lang.NumberFormatException: For input string: "e164" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Long.parseLong(Long.java:589) > at java.lang.Long.parseLong(Long.java:631) > at > org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) > ... 15 more > {code} > For reference, it looks like YARN-2182 appears in Hadoop 2.6 to have changed > container ID formats and a new API call came out > [{{ContainerId.fromString}}|https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/records/ContainerId.html#fromString(java.lang.String)]. >
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Priority: Trivial (was: Major) > Update to use ContainerId.fromString > > > Key: TWILL-254 > URL: https://issues.apache.org/jira/browse/TWILL-254 > Project: Apache Twill > Issue Type: Bug > Components: yarn >Affects Versions: 0.12.0 >Reporter: Clay B. >Priority: Trivial > > Hello, trying to use 0.12 or master, I get the following on a Hortonworks > 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by > specifically building for Hadoop 2.6+ via {{mvn package > -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) > I have a patch to move us away from the deprecated (see YARN-1942) call in > the stack trace anyhow and provide Hadoop 2.6 as a supported profile. > {code:java} > Exception in thread "main" java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) > Caused by: java.lang.RuntimeException: > java.lang.reflect.InvocationTargetException > at com.google.common.base.Throwables.propagate(Throwables.java:160) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) > ... 5 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) > ... 7 more > Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: > container_e164_1516738752188_0024_01_01 > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) > at > org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) > at > org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) > at > org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) > ... 12 more > Caused by: java.lang.NumberFormatException: For input string: "e164" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Long.parseLong(Long.java:589) > at java.lang.Long.parseLong(Long.java:631) > at > org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) > ... 15 more > {code} > For reference, it looks like YARN-2182 appears in Hadoop 2.6 to have changed > container ID formats and a new API call came out > [{{ContainerId.fromString}}|https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/records/ContainerId.html#fromString(java.lang.String)]. > Meanwhile, the container ID format change was not reflected, of course, in > older versions of Hadoop as clarified in YARN-3575. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Description: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) I have a patch to move us away from the deprecated (see YARN-1942) call in the stack trace anyhow and provide Hadoop 2.6 as a supported profile. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) ... 7 more Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: container_e164_1516738752188_0024_01_01 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) at org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) ... 12 more Caused by: java.lang.NumberFormatException: For input string: "e164" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) ... 15 more {code} was: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) I have a patch to move us away from the deprecated )YARN-1942) call in the stack trace anyhow. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Description: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) I have a patch to move us away from the deprecated (see YARN-1942) call in the stack trace anyhow and provide Hadoop 2.6 as a supported profile. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) ... 7 more Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: container_e164_1516738752188_0024_01_01 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) at org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) ... 12 more Caused by: java.lang.NumberFormatException: For input string: "e164" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) ... 15 more {code} For reference, it looks like YARN-2182 appears in Hadoop 2.6 to have changed container ID formats and a new API call came out [{{ContainerId.fromString}}|https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/records/ContainerId.html#fromString(java.lang.String)]. Meanwhile, the container ID format change was not reflected, of course, in older versions of Hadoop as clarified in YARN-3575. was: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) I have a patch to move us away from the deprecated (see YARN-1942) call in the stack trace anyhow and provide Hadoop 2.6 as a supported profile. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString (YARN-1942)
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Description: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but being a bit slow to realize what was up, I have a patch to move us away from the deprecated call in the stack trace anyhow. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) ... 7 more Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: container_e164_1516738752188_0024_01_01 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) at org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) ... 12 more Caused by: java.lang.NumberFormatException: For input string: "e164" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) ... 15 more {code} was: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but being a bit slow to realize what was up, I have a patch to move us away from the deprecated call in the stack trace anyhow. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString (YARN-1942)
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Description: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) I have a patch to move us away from the deprecated )YARN-1942) call in the stack trace anyhow. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) ... 7 more Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: container_e164_1516738752188_0024_01_01 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) at org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) ... 12 more Caused by: java.lang.NumberFormatException: For input string: "e164" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) ... 15 more {code} was: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but being a bit slow to realize what was up, I have a patch to move us away from the deprecated call in the stack trace anyhow. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Summary: Update to use ContainerId.fromString (was: Update to use ContainerId.fromString (YARN-1942)) > Update to use ContainerId.fromString > > > Key: TWILL-254 > URL: https://issues.apache.org/jira/browse/TWILL-254 > Project: Apache Twill > Issue Type: Bug > Components: yarn >Affects Versions: 0.12.0 >Reporter: Clay B. >Priority: Major > > Hello, trying to use 0.12 or master, I get the following on a Hortonworks > 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by > specifically building for Hadoop 2.6+ via {{mvn package > -Dhadoop.version=2.6.0}} but I was being a bit slow to realize what was up,) > I have a patch to move us away from the deprecated )YARN-1942) call in the > stack trace anyhow. > {code:java} > Exception in thread "main" java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) > Caused by: java.lang.RuntimeException: > java.lang.reflect.InvocationTargetException > at com.google.common.base.Throwables.propagate(Throwables.java:160) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) > ... 5 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) > ... 7 more > Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: > container_e164_1516738752188_0024_01_01 > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) > at > org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) > at > org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) > at > org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) > ... 12 more > Caused by: java.lang.NumberFormatException: For input string: "e164" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Long.parseLong(Long.java:589) > at java.lang.Long.parseLong(Long.java:631) > at > org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) > ... 15 more > {code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString (YARN-1942)
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Description: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but being a bit slow to realize what was up, I have a patch to move us away from the deprecated call in the stack trace anyhow. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) ... 7 more Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: container_e164_1516738752188_0024_01_01 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) at org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) ... 12 more Caused by: java.lang.NumberFormatException: For input string: "e164" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) ... 15 more {code} It looks like YARN-2182 appears in Hadoop 2.6 to have changed container ID formats and a new API call came out [{{ContainerId.fromString}}|https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/records/ContainerId.html#fromString(java.lang.String)]. Meanwhile, the container ID format change was not reflected of course in older versions of Hadoop in (the deprecated) [{{ConverterUtils.toContainerId|. YARN-3575 of course says there is }} was: Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but being a bit slow to realize what was up, I have a patch to move us away from the deprecated call in the stack trace. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException
[jira] [Created] (TWILL-254) YARN-1942
Clay B. created TWILL-254: - Summary: YARN-1942 Key: TWILL-254 URL: https://issues.apache.org/jira/browse/TWILL-254 Project: Apache Twill Issue Type: Bug Components: yarn Affects Versions: 0.12.0 Reporter: Clay B. Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via {{mvn package -Dhadoop.version=2.6.0}} but being a bit slow to realize what was up, I have a patch to move us away from the deprecated call in the stack trace. {code:java} Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) ... 5 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) ... 7 more Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: container_e164_1516738752188_0024_01_01 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) at org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) ... 12 more Caused by: java.lang.NumberFormatException: For input string: "e164" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) ... 15 more {code} It looks like YARN-2182 appears in Hadoop 2.6 to have changed container ID formats and a new API call came out [{{ContainerId.fromString}}|https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/records/ContainerId.html#fromString(java.lang.String)]. Meanwhile, the container ID format change was not reflected of course in older versions of Hadoop in (the deprecated) [{{ConverterUtils.toContainerId|. YARN-3575 of course says there is }} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (TWILL-254) Update to use ContainerId.fromString (YARN-1942)
[ https://issues.apache.org/jira/browse/TWILL-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated TWILL-254: -- Summary: Update to use ContainerId.fromString (YARN-1942) (was: YARN-1942) > Update to use ContainerId.fromString (YARN-1942) > > > Key: TWILL-254 > URL: https://issues.apache.org/jira/browse/TWILL-254 > Project: Apache Twill > Issue Type: Bug > Components: yarn >Affects Versions: 0.12.0 >Reporter: Clay B. >Priority: Major > > Hello, trying to use 0.12 or master, I get the following on a Hortonworks > 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by > specifically building for Hadoop 2.6+ via {{mvn package > -Dhadoop.version=2.6.0}} but being a bit slow to realize what was up, I have > a patch to move us away from the deprecated call in the stack trace. > {code:java} > Exception in thread "main" java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73) > Caused by: java.lang.RuntimeException: > java.lang.reflect.InvocationTargetException > at com.google.common.base.Throwables.propagate(Throwables.java:160) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85) > at > org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71) > ... 5 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) > ... 7 more > Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: > container_e164_1516738752188_0024_01_01 > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) > at > org.apache.twill.internal.yarn.AbstractYarnAMClient.(AbstractYarnAMClient.java:83) > at > org.apache.twill.internal.yarn.Hadoop21YarnAMClient.(Hadoop21YarnAMClient.java:65) > at > org.apache.twill.internal.yarn.Hadoop22YarnAMClient.(Hadoop22YarnAMClient.java:34) > ... 12 more > Caused by: java.lang.NumberFormatException: For input string: "e164" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Long.parseLong(Long.java:589) > at java.lang.Long.parseLong(Long.java:631) > at > org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) > ... 15 more > {code} > > It looks like YARN-2182 appears in Hadoop 2.6 to have changed container ID > formats and a new API call came out > [{{ContainerId.fromString}}|https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/records/ContainerId.html#fromString(java.lang.String)]. > Meanwhile, the container ID format change was not reflected of course in > older versions of Hadoop in (the deprecated) > [{{ConverterUtils.toContainerId|. YARN-3575 of course says there is }} -- This message was sent by Atlassian JIRA (v7.6.3#76005)