[jira] [Updated] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
[ https://issues.apache.org/jira/browse/YARN-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10265: Description: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.48 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.48 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. was: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.48 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.48 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final|https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. > Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue > -- > > Key: YARN-10265 > URL: https://issues.apache.org/jira/browse/YARN-10265 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > > In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a > workaround using an non-official released netty-4.1.48 to fix the ARM support > issue. but just a few hours agon, Netty has release the 4.1.50 version which > is officially support ARM platform, please see: > [https://github.com/netty/netty/pull/9804] > > netty-4.1.50.Final release: > [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] > commits from netty-4.1.48 to netty-4.1.50: > [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final] > So, now it is better to upgrade the netty-dependency version of Hadoop to > 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
[ https://issues.apache.org/jira/browse/YARN-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10265: Description: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.48 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.48 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final|https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. was: In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.49 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.49 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. > Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue > -- > > Key: YARN-10265 > URL: https://issues.apache.org/jira/browse/YARN-10265 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > > In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a > workaround using an non-official released netty-4.1.48 to fix the ARM support > issue. but just a few hours agon, Netty has release the 4.1.50 version which > is officially support ARM platform, please see: > [https://github.com/netty/netty/pull/9804] > > netty-4.1.50.Final release: > [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] > commits from netty-4.1.48 to netty-4.1.50: > [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final|https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] > So, now it is better to upgrade the netty-dependency version of Hadoop to > 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
liusheng created YARN-10265: --- Summary: Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue Key: YARN-10265 URL: https://issues.apache.org/jira/browse/YARN-10265 Project: Hadoop YARN Issue Type: Bug Reporter: liusheng In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a workaround using an non-official released netty-4.1.49 to fix the ARM support issue. but just a few hours agon, Netty has release the 4.1.50 version which is officially support ARM platform, please see: [https://github.com/netty/netty/pull/9804] netty-4.1.50.Final release: [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] commits from netty-4.1.49 to netty-4.1.50: [https://github.com/netty/netty/compare/netty-4.1.49.Final...netty-4.1.50.Final] So, now it is better to upgrade the netty-dependency version of Hadoop to 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105940#comment-17105940
]
liusheng commented on YARN-9898:
The 2 above failed tests I have run locally without this patch applied and
still fail
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch,
> YARN-9898.003.patch, YARN-9898.004.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105116#comment-17105116
]
liusheng commented on YARN-9898:
I have no idea about the Jenkins results :(..
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch,
> YARN-9898.003.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17104027#comment-17104027
]
liusheng commented on YARN-9898:
I have added a saperated HDFS jira:
https://issues.apache.org/jira/browse/HDFS-15098
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch,
> YARN-9898.003.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17104023#comment-17104023
]
liusheng commented on YARN-9898:
[~ayushtkn], thanks, sure, let me try.
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103056#comment-17103056
]
liusheng commented on YARN-9898:
yes, thank you, I have also tested OK locally before uploading this patch, and
this patch have no side affect on x86 functionalities. not sure about the CI
results.
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102152#comment-17102152
]
liusheng commented on YARN-9898:
I have no idea about the jenkins rerults means.. :(
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng updated YARN-9898:
---
Attachment: (was: YARN-9898.001.patch)
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng updated YARN-9898:
---
Attachment: (was: YARN-9898.001.patch)
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng updated YARN-9898:
---
Attachment: (was: YARN-9898.001.patch)
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng reassigned YARN-9898:
--
Assignee: liusheng
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008754#comment-17008754
]
liusheng commented on YARN-9898:
we have tried to push netty to support on ARM platform:
https://github.com/netty/netty/pull/9804
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Priority: Major
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000521#comment-17000521 ] liusheng commented on YARN-10042: - Thanks a lot for your review [~tangzhankun], [~cheersyang] would you please take a look ? thank you > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: YARN-10042.001.patch) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log, > yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: YARN-10042.001.patch) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: YARN-10042.001.patch > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10042.001.patch, YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: yarn_csi_tests_x86_64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log, > yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: yarn_csi_tests_x86_64_grpc_1.26.0.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: hadoop_build_x86_64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log, yarn_csi_tests_aarch64_grpc_1.26.0.log, > yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: hadoop_build_x86_64_grpc_1.26.0.log.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Description: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing, please see the attachment, they are: log of building on aarch64, log of building on x86_64, log of running tests of yarn csi on aarch64, log of running tests of yarn csi on x86_64. was: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: yarn_csi_tests_aarch64_grpc_1.26.0.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: yarn_csi_tests_aarch64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: hadoop_build_aarch64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_aarch64_grpc_1.26.0.log, > hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: yarn_csi_tests_x86_64_grpc_1.26.0.log hadoop_build_x86_64_grpc_1.26.0.log.log yarn_csi_tests_aarch64_grpc_1.26.0.log hadoop_build_aarch64_grpc_1.26.0.log > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Attachment: (was: hadoop_build_aarch64_grpc_1.26.0.log) > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: hadoop_build_x86_64_grpc_1.26.0.log.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated YARN-10042: Description: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. was: For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
liusheng created YARN-10042: --- Summary: Uupgrade grpc-xxx depdencies to 1.26.0 Key: YARN-10042 URL: https://issues.apache.org/jira/browse/YARN-10042 Project: Hadoop YARN Issue Type: Bug Reporter: liusheng For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the grpc-java repo has support aarch64 platform and release in 1.26.0 in maven central. see: [https://github.com/grpc/grpc-java/pull/6496] [https://search.maven.org/search?q=g:io.grpc] It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 version. both x86_64 and aarch64 server are building OK accroding to my testing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9876) TimelineService Hbase tests are failing
[
https://issues.apache.org/jira/browse/YARN-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16996906#comment-16996906
]
liusheng commented on YARN-9876:
Hi,
Anyone can help to care about this issue ?
> TimelineService Hbase tests are failing
> ---
>
> Key: YARN-9876
> URL: https://issues.apache.org/jira/browse/YARN-9876
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineservice
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> TimelineService Hbase tests are failing
> {code}
> hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain
> hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun
> hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity
> hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps
> hadoop.yarn.server.timelineservice.storage.TestTimelineWriterHBaseDown
> hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction
>
>
> hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
>
> hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema
> hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities
> hadoop.yarn.server.timelineservice.storage.TestTimelineReaderHBaseDown
> {code}
> Logs:
> {code}
> [ERROR]
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain
> Time elapsed: 208.136 s <<< ERROR!
> java.io.IOException: Shutting down
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:236)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:98)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1089)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1045)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:917)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:911)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:840)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain.setupBeforeClass(TestHBaseTimelineStorageDomain.java:55)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Caused by: java.lang.RuntimeException: Master not initialized after 20ms
> seconds
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:230)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:414)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:229)
> ... 25 more
> 2019-10-06 22:27:04,233 FATAL [192.168.1.180:52250.activeMasterManager]
> master.HMaster (HMaster.java:run(2039)) - Failed to become active master
> java.lang.NoClassDefFoundError: com/google/protobuf/LiteralByteString
> at java.lang.ClassLoader.defineClass1(Native Method)
> at
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16959353#comment-16959353
]
liusheng commented on YARN-9511:
Hi [~adam.antal],
I am using a Ubuntu 16.04 server with JDK 8(for building and testing),
{code:java}
zuul@ubuntu:~$ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
{code}
The *umask* is for determining the default permissions(read,write, executable)
of newly created file/directory. it take effect on current login session if set
by 'umask' command.
> [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException:
> The remote jarfile should not be writable by group or others. The current
> Permission is 436
> ---
>
> Key: YARN-9511
> URL: https://issues.apache.org/jira/browse/YARN-9511
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Siyao Meng
>Assignee: Szilard Nemeth
>Priority: Major
>
> Found in maven JDK 11 unit test run. Compiled on JDK 8.
> {code}
> [ERROR]
> testRemoteAuxServiceClassPath(org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices)
> Time elapsed: 0.551 s <<<
> ERROR!org.apache.hadoop.yarn.exceptions.YarnRuntimeException: The remote
> jarfile should not be writable by group or others. The current Permission is
> 436
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:202)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices.testRemoteAuxServiceClassPath(TestAuxServices.java:268)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9897) Add an Aarch64 CI for YARN
[
https://issues.apache.org/jira/browse/YARN-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957484#comment-16957484
]
liusheng edited comment on YARN-9897 at 10/23/19 1:34 AM:
--
Hi [~eyang],
I have tried your the two tests suggested in your comment, looks like both are
success.
{code:java}
[INFO] C M A K E B U I L D E RT E S T
[INFO] ---
[INFO] cetest: running
/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest
--gtest_filter=-Perf.
--gtest_output=xml:/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml
[INFO] with extra environment variables {}
[INFO] STATUS: SUCCESS after 154 millisecond(s).
[INFO] ---
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 01:01 min
[INFO] Finished at: 2019-10-23T01:29:41Z
[INFO]
{code}
{code:java}
[INFO] ---
[INFO] C M A K E B U I L D E RT E S T
[INFO] ---
[INFO] test-container-executor: running
/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/target/usr/local/bin/test-container-executor
[INFO] with extra environment variables {}
[INFO] STATUS: SUCCESS after 5968 millisecond(s).
[INFO] ---
[INFO]
[INFO] --- hadoop-maven-plugins:3.3.0-SNAPSHOT:cmake-test (cetest) @
hadoop-yarn-server-nodemanager ---
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 01:07 min
[INFO] Finished at: 2019-10-23T01:32:28Z
[INFO]
{code}
Does this look good to you ?
was (Author: seanlau):
Hi [~eyang],
Looks like both these two tests are OK, see:
{code:java}
[INFO] C M A K E B U I L D E RT E S T
[INFO] ---
[INFO] cetest: running
/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest
--gtest_filter=-Perf.
--gtest_output=xml:/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml
[INFO] with extra environment variables {}
[INFO] STATUS: SUCCESS after 154 millisecond(s).
[INFO] ---
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 01:01 min
[INFO] Finished at: 2019-10-23T01:29:41Z
[INFO]
{code}
> Add an Aarch64 CI for YARN
> --
>
> Key: YARN-9897
> URL: https://issues.apache.org/jira/browse/YARN-9897
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: build, test
>Reporter: Zhenyu Zheng
>Priority: Major
> Attachments: hadoop_build.log
>
>
> As YARN is the resource manager of Hadoop and there are large number of other
> software that also uses YARN for resource management. The capability of
> running YARN on platforms with different architecture and managing hardware
> resources with different architecture could be very important and useful.
> Aarch64(ARM) architecture is currently the dominate architecture in small
> devices like phone, IOT devices, security cameras, drones etc. With the
> increasing compuiting capability and the increasing connection speed like 5G
> network, there could be greate posibility and opportunity for world chaging
> inovations and new market if we can managing and make use of those devices as
> well.
> Currently, all YARN CIs are based on x86 architecture and we have been
> performing tests on Aarch64 and proposing possible solutions for problems we
> have meet, like:
> https://issues.apache.org/jira/browse/HADOOP-16614
> we have done all YARN tests and it turns out there are only a few problems,
> and we can provide possible solutions for discussion.
> We want to propose to add an Aarch64 CI for
[jira] [Commented] (YARN-9897) Add an Aarch64 CI for YARN
[
https://issues.apache.org/jira/browse/YARN-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957484#comment-16957484
]
liusheng commented on YARN-9897:
Hi [~eyang],
Looks like both these two tests are OK, see:
{code:java}
[INFO] C M A K E B U I L D E RT E S T
[INFO] ---
[INFO] cetest: running
/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest
--gtest_filter=-Perf.
--gtest_output=xml:/home/zuul/src/github.com/liusheng/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml
[INFO] with extra environment variables {}
[INFO] STATUS: SUCCESS after 154 millisecond(s).
[INFO] ---
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 01:01 min
[INFO] Finished at: 2019-10-23T01:29:41Z
[INFO]
{code}
> Add an Aarch64 CI for YARN
> --
>
> Key: YARN-9897
> URL: https://issues.apache.org/jira/browse/YARN-9897
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: build, test
>Reporter: Zhenyu Zheng
>Priority: Major
> Attachments: hadoop_build.log
>
>
> As YARN is the resource manager of Hadoop and there are large number of other
> software that also uses YARN for resource management. The capability of
> running YARN on platforms with different architecture and managing hardware
> resources with different architecture could be very important and useful.
> Aarch64(ARM) architecture is currently the dominate architecture in small
> devices like phone, IOT devices, security cameras, drones etc. With the
> increasing compuiting capability and the increasing connection speed like 5G
> network, there could be greate posibility and opportunity for world chaging
> inovations and new market if we can managing and make use of those devices as
> well.
> Currently, all YARN CIs are based on x86 architecture and we have been
> performing tests on Aarch64 and proposing possible solutions for problems we
> have meet, like:
> https://issues.apache.org/jira/browse/HADOOP-16614
> we have done all YARN tests and it turns out there are only a few problems,
> and we can provide possible solutions for discussion.
> We want to propose to add an Aarch64 CI for YARN to promote the support for
> YARN on Aarch64 platforms. We are willing to provide machines to the current
> CI system and manpower to mananging the CI and fxing problems that occours.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9897) Add an Aarch64 CI for YARN
[
https://issues.apache.org/jira/browse/YARN-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956640#comment-16956640
]
liusheng commented on YARN-9897:
Hi [~eyang],
Yes, we have built Hadoop with *-Pnative* flag, without the
container-executor.additional_cflags, seems everything is OK, here is my
building command:
{code:java}
mvn clean install -e -B -Pdist,native -Dtar -DskipTests -Dmaven.javadoc.skip
{code}
and attach my building log
> Add an Aarch64 CI for YARN
> --
>
> Key: YARN-9897
> URL: https://issues.apache.org/jira/browse/YARN-9897
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: build, test
>Reporter: Zhenyu Zheng
>Priority: Major
>
> As YARN is the resource manager of Hadoop and there are large number of other
> software that also uses YARN for resource management. The capability of
> running YARN on platforms with different architecture and managing hardware
> resources with different architecture could be very important and useful.
> Aarch64(ARM) architecture is currently the dominate architecture in small
> devices like phone, IOT devices, security cameras, drones etc. With the
> increasing compuiting capability and the increasing connection speed like 5G
> network, there could be greate posibility and opportunity for world chaging
> inovations and new market if we can managing and make use of those devices as
> well.
> Currently, all YARN CIs are based on x86 architecture and we have been
> performing tests on Aarch64 and proposing possible solutions for problems we
> have meet, like:
> https://issues.apache.org/jira/browse/HADOOP-16614
> we have done all YARN tests and it turns out there are only a few problems,
> and we can provide possible solutions for discussion.
> We want to propose to add an Aarch64 CI for YARN to promote the support for
> YARN on Aarch64 platforms. We are willing to provide machines to the current
> CI system and manpower to mananging the CI and fxing problems that occours.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956634#comment-16956634
]
liusheng commented on YARN-9511:
Hi [~adam.antal],
Here are my steps:
1. Confirm current *umask* value of system, mine is *0002*
{code:java}
$ umask
0002{code}
2. Run unit tests of *TestAuxServices*, and will reproduce the problems
described in this issue.
{code:java}
$ cd hadoop-yarn-project/
$ mvn test -Dtest=TestAuxServices{code}
3. modify the permissions of hadoop source code directory to remove other and
group wirte permission and the system's umask to *022*
{code:java}
$ chmod go-w hadoop/ -R
$ umask 022
$ umask
0022{code}
4. Re-run the unitests of *TestAuxServices* again, all the unit tests can pass
{code:java}
$ cd hadoop-yarn-project/
$ mvn test -Dtest=TestAuxServices{code}
> [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException:
> The remote jarfile should not be writable by group or others. The current
> Permission is 436
> ---
>
> Key: YARN-9511
> URL: https://issues.apache.org/jira/browse/YARN-9511
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Siyao Meng
>Assignee: Szilard Nemeth
>Priority: Major
>
> Found in maven JDK 11 unit test run. Compiled on JDK 8.
> {code}
> [ERROR]
> testRemoteAuxServiceClassPath(org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices)
> Time elapsed: 0.551 s <<<
> ERROR!org.apache.hadoop.yarn.exceptions.YarnRuntimeException: The remote
> jarfile should not be writable by group or others. The current Permission is
> 436
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:202)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices.testRemoteAuxServiceClassPath(TestAuxServices.java:268)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng updated YARN-9898:
---
Description:
Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
io.netty maven repo, cannot support ARM platform.
When run the test *TestCsiClient.testIdentityService* on ARM server, it will
raise error like following:
{code:java}
Caused by: java.io.FileNotFoundException:
META-INF/native/libnetty_transport_native_epoll_aarch_64.so
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at
java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
at java.security.AccessController.doPrivileged(Native
Method)
at
io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
... 46 more
{code}
was:
Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
io.netty maven repo, cannot support ARM platform.
When run the test *TestCsiClient.testIdentityService*, it will raise error like
following:
{code:java}
Caused by: java.io.FileNotFoundException:
META-INF/native/libnetty_transport_native_epoll_aarch_64.so
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at
java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
at java.security.AccessController.doPrivileged(Native
Method)
at
io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
at
[jira] [Updated] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng updated YARN-9898:
---
Description:
Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
io.netty maven repo, cannot support ARM platform.
When run the test *TestCsiClient.testIdentityService*, it will raise error like
following:
{code:java}
Caused by: java.io.FileNotFoundException:
META-INF/native/libnetty_transport_native_epoll_aarch_64.so
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at
java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
at java.security.AccessController.doPrivileged(Native
Method)
at
io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
... 46 more
{code}
was:
{code:java}
Caused by: java.io.FileNotFoundException:
META-INF/native/libnetty_transport_native_epoll_aarch_64.so
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at
java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
at java.security.AccessController.doPrivileged(Native
Method)
at
io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
... 46 more
{code}
Summary: Dependency netty-all-4.1.27.Final doesn't support ARM platform
(was: Dependency netty-all-4.1.27.Final doesn)
> Dependency
[jira] [Created] (YARN-9898) Dependency netty-all-4.1.27.Final doesn
liusheng created YARN-9898:
--
Summary: Dependency netty-all-4.1.27.Final doesn
Key: YARN-9898
URL: https://issues.apache.org/jira/browse/YARN-9898
Project: Hadoop YARN
Issue Type: Bug
Reporter: liusheng
{code:java}
Caused by: java.io.FileNotFoundException:
META-INF/native/libnetty_transport_native_epoll_aarch_64.so
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
at
io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
... 45 more
Suppressed: java.lang.UnsatisfiedLinkError: no
netty_transport_native_epoll_aarch_64 in java.library.path
at
java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at
io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
at java.security.AccessController.doPrivileged(Native
Method)
at
io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
at
io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
... 46 more
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949890#comment-16949890
]
liusheng edited comment on YARN-9511 at 10/12/19 3:51 AM:
--
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
*TestAuxServices*, and will cause *2 Errors 9 Failures*, see:
{code:java}
Failures:
TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0>
TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but
was:<0>
TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6>
but was:<1>
TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but
was:<1>
TestAuxServices.testCustomizedAuxServiceClassPath:416
TestAuxServices.testManualReload:919 expected:<2> but was:<0>
TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar
is wrong.Should throw out exception.
TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0>
TestAuxServices.testValidAuxServiceName:698 Should receive the exception.
Errors:
TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement
TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote
jar...
{code}
After debuging, I found all these issues are directly or indirectly related
with files permissions.
there are two situations when running these tests:
# *useManifest* enabled, when running tests with useManifest enabled, the
tests will check and use the manifest file:
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code}
this file and all its parents directories must not be writable by group or
others, see:
{code:java}
private boolean checkManifestPermissions(FileStatus status) throws
IOException {
if ((status.getPermission().toShort() & 0022) != 0) {
LOG.error("Manifest file and parents must not be writable by group or " +
"others. The current Permission of " + status.getPath() + " is " +
status.getPermission());
return false;
}
Path parent = status.getPath().getParent();
if (parent == null) {
return true;
}
return checkManifestPermissions(manifestFS.getFileStatus(parent));
}{code}
# *useManifest not* enabled, when running tests with useManifest enabled,
tests will use a *test-runjar.jar* file
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar
{code}
related code for checking its permission:
{code:java}
private Path maybeDownloadJars(String sName, String className, String
remoteFile, AuxServiceFile.TypeEnum type, Configuration conf)
throws IOException {
// load AuxiliaryService from remote classpath
FileContext localLFS = getLocalFileContext(conf);
// create NM aux-service dir in NM localdir if it does not exist.
Path nmAuxDir = dirsHandler.getLocalPathForWrite("."
+ Path.SEPARATOR + NM_AUX_SERVICE_DIR);
if (!localLFS.util().exists(nmAuxDir)) {
try {
localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true);
} catch (IOException ex) {
throw new YarnRuntimeException("Fail to create dir:"
+ nmAuxDir.toString(), ex);
}
}
Path src = new Path(remoteFile);
FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf);
FileStatus scFileStatus = remoteLFS.getFileStatus(src);
if (!scFileStatus.getOwner().equals(
this.userUGI.getShortUserName())) {
throw new YarnRuntimeException("The remote jarfile owner:"
+ scFileStatus.getOwner() + " is not the same as the NM user:"
+ this.userUGI.getShortUserName() + ".");
}
if ((scFileStatus.getPermission().toShort() & 0022) != 0) {
throw new YarnRuntimeException("The remote jarfile should not "
+ "be writable by group or others. "
+ "The current Permission is "
+ scFileStatus.getPermission().toShort());
}
{code}
According to the above reasons, I have tried to change *manifest.txt* file
parents directories without *writeable* permission of group and others. and
change the *umask to 022*, which will effect new created file and directories
permissions, because the *manifest.txt* and *run-tests.jar* will be new created
when running tests.
{code:java}
chmod go-w yourpath/hadoop/ -R
umask 022
umask
{code}
After doing above and re-run tests of *TestAuxServices*, all the tests can
pass.
Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of
Hadoop or not. could you please give some suggestions?
Thanks.
was (Author: seanlau):
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949890#comment-16949890
]
liusheng edited comment on YARN-9511 at 10/12/19 3:51 AM:
--
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
*TestAuxServices*, and will cause *2 Errors 9 Failures*, see:
{code:java}
Failures:
TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0>
TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but
was:<0>
TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6>
but was:<1>
TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but
was:<1>
TestAuxServices.testCustomizedAuxServiceClassPath:416
TestAuxServices.testManualReload:919 expected:<2> but was:<0>
TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar
is wrong.Should throw out exception.
TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0>
TestAuxServices.testValidAuxServiceName:698 Should receive the exception.
Errors:
TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement
TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote
jar...
{code}
After debuging, I found all these issues are directly or indirectly related
with files permissions.
there are two situations when running these tests:
# *useManifest* enabled, when running tests with useManifest enabled, the
tests will check and use the manifest file:
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code}
this file and all its parents directories must not be writable by group or
others, see:
{code:java}
private boolean checkManifestPermissions(FileStatus status) throws
IOException {
if ((status.getPermission().toShort() & 0022) != 0) {
LOG.error("Manifest file and parents must not be writable by group or " +
"others. The current Permission of " + status.getPath() + " is " +
status.getPermission());
return false;
}
Path parent = status.getPath().getParent();
if (parent == null) {
return true;
}
return checkManifestPermissions(manifestFS.getFileStatus(parent));
}{code}
# *useManifest not* enabled, when running tests with useManifest enabled,
tests will use a *test-runjar.jar* file
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar
{code}
related code for checking its permission:
{code:java}
private Path maybeDownloadJars(String sName, String className, String
remoteFile, AuxServiceFile.TypeEnum type, Configuration conf)
throws IOException {
// load AuxiliaryService from remote classpath
FileContext localLFS = getLocalFileContext(conf);
// create NM aux-service dir in NM localdir if it does not exist.
Path nmAuxDir = dirsHandler.getLocalPathForWrite("."
+ Path.SEPARATOR + NM_AUX_SERVICE_DIR);
if (!localLFS.util().exists(nmAuxDir)) {
try {
localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true);
} catch (IOException ex) {
throw new YarnRuntimeException("Fail to create dir:"
+ nmAuxDir.toString(), ex);
}
}
Path src = new Path(remoteFile);
FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf);
FileStatus scFileStatus = remoteLFS.getFileStatus(src);
if (!scFileStatus.getOwner().equals(
this.userUGI.getShortUserName())) {
throw new YarnRuntimeException("The remote jarfile owner:"
+ scFileStatus.getOwner() + " is not the same as the NM user:"
+ this.userUGI.getShortUserName() + ".");
}
if ((scFileStatus.getPermission().toShort() & 0022) != 0) {
throw new YarnRuntimeException("The remote jarfile should not "
+ "be writable by group or others. "
+ "The current Permission is "
+ scFileStatus.getPermission().toShort());
}
{code}
According to the above reasons, I have tried to change *manifest.txt* file
parents directories without *writeable* permission of group and others. and
change the *umask to 022*, which will effect new created file and directories
permissions, because the *manifest.txt* and *run-tests.jar* will be new created
when running tests.
{code:java}
chmod go-w yourpath/hadoop/ -R
umask 022
umask
{code}
After doing above and re-run tests of *TestAuxServices*, all the tests can
pass.
Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of
Hadoop or not. could you please give some suggestions?
Thanks.
was (Author: seanlau):
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949890#comment-16949890
]
liusheng edited comment on YARN-9511 at 10/12/19 3:50 AM:
--
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
*TestAuxServices*, and will cause *2 Errors 9 Failures*, see:
{code:java}
Failures:
TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0>
TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but
was:<0>
TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6>
but was:<1>
TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but
was:<1>
TestAuxServices.testCustomizedAuxServiceClassPath:416
TestAuxServices.testManualReload:919 expected:<2> but was:<0>
TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar
is wrong.Should throw out exception.
TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0>
TestAuxServices.testValidAuxServiceName:698 Should receive the exception.
Errors:
TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement
TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote
jar...
{code}
After debuging, I found all these issues are directly or indirectly related
with files permissions.
there are two situations when running these tests:
# *useManifest* enabled, when running tests with useManifest enabled, the
tests will check and use the manifest file:
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code}
this file and all its parents directories must not be writable by group or
others, see:
{code:java}
private boolean checkManifestPermissions(FileStatus status) throws
IOException {
if ((status.getPermission().toShort() & 0022) != 0) {
LOG.error("Manifest file and parents must not be writable by group or " +
"others. The current Permission of " + status.getPath() + " is " +
status.getPermission());
return false;
}
Path parent = status.getPath().getParent();
if (parent == null) {
return true;
}
return checkManifestPermissions(manifestFS.getFileStatus(parent));
}{code}
**
# *useManifest not* enabled, when running tests with useManifest enabled,
tests will use a *test-runjar.jar* file
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar
{code}
related code for checking its permission:
{code:java}
private Path maybeDownloadJars(String sName, String className, String
remoteFile, AuxServiceFile.TypeEnum type, Configuration conf)
throws IOException {
// load AuxiliaryService from remote classpath
FileContext localLFS = getLocalFileContext(conf);
// create NM aux-service dir in NM localdir if it does not exist.
Path nmAuxDir = dirsHandler.getLocalPathForWrite("."
+ Path.SEPARATOR + NM_AUX_SERVICE_DIR);
if (!localLFS.util().exists(nmAuxDir)) {
try {
localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true);
} catch (IOException ex) {
throw new YarnRuntimeException("Fail to create dir:"
+ nmAuxDir.toString(), ex);
}
}
Path src = new Path(remoteFile);
FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf);
FileStatus scFileStatus = remoteLFS.getFileStatus(src);
if (!scFileStatus.getOwner().equals(
this.userUGI.getShortUserName())) {
throw new YarnRuntimeException("The remote jarfile owner:"
+ scFileStatus.getOwner() + " is not the same as the NM user:"
+ this.userUGI.getShortUserName() + ".");
}
if ((scFileStatus.getPermission().toShort() & 0022) != 0) {
throw new YarnRuntimeException("The remote jarfile should not "
+ "be writable by group or others. "
+ "The current Permission is "
+ scFileStatus.getPermission().toShort());
}
{code}
According to the above reasons, I have tried to change *manifest.txt* file
parents directories without *writeable* permission of group and others. and
change the *umask to 022*, which will effect new created file and directories
permissions, because the *manifest.txt* and *run-tests.jar* will be new created
when running tests.
{code:java}
chmod go-w yourpath/hadoop/ -R
umask 022
umask
{code}
After doing above and re-run tests of *TestAuxServices*, all the tests can
pass.
Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of
Hadoop or not. could you please give some suggestions?
Thanks.
was (Author: seanlau):
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
[jira] [Comment Edited] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949890#comment-16949890
]
liusheng edited comment on YARN-9511 at 10/12/19 3:49 AM:
--
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
*TestAuxServices*, and will cause *2 Errors 9 Failures*, see:
{code:java}
Failures:
TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0>
TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but
was:<0>
TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6>
but was:<1>
TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but
was:<1>
TestAuxServices.testCustomizedAuxServiceClassPath:416
TestAuxServices.testManualReload:919 expected:<2> but was:<0>
TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar
is wrong.Should throw out exception.
TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0>
TestAuxServices.testValidAuxServiceName:698 Should receive the exception.
Errors:
TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement
TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote
jar...
{code}
After debuging, I found all these issues are directly or indirectly related
with files permissions.
there are two situations when running these tests:
# *useManifest* enabled, when running tests with useManifest enabled, the
tests will check and use the manifest file:
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code}
this file and all its parents directories must not be writable by group or
others, see:
{code:java}
private boolean checkManifestPermissions(FileStatus status) throws
IOException {
if ((status.getPermission().toShort() & 0022) != 0) {
LOG.error("Manifest file and parents must not be writable by group or " +
"others. The current Permission of " + status.getPath() + " is " +
status.getPermission());
return false;
}
Path parent = status.getPath().getParent();
if (parent == null) {
return true;
}
return checkManifestPermissions(manifestFS.getFileStatus(parent));
}{code}
**
# *useManifest not* enabled, when running tests with useManifest enabled,
tests will use a *test-runjar.jar* file
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/test-runjar.jar
{code}
related code for checking its permission:
{code:java}
private Path maybeDownloadJars(String sName, String className, String
remoteFile, AuxServiceFile.TypeEnum type, Configuration conf)
throws IOException {
// load AuxiliaryService from remote classpath
FileContext localLFS = getLocalFileContext(conf);
// create NM aux-service dir in NM localdir if it does not exist.
Path nmAuxDir = dirsHandler.getLocalPathForWrite("."
+ Path.SEPARATOR + NM_AUX_SERVICE_DIR);
if (!localLFS.util().exists(nmAuxDir)) {
try {
localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true);
} catch (IOException ex) {
throw new YarnRuntimeException("Fail to create dir:"
+ nmAuxDir.toString(), ex);
}
}
Path src = new Path(remoteFile);
FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf);
FileStatus scFileStatus = remoteLFS.getFileStatus(src);
if (!scFileStatus.getOwner().equals(
this.userUGI.getShortUserName())) {
throw new YarnRuntimeException("The remote jarfile owner:"
+ scFileStatus.getOwner() + " is not the same as the NM user:"
+ this.userUGI.getShortUserName() + ".");
}
if ((scFileStatus.getPermission().toShort() & 0022) != 0) {
throw new YarnRuntimeException("The remote jarfile should not "
+ "be writable by group or others. "
+ "The current Permission is "
+ scFileStatus.getPermission().toShort());
}
{code}
According to the above reasons, I have tried to change *manifest.txt* file
parents directories without *writeable* permission of group and others. and
change the *umask to 077*, which will effect new created file and directories
permissions, because the *manifest.txt* and *run-tests.jar* will be new created
when running tests.
{code:java}
chmod go-w yourpath/hadoop/ -R
umask 022
umask
{code}
After doing above and re-run tests of *TestAuxServices*, all the tests can
pass.
Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of
Hadoop or not. could you please give some suggestions?
Thanks.
was (Author: seanlau):
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949890#comment-16949890
]
liusheng commented on YARN-9511:
Hi [~snemeth] [~adam.antal],
Thank you both for care about this issue. I have take some time tried to find
the reason of this issue. This issue will effect the tests of
*TestAuxServices*, and will cause *2 Errors 9 Failures*, see:
{code:java}
Failures:
TestAuxServices.testAuxServiceRecoverySetup:717 expected:<2> but was:<0>
TestAuxServices.testAuxServicesManifestPermissions:874 expected:<2> but
was:<0>
TestAuxServices.testAuxServicesMeta:638 Invalid mix of services expected:<6>
but was:<1>
TestAuxServices.testAuxServices:610 Invalid mix of services expected:<6> but
was:<1>
TestAuxServices.testCustomizedAuxServiceClassPath:416
TestAuxServices.testManualReload:919 expected:<2> but was:<0>
TestAuxServices.testRemoteAuxServiceClassPath:313 The permission of the jar
is wrong.Should throw out exception.
TestAuxServices.testRemoveManifest:897 expected:<2> but was:<0>
TestAuxServices.testValidAuxServiceName:698 Should receive the exception.
Errors:
TestAuxServices.testAuxUnexpectedStop:664 » NoSuchElement
TestAuxServices.testRemoteAuxServiceClassPath:334 » YarnRuntime The remote
jar...
{code}
After debuging, I found all these issues are directly or indirectly related
with files permissions.
there are two situations when running these tests:
# *useManifest* enabled, when running tests with useManifest enabled, the
tests will check and use the manifest file:
{code:java}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-dir/TestAuxServices/manifest.txt{code}
this file and all its parents directories must not be writable by group or
others, see:
{code:java}
private boolean checkManifestPermissions(FileStatus status) throws
IOException {
if ((status.getPermission().toShort() & 0022) != 0) {
LOG.error("Manifest file and parents must not be writable by group or " +
"others. The current Permission of " + status.getPath() + " is " +
status.getPermission());
return false;
}
Path parent = status.getPath().getParent();
if (parent == null) {
return true;
}
return checkManifestPermissions(manifestFS.getFileStatus(parent));
}{code}
# *useManifest not* enabled, when running tests with useManifest enabled,
tests will use a test-jar.jar file
{code:java}
private Path maybeDownloadJars(String sName, String className, String
remoteFile, AuxServiceFile.TypeEnum type, Configuration conf)
throws IOException {
// load AuxiliaryService from remote classpath
FileContext localLFS = getLocalFileContext(conf);
// create NM aux-service dir in NM localdir if it does not exist.
Path nmAuxDir = dirsHandler.getLocalPathForWrite("."
+ Path.SEPARATOR + NM_AUX_SERVICE_DIR);
if (!localLFS.util().exists(nmAuxDir)) {
try {
localLFS.mkdir(nmAuxDir, NM_AUX_SERVICE_DIR_PERM, true);
} catch (IOException ex) {
throw new YarnRuntimeException("Fail to create dir:"
+ nmAuxDir.toString(), ex);
}
}
Path src = new Path(remoteFile);
FileContext remoteLFS = getRemoteFileContext(src.toUri(), conf);
FileStatus scFileStatus = remoteLFS.getFileStatus(src);
if (!scFileStatus.getOwner().equals(
this.userUGI.getShortUserName())) {
throw new YarnRuntimeException("The remote jarfile owner:"
+ scFileStatus.getOwner() + " is not the same as the NM user:"
+ this.userUGI.getShortUserName() + ".");
}
if ((scFileStatus.getPermission().toShort() & 0022) != 0) {
throw new YarnRuntimeException("The remote jarfile should not "
+ "be writable by group or others. "
+ "The current Permission is "
+ scFileStatus.getPermission().toShort());
}
{code}
According to the above reasons, I have tried to change *manifest.txt* file
parents directories without *writeable* permission of group and others. and
change the *umask to 077*, which will effect new created file and directories
permissions, because the *manifest.txt* and *run-tests.jar* will be new created
when running tests.
{code:java}
chmod go-w yourpath/hadoop/ -R
umask 022
umask
{code}
After doing above and re-run tests of *TestAuxServices*, all the tests can
pass.
Actually, I am a new comer to Hadoop, so I am not sure whether this is a bug of
Hadoop or not. could you please give some suggestions?
Thanks.
> [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException:
> The remote jarfile should not be writable by group or others. The current
> Permission is 436
> ---
>
> Key: YARN-9511
> URL:
[jira] [Commented] (YARN-9511) [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException: The remote jarfile should not be writable by group or others. The current Permission is 436
[
https://issues.apache.org/jira/browse/YARN-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922211#comment-16922211
]
liusheng commented on YARN-9511:
Hi,
Any update about this issue ?
> [JDK11] TestAuxServices#testRemoteAuxServiceClassPath YarnRuntimeException:
> The remote jarfile should not be writable by group or others. The current
> Permission is 436
> ---
>
> Key: YARN-9511
> URL: https://issues.apache.org/jira/browse/YARN-9511
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Siyao Meng
>Assignee: Szilard Nemeth
>Priority: Major
>
> Found in maven JDK 11 unit test run. Compiled on JDK 8.
> {code}
> [ERROR]
> testRemoteAuxServiceClassPath(org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices)
> Time elapsed: 0.551 s <<<
> ERROR!org.apache.hadoop.yarn.exceptions.YarnRuntimeException: The remote
> jarfile should not be writable by group or others. The current Permission is
> 436
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:202)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.TestAuxServices.testRemoteAuxServiceClassPath(TestAuxServices.java:268)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
