Mohit Aggarwal created FLINK-31612: -------------------------------------- Summary: ClassNotFoundException when using GCS path as HA directory. Key: FLINK-31612 URL: https://issues.apache.org/jira/browse/FLINK-31612 Project: Flink Issue Type: Bug Environment: Flink Kuberenetes operator: 1.4
Flink version: 1.17 GKE Kubernetes cluster. Reporter: Mohit Aggarwal Hi, When I am trying to run Flink job in HA mode with GCS path as a HA directory (eg: [gs://flame-poc/ha]) or while starting a job from checkpoints in GCS I am getting following exception: {code:java} Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback not found at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2688) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2712) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.security.Groups.<init>(Groups.java:107) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.security.Groups.<init>(Groups.java:102) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:451) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:338) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:300) ~[?:?] at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:575) ~[?:?] at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.getUgiUserName(GoogleHadoopFileSystemBase.java:1226) ~[?:?] at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.listStatus(GoogleHadoopFileSystemBase.java:858) ~[?:?] at org.apache.flink.fs.gs.org.apache.flink.runtime.fs.hdfs.HadoopFileSystem.listStatus(HadoopFileSystem.java:170) ~[?:?] {code} {*}Observations{*}: While using File system as a HA path and GCS as checkpointing directory the job is able to write checkpoints to GCS checkpoint path. After debugging what I found was all the *org.apache.hadoop* paths are shaded to {*}org.apache.flink.fs.shaded.hadoop3.org{*}{*}.apache.hadoop{*}. Ideally the code should look for {*}org.apache.flink.fs.shaded.hadoop3.org{*}{*}.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback{*} instead of *org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.* I think it is not getting shaded over here [https://github.com/apache/hadoop/blob/branch-3.3.4/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Groups.java#L108] As a workaround I rebuilt flink-gs-fs-hadoop plugin removing this relocation and it worked for me. {code:java} <relocation> <pattern>org.apache.hadoop</pattern> <shadedPattern>org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop</shadedPattern> </relocation> {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)