[
https://issues.apache.org/jira/browse/KUDU-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Henke resolved KUDU-2796.
-------------------------------
Resolution: Fixed
Fix Version/s: 1.10.0
Resolved via
[https://github.com/apache/kudu/commit/963a358c7745eb5173e5168c6072242a400b2184]
> Restore job mangles HDFS paths
> ------------------------------
>
> Key: KUDU-2796
> URL: https://issues.apache.org/jira/browse/KUDU-2796
> Project: Kudu
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Grant Henke
> Assignee: Grant Henke
> Priority: Major
> Labels: backup
> Fix For: 1.10.0
>
>
> It looks like there is an issue with certain HDFS deployments where
> converting to and from a Java and HDFS path can break the job.
> Here is a sample stack trace:
> {noformat}
> 19/04/23 10:43:24 ERROR yarn.ApplicationMaster: User class threw exception:
> org.apache.spark.sql.AnalysisException: Path does not exist:
> hdfs://ns1/ns1/kudu-backup-tests/default.loadgen_auto_0c24803e79b04c4eb010abaff6a290b9/1556039820974;
> org.apache.spark.sql.AnalysisException: Path does not exist:
> hdfs://ns1/ns1/kudu-backup-tests/default.loadgen_auto_0c24803e79b04c4eb010abaff6a290b9/1556039820974;
> at
> org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:557)
> at
> org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:544)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at scala.collection.immutable.List.foreach(List.scala:392)
> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.immutable.List.flatMap(List.scala:355)
> at
> org.apache.spark.sql.execution.datasources.DataSource.org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary(DataSource.scala:544)
> at
> org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:358)
> at
> org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178)
> at
> org.apache.kudu.backup.KuduRestore$$anonfun$run$1$$anonfun$apply$1.apply(KuduRestore.scala:79)
> at
> org.apache.kudu.backup.KuduRestore$$anonfun$run$1$$anonfun$apply$1.apply(KuduRestore.scala:58)
> at scala.collection.immutable.List.foreach(List.scala:392)
> at
> org.apache.kudu.backup.KuduRestore$$anonfun$run$1.apply(KuduRestore.scala:58)
> at
> org.apache.kudu.backup.KuduRestore$$anonfun$run$1.apply(KuduRestore.scala:55)
> at scala.collection.immutable.List.foreach(List.scala:392)
> at org.apache.kudu.backup.KuduRestore$.run(KuduRestore.scala:55)
> at org.apache.kudu.backup.KuduRestore$.main(KuduRestore.scala:166)
> at org.apache.kudu.backup.KuduRestore.main(KuduRestore.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:673){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
