[
https://issues.apache.org/jira/browse/PHOENIX-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355485#comment-16355485
]
Josh Mahonin commented on PHOENIX-4056:
---------------------------------------
I don't have much time to do more than a cursory review right now,
unfortunately. Overall I like the approach, but:
a) What happens if 'mapred.output.dir' isn't set? Or is that always the case?
org.apache.hadoop.conf.Configuration::get() will return a null in that case. Is
Spark OK with that? Maybe another idea is to wrap it up in a scala Option,
something like:
{{Option(conf.get('mapred.output.dir')).getOrElse("")}}
a) Any ideas on if this is backwards compatible down to Spark 2.0? It maybe an
impossible goal, but it would be nice to stay binary compatible with the whole
2.x range, if possible.
> java.lang.IllegalArgumentException: Can not create a Path from an empty string
> ------------------------------------------------------------------------------
>
> Key: PHOENIX-4056
> URL: https://issues.apache.org/jira/browse/PHOENIX-4056
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.11.0
> Environment: CDH5.12
> Phoenix:4.11
> HBase:1.2
> Spark: 2.2.0
> phoenix-spark.version:4.11.0-HBase-1.2
> Reporter: Jepson
> Priority: Major
> Labels: features, patch, test
> Attachments: PHOENIX-4056.patch
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> 1.use the configuration of server and client(scala project)
> <property>
> <name>phoenix.schema.isNamespaceMappingEnabled</name>
> <value>true</value>
> </property>
> <property>
> <name>phoenix.schema.mapSystemTablesToNamespace</name>
> <value>true</value>
> </property>
> 2.The Code:
> {code:java}
> resultDF.write
> .format("org.apache.phoenix.spark")
> .mode(SaveMode.Overwrite)
> .option("table", "JYDW.ADDRESS_ORDERCOUNT")
> .option("zkUrl","192.168.1.40,192.168.1.41,192.168.1.42:2181")
> .save()
> {code}
> 3.Throw this error,help to fix it,thankyou :
> 7/08/02 01:07:25 INFO DAGScheduler: Job 6 finished: runJob at
> SparkHadoopMapReduceWriter.scala:88, took 7.990715 s
> 17/08/02 01:07:25 ERROR SparkHadoopMapReduceWriter: Aborting job
> job_20170802010717_0079.
> {color:#59afe1}*java.lang.IllegalArgumentException: Can not create a Path
> from an empty string*{color}
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
> at org.apache.hadoop.fs.Path.<init>(Path.java:134)
> at org.apache.hadoop.fs.Path.<init>(Path.java:88)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:58)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:132)
> at
> org.apache.spark.internal.io.SparkHadoopMapReduceWriter$.write(SparkHadoopMapReduceWriter.scala:101)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1084)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1003)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:994)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:59)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:28)
> at
> org.apache.phoenix.spark.DefaultSource.createRelation(DefaultSource.scala:47)
> at
> org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:472)
> at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:48)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:117)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:117)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:138)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:135)
> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:116)
> at
> org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:92)
> at
> org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:92)
> at
> org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:610)
> at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:233)
> at com.jiuye.SparkPhoenixTest2$.main(SparkPhoenixTest2.scala:62)
> at com.jiuye.SparkPhoenixTest2.main(SparkPhoenixTest2.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Exception in thread "main" java.lang.IllegalArgumentException: Can not create
> a Path from an empty string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
