[ https://issues.apache.org/jira/browse/SPARK-46034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
angerszhu updated SPARK-46034: ------------------------------ Description: For below case, it failed with FileNotFoundException {code:java} add jar hdfs://path/search_hadoop_udf-1.0.0-SNAPSHOT.jar; add file hdfs://path/feature_map.txt; CREATE or replace TEMPORARY FUNCTION si_to_fn AS "com.shopee.deep.data_mart.udf.SlotIdToFeatName"; select si_to_fn(k, './feature_map.txt') as feat_name from ( select 'slot_8116' as k union all select 'slot_2219' as k) A; {code} It was called that user use valued-table then the task will running on driver, but driver didn't copy this file to the root path, then failed. > SparkContext add file should also copy file to local root path > -------------------------------------------------------------- > > Key: SPARK-46034 > URL: https://issues.apache.org/jira/browse/SPARK-46034 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 3.2.1 > Reporter: angerszhu > Priority: Major > > For below case, it failed with FileNotFoundException > {code:java} > add jar hdfs://path/search_hadoop_udf-1.0.0-SNAPSHOT.jar; > add file hdfs://path/feature_map.txt; > CREATE or replace TEMPORARY FUNCTION si_to_fn AS > "com.shopee.deep.data_mart.udf.SlotIdToFeatName"; > select si_to_fn(k, './feature_map.txt') as feat_name > from ( > select 'slot_8116' as k > union all > select 'slot_2219' as k) > A; {code} > It was called that user use valued-table then the task will running on > driver, but driver didn't copy this file to the root path, then failed. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org