I’ve created a jira issue for this https://issues.apache.org/jira/browse/SPARK-4967
Originally we want to support multiple parquet file paths scanning as I guess, and those file paths are in a single string separated by comma internally, however I didn’t find any public example says we support multiple parquet files for API sqlContext.parquetFile, we need to think how to support multiple paths in some other way. Cheng Hao From: Michael Armbrust [mailto:[email protected]] Sent: Thursday, December 25, 2014 1:01 PM To: Daniel Siegmann Cc: [email protected] Subject: Re: Escape commas in file names No, there is not. Can you open a JIRA? On Tue, Dec 23, 2014 at 6:33 PM, Daniel Siegmann <[email protected]<mailto:[email protected]>> wrote: I am trying to load a Parquet file which has a comma in its name. Yes, this is a valid file name in HDFS. However, sqlContext.parquetFile interprets this as a comma-separated list of parquet files. Is there any way to escape the comma so it is treated as part of a single file name? -- Daniel Siegmann, Software Developer Velos Accelerating Machine Learning 54 W 40th St, New York, NY 10018 E: [email protected]<mailto:[email protected]> W: www.velos.io<http://www.velos.io>
