Github user sujith71955 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22396#discussion_r217673140
  
    --- Diff: docs/sql-programming-guide.md ---
    @@ -1898,6 +1898,7 @@ working with timestamps in `pandas_udf`s to get the 
best performance, see
       - Since Spark 2.4, File listing for compute statistics is done in 
parallel by default. This can be disabled by setting 
`spark.sql.parallelFileListingInStatsComputation.enabled` to `False`.
       - Since Spark 2.4, Metadata files (e.g. Parquet summary files) and 
temporary files are not counted as data files when calculating table size 
during Statistics computation.
       - Since Spark 2.4, empty strings are saved as quoted empty strings `""`. 
In version 2.3 and earlier, empty strings are equal to `null` values and do not 
reflect to any characters in saved CSV files. For example, the row of `"a", 
null, "", 1` was writted as `a,,,1`. Since Spark 2.4, the same row is saved as 
`a,,"",1`. To restore the previous behavior, set the CSV option `emptyValue` to 
empty (not quoted) string.  
    +  - Since Spark 2.4 load command from local filesystem supports wildcards 
in the folder level paths(e.g. LOAD DATA LOCAL INPATH 'tmp/folder*/). Now 
onwards normal space convention can be used in folder/file names (e.g. LOAD 
DATA INPATH 'tmp/folderName/file Name.csv), Older versions space in folder/file 
names has been represented using '%20'(e.g. LOAD DATA INPATH 
'tmp/folderName/myFile%20Name.csv), this usage will not be supported from spark 
2.4 version.
    --- End diff --
    
    Is it specific to the local file system? << Yes, , its specific to local 
file system as in hdfs user can provide wildcard character in folder level 
also, for local file system folder level support was not there and error will 
be thrown)
    Can this text add a quick example of using ? too?<< Yes i added the same>>



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to