HyukjinKwon commented on a change in pull request #31489: URL: https://github.com/apache/spark/pull/31489#discussion_r571539866
########## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ########## @@ -825,6 +825,26 @@ class DataFrameReader private[sql](sparkSession: SparkSession) extends Logging { * must be in the following form: YYYY-MM-DDTHH:mm:ss (e.g. 2020-06-01T13:00:00)</li> * <li>`recursiveFileLookup`: recursively scan a directory for files. Using this option * disables partition discovery</li> + * <li>`datetimeRebaseMode`: the rebasing mode for the values of the `DATE`, `TIMESTAMP_MICROS`, + * `TIMESTAMP_MILLIS` logical types from the Julian to Proleptic Gregorian calendar: + * <ul> + * <li>`EXCEPTION` : Spark fails in reads of ancient dates/timestamps that are ambiguous + * between the two calendars</li> + * <li>`CORRECTED` : loading of dates/timestamps without rebasing</li> + * <li>`LEGACY` : perform rebasing of ancient dates/timestamps from the Julian to Proleptic + * Gregorian calendar</li> + * </ul> + * </li> + * <li>`int96RebaseMode`: the rebasing mode for `INT96` timestamps from the Julian to Proleptic + * Gregorian calendar: Review comment: Shall we mention that the default is the one set in `spark.sql.legacy.parquet.datetimeRebaseModeInRead`? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org