lidavidm commented on a change in pull request #11358:
URL: https://github.com/apache/arrow/pull/11358#discussion_r733617001
##########
File path: docs/source/cpp/csv.rst
##########
@@ -190,6 +190,70 @@ dictionary-encoded string-like array. It switches to a
plain string-like
array when the threshold in :member:`ConvertOptions::auto_dict_max_cardinality`
is reached.
+Timestamp inference/parsing
+---------------------------
+
+If type inference is enabled, the CSV reader first tries to interpret
+string-like columns as timestamps. If all rows have some zone offset
+(e.g. ``Z`` or ``+0100``), even if the offsets are inconsistent, then the
+inferred type will be UTC timestamp. If no rows have a zone offset, then the
+inferred type will be timestamp without timezone. A mix of rows with/without
+offsets will result in a string column.
+
+If the type is explicitly specified as a timestamp without timezone ("naive"),
+then the reader will error on values with zone offsets in that column. Else, if
+the type is timestamp with timezone, the column values must either all have
+zone offsets or all lack zone offsets. In the former case, values are
+unambiguous, since each row specifies a precise time in UTC, but in the latter
+case, Arrow will currently interpret the timestamps as specifying values in UTC
+(i.e. as if they had the zone offset "Z" or "+0000"), *not* as values in the
+local time of the timezone.
Review comment:
Talking to Neal offline it looks like the test isn't really meant to
check this case, plus he noted we could always start with an error and make it
more implicit later - so I'll roll these changes back (and update the table
below as noted by Joris).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
