[
https://issues.apache.org/jira/browse/PARQUET-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masayuki Takahashi updated PARQUET-1297:
----------------------------------------
Description:
Arrow's 'Timestamp' definition is below:
{code:java}
{
"name" : "timestamp",
"unit" : "SECOND|MILLISECOND|MICROSECOND|NANOSECOND"
}
{code}
[http://arrow.apache.org/docs/metadata.html]
But Parquet only supports 'TIMESTAMP_MILLIS' and 'TIMESTAMP_MICROS'.
[https://github.com/Apache/parquet-format/blob/master/LogicalTypes.md]
Therefore SchemaConverter should not convert from Timestamp(TimeUnit.SECOND)
and Timestamp(TimeUnit.NANOSECOND) of Arrow to Parquet.
Related:
https://issues.apache.org/jira/browse/PARQUET-1285
was:
Arrow's 'Timestamp' definition is below:
{code:java}
{ "name" : "time", "unit" : "SECOND|MILLISECOND|MICROSECOND|NANOSECOND",}
{code}
[http://arrow.apache.org/docs/metadata.html]
But Parquet only supports 'TIMESTAMP_MILLIS' and 'TIMESTAMP_MICROS'.
[https://github.com/Apache/parquet-format/blob/master/LogicalTypes.md]
Therefore SchemaConverter should not convert from Timestamp(TimeUnit.SECOND)
and Timestamp(TimeUnit.NANOSECOND) of Arrow to Parquet.
Related:
https://issues.apache.org/jira/browse/PARQUET-1285
> [Java] SchemaConverter should not convert from Timestamp(TimeUnit.SECOND) and
> Timestamp(TimeUnit.NANOSECOND) of Arrow
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: PARQUET-1297
> URL: https://issues.apache.org/jira/browse/PARQUET-1297
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Reporter: Masayuki Takahashi
> Assignee: Masayuki Takahashi
> Priority: Minor
> Fix For: 1.10.0
>
>
> Arrow's 'Timestamp' definition is below:
> {code:java}
> {
> "name" : "timestamp",
> "unit" : "SECOND|MILLISECOND|MICROSECOND|NANOSECOND"
> }
> {code}
> [http://arrow.apache.org/docs/metadata.html]
> But Parquet only supports 'TIMESTAMP_MILLIS' and 'TIMESTAMP_MICROS'.
> [https://github.com/Apache/parquet-format/blob/master/LogicalTypes.md]
> Therefore SchemaConverter should not convert from Timestamp(TimeUnit.SECOND)
> and Timestamp(TimeUnit.NANOSECOND) of Arrow to Parquet.
> Related:
> https://issues.apache.org/jira/browse/PARQUET-1285
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
