[
https://issues.apache.org/jira/browse/AIRFLOW-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799814#comment-16799814
]
ASF GitHub Bot commented on AIRFLOW-3541:
-----------------------------------------
Fokko commented on pull request #4553: [AIRFLOW-3541] Add Avro logical type
conversion to bigquery hook
URL: https://github.com/apache/airflow/pull/4553
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Include --useAvroLogicalTypes setting for the Airflow operator
> GoogleCloudStorageToBigQueryOperator
> ---------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-3541
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3541
> Project: Apache Airflow
> Issue Type: New Feature
> Components: operators
> Environment: Composer
> Reporter: Monica Parpi
> Priority: Major
> Labels: pull-request-available
>
> Getting an error [1] when loading data from GCS into BigQuery using the
> GoogleCloudStorageToBigQueryOperator for a Avro schema containing this column:
> {
> "name": "date_requested",
> "type": "int",
> "logicalType": "date",
> }
> This column contains the number of days since the epoch as indicated here [2].
> It is possible to load this data into a DATE column by using the
> "--useAvroLogicalTypes" parameter when initiating a load using the CLI [3]
> It would be very useful if this setting could be enabled via the Airflow
> operator that performs the equivalent data load [4].
> Thank you!
> [ [1]
> https://issuetracker.google.com/35905894|https://issuetracker.google.com/35905894]
>
> [[2]
> http://avro.apache.org/docs/current/spec.html#Date|http://avro.apache.org/docs/current/spec.html#Date]
> [3]
> [https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.useAvroLogicalTypes]
> [4]
> [https://airflow.apache.org/_modules/airflow/contrib/operators/gcs_to_bq.html#GoogleCloudStorageToBigQueryOperator]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
