[ https://issues.apache.org/jira/browse/AIRFLOW-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaxil Naik closed AIRFLOW-6891. ------------------------------- Resolution: Cannot Reproduce I couldn't reproduce the issue from StackOverflow user > GCS to BQ operator fails when JSON is the source format > ------------------------------------------------------- > > Key: AIRFLOW-6891 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6891 > Project: Apache Airflow > Issue Type: Bug > Components: gcp > Affects Versions: 1.10.9 > Reporter: Kaxil Naik > Assignee: Kaxil Naik > Priority: Major > > From > https://stackoverflow.com/questions/60358764/airflow-gcs-to-bq-operator-fails-when-json-is-the-source-format > I have a GoogleCloudStorageToBigQueryOperator operator running on airflow in > a dag. It works perfect when working CSV files... I am now trying to ingest a > JSON file, and I'm receiving errors: such like: > *skipLeadingRows* is not a valid src_fmt_configs for type > *NEWLINE_DELIMITED_JSON* > The weird thing is that I'm not calling *skipLeadingRows* in my calling. as > below: > > {noformat} > load_Users_to_GBQ = GoogleCloudStorageToBigQueryOperator( > task_id='Table1_GCS_to_GBQ', > bucket='bucket1', > source_objects=['table*.json'], > source_format='NEWLINE_DELIMITED_JSON', > destination_project_dataset_table='DB.table1', > autodetect=False, > schema_fields=[ > {'name': 'fieldid', 'type': 'integer', 'mode': 'NULLABLE'}, > {'name': 'filed2', 'type': 'integer', 'mode': 'NULLABLE'}, > {'name': 'field3', 'type': 'string', 'mode': 'NULLABLE'}, > {'name': 'field4', 'type': 'string', 'mode': 'NULLABLE'}, > {'name': 'field5', 'type': 'string', 'mode': 'NULLABLE'} > ], > write_disposition='WRITE_TRUNCATE', > google_cloud_storage_conn_id='Conn1', > bigquery_conn_id='Conn1', > dag=dag) > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)