[
https://issues.apache.org/jira/browse/ARROW-6081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897125#comment-16897125
]
Joris Van den Bossche commented on ARROW-6081:
----------------------------------------------
The final error comes from bigquery, so you might want to also report it to
them that they should handle arrow errors in a clearer way.
The underlying error raised from arrow is "pyarrow.lib.ArrowInvalid: Nested
column branch had multiple children". It is hard to say for sure without a
reproducible example, but I suppose this is related to the current limitation
of the arrow parquet reader regarding nested columns with multiple children.
See eg ARROW-1644, ARROW-1599
> FileNotFoundError: [Errno 2] No such file or directory:
> '/tmp/tmptb2ao6te_job_6e0a8ca1.parquet'
> -----------------------------------------------------------------------------------------------
>
> Key: ARROW-6081
> URL: https://issues.apache.org/jira/browse/ARROW-6081
> Project: Apache Arrow
> Issue Type: Bug
> Reporter: David Draper
> Priority: Major
>
> Any idea on how to fix this error?
>
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.6/site-packages/google/cloud/bigquery/client.py",
> line 1530, in load_table_from_dataframe
> dataframe.to_parquet(tmppath)
> File "/usr/local/lib64/python3.6/site-packages/pandas/core/frame.py", line
> 2203, in to_parquet
> partition_cols=partition_cols, **kwargs)
> File "/usr/local/lib64/python3.6/site-packages/pandas/io/parquet.py", line
> 252, in to_parquet
> partition_cols=partition_cols, **kwargs)
> File "/usr/local/lib64/python3.6/site-packages/pandas/io/parquet.py", line
> 122, in write
> coerce_timestamps=coerce_timestamps, **kwargs)
> File "/usr/local/lib64/python3.6/site-packages/pyarrow/parquet.py", line
> 1270, in write_table
> writer.write_table(table, row_group_size=row_group_size)
> File "/usr/local/lib64/python3.6/site-packages/pyarrow/parquet.py", line
> 426, in write_table
> self.writer.write_table(table, row_group_size=row_group_size)
> File "pyarrow/_parquet.pyx", line 1311, in
> pyarrow._parquet.ParquetWriter.write_table
> File "pyarrow/error.pxi", line 85, in pyarrow.lib.check_status
> pyarrow.lib.ArrowInvalid: Nested column branch had multiple children
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/var/cache/tomcat/temp/interpreter-2169813765840716657.tmp", line 84,
> in <module>
> client.load_table_from_dataframe(appended_data,
> table_ref,job_config=job_config).result()
> File
> "/usr/local/lib/python3.6/site-packages/google/cloud/bigquery/client.py",
> line 1546, in load_table_from_dataframe
> os.remove(tmppath)
> FileNotFoundError: [Errno 2] No such file or directory:
> '/tmp/tmptb2ao6te_job_6e0a8ca1.parquet'
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
