[
https://issues.apache.org/jira/browse/ARROW-17866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610657#comment-17610657
]
Joris Van den Bossche commented on ARROW-17866:
-----------------------------------------------
[~meystingray] thanks for the report!
This sounds very similar as ARROW-17137 (that also mentions a possible
workaround for now).
> [Python] List child array invalid
> ---------------------------------
>
> Key: ARROW-17866
> URL: https://issues.apache.org/jira/browse/ARROW-17866
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 9.0.0
> Reporter: Sean Conroy
> Priority: Major
>
> This issue happens for all the versions of pyarrow I checked (9.0.0, 7.0.0,
> 6.0.0, 6.0.1).
> Running on Windows 11.
> {code:java}
> log.to_feather(log_fname)
> Traceback (most recent call last):
> File "G:\My
> Drive\ds-atcore-etl\venv\lib\site-packages\IPython\core\interactiveshell.py",
> line 3444, in run_code
> exec(code_obj, self.user_global_ns, self.user_ns)
> File "<ipython-input-14-5d807cc56571>", line 1, in <module>
> log.to_feather(log_fname)
> File "G:\My
> Drive\ds-atcore-etl\venv\lib\site-packages\pandas\util\_decorators.py", line
> 207, in wrapper
> return func(*args, **kwargs)
> File "G:\My
> Drive\ds-atcore-etl\venv\lib\site-packages\pandas\core\frame.py", line 2519,
> in to_feather
> to_feather(self, path, **kwargs)
> File "G:\My
> Drive\ds-atcore-etl\venv\lib\site-packages\pandas\io\feather_format.py", line
> 87, in to_feather
> feather.write_feather(df, handles.handle, **kwargs)
> File "G:\My Drive\ds-atcore-etl\venv\lib\site-packages\pyarrow\feather.py",
> line 164, in write_feather
> table = Table.from_pandas(df, preserve_index=preserve_index)
> File "pyarrow\table.pxi", line 3495, in pyarrow.lib.Table.from_pandas
> File "pyarrow\table.pxi", line 3597, in pyarrow.lib.Table.from_arrays
> File "pyarrow\table.pxi", line 2793, in pyarrow.lib.Table.validate
> File "pyarrow\error.pxi", line 100, in pyarrow.lib.check_status
> pyarrow.lib.ArrowInvalid: Column 13: In chunk 0: Invalid: List child array
> invalid: Invalid: Struct child array #0 has length smaller than expected for
> struct array (67186731 < 67186732) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
