[
https://issues.apache.org/jira/browse/ARROW-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jacques updated ARROW-2591:
---------------------------
Description:
Context Is the following: I am currently dealing with sparse column
serialization in parquet. In some cases, many lines are empty I can also have
columns containing only empty lists.
However I got a segmentation fault when I try to write in parquet thoses
columns filled only with empty lists.
Here is a simple code snipet reproduces the segmentation fault I had:
{noformat}
In [1]: import pyarrow as pa
In [2]: import pyarrow.parquet as pq
In [3]: pa_ar = pa.array([[],[]],pa.list_(pa.int32()))
In [4]: table = pa.Table.from_arrays([pa_ar],["test"])
In [5]: pq.write_table(
...: table=table,
...: where="test.parquet",
...: compression="snappy",
...: flavor="spark"
...: )
Segmentation fault
{noformat}
May I have it fixed?
Best
Jacques
was:
Context Is the following: I am currently dealing with sparse column
serialization in parquet. In some cases, many of this
When trying this simple code snipet I got a segmentation fault
{noformat}
In [1]: import pyarrow as pa
In [2]: import pyarrow.parquet as pq
In [3]: pa_ar = pa.array([[],[]],pa.list_(pa.int32()))
In [4]: table = pa.Table.from_arrays([pa_ar],["test"])
In [5]: pq.write_table(
...: table=table,
...: where="test.parquet",
...: compression="snappy",
...: flavor="spark"
...: )
Segmentation fault
{noformat}
May I have it fixed.
Best
Jacques
> [Python] Segmentationfault issue in pq.write_table
> --------------------------------------------------
>
> Key: ARROW-2591
> URL: https://issues.apache.org/jira/browse/ARROW-2591
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 0.8.0, 0.9.0
> Reporter: jacques
> Priority: Major
>
> Context Is the following: I am currently dealing with sparse column
> serialization in parquet. In some cases, many lines are empty I can also have
> columns containing only empty lists.
> However I got a segmentation fault when I try to write in parquet thoses
> columns filled only with empty lists.
> Here is a simple code snipet reproduces the segmentation fault I had:
> {noformat}
> In [1]: import pyarrow as pa
> In [2]: import pyarrow.parquet as pq
> In [3]: pa_ar = pa.array([[],[]],pa.list_(pa.int32()))
> In [4]: table = pa.Table.from_arrays([pa_ar],["test"])
> In [5]: pq.write_table(
> ...: table=table,
> ...: where="test.parquet",
> ...: compression="snappy",
> ...: flavor="spark"
> ...: )
> Segmentation fault
> {noformat}
> May I have it fixed?
> Best
> Jacques
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
