Repository: arrow Updated Branches: refs/heads/master 3d285c69d -> d9f895ebd
ARROW-1446: [Python] Add (very slow) large memory unit test for int32 overflow in PARQUET-1090 Bug fixed in https://github.com/apache/parquet-cpp/pull/389 Author: Wes McKinney <[email protected]> Closes #1055 from wesm/ARROW-1446 and squashes the following commits: 97c634c9 [Wes McKinney] Add (very slow) large memory unit test for PARQUET-1090 Project: http://git-wip-us.apache.org/repos/asf/arrow/repo Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/d9f895eb Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/d9f895eb Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/d9f895eb Branch: refs/heads/master Commit: d9f895ebde94a21bd2bf91085a186683fb521e48 Parents: 3d285c6 Author: Wes McKinney <[email protected]> Authored: Wed Sep 6 19:27:23 2017 -0400 Committer: Wes McKinney <[email protected]> Committed: Wed Sep 6 19:27:23 2017 -0400 ---------------------------------------------------------------------- python/pyarrow/tests/test_parquet.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/arrow/blob/d9f895eb/python/pyarrow/tests/test_parquet.py ---------------------------------------------------------------------- diff --git a/python/pyarrow/tests/test_parquet.py b/python/pyarrow/tests/test_parquet.py index fa9455b..ae5c28f 100644 --- a/python/pyarrow/tests/test_parquet.py +++ b/python/pyarrow/tests/test_parquet.py @@ -1239,3 +1239,17 @@ def test_write_to_dataset_with_partitions(tmpdir): @parquet def test_write_to_dataset_no_partitions(tmpdir): _test_write_to_dataset_no_partitions(str(tmpdir)) + + [email protected]_memory +@parquet +def test_large_table_int32_overflow(): + size = np.iinfo('int32').max + 1 + + arr = np.ones(size, dtype='uint8') + + parr = pa.Array.from_pandas(arr, type=pa.uint8()) + + table = pa.Table.from_arrays([parr], names=['one']) + f = io.BytesIO() + _write_table(table, f)
