syun64 commented on code in PR #929:
URL: https://github.com/apache/iceberg-python/pull/929#discussion_r1684298458
##########
tests/integration/test_writes/test_writes.py:
##########
@@ -401,12 +402,12 @@ def
test_python_writes_with_small_and_large_types_spark_reads(
assert arrow_table_on_read.schema == pa.schema([
pa.field("foo", pa.large_string()),
pa.field("id", pa.int32()),
- pa.field("name", pa.large_string()),
+ pa.field("name", pa.string()),
pa.field(
"address",
pa.struct([
- pa.field("street", pa.large_string()),
- pa.field("city", pa.large_string()),
+ pa.field("street", pa.string()),
Review Comment:
@raulcd - It wasn't a bug, but actually an intentional change for the time
being. If we update to PyArrow 17.0.0 we will be able to revert that change,
and let the encoding in the parquet file dictate whether the table should be
read as a large or small type for the Table API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
