jingyi-zhao-01 commented on issue #45590:
URL: https://github.com/apache/arrow/issues/45590#issuecomment-4960260938
I can still reproduce this on current Arrow main / local PyArrow dev build:
- pyarrow `26.0.0.dev3+gb03b4c539`
This does not require the original CSV file; a table with duplicate field
names is enough:
```python
import pyarrow as pa
table = pa.table([[1, 2], [3, 4], [5, 6]], names=["keep", "dup", "dup"])
print(table.column_names)
print(table.schema.get_field_index("dup"))
print(table.schema.get_all_field_indices("dup"))
print(table.select([0]))
print(table.drop_columns(["dup"]))
```
Observed output:
```text
['keep', 'dup', 'dup']
-1
[1, 2]
pyarrow.Table
keep: int64
----
keep: [[1,2]]
KeyError: "Column 'dup' not found"
```
So the issue is still consistent with the earlier analysis: `drop_columns()`
uses `Schema.get_field_index()`, which returns `-1` for an ambiguous duplicate
field name, even though `get_all_field_indices("dup")` can identify the
matching columns.
Minimal ask: should `drop_columns("dup")` remove all columns with that name
when the schema contains duplicates, matching the user-facing meaning of
dropping by field name? If yes, I can take a shot at a small test + narrow fix
around `drop_columns()` / `drop()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
