mgmarino commented on issue #216:
URL: https://github.com/apache/iceberg-python/issues/216#issuecomment-1899935248
It seems that this is also required to enable AWS Athena to query a table
created with `pyiceberg` and not just useful for UI/CLI tools.
### Example
Running a test to generate a table on glue:
```python
from pyiceberg.schema import Schema
from pyiceberg.types import (
StringType,
NestedField,
)
schema = Schema(
NestedField(field_id=1, name="x", field_type=StringType(),
required=False),
)
catalog.create_namespace("test_pyiceberg",
properties=dict(location="s3://tado-data-scratch/test_pyiceberg"))
catalog.create_table(
identifier="test_pyiceberg.test",
schema=schema,
)
import pyarrow as pa
import pandas as pd
t = catalog.load_table("test_pyiceberg.test")
to_append = pa.Table.from_pandas(pd.DataFrame([dict(x="hello!")]))
t.append(to_append)
t.scan().to_pandas()
# x
# 0 hello!
```
The subsequent Athena query:
```sql
SELECT * FROM test_pyiceberg.test
```
fails with
```
Reason: COLUMN_NOT_FOUND: line 1:8: SELECT * not allowed from relation that
has no columns
```
At this point, updating the schema manually in Glue then results in being
able to successfully query Athena.
### Other comments
As a side note, querying the schema via Athena still works, e.g. before
updating the schema in Glue
```sql
SHOW COLUMNS FROM test_pyiceberg.test
```
will succeed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
