[
https://issues.apache.org/jira/browse/HIVE-25871?focusedWorklogId=711573&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-711573
]
ASF GitHub Bot logged work on HIVE-25871:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Jan/22 18:53
Start Date: 19/Jan/22 18:53
Worklog Time Spent: 10m
Work Description: boroknagyz commented on pull request #2948:
URL: https://github.com/apache/hive/pull/2948#issuecomment-1016768717
> > HiveSchemaConverter assigned field ids starting from 0, while Iceberg
assigns field ids starting from 1. This caused test failures because the name
mapping had wrong field ids. So in the second commit I fixed
HiveSchemaConverter as well.
>
> If we provide a schema where the id is starting from 0, would that mean
that the schema is not correct? Or this is just to be more aligned with the way
Iceberg generates ids?
Seems like Iceberg doesn't respect the field ids during table creation, it
always reassigns them.
E.g. I was trying to create a table with this schema:
`iceberg.mr.table.schema={"type":"struct","schema-id":0,"fields":[{"id":0,"name":"a","required":false,"type":"int"}]}`
or
`iceberg.mr.table.schema={"type":"struct","schema-id":0,"fields":[{"id":5,"name":"a","required":false,"type":"int"}]}`
And Iceberg created the following schema:
`table {
1: a: optional int
}`
OTOH, I don't think the value 0 is invalid. The spec reserves only these ids:
https://iceberg.apache.org/#spec/#reserved-field-ids
So I edited the Iceberg snapshot file and rewrote the field ids to start
from 0, then inserted data files and queried the table. Iceberg didn't complain
about the 0 field id.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 711573)
Time Spent: 50m (was: 40m)
> Hive should set name mapping table property for migrated Iceberg tables
> -----------------------------------------------------------------------
>
> Key: HIVE-25871
> URL: https://issues.apache.org/jira/browse/HIVE-25871
> Project: Hive
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Hive should set the name-mapping table property during table migration.
> It would be useful for [column
> projection|https://iceberg.apache.org/#spec/#column-projection] for files
> without field ids.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
