jlloh commented on issue #9256:
URL: https://github.com/apache/hudi/issues/9256#issuecomment-1651495991
> P.S. Can you please help to check if the configs below contain the new
column spec definition that was added via Spark?
```
getOrderedColumnExpr()
AppConfig.output.hudiSink.targetTable
```
`AppConfig.output.hudiSink.targetTable` is a string of the temp table that
the Flink append only job writes to. Using your example, it would be
`wt_test_hudi`.
`getOrderedColumnExpr()` generates a select expression in flink to ensure
the order of the columns inserted into the hudi table in Flink. I.e. `select
col_1, col_2, col_2`, so it should definitely contain the new column as well.
One thing I noticed differs in my configuration vs yours is that my Flink
table is not connected to the Hive metastore:
```
'hive_sync.enable'='true',
'hive_sync.metastore.uris' = 'thrift://hive.metastore.io:9083',
'hive_sync.use_jdbc' = 'false',
'hive_sync.mode' = 'hms',
'hive_sync.db' = 'dev_hudi',
'hive_sync.table' = 'hfse_flink_writes'
```
All these would be false.
When I did my alter table add partition, I did not set the session property,
but instead I did:
```
alter = """
alter table my_hive_schema.table SET TBLPROPERTIES ('hoodie.metadata.enable'
= 'false', 'hoodie.schema.on.read.enable' = 'true')
"""
spark.sql(alter).show()
```
Before I did a alter table add partition. Not sure if this is equivalent.
Also, one thing I noticed is that `hoodie.properties` does not change to
reflect the presence of my new columns. Let me know if you still want me to
test a primitive, non-nested column.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
