MikeMccree closed issue #10273: [SUPPORT] - Issues after upgrading EMR & Hudi
URL: https://github.com/apache/hudi/issues/10273
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
T
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1855943806
Hi @ad1happy2go yes, confirmed it is syncing. I see the DB, tables and data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
ad1happy2go commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1855513194
@MikeMccree Are you sure after removing this is it syncing to Glue Catalog.
Did you confirmed the tables ?
--
This is an automated message from the Apache Git Service.
To respon
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1853396088
@ad1happy2go Which logs would you like to see?
Also - after more playing around with the configs, I discovered the below:
```
# 'hoodie.meta.sync.client.tool.cl
ad1happy2go commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1852540846
Can you provide us the logs to look into it more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1852522867
@ad1happy2go Something else interesting to note. If I manually create the DB
```
database_name = "michael_test"
# Create the database
spark.sql(f"CREATE DATABASE IF N
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1852492519
Hi @ad1happy2go
_Also did you tried with explicitly defining the Glue Sync Tool?_ Yes I had
it in while running all my tests
I have added both of these configurations
ad1happy2go commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1852436464
@MikeMccree Do you have this database in glue, If yes then your setup might
not be accessing glue at all.
You can use
`[hoodie.datasource.hive_sync.auto_create_database](https
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1852429920
@ad1happy2go After more toying around I managed to get rid of the above
exceptions by being specific about the JARS I am submitting along with my
spark-submit. The problem now is I
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1850461094
Hi @ad1happy2go
Thanks for the above - I think my real question is the following "Are there
any additional config changes I need to make to my script to upgrade from hudi
0
ad1happy2go commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1849968200
@MikeMccree Do you want to sync your table with Glue catalog. If yes, can
you set "hoodie.meta.sync.client.tool.class" as
"org.apache.hudi.aws.sync.AwsGlueCatalogSyncTool").
--
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1849816260
@ad1happy2go The only config I am using is mentioned above, but here it is
again for you:
`hudi_streaming_count_options = { 'hoodie.database.name': database_name,
'hoodie.ta
ad1happy2go commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1849361768
@MikeMccree Can you let us know the hive sync configurations you are using?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
MikeMccree commented on issue #10273:
URL: https://github.com/apache/hudi/issues/10273#issuecomment-1847137840
You need any more info to assist?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
14 matches
Mail list logo
