[jira] [Created] (HUDI-5457) Configuration documentation for hoodie.datasource.write.operation needs to be updated

2022-12-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5457: - Summary: Configuration documentation for hoodie.datasource.write.operation needs to be updated Key: HUDI-5457 URL: https://issues.apache.org/jira/browse/HUDI-5457

[jira] [Created] (HUDI-5452) Spark-sql long datatype conversion to bigint in hive causes issues with alter table

2022-12-21 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5452: - Summary: Spark-sql long datatype conversion to bigint in hive causes issues with alter table Key: HUDI-5452 URL: https://issues.apache.org/jira/browse/HUDI-5452

[jira] [Commented] (HUDI-5419) Spark-SQL tests persist configs

2022-12-20 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649907#comment-17649907 ] Jonathan Vexler commented on HUDI-5419: --- Here is my modified test function from

[jira] [Assigned] (HUDI-5419) Spark-SQL tests persist configs

2022-12-20 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-5419: - Assignee: (was: Jonathan Vexler) > Spark-SQL tests persist configs >

[jira] [Updated] (HUDI-5419) Spark-SQL tests persist configs

2022-12-20 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5419: -- Priority: Minor (was: Major) > Spark-SQL tests persist configs >

[jira] [Updated] (HUDI-5419) Spark-SQL tests persist configs

2022-12-20 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5419: -- Status: Open (was: In Progress) > Spark-SQL tests persist configs >

[jira] [Commented] (HUDI-5419) Spark-SQL tests persist configs

2022-12-20 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649903#comment-17649903 ] Jonathan Vexler commented on HUDI-5419: --- [^logs_24157.zip] Are the logs for the last gh actions test

[jira] [Updated] (HUDI-5419) Spark-SQL tests persist configs

2022-12-20 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5419: -- Attachment: logs_24157.zip > Spark-SQL tests persist configs >

[jira] [Updated] (HUDI-5418) Spark Sql Guide says that precombine field is only required for MOR but it is always required

2022-12-19 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5418: -- Sprint: 2022/12/12 > Spark Sql Guide says that precombine field is only required for MOR but it

[jira] [Updated] (HUDI-5419) Spark-SQL tests persist configs

2022-12-19 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5419: -- Status: In Progress (was: Open) > Spark-SQL tests persist configs >

[jira] [Updated] (HUDI-5418) Spark Sql Guide says that precombine field is only required for MOR but it is always required

2022-12-19 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5418: -- Status: Patch Available (was: In Progress) > Spark Sql Guide says that precombine field is

[jira] [Updated] (HUDI-5418) Spark Sql Guide says that precombine field is only required for MOR but it is always required

2022-12-19 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5418: -- Status: In Progress (was: Open) > Spark Sql Guide says that precombine field is only required

[jira] [Updated] (HUDI-5419) Spark-SQL tests persist configs

2022-12-19 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5419: -- Sprint: 2022/12/12 > Spark-SQL tests persist configs > > >

[jira] [Closed] (HUDI-5390) Site does not tell users to use master branch for docker demo for m1 mac

2022-12-19 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5390. - Resolution: Fixed > Site does not tell users to use master branch for docker demo for m1 mac >

[jira] [Created] (HUDI-5419) Spark-SQL tests persist configs

2022-12-19 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5419: - Summary: Spark-SQL tests persist configs Key: HUDI-5419 URL: https://issues.apache.org/jira/browse/HUDI-5419 Project: Apache Hudi Issue Type: Test

[jira] [Created] (HUDI-5418) Spark Sql Guide says that precombine field is only required for MOR but it is always required

2022-12-19 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5418: - Summary: Spark Sql Guide says that precombine field is only required for MOR but it is always required Key: HUDI-5418 URL: https://issues.apache.org/jira/browse/HUDI-5418

[jira] [Updated] (HUDI-5390) Site does not tell users to use master branch for docker demo for m1 mac

2022-12-14 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5390: -- Status: In Progress (was: Open) > Site does not tell users to use master branch for docker

[jira] [Updated] (HUDI-5390) Site does not tell users to use master branch for docker demo for m1 mac

2022-12-14 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5390: -- Status: Patch Available (was: In Progress) > Site does not tell users to use master branch for

[jira] [Created] (HUDI-5390) Site does not tell users to use master branch for docker demo for m1 mac

2022-12-14 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5390: - Summary: Site does not tell users to use master branch for docker demo for m1 mac Key: HUDI-5390 URL: https://issues.apache.org/jira/browse/HUDI-5390 Project:

[jira] [Updated] (HUDI-5261) Use proper parallelism for engine context APIs

2022-12-13 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5261: -- Status: Patch Available (was: In Progress) > Use proper parallelism for engine context APIs >

[jira] [Updated] (HUDI-5262) When creating table in spark-sql setting wrong keygenerator config does not warn

2022-12-13 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5262: -- Sprint: 2022/12/06 > When creating table in spark-sql setting wrong keygenerator config does

[jira] [Updated] (HUDI-5376) Update quickstart guide for hudi hoodie.datasource.write.keygenerator.class spark-sql change

2022-12-13 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5376: -- Sprint: 2022/12/06 > Update quickstart guide for hudi

[jira] [Updated] (HUDI-5376) Update quickstart guide for hudi hoodie.datasource.write.keygenerator.class spark-sql change

2022-12-13 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5376: -- Status: In Progress (was: Open) > Update quickstart guide for hudi

[jira] [Updated] (HUDI-5376) Update quickstart guide for hudi hoodie.datasource.write.keygenerator.class spark-sql change

2022-12-13 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5376: -- Status: Patch Available (was: In Progress) > Update quickstart guide for hudi

[jira] [Created] (HUDI-5376) Update quickstart guide for hudi hoodie.datasource.write.keygenerator.class spark-sql change

2022-12-12 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5376: - Summary: Update quickstart guide for hudi hoodie.datasource.write.keygenerator.class spark-sql change Key: HUDI-5376 URL: https://issues.apache.org/jira/browse/HUDI-5376

[jira] [Updated] (HUDI-5262) When creating table in spark-sql setting wrong keygenerator config does not warn

2022-12-12 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5262: -- Status: Patch Available (was: In Progress) > When creating table in spark-sql setting wrong

[jira] [Updated] (HUDI-5262) When creating table in spark-sql setting wrong keygenerator config does not warn

2022-12-12 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5262: -- Status: In Progress (was: Open) > When creating table in spark-sql setting wrong keygenerator

[jira] [Assigned] (HUDI-5262) When creating table in spark-sql setting wrong keygenerator config does not warn

2022-12-12 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-5262: - Assignee: Jonathan Vexler > When creating table in spark-sql setting wrong keygenerator

[jira] [Closed] (HUDI-5359) If someone sets multiple keys as primary in spark-sql and doesn't set a keygen it will default to simple instead of complex

2022-12-12 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5359. - Resolution: Cannot Reproduce > If someone sets multiple keys as primary in spark-sql and doesn't

[jira] [Created] (HUDI-5359) If someone sets multiple keys as primary in spark-sql and doesn't set a keygen it will default to simple instead of complex

2022-12-09 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5359: - Summary: If someone sets multiple keys as primary in spark-sql and doesn't set a keygen it will default to simple instead of complex Key: HUDI-5359 URL:

[jira] [Updated] (HUDI-5359) If someone sets multiple keys as primary in spark-sql and doesn't set a keygen it will default to simple instead of complex

2022-12-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5359: -- Sprint: 2022/12/06 > If someone sets multiple keys as primary in spark-sql and doesn't set a >

[jira] [Commented] (HUDI-5305) Detect concurrent writes during compaction and clustering if they shouldn't happen

2022-12-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645396#comment-17645396 ] Jonathan Vexler commented on HUDI-5305: --- I think this issue is getting moved to someone else because

[jira] [Updated] (HUDI-5304) Manage spark-sql core flow tests

2022-12-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5304: -- Status: In Progress (was: Open) > Manage spark-sql core flow tests >

[jira] [Updated] (HUDI-5304) Manage spark-sql core flow tests

2022-12-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5304: -- Status: Patch Available (was: In Progress) > Manage spark-sql core flow tests >

[jira] [Commented] (HUDI-5231) Address checkstyle warnings while building hudi

2022-12-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645395#comment-17645395 ] Jonathan Vexler commented on HUDI-5231: --- We probably need a longer term fix, but think we will

[jira] [Closed] (HUDI-5304) Manage spark-sql core flow tests

2022-12-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5304. - Resolution: Fixed > Manage spark-sql core flow tests > > >

[jira] [Updated] (HUDI-5321) Fix Bulk Insert ColumnSortPartitioners

2022-12-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5321: -- Status: Patch Available (was: In Progress) > Fix Bulk Insert ColumnSortPartitioners >

[jira] [Updated] (HUDI-5261) Use proper parallelism for engine context APIs

2022-12-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5261: -- Status: In Progress (was: Open) > Use proper parallelism for engine context APIs >

[jira] [Commented] (HUDI-5261) Use proper parallelism for engine context APIs

2022-12-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645041#comment-17645041 ] Jonathan Vexler commented on HUDI-5261: --- TimelineServerPerf has numExecuters with a default of 10

[jira] [Commented] (HUDI-5261) Use proper parallelism for engine context APIs

2022-12-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645037#comment-17645037 ] Jonathan Vexler commented on HUDI-5261: --- FileSystemBackedTableMetadata has config  {code:java}

[jira] [Commented] (HUDI-5261) Use proper parallelism for engine context APIs

2022-12-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645032#comment-17645032 ] Jonathan Vexler commented on HUDI-5261: --- I see in this guide

[jira] [Updated] (HUDI-5321) Fix Bulk Insert ColumnSortPartitioners

2022-12-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5321: -- Status: In Progress (was: Open) > Fix Bulk Insert ColumnSortPartitioners >

[jira] [Updated] (HUDI-5335) Make metasync to multiple catalogs multithreaded

2022-12-06 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5335: -- Description: If you have multiple catalogs, metasync to each one happens sequentially. If a

[jira] [Created] (HUDI-5335) Make metasync to multiple catalogs multithreaded

2022-12-06 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5335: - Summary: Make metasync to multiple catalogs multithreaded Key: HUDI-5335 URL: https://issues.apache.org/jira/browse/HUDI-5335 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-5262) When creating table in spark-sql setting wrong keygenerator config does not warn

2022-12-06 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5262: -- Summary: When creating table in spark-sql setting wrong keygenerator config does not warn

[jira] [Updated] (HUDI-5305) Detect concurrent writes during compaction and clustering if they shouldn't happen

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5305: -- Status: In Progress (was: Open) > Detect concurrent writes during compaction and clustering if

[jira] [Updated] (HUDI-5305) Detect concurrent writes during compaction and clustering if they shouldn't happen

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5305: -- Sprint: 2022/11/29 > Detect concurrent writes during compaction and clustering if they

[jira] [Updated] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5295: -- Status: In Progress (was: Open) > With multiple meta syncs, one meta sync failure should not

[jira] [Updated] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5295: -- Status: Patch Available (was: In Progress) > With multiple meta syncs, one meta sync failure

[jira] [Updated] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5295: -- Sprint: 2022/11/29 > With multiple meta syncs, one meta sync failure should not impact other

[jira] [Created] (HUDI-5305) Detect concurrent writes during compaction and clustering if they shouldn't happen

2022-11-30 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5305: - Summary: Detect concurrent writes during compaction and clustering if they shouldn't happen Key: HUDI-5305 URL: https://issues.apache.org/jira/browse/HUDI-5305

[jira] [Created] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-11-29 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5295: - Summary: With multiple meta syncs, one meta sync failure should not impact other meta syncs. Key: HUDI-5295 URL: https://issues.apache.org/jira/browse/HUDI-5295

[jira] [Commented] (HUDI-4745) Fix flaky: ITTestDataStreamWrite.testWriteCopyOnWriteWithClustering

2022-11-28 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640076#comment-17640076 ] Jonathan Vexler commented on HUDI-4745: --- I think this run

[jira] [Commented] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637444#comment-17637444 ] Jonathan Vexler commented on HUDI-5231: --- Ethan claims to have a fix for hudi-common so skip those >

[jira] [Updated] (HUDI-5269) Enhancing core user flow tests for spark-sql writes

2022-11-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5269: -- Status: In Progress (was: Open) > Enhancing core user flow tests for spark-sql writes >

[jira] [Updated] (HUDI-5269) Enhancing core user flow tests for spark-sql writes

2022-11-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5269: -- Status: Patch Available (was: In Progress) > Enhancing core user flow tests for spark-sql

[jira] [Updated] (HUDI-5269) Enhancing core user flow tests for spark-sql writes

2022-11-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5269: -- Sprint: 2022/11/15 > Enhancing core user flow tests for spark-sql writes >

[jira] [Created] (HUDI-5269) Enhancing core user flow tests for spark-sql writes

2022-11-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5269: - Summary: Enhancing core user flow tests for spark-sql writes Key: HUDI-5269 URL: https://issues.apache.org/jira/browse/HUDI-5269 Project: Apache Hudi

[jira] [Created] (HUDI-5268) Warn when configs are set in spark-sql that don't do anything

2022-11-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5268: - Summary: Warn when configs are set in spark-sql that don't do anything Key: HUDI-5268 URL: https://issues.apache.org/jira/browse/HUDI-5268 Project: Apache Hudi

[jira] [Created] (HUDI-5267) Improve Documentation for spark-sql

2022-11-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5267: - Summary: Improve Documentation for spark-sql Key: HUDI-5267 URL: https://issues.apache.org/jira/browse/HUDI-5267 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-5263) Setting partitioned by (partition_path) with nonpartitioned keygenerator in spark-sql will cause the colum to be null

2022-11-22 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5263: -- Description: When creating the table, for example:  {code:java} create table hudi_cow_pt_tbl (

[jira] [Created] (HUDI-5266) Incremental Query for spark-sql

2022-11-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5266: - Summary: Incremental Query for spark-sql Key: HUDI-5266 URL: https://issues.apache.org/jira/browse/HUDI-5266 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-5265) Time travel query for

2022-11-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5265: - Summary: Time travel query for https://issues.apache.org/jira/browse/HUDI-5265 Project: Apache Hudi Issue Type: New Feature Components: spark-sql Reporter: Jonathan

[jira] [Created] (HUDI-5263) Setting partitioned by (partition_path) with nonpartitioned keygenerator in spark-sql will cause the colum to be null

2022-11-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5263: - Summary: Setting partitioned by (partition_path) with nonpartitioned keygenerator in spark-sql will cause the colum to be null Key: HUDI-5263 URL:

[jira] [Created] (HUDI-5262) When creating table in spark-sql setting wrong keygenerator class does not warn

2022-11-22 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5262: - Summary: When creating table in spark-sql setting wrong keygenerator class does not warn Key: HUDI-5262 URL: https://issues.apache.org/jira/browse/HUDI-5262

[jira] [Updated] (HUDI-5257) Spark-Sql duplicates and re-uses record keys under certain configs and use cases

2022-11-21 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5257: -- Description: On a new table with primary key  _row_key and partitioned by partition_path, if

[jira] [Created] (HUDI-5257) Spark-Sql duplicates and re-uses record keys under certain configs and use cases

2022-11-21 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5257: - Summary: Spark-Sql duplicates and re-uses record keys under certain configs and use cases Key: HUDI-5257 URL: https://issues.apache.org/jira/browse/HUDI-5257

[jira] [Updated] (HUDI-5242) Do not fail Meta sync in Deltastreamer when inline table service fails

2022-11-18 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5242: -- Sprint: 2022/11/15 (was: 2022/11/29) > Do not fail Meta sync in Deltastreamer when inline

[jira] [Updated] (HUDI-5242) Do not fail Meta sync in Deltastreamer when inline table service fails

2022-11-18 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5242: -- Sprint: 2022/11/29 > Do not fail Meta sync in Deltastreamer when inline table service fails >

[jira] [Updated] (HUDI-5242) Do not fail Meta sync in Deltastreamer when inline table service fails

2022-11-18 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5242: -- Status: In Progress (was: Open) > Do not fail Meta sync in Deltastreamer when inline table

[jira] [Updated] (HUDI-5242) Do not fail Meta sync in Deltastreamer when inline table service fails

2022-11-18 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5242: -- Status: Patch Available (was: In Progress) > Do not fail Meta sync in Deltastreamer when

[jira] [Created] (HUDI-5242) Do not fail Meta sync in Deltastreamer when inline table service fails

2022-11-18 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5242: - Summary: Do not fail Meta sync in Deltastreamer when inline table service fails Key: HUDI-5242 URL: https://issues.apache.org/jira/browse/HUDI-5242 Project: Apache

[jira] [Commented] (HUDI-4984) Presto and Trino don't work in Docker Demo M1 mac

2022-11-17 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635543#comment-17635543 ] Jonathan Vexler commented on HUDI-4984: --- Upped docker cpu to 7 cores and tried .271 and .275 presto

[jira] [Comment Edited] (HUDI-4984) Presto and Trino don't work in Docker Demo M1 mac

2022-11-17 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635518#comment-17635518 ] Jonathan Vexler edited comment on HUDI-4984 at 11/17/22 7:33 PM: - For

[jira] [Commented] (HUDI-4984) Presto and Trino don't work in Docker Demo M1 mac

2022-11-17 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635520#comment-17635520 ] Jonathan Vexler commented on HUDI-4984: --- Not really sure where to go with presto, but with trino we

[jira] [Commented] (HUDI-4984) Presto and Trino don't work in Docker Demo M1 mac

2022-11-17 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635518#comment-17635518 ] Jonathan Vexler commented on HUDI-4984: --- For presto,  We tried using the arm version of the image

[jira] [Commented] (HUDI-4984) Presto and Trino don't work in Docker Demo M1 mac

2022-11-17 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635512#comment-17635512 ] Jonathan Vexler commented on HUDI-4984: --- Using the existing images, when we try to use trino we get

[jira] [Commented] (HUDI-4984) Presto and Trino don't work in Docker Demo M1 mac

2022-11-16 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635010#comment-17635010 ] Jonathan Vexler commented on HUDI-4984: --- After adding presto back to the config, I was able to open

[jira] [Updated] (HUDI-4967) Improve docs for meta sync with TimestampBasedKeyGenerator

2022-11-11 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4967: -- Status: Patch Available (was: In Progress) > Improve docs for meta sync with

[jira] [Closed] (HUDI-5184) Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from HoodiePySparkQuickstart.py

2022-11-11 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5184. - Resolution: Fixed > Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from >

[jira] [Updated] (HUDI-4967) Improve docs for meta sync with TimestampBasedKeyGenerator

2022-11-11 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4967: -- Status: In Progress (was: Open) > Improve docs for meta sync with TimestampBasedKeyGenerator >

[jira] [Closed] (HUDI-5036) Update contribution guide based on PR validation

2022-11-11 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5036. - Resolution: Fixed > Update contribution guide based on PR validation >

[jira] [Created] (HUDI-5192) GH actions and azure ci tests run even for trivial fixes

2022-11-10 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5192: - Summary: GH actions and azure ci tests run even for trivial fixes Key: HUDI-5192 URL: https://issues.apache.org/jira/browse/HUDI-5192 Project: Apache Hudi

[jira] [Closed] (HUDI-5056) Add support to DELETE_PARTITIONS w/ wild card

2022-11-10 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5056. - Resolution: Fixed > Add support to DELETE_PARTITIONS w/ wild card >

[jira] [Closed] (HUDI-5171) Ensure validateTableConfig also checks for partition path field value switch

2022-11-10 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-5171. - Resolution: Fixed > Ensure validateTableConfig also checks for partition path field value switch

[jira] [Resolved] (HUDI-4888) Add validation to block COW table to use consistent hashing bucket index

2022-11-10 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler resolved HUDI-4888. --- > Add validation to block COW table to use consistent hashing bucket index >

[jira] [Closed] (HUDI-4888) Add validation to block COW table to use consistent hashing bucket index

2022-11-10 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-4888. - Resolution: Fixed > Add validation to block COW table to use consistent hashing bucket index >

[jira] [Updated] (HUDI-5184) Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from HoodiePySparkQuickstart.py

2022-11-10 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5184: -- Status: Patch Available (was: In Progress) > Remove export PYSPARK_SUBMIT_ARGS="--master

[jira] [Updated] (HUDI-5184) Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from HoodiePySparkQuickstart.py

2022-11-10 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5184: -- Status: In Progress (was: Open) > Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from

[jira] [Updated] (HUDI-4888) Add validation to block COW table to use consistent hashing bucket index

2022-11-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4888: -- Status: Patch Available (was: In Progress) > Add validation to block COW table to use

[jira] [Commented] (HUDI-4990) Parallelize deduplication in CLI tool

2022-11-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631215#comment-17631215 ] Jonathan Vexler commented on HUDI-4990: --- We are stuck on this for now because we can't run the

[jira] [Commented] (HUDI-5056) Add support to DELETE_PARTITIONS w/ wild card

2022-11-09 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631138#comment-17631138 ] Jonathan Vexler commented on HUDI-5056: --- Fix should be ready but azure ci is failing for a seemingly

[jira] [Created] (HUDI-5184) Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from HoodiePySparkQuickstart.py

2022-11-08 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5184: - Summary: Remove export PYSPARK_SUBMIT_ARGS="--master local[*]" from HoodiePySparkQuickstart.py Key: HUDI-5184 URL: https://issues.apache.org/jira/browse/HUDI-5184

[jira] [Updated] (HUDI-4888) Add validation to block COW table to use consistent hashing bucket index

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-4888: -- Status: In Progress (was: Open) > Add validation to block COW table to use consistent hashing

[jira] [Updated] (HUDI-5036) Update contribution guide based on PR validation

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5036: -- Status: Patch Available (was: In Progress) > Update contribution guide based on PR validation

[jira] [Updated] (HUDI-5036) Update contribution guide based on PR validation

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5036: -- Status: In Progress (was: Open) > Update contribution guide based on PR validation >

[jira] [Updated] (HUDI-5171) Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5171: -- Status: Patch Available (was: In Progress) > Ensure validateTableConfig also checks for

[jira] [Updated] (HUDI-5171) Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5171: -- Status: In Progress (was: Open) > Ensure validateTableConfig also checks for partition path

[jira] [Created] (HUDI-5180) Get Involved on the website has broken links

2022-11-08 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5180: - Summary: Get Involved on the website has broken links Key: HUDI-5180 URL: https://issues.apache.org/jira/browse/HUDI-5180 Project: Apache Hudi Issue Type:

[jira] [Assigned] (HUDI-5171) Ensure validateTableConfig also checks for partition path field value switch

2022-11-08 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler reassigned HUDI-5171: - Assignee: Jonathan Vexler (was: sivabalan narayanan) > Ensure validateTableConfig also

<    1   2   3   4   5   6   7   >