[ https://issues.apache.org/jira/browse/HUDI-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan updated HUDI-5928: -------------------------------------- Fix Version/s: 0.13.1 > PostWriteTermination Strategy does not shutdown the spark job completely > ------------------------------------------------------------------------ > > Key: HUDI-5928 > URL: https://issues.apache.org/jira/browse/HUDI-5928 > Project: Apache Hudi > Issue Type: Bug > Components: writer-core > Reporter: sivabalan narayanan > Assignee: sivabalan narayanan > Priority: Critical > Fix For: 0.13.1 > > > When post write termination strategy is used, spark job is not fully > shutdown. > {code:java} > 23/03/13 11:16:06 INFO DFSPathSelector: Root path => /tmp/input_dataset/ > source limit => 30000000 > 23/03/13 11:16:06 INFO DeltaSync: No new data, source checkpoint has not > changed. Nothing to commit. Old checkpoint=(Option{val=1678730775000}). New > Checkpoint=(1678730775000) > 23/03/13 11:16:06 INFO NoNewDataTerminationStrategy: Shutting down on > continuous mode as there is no new data for 3 > 23/03/13 11:16:12 INFO HoodieAsyncService: Waiting for next instant up to 10 > seconds > 23/03/13 11:16:22 INFO HoodieAsyncService: Waiting for next instant up to 10 > seconds > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 410 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 638 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 567 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 618 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 591 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 577 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 620 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 539 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 424 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 429 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 516 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 622 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 503 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 446 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 460 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 547 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 606 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 435 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 625 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 601 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 630 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 405 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 495 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 500 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 559 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 520 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 621 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 588 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 431 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 423 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 534 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 603 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 631 > 23/03/13 11:16:31 INFO BlockManagerInfo: Removed broadcast_20_piece0 on > 192.168.1.7:60400 in memory (size: 94.3 KB, free: 4.1 GB) > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 537 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 568 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 597 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 562 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 535 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 481 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 611 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 585 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 640 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 515 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 645 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 609 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 581 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 530 > 23/03/13 11:16:31 INFO BlockManagerInfo: Removed broadcast_23_piece0 on > 192.168.1.7:60400 in memory (size: 25.7 KB, free: 4.1 GB) > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 602 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 650 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 469 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 517 > 23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 542 {code} > Job is just stuck there. -- This message was sent by Atlassian Jira (v8.20.10#820010)