[GitHub] [hudi] chenshzh commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants
chenshzh commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1366406000 > Hello @chenshzh , do you think this is still an issue here ? We can have an offline talk if possible, are you in the Hudi DingTalk group now ? Yes, I'm in the group now, and would be glad to discuss it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] chenshzh commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants
chenshzh commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1365612404 > * FAILURE @alexeykudinkin would you help review the CI failure once more? I have rebased to the latest and it seems not the problem of this PR's specific changes ? Because we find it failed in `hudi-utilities` and quite many recent PRs' pipeline failures are all about this module. ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project hudi-utilities_2.11: There are test failures. [ERROR] [ERROR] Please refer to /home/vsts/work/1/s/hudi-utilities/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. [ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd /home/vsts/work/1/s/hudi-utilities && /usr/lib/jvm/temurin-8-jdk-amd64/jre/bin/java -Xmx2g org.apache.maven.surefire.booter.ForkedBooter /home/vsts/work/1/s/hudi-utilities/target/surefire 2022-12-26T06-54-12_966-jvmRun1 surefire3510960863127007832tmp surefire_87691856527070078480tmp [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 255 [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd /home/vsts/work/1/s/hudi-utilities && /usr/lib/jvm/temurin-8-jdk-amd64/jre/bin/java -Xmx2g org.apache.maven.surefire.booter.ForkedBooter /home/vsts/work/1/s/hudi-utilities/target/surefire 2022-12-26T06-54-12_966-jvmRun1 surefire3510960863127007832tmp surefire_87691856527070078480tmp [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 255 [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011) [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857) [ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) [ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192) [ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105) [ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:960) [ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293) [ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:196) [ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [ERROR] at java.lang.reflect.Method.invoke(Method.java:498) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406) [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347) [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR]
[GitHub] [hudi] chenshzh commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants
chenshzh commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1364929996 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] chenshzh commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants
chenshzh commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1345809714 > @chenshzh would you please rebase to latest master first？ @leesf updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] chenshzh commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants
chenshzh commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1172318760 @danny0405 updated already, pls take a review to see whether it's ok. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] chenshzh commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants
chenshzh commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1169577544 Fix conflicts with https://issues.apache.org/jira/projects/HUDI/issues/HUDI-4311 , which also has attempted to fix the rollback scene. But it might not be complete because rollback could happen during runtime, such as StreamWriteOperatorCoordinator#notifyCheckpointComplete when commitInstant, and it won't crash the job for CkpMeta to rebootstrap. Would it be better to be linked with the actual rollback operation? So I re-added the bootstrap startInstant logics -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
