kkrugler commented on issue #8641: URL: https://github.com/apache/pinot/issues/8641#issuecomment-1144092481
@Jackie-Jiang - I was thinking about the `terminateOnFailure` flag a bit. The other segment generating code (Spark & Hadoop) already terminate on failure implicitly, as the task (and thus job) will fail if there's a segment generation failure. I guess there's some small value in finding out all of the segment generation failures in one pass (so not terminating on failure), but in my experience this hasn't been an issue for our Hadoop jobs, so not sure how much value is added by being able to control that behavior. And to be consistent, I think we'd need to change Hadoop & Spark code to support NOT terminating on failure if that flag isn't set, yes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
