[ https://issues.apache.org/jira/browse/YARN-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802642#comment-17802642 ]
Shilun Fan commented on YARN-11054: ----------------------------------- Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0. > Alleviate LocalJobRunnerMetricName Conflicts > -------------------------------------------- > > Key: YARN-11054 > URL: https://issues.apache.org/jira/browse/YARN-11054 > Project: Hadoop YARN > Issue Type: Improvement > Affects Versions: 3.3.1 > Reporter: Xingjun Hao > Priority: Minor > Labels: pull-request-available > Fix For: 3.3.1 > > Time Spent: 20m > Remaining Estimate: 0h > > In some scenarios, Sqoop will use LocalJobRuner (YarnLocal mode) to run a lot > of jobs, assuming 2 million jobs have been run, LocalJobRunner MetricName > generated by nextInt function is in the range of (0, 2147483647), > Then the probability of conflict is about 2000/2147483647 = 1/1000, which > means that an average of 1 task will fail for every 1000 jobs run. > If LocalJobRunner MetricName is generated by nextLong() whose range is (0, > 9223372036854775807), considering that Long's range is 1 billion times that > of Int, the probability of a new MetricName conflicts is also reduced by one > trillionth times. > the probability of conflict also goes to 1/500000000 from 1/1000. ( under the > situation that there are about 200million jobs have been run) -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org