[ https://issues.apache.org/jira/browse/HADOOP-16810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Hussein updated HADOOP-16810: ----------------------------------- Description: I was investigating a JUnit test (MAPREDUCE-7079 :TestMRIntermediateDataEncryption is failing in precommit builds) that was consistently hanging on Linux VMs and failing Mapreduce pre-builds. I found that the test hangs slows or hangs indefinitely whenever Java reads the random file. I explored two different ways to get that test case to work properly on my local Linux VM running rel7: # The haveged service seeds a system's random source by executing a loop repeatedly and using the differences in the processor's time stamp counter. It ensures entropy never drops below 1000. # To install "haveged" and "rng-tools" on the virtual machine running Rel7. Then, start rngd service {{sudo service rngd start}} . This will fix the problem for all the components on the image including java, native and any other component. # Change java configuration to load urandom {code:bash} sudo vim $JAVA_HOME/jre/lib/security/java.security ## Change the line “securerandom.source=file:/dev/random” to read: securerandom.source=file:/dev/./urandom {code} The first solution is better because this will fix the problem for everything that requires SSL/TLS or other services that depend upon encryption. Since the precommit build runs on Docker, then it would be best to mount {{/dev/urandom}} from the host as {{/dev/random}} into the container: {code:java} docker run -v /dev/urandom:/dev/random {code} For Yetus, we need to add the mount to the {{DOCKER_EXTRAARGS}} as follows: {code:java} DOCKER_EXTRAARGS+=("-v" "/dev/urandom:/dev/random") {code} ... was: I was investigating a JUnit test (MAPREDUCE-7079 :TestMRIntermediateDataEncryption is failing in precommit builds) that was consistently hanging on Linux VMs and failing Mapreduce pre-builds. I found that the test hangs slows or hangs indefinitely whenever Java reads the random file. I explored two different ways to get that test case to work properly on my local Linux VM running rel7: # To install "haveged" and "rng-tools" on the virtual machine running Rel7. Then, start rngd service {{sudo service rngd start}} . This will fix the problem for all the components on the image including java, native and any other component. # Change java configuration to load urandom {code:bash} sudo vim $JAVA_HOME/jre/lib/security/java.security ## Change the line “securerandom.source=file:/dev/random” to read: securerandom.source=file:/dev/./urandom {code} The first solution is better because this will fix the problem for everything that requires SSL/TLS or other services that depend upon encryption. Since the precommit build runs on Docker, then it would be best to mount {{/dev/urandom}} from the host as {{/dev/random}} into the container: {code:java} docker run -v /dev/urandom:/dev/random {code} For Yetus, we need to add the mount to the {{DOCKER_EXTRAARGS}} as follows: {code:java} DOCKER_EXTRAARGS+=("-v" "/dev/urandom:/dev/random") {code} ... > Increase entropy to improve cryptographic randomness on precommit Linux VMs > --------------------------------------------------------------------------- > > Key: HADOOP-16810 > URL: https://issues.apache.org/jira/browse/HADOOP-16810 > Project: Hadoop Common > Issue Type: Bug > Reporter: Ahmed Hussein > Assignee: Allen Wittenauer > Priority: Major > > I was investigating a JUnit test (MAPREDUCE-7079 > :TestMRIntermediateDataEncryption is failing in precommit builds) that was > consistently hanging on Linux VMs and failing Mapreduce pre-builds. > I found that the test hangs slows or hangs indefinitely whenever Java reads > the random file. > I explored two different ways to get that test case to work properly on my > local Linux VM running rel7: > # The haveged service seeds a system's random source by executing a loop > repeatedly and using the differences in the processor's time stamp counter. > It ensures entropy never drops below 1000. > # To install "haveged" and "rng-tools" on the virtual machine running Rel7. > Then, start rngd service {{sudo service rngd start}} . This will fix the > problem for all the components on the image including java, native and any > other component. > # Change java configuration to load urandom > {code:bash} > sudo vim $JAVA_HOME/jre/lib/security/java.security > ## Change the line “securerandom.source=file:/dev/random” to read: > securerandom.source=file:/dev/./urandom > {code} > The first solution is better because this will fix the problem for everything > that requires SSL/TLS or other services that depend upon encryption. > Since the precommit build runs on Docker, then it would be best to mount > {{/dev/urandom}} from the host as {{/dev/random}} into the container: > {code:java} > docker run -v /dev/urandom:/dev/random > {code} > For Yetus, we need to add the mount to the {{DOCKER_EXTRAARGS}} as follows: > {code:java} > DOCKER_EXTRAARGS+=("-v" "/dev/urandom:/dev/random") > {code} > ... -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org