Hi Dongjin, Thanks for doing this research. I don't see building-druid.png attached to your mail -- maybe the ASF mailing list ate it? Please do send in a pull request to update the documentation. Fwiw I think 8GB should be enough memory with the right settings, since our Travis CI build environment only has about 7.5GB. We use the sudo-enabled trusty environment described here: https://docs.travis-ci.com/user/reference/overview/. Our Travis config is here: https://github.com/apache/incubator-druid/blob/master/.travis.yml
Separately it would also be interesting to see what this memory is being used for during the build. I wonder if it's unavoidable or if some of the memory use is 'silly' and can be reduced. On Tue, Jul 24, 2018 at 9:46 PM Dongjin Lee <dong...@apache.org> wrote: > After some experiments, I figured out the following: > > 1. Druid uses above 8gb of memory for testing. (building-druid.png) > 2. With 8gb(physical)+4gb(swap) of memory, the test succeeds regardless of > maven version (3.3.9, 3.5.2, 3.5.4) or MAVEN_OPTS. However, with > 8gb(physical)+2gb(swap) of memory[^1], some tests failed. The list of > failing tests differs between maven 3.3.9 and 3.5.2. > > In short, retaining sufficient memory solved the problem - *It seems like > 12gb of memory is a recommended setting for building druid.* (I guess > lots of you are working with the MacBook Pro with 16gb RAM, right? In that > case, you must not have encountered this problem.) > > If you are okay, may I update the building documentation for the newbies > like me? > > Thanks, > Dongjin > > +1. While building Druid, I found another problem. But this issue should > be discussed in another thread. > > [^1]: You know, the other processes also occupy the memory. > > > On Tue, Jul 24, 2018 at 3:07 AM Jihoon Son <jihoon...@apache.org> wrote: > >> I'm also using Maven 3.5.2 and not using any special configurations for >> Maven, but I have never seen that error too. >> Most of our Travis jobs have been working with only 512 MB of direct >> memory. Only the 'strict compilation' Travis job requires 3 GB of memory. >> >> I think it's worthwhile to look into this more. Maybe we somehow use more >> memory when we run all tests by 'mvn install'. Maybe this relates to the >> frequent transient failures of 'processing module test', one of our Travis >> jobs. >> >> Jihoon >> >> On Mon, Jul 23, 2018 at 9:32 AM Gian Merlino <g...@apache.org> wrote: >> >> > Interesting. Fwiw, I am using Maven 3.5.2 for building Druid and it has >> > been working for for me. I don't think I"m using any special Maven >> > overrides (at least, I don't see anything interesting in my ~/.m2 >> directory >> > or in my environment variables). It might have to do with how much >> memory >> > our machines have? I do most of my builds on a Mac with 16GB RAM. Maybe >> try >> > checking .travis.yml in the druid repo. It sets -Xmx3000m for mvn >> install >> > commands, which might be needed for more low memory environments. >> > >> > $ mvn --version >> > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; >> > 2017-10-18T00:58:13-07:00) >> > Maven home: /usr/local/Cellar/maven/3.5.2/libexec >> > Java version: 1.8.0_161, vendor: Oracle Corporation >> > Java home: >> > /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre >> > Default locale: en_US, platform encoding: UTF-8 >> > OS name: "mac os x", version: "10.13.5", arch: "x86_64", family: "mac" >> > >> > On Mon, Jul 23, 2018 at 6:40 AM Dongjin Lee <dong...@apache.org> wrote: >> > >> > > Finally, it seems like I found the reason. It was a composition of >> > several >> > > problems: >> > > >> > > - Druid should not be built with maven 3.5.x. With 3.5.2, Test suites >> > like >> > > `GroupByQueryRunnerFailureTest` fails. After I switched into 3.3.9 >> which >> > is >> > > built in the latest version of IntelliJ, those errors disappeared. It >> > seems >> > > like maven 3.5.x is not stable yet - it applied a drastic change, and >> it >> > is >> > > also why they skipped 3.4.x. >> > > - It seems like Druid requires some MaxDirectMemorySize configuration >> for >> > > some test suites. With some JVM parameter like >> > `-XX:MaxDirectMemorySize=4g` >> > > some test suites were passed, but not all. I am now trying the other >> > > options with enlarged swap space. >> > > >> > > Question: How much MaxDirectMemorySize configuration are you using? >> > > >> > > Best, >> > > Dongjin >> > > >> > > On Sat, Jul 21, 2018 at 3:01 AM Jihoon Son <ghoon...@gmail.com> >> wrote: >> > > >> > > > Hi Dongjin, >> > > > >> > > > that is weird. It looks like the vm crashed because of out of memory >> > > while >> > > > testing. >> > > > It might be a real issue or not. >> > > > Have you set any memory configuration for your maven? >> > > > >> > > > Jihoon >> > > > >> > > > On Thu, Jul 19, 2018 at 7:09 PM Dongjin Lee <dong...@apache.org> >> > wrote: >> > > > >> > > > > Hi Jihoon, >> > > > > >> > > > > I ran `mvn clean package` following development/build >> > > > > < >> > > > > >> > > > >> > > >> > >> https://github.com/apache/incubator-druid/blob/master/docs/content/development/build.md >> > > > > > >> > > > > . >> > > > > >> > > > > Dongjin >> > > > > >> > > > > On Fri, Jul 20, 2018 at 12:30 AM Jihoon Son <jihoon...@apache.org >> > >> > > > wrote: >> > > > > >> > > > > > Hi Dongjin, >> > > > > > >> > > > > > what maven command did you run? >> > > > > > >> > > > > > Jihoon >> > > > > > >> > > > > > On Wed, Jul 18, 2018 at 10:38 PM Dongjin Lee < >> dong...@apache.org> >> > > > wrote: >> > > > > > >> > > > > > > Hello. I am trying to build druid, but it fails. My >> environment >> > is >> > > > like >> > > > > > the >> > > > > > > following: >> > > > > > > >> > > > > > > - CPU: Intel(R) Core(TM) i7-7560U CPU @ 2.40GHz >> > > > > > > - RAM: 7704 MB >> > > > > > > - OS: ubuntu 18.04 >> > > > > > > - JDK: openjdk version "1.8.0_171" (default configuration, >> with >> > > > > > MaxHeapSize >> > > > > > > = 1928 MB) >> > > > > > > - Branch: master (commit: cd8ea3d) >> > > > > > > >> > > > > > > The error message I got is: >> > > > > > > >> > > > > > > [INFO] >> > > > > > > > >> > > > > > >> > > > >> > ------------------------------------------------------------------------ >> > > > > > > > [INFO] Reactor Summary: >> > > > > > > > [INFO] >> > > > > > > > [INFO] io.druid:druid ..................................... >> > > > SUCCESS [ >> > > > > > > > 50.258 s] >> > > > > > > > [INFO] java-util .......................................... >> > > SUCCESS >> > > > > > > [03:57 >> > > > > > > > min] >> > > > > > > > [INFO] druid-api .......................................... >> > > > SUCCESS [ >> > > > > > > > 22.694 s] >> > > > > > > > [INFO] druid-common ....................................... >> > > > SUCCESS [ >> > > > > > > > 14.083 s] >> > > > > > > > [INFO] druid-hll .......................................... >> > > > SUCCESS [ >> > > > > > > > 17.126 s] >> > > > > > > > [INFO] extendedset ........................................ >> > > > SUCCESS [ >> > > > > > > > 10.856 s] >> > > > > > > > >> > > > > > > > *[INFO] druid-processing ................................... >> > > > FAILURE >> > > > > > > > [04:36 min]*[INFO] druid-aws-common >> > > > > ................................... >> > > > > > > > SKIPPED >> > > > > > > > [INFO] druid-server ....................................... >> > > SKIPPED >> > > > > > > > [INFO] druid-examples ..................................... >> > > SKIPPED >> > > > > > > > ... >> > > > > > > > [INFO] >> > > > > > > > >> > > > > > >> > > > >> > ------------------------------------------------------------------------ >> > > > > > > > [INFO] BUILD FAILURE >> > > > > > > > [INFO] >> > > > > > > > >> > > > > > >> > > > >> > ------------------------------------------------------------------------ >> > > > > > > > [INFO] Total time: 10:29 min >> > > > > > > > [INFO] Finished at: 2018-07-19T13:23:31+09:00 >> > > > > > > > [INFO] Final Memory: 88M/777M >> > > > > > > > [INFO] >> > > > > > > > >> > > > > > >> > > > >> > ------------------------------------------------------------------------ >> > > > > > > > >> > > > > > > > *[ERROR] Failed to execute goal >> > > > > > > > org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test >> > > > > > (default-test) >> > > > > > > > on project druid-processing: Execution default-test of goal >> > > > > > > > org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test >> > > failed: >> > > > > The >> > > > > > > > forked VM terminated without properly saying goodbye. VM >> crash >> > or >> > > > > > > > System.exit called?*[ERROR] Command was /bin/sh -c cd >> > > > > > > > /home/djlee/workspace/java/druid/processing && >> > > > > > > > /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx3000m >> > > > > > > -Duser.language=en >> > > > > > > > -Duser.country=US -Dfile.encoding=UTF-8 -Duser.timezone=UTC >> > > > > > > > >> > > -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager >> > > > > > > > -Ddruid.indexing.doubleStorage=double -jar >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> /home/djlee/workspace/java/druid/processing/target/surefire/surefirebooter1075382243904099051.jar >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> /home/djlee/workspace/java/druid/processing/target/surefire/surefire5444459351134757209tmp >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> /home/djlee/workspace/java/druid/processing/target/surefire/surefire_5173894389718744688tmp >> > > > > > > >> > > > > > > >> > > > > > > It seems like it fails when it runs tests on >> `druid-processing` >> > > > module >> > > > > > but >> > > > > > > I can't certain. Is there anyone who can give me some hints? >> > Thanks >> > > > in >> > > > > > > advance. >> > > > > > > >> > > > > > > Best, >> > > > > > > Dongjin >> > > > > > > >> > > > > > > -- >> > > > > > > *Dongjin Lee* >> > > > > > > >> > > > > > > *A hitchhiker in the mathematical world.* >> > > > > > > >> > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr >> > > > > > > <http://github.com/dongjinleekr>linkedin: >> > > > > > kr.linkedin.com/in/dongjinleekr >> > > > > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare: >> > > > > > > www.slideshare.net/dongjinleekr >> > > > > > > <http://www.slideshare.net/dongjinleekr>* >> > > > > > > >> > > > > > >> > > > > > -- >> > > > > > *Dongjin Lee* >> > > > > > >> > > > > > *A hitchhiker in the mathematical world.* >> > > > > > >> > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr >> > > > > > <http://github.com/dongjinleekr>linkedin: >> > > > > kr.linkedin.com/in/dongjinleekr >> > > > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare: >> > > > > www.slideshare.net/dongjinleekr >> > > > > > <http://www.slideshare.net/dongjinleekr>* >> > > > > > >> > > > > >> > > > >> > > > -- >> > > > *Dongjin Lee* >> > > > >> > > > *A hitchhiker in the mathematical world.* >> > > > >> > > > *github: <http://goog_969573159/>github.com/dongjinleekr >> > > > <http://github.com/dongjinleekr>linkedin: >> > > kr.linkedin.com/in/dongjinleekr >> > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare: >> > > www.slideshare.net/dongjinleekr >> > > > <http://www.slideshare.net/dongjinleekr>* >> > > > >> > > >> > >> >> -- >> *Dongjin Lee* >> >> *A hitchhiker in the mathematical world.* >> >> *github: <http://goog_969573159/>github.com/dongjinleekr >> <http://github.com/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr >> <http://kr.linkedin.com/in/dongjinleekr>slideshare: >> www.slideshare.net/dongjinleekr >> <http://www.slideshare.net/dongjinleekr>* >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > For additional commands, e-mail: dev-h...@druid.apache.org
