> On Feb. 26, 2015, 3:07 a.m., Gwen Shapira wrote:
> > Ah, now I get what "dedupe buffer" refers to :)
> >
> > I never managed to reproduce the oom, but this looks like the right fix.
>
> Ewen Cheslack-Postava wrote:
> This happened consistently for me, which provided good motivation to
> figure out how to fix it :) It looks like the tests use the default JVM
> settings (build.gradle has overrides for compile, but not for test), so the
> 500 MB allocation was always failing since it obviously exceeds the default
> 128 MB heap. Now I'm wondering if other people are running with different
> settings. Maybe if you have a gradle daemon running that was started with a
> bigger max heap size?
>
> Gwen Shapira wrote:
> My gradle.properties has:
> org.gradle.jvmargs=-XX:MaxPermSize=512m -Xmx1024m
>
> I take it that yours doesn't?
>
> The builds in Apache's jenkins didn't seem to fail on that test either...
>
> Ewen Cheslack-Postava wrote:
> The one checked into the repo does, and I double checked that I'm not
> overriding that setting in my ~/.gradle/gradle.properties. But as far as I
> can tell that only applies to the gradle daemon process. It forks off another
> process to run the tests, which doesn't inherit those settings. If I add this
> to the build.gradle file next to the overrides for ScalaCompile tasks:
>
> tasks.withType(JavaForkOptions) {
> jvmArgs = ['-XX:MaxPermSize=512m', '-Xmx1024m']
> }
>
> then the forked process gets those settings and the test passes without
> this patch.
>
> Gwen Shapira wrote:
> huh. so I have no idea why the tests pass on my setup...
>
> Ewen Cheslack-Postava wrote:
> I checked my setup by grabbing info from visualvm while the tests were
> running. Maybe double check the processes running for you when you run the
> tests, and what settings those processes have had passed to them?
>
> I don't think it's critical since the immediate issue is resolved by the
> patch, but it would be nice to know why our test environments are
> inconsistent.
>
> Gwen Shapira wrote:
> First, I didn't know about visualvm. Its very sweet :)
>
> Looking at it, the GradleDaemon had the flags I specified above and it
> had 500M RAM allocated throughout the test. Used mem for this process grew
> gradually to 250M during the test execution.
> GradleWorkerMain actually ran the tests. It had no memory-related JVM
> flags. It started with 250M allocated and jumped to 750M when
> testDeleteTopicWithCleaner started (with 522M used).
>
> With that info, I suspect that the default JVM settings on my Mac OS X
> Yosemite, JVM 1.7.0.67 were different.
>
> I looked at: java -XX:+PrintFlagsFinal -version
> and found:
> uintx MaxHeapSize := 4294967296
> {product}
>
> Thats pretty large :)
Ok, so maybe this was a JDK6 -> JDK7 change. I'm on JDK6 since we're still
supporting it. Same output for me:
uintx MaxHeapSize = 132120576 {product}
So mystery solved. These things would probably get caught more easily if we had
decent test coverage across multiple JDK versions...
- Ewen
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31447/#review74200
-----------------------------------------------------------
On Feb. 25, 2015, 11:27 p.m., Ewen Cheslack-Postava wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31447/
> -----------------------------------------------------------
>
> (Updated Feb. 25, 2015, 11:27 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1881
> https://issues.apache.org/jira/browse/KAFKA-1881
>
>
> Repository: kafka
>
>
> Description
> -------
>
> KAFKA-1881: Fix out of memory error in testDeleteTopicWithCleaner by making
> the dedupe buffer much smaller.
>
>
> Diffs
> -----
>
> core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala
> 0cbd72684a0cc6fe9f1fba2dc834de675b2eb994
>
> Diff: https://reviews.apache.org/r/31447/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ewen Cheslack-Postava
>
>
