This is a shot in the dark, but I notice you are running with -Xmx100g.
You might try a less aggressive heap allocation.
The stochastic nature of the failures might be due to contention over
system resources.
It looks like some of your runs are, in fact, growing the java heap to 100g.
-Bob
On 8/21/14, 12:18 PM, Nils Homer wrote:
This is certainly a strange issue. Perhaps there is a problem using
the intel deflator with your JVM and environment. Could you retry by
adding to your java command "-Dtry_use_intel_deflater=false"? You
could also move the "libIntelDeflater.so" away from the path. You
should see a difference in the version line that the tool outputs to
stderr:
amd64; Java HotSpot(TM) 64-Bit Server VM 1.7.0_67-b01; Picard version:
1.118(2329276ea55d31ab6b19bab55b9ee7b51e4a446e_1406559781) IntelDeflater
The IntelDeflator should say JdkDeflator if you turned that feature
off properly.
Nils
On Wed, Aug 20, 2014 at 7:21 PM, Thomas Sibley <[email protected]
<mailto:[email protected]>> wrote:
On Aug 20, 2014, at 15:58 , Thomas Sibley <[email protected]
<mailto:[email protected]>> wrote:
> On Aug 20, 2014, at 14:25 , Thomas Sibley <[email protected]
<mailto:[email protected]>> wrote:
>> I'm running into problems using Picard's MarkDuplicates
utility. Sometimes I get a JVM crash, sometimes the process just
seems to hang. After the initial startup, strace shows nothing
but sched_yield() and futex() calls. While the input data which
hangs has ~900k reads, MarkDuplicates runs in about 1min on ~400k
reads from a different input file.
>>
>> Example informational output from both crashing and hanging
runs are attached, as well as the crash log of the JVM.
>>
>> $ java -version
>> java version "1.7.0_67"
>> Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
>> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
>
> Additional information: jconsole never connects to the java
process and jstack -F produces the attached thread trace.
The process I ran jstack on just completed without error, after
130 minutes. It took a few retry attempts to get a run all the
way through without a JVM crash. Attached is the informational
output from the MarkDuplicates command.
I'm still interested in tracking down the reason for the JVM
crashes, if anyone has pointers or ideas.
I'd also like to know more about what affects the runtime of
MarkDuplicates. It's a little wild to me that a ~2 fold increase
in read number alone might produce a runtime difference from 1
minute to 130 minutes. Are there other properties of the input
data that would affect this?
Thanks,
Thomas
------------------------------------------------------------------------------
Slashdot TV.
Video for Nerds. Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
Samtools-help mailing list
[email protected]
<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/samtools-help
------------------------------------------------------------------------------
Slashdot TV.
Video for Nerds. Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
Samtools-help mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/samtools-help
------------------------------------------------------------------------------
Slashdot TV.
Video for Nerds. Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
Samtools-help mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/samtools-help
