Hi,
I have the following problem when trying to execute
cCreateSequenceDictionary.jar on an ubuntu linux server:
Exception in thread "main" java.lang.OutOfMemoryError: Requested array size
exceeds VM limit
The genome is rather big (around 12G). I have tried controlling memory usage
with:
java -jar -Xmx <value> …cmd
with <value>(s) 4G, 12G, 14G, 30G, 100G, 400G, 500G
The error stays the same (system call attached, as well as error message)
I have also tried setting MAX_RECORDS_IN_RAM to 5,000,000 or 20,000,000
Additional information:
######## Java Version ############
bastian@[hostname redacted]:~$ java -version
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)
######## Server system ############
Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.8.0-29-generic x86_64)
######## Genome ##########
12666745333 Jul 16 19:41 picea_abies.master-rna-scaff.nov2012.fa
######## Server hardware ###########
bastian@[hostname redacted]:~$ free
total used free shared buffers cached
Mem: 528344012 244740016 283603996 0 164596 224830244
-/+ buffers/cache: 19745176 508598836
Swap: 31583228 190760 31392468
######## The system has 64 CPUs: ##########
cat /proc/cpuinfo
.
.
.
processor : 63
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD Opteron(tm) Processor 6386 SE
stepping : 0
microcode : 0x6000822
cpu MHz : 2792.892
cache size : 2048 KB
physical id : 3
siblings : 16
core id : 7
cpu cores : 8
apicid : 143
initial apicid : 111
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb
rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf
pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c
lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core arat
cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid
decodeassists pausefilter pfthreshold bmi1
bogomips : 5586.23
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro
######## File system ############
df:
[IP address redacted]:/export/[hostname redacted] 61201197056 29745669120
31455527936 49% /mnt/[hostname redacted]
######## Command line call ##########
[Sat Jul 19 23:57:33 CEST 2014] picard.sam.CreateSequenceDictionary
REFERENCE=picea_abies.master-rna-scaff.nov2012.fa
OUTPUT=picea_abies.master-rna-scaff.nov2012.dict MAX_RECORDS_IN_RAM=20000000
TRUNCATE_NAMES_AT_WHITESPACE=true NUM_SEQUENCES=2147483647 VERBOSITY=INFO
QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 CREATE_INDEX=false
CREATE_MD5_FILE=false
[Sat Jul 19 23:57:33 CEST 2014] Executing as bastian@[hostname redacted] on
Linux 3.8.0-29-generic amd64; Java HotSpot(TM) 64-Bit Server VM 1.7.0_55-b13;
Picard version: 1.117(107391d3f3e72b31589868c250262ca79659f577_1405353489)
JdkDeflater
[Sun Jul 20 00:18:57 CEST 2014] picard.sam.CreateSequenceDictionary done.
Elapsed time: 21.40 minutes.
Runtime.totalMemory()=19666042880
To get help, see http://picard.sourceforge.net/index.shtml#GettingHelp
Exception in thread "main" java.lang.OutOfMemoryError: Requested array size
exceeds VM limit
at java.util.Arrays.copyOf(Arrays.java:2367)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:535)
at java.lang.StringBuffer.append(StringBuffer.java:322)
at java.io.StringWriter.write(StringWriter.java:94)
at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
at java.io.BufferedWriter.write(BufferedWriter.java:230)
at java.io.Writer.write(Writer.java:157)
at java.io.Writer.append(Writer.java:227)
at
htsjdk.samtools.SAMTextHeaderCodec.println(SAMTextHeaderCodec.java:392)
at
htsjdk.samtools.SAMTextHeaderCodec.writeSQLine(SAMTextHeaderCodec.java:447)
at
htsjdk.samtools.SAMTextHeaderCodec.encode(SAMTextHeaderCodec.java:371)
at
htsjdk.samtools.SAMTextHeaderCodec.encode(SAMTextHeaderCodec.java:356)
at
htsjdk.samtools.SAMFileWriterImpl.setHeader(SAMFileWriterImpl.java:135)
at
htsjdk.samtools.SAMFileWriterFactory.makeSAMWriter(SAMFileWriterFactory.java:212)
at
picard.sam.CreateSequenceDictionary.doWork(CreateSequenceDictionary.java:124)
at
picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:183)
at
picard.sam.CreateSequenceDictionary.main(CreateSequenceDictionary.java:97)
------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Samtools-help mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/samtools-help
