Go into the source tree and edit the file: src/lib/memalloc.c change the definition of the maxAlloc variable:
static size_t maxAlloc = ... Currently it allows 16 Gb in a single allocation. Add a factor of 4 to make it 64 Gb. --Hiram ----- Original Message ----- From: "Shankar Ajay Subramanian" <[email protected]> To: "Galt Barber" <[email protected]> Cc: [email protected] Sent: Tuesday, March 15, 2011 5:32:41 PM Subject: Re: [Genome] bedGraphToBigWig memory issue Hi Galt, Thanks for your response. I updated the executable yesterday with what appeared to be a new version (updated Mar 14 on the UCSC website) but I still end up getting a similar error: needLargeMem: trying to allocate 34345569624 bytes (limit: 17179869184) I forgot to mention that the program writes a partial bigWig file (roughly 6GB) before aborting. Let me know if you need additional information for debugging/diagnosis. Shankar On Tue, Mar 15, 2011 at 8:23 PM, Galt Barber <[email protected]> wrote: > > Hi, Shankar > > I tried to reproduce this error, but was unable to. > I used a 34GB bedgraph file and had no trouble. > > Since this problem may have been fixed somewhat recently, > I suggest you try downloading a fresh version > of bedGraphToBigWig and run it again. > > -Galt > > > 3/14/2011 8:29 PM, Shankar Ajay Subramanian: > > Hello, >> >> I'm trying to create a per-base coverage track from whole-genome >> sequencing >> data and I opted to use the bigWig format to display this on the UCSC >> browser. I downloaded the 64-bit linux 'bedGraphToBigWig' binary utility >> to >> do so (Feb-18 version). >> >> The original read data are in BAM format (200GB), which I then converted >> to >> bedGraph (49GB). When I tried to convert intermediate bedGraph file to >> bigWig I end up getting the following error even though the process runs >> on >> a machine with>100G RAM: >> >> needLargeMem: trying to allocate 34345569624 bytes (limit: 4294967296) >> >> From talking to our sys admin it appears that the utility calls a library >>> >> where a hard-coded memory limit is specified for a 64-bit machine >> (memalloc.c). Is there any way to get around this? I considered >> down-sampling but didn't know if that would help since the bedGraph file >> is >> likely to contain nearly as many data-points with Column4 values being >> lower. >> >> Appreciate your help. >> >> Best, >> Shankar >> >> Postdoctoral fellow >> NIH/NHGRI >> > > _______________________________________________ Genome maillist - [email protected] https://lists.soe.ucsc.edu/mailman/listinfo/genome _______________________________________________ Genome maillist - [email protected] https://lists.soe.ucsc.edu/mailman/listinfo/genome
