Yes. I was thinking that perhaps a call or two to needlargemem should have called needhugemem instead.
However since we could not reproduce it thar makes it hard to be sure. -galt On Mar 15, 2011, at 6:16 PM, Hiram Clawson <[email protected]> wrote: > > Go into the source tree and edit the file: src/lib/memalloc.c > change the definition of the maxAlloc variable: > > static size_t maxAlloc = ... > > Currently it allows 16 Gb in a single allocation. Add a factor > of 4 to make it 64 Gb. > > --Hiram > > ----- Original Message ----- > From: "Shankar Ajay Subramanian" <[email protected]> > To: "Galt Barber" <[email protected]> > Cc: [email protected] > Sent: Tuesday, March 15, 2011 5:32:41 PM > Subject: Re: [Genome] bedGraphToBigWig memory issue > > Hi Galt, > > Thanks for your response. I updated the executable yesterday with what > appeared to be a new version (updated Mar 14 on the UCSC website) > but I > still end up getting a similar error: > > needLargeMem: trying to allocate 34345569624 bytes (limit: > 17179869184) > > I forgot to mention that the program writes a partial bigWig file > (roughly > 6GB) before aborting. > > Let me know if you need additional information for debugging/ > diagnosis. > > Shankar > > On Tue, Mar 15, 2011 at 8:23 PM, Galt Barber <[email protected]> > wrote: > >> >> Hi, Shankar >> >> I tried to reproduce this error, but was unable to. >> I used a 34GB bedgraph file and had no trouble. >> >> Since this problem may have been fixed somewhat recently, >> I suggest you try downloading a fresh version >> of bedGraphToBigWig and run it again. >> >> -Galt >> >> >> 3/14/2011 8:29 PM, Shankar Ajay Subramanian: >> >> Hello, >>> >>> I'm trying to create a per-base coverage track from whole-genome >>> sequencing >>> data and I opted to use the bigWig format to display this on the >>> UCSC >>> browser. I downloaded the 64-bit linux 'bedGraphToBigWig' binary >>> utility >>> to >>> do so (Feb-18 version). >>> >>> The original read data are in BAM format (200GB), which I then >>> converted >>> to >>> bedGraph (49GB). When I tried to convert intermediate bedGraph >>> file to >>> bigWig I end up getting the following error even though the >>> process runs >>> on >>> a machine with>100G RAM: >>> >>> needLargeMem: trying to allocate 34345569624 bytes (limit: 4294967296 >>> ) >>> >>> From talking to our sys admin it appears that the utility calls a >>> library >>>> >>> where a hard-coded memory limit is specified for a 64-bit machine >>> (memalloc.c). Is there any way to get around this? I considered >>> down-sampling but didn't know if that would help since the >>> bedGraph file >>> is >>> likely to contain nearly as many data-points with Column4 values >>> being >>> lower. >>> >>> Appreciate your help. >>> >>> Best, >>> Shankar >>> >>> Postdoctoral fellow >>> NIH/NHGRI >>> >> >> > _______________________________________________ > Genome maillist - [email protected] > https://lists.soe.ucsc.edu/mailman/listinfo/genome _______________________________________________ Genome maillist - [email protected] https://lists.soe.ucsc.edu/mailman/listinfo/genome
