Hi Shankar, Another suggestion: instead of bedGraph, try the more concise fixedStep wiggle format, and wigToBigWig, as described here:
https://lists.soe.ucsc.edu/pipermail/genome/2010-December/024318.html Hope that helps, Angie ----- "Galt Barber" <[email protected]> wrote: > From: "Galt Barber" <[email protected]> > To: "Hiram Clawson" <[email protected]> > Cc: [email protected], "Shankar Ajay Subramanian" <[email protected]> > Sent: Tuesday, March 15, 2011 7:54:42 PM GMT -08:00 US/Canada Pacific > Subject: Re: [Genome] bedGraphToBigWig memory issue > > Yes. I was thinking that perhaps > a call or two to needlargemem > should have called needhugemem > instead. > > However since we could not reproduce it thar makes it hard to be > sure. > > -galt > > On Mar 15, 2011, at 6:16 PM, Hiram Clawson <[email protected]> > wrote: > > > > > Go into the source tree and edit the file: src/lib/memalloc.c > > change the definition of the maxAlloc variable: > > > > static size_t maxAlloc = ... > > > > Currently it allows 16 Gb in a single allocation. Add a factor > > of 4 to make it 64 Gb. > > > > --Hiram > > > > ----- Original Message ----- > > From: "Shankar Ajay Subramanian" <[email protected]> > > To: "Galt Barber" <[email protected]> > > Cc: [email protected] > > Sent: Tuesday, March 15, 2011 5:32:41 PM > > Subject: Re: [Genome] bedGraphToBigWig memory issue > > > > Hi Galt, > > > > Thanks for your response. I updated the executable yesterday with > what > > appeared to be a new version (updated Mar 14 on the UCSC website) > > but I > > still end up getting a similar error: > > > > needLargeMem: trying to allocate 34345569624 bytes (limit: > > 17179869184) > > > > I forgot to mention that the program writes a partial bigWig file > > (roughly > > 6GB) before aborting. > > > > Let me know if you need additional information for debugging/ > > diagnosis. > > > > Shankar > > > > On Tue, Mar 15, 2011 at 8:23 PM, Galt Barber <[email protected]> > > wrote: > > > >> > >> Hi, Shankar > >> > >> I tried to reproduce this error, but was unable to. > >> I used a 34GB bedgraph file and had no trouble. > >> > >> Since this problem may have been fixed somewhat recently, > >> I suggest you try downloading a fresh version > >> of bedGraphToBigWig and run it again. > >> > >> -Galt > >> > >> > >> 3/14/2011 8:29 PM, Shankar Ajay Subramanian: > >> > >> Hello, > >>> > >>> I'm trying to create a per-base coverage track from whole-genome > >>> sequencing > >>> data and I opted to use the bigWig format to display this on the > > >>> UCSC > >>> browser. I downloaded the 64-bit linux 'bedGraphToBigWig' binary > > >>> utility > >>> to > >>> do so (Feb-18 version). > >>> > >>> The original read data are in BAM format (200GB), which I then > >>> converted > >>> to > >>> bedGraph (49GB). When I tried to convert intermediate bedGraph > >>> file to > >>> bigWig I end up getting the following error even though the > >>> process runs > >>> on > >>> a machine with>100G RAM: > >>> > >>> needLargeMem: trying to allocate 34345569624 bytes (limit: > 4294967296 > >>> ) > >>> > >>> From talking to our sys admin it appears that the utility calls a > > >>> library > >>>> > >>> where a hard-coded memory limit is specified for a 64-bit machine > >>> (memalloc.c). Is there any way to get around this? I considered > >>> down-sampling but didn't know if that would help since the > >>> bedGraph file > >>> is > >>> likely to contain nearly as many data-points with Column4 values > > >>> being > >>> lower. > >>> > >>> Appreciate your help. > >>> > >>> Best, > >>> Shankar > >>> > >>> Postdoctoral fellow > >>> NIH/NHGRI > >>> > >> > >> > > _______________________________________________ > > Genome maillist - [email protected] > > https://lists.soe.ucsc.edu/mailman/listinfo/genome > _______________________________________________ > Genome maillist - [email protected] > https://lists.soe.ucsc.edu/mailman/listinfo/genome _______________________________________________ Genome maillist - [email protected] https://lists.soe.ucsc.edu/mailman/listinfo/genome
