Hi John, I'm so glad you got it to work, and thank you for sharing your changes. I will work those into hgc.c.
Angie ----- "John Hayward" <[email protected]> wrote: > From: "John Hayward" <[email protected]> > To: "John Hayward" <[email protected]>, "Angie Hinrichs" > <[email protected]>, "Kyle Tretina" <[email protected]> > Cc: "UCSC" <[email protected]> > Sent: Tuesday, June 8, 2010 8:51:03 AM GMT -08:00 US/Canada Pacific > Subject: RE: [Genome] Fwd: Fwd: Problem with Mirror > > > > Hi Angie, > We were able to get our queries to work after making changes to > getSnpSeqFileOffset > function in hgc.c. It appears that the original code does not work in a 32 > bit environment. Here are the changes we needed to do to have it work in our > setting: > 1) Line 23137 change declaration of offset > === was > long offset = 0; > === to > long long offset = 0; > > 2) Line 23135 change call of conversion of returned sql offset: > ==== was > offset = sqlUnsignedLong(row[0]) > === to > offset = sqlLongLong(row[0]) > > I don't know if there are other 32/64 issues but it may be helpful to others > working in a 32 bit environment to have these adjustments made in the code. > > Thanks for your help in locating this problem. > > johnh... > > > From: John Hayward > Sent: Tuesday, June 08, 2010 9:51 AM > To: Angie Hinrichs; Kyle Tretina > Cc: UCSC > Subject: RE: [Genome] Fwd: Fwd: Problem with Mirror > > > > The problem seems to be a 64/32 bit issue. > I rebuilt the binaries and observed the flags for building (here is a > snippet) > .... > gcc -O -g -Wall -Werror -Wformat -Wimplicit -Wreturn-type -Wuninitialized > -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DMACHTYPE_i486 > -Wall -Werror -Wformat -Wimplicit -Wreturn-type -Wuninitialized -I../inc > -I../../inc -I../../../inc -I../../../../inc -I../../../../../inc -o > hgLoadWiggle.o -c hgLoadWiggle.c > .... > > The error on the web page we see is: > ===== > Expected FASTA header, got this line: > ATAATACATT TTAATTTTCT AGAAAGTAAT GGTATGAAGT TCCAACTTTT TTTAAAAAAG ACTTTTTTTC > at offset 344476137 in file /gbdb/hg18/snp/snp130.fa > ====== > The offset you are seeing from the database is 17524345321 > > Here is a bc script: > ===== > bc 1.06.94 > Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, > Inc. > This is free software with ABSOLUTELY NO WARRANTY. > For details type `warranty'. > obase=16 > 15 > F > 344476137 // this is the offset reported in the error > 148849E9 > 17524345321 // this is the offset according to the database > 4148849E9 // if you truncate this value to 32 bits (last 8 hex digits) then > you get the value > // above > ====== > This shows that the offset in the database is more than 32 bits and when > truncated to 32 bits is the same value reported in the error message. > > The operating system is: > Linux ubuntu 2.6.31-302-ec2 #7-Ubuntu SMP Tue Oct 13 19:06:04 UTC 2009 i686 > GNU/Linux > Ubuntu 10.04 (lucid) > > Now we need to find out where the offset is being truncated - it seems that > the flags for file offsets are in place but I wonder if there are some other > 32 bit issues in the source. > > I recall when originally building there were some warnings which required > some changes to compile. I think the next step is to repeat those steps and > make sure there are no 32/64 bit issues. > > johnh... > > > From: Angie Hinrichs [[email protected]] > Sent: Monday, June 07, 2010 5:26 PM > To: Kyle Tretina > Cc: UCSC; John Hayward > Subject: Re: [Genome] Fwd: Fwd: Problem with Mirror > > > Hi Kyle, > > I think I know what it is now: snp130.fa is an extremely large file, and > either your operating system can't handle its size or perhaps the flags > "-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE" were omitted from the build? > (our makefiles all include kent/src/inc/common.mk, which defines the variable > HG_DEFS to include those, and our compile rule uses $HG_DEFS) > > snp130.fa is a whopping 18,668,682,162 bytes. (18GB or 17GiB depending on how > you count) If your filesystem can handle only 32-bit file offsets (or even > 36), that is just too big. > > rs72477211 's offset ("select file_offset from snp130Seq where acc = > 'rs72477211'") is 17524345321, which is (16 * 1024 * 1024 * 1024) + > 344476137, and 344476137 is the file offset that you see in the error message > because the top bit of the offset is lost. > > What operating system and compiler are you using? > > Angie > > > ----- "Kyle Tretina" <[email protected]> wrote: > > From: "Kyle Tretina" <[email protected]> > > To: "Angie Hinrichs" <[email protected]>, "UCSC" <[email protected]>, > > "John Hayward" <[email protected]> > > Sent: Sunday, June 6, 2010 10:59:57 AM GMT -08:00 US/Canada Pacific > > Subject: Re: [Genome] Fwd: Fwd: Problem with Mirror > > > > Hello, > > > > Note: this was in database hg19 using the command: select settings from trackDb where tableName = "snp130"; ======================================================================= > > +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | settings | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | chimpDb panTro2 chimpOrangMacOrthoTable snp130OrthoPt2Pa2Rm2 codingAnnoLabel_snp130CodingDbSnp dbSNP codingAnnotations snp130CodingDbSnp, defaultGeneTracks knownGene group varRep hapmapPhase III longLabel Simple Nucleotide Polymorphisms (dbSNP build 130) macaqueDb rheMac2 maxWindowToDraw 10000000 orangDb ponAbe2 priority 100.094 shortLabel SNPs (130) snpSeqFile /gbdb/hg18/snp/snp130.fa track snp130 type bed 6 + url http://www.ncbi.nlm.nih.gov/SNP/snp_ref.cgi?type=rs&rs=$$ urlLabel dbSNP: visibility dense | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.07 sec) > > > > ======================================================================= (This is the error message for rs72477211) > > Error(s): • Expected FASTA header, got this line: > > ATAATACATT TTAATTTTCT AGAAAGTAAT GGTATGAAGT TCCAACTTTT TTTAAAAAAG ACTTTTTTTC > > at offset 344476137 in file /gbdb/hg18/snp/snp130.fa ======================================================================= > > So rs242 does work. When I enter it into my mirror, I am getting the same output as your website. I was shocked when I saw it, because this was the first one I had entered that did not return an error message. > > > > Kyle Tretina Wheaton College > > > > > > _______________________________________________ Genome maillist - [email protected] https://lists.soe.ucsc.edu/mailman/listinfo/genome
