Galt, Thanks for getting back to me.
Are there any plans to return to just requesting the small chunks for bigWig and bigBed? This would definitely save some work that's done on my server, and I imagine might speed up things on your end as my server would be free to serve up the next chunk instead reading and sending the rest of the file on every request. Thanks again, Regards, Tim ________________________________________ From: Galt Barber [[email protected]] Sent: Friday, May 07, 2010 2:56 PM To: Charnecki, Timothy A Cc: [email protected] Subject: Re: [Genome] Open-ended Range in GET header requests for bigWig files By the way, it is more accurate to say that we added this change to support BAM files (not for FTP). BigBed and BigWig files were designed from the beginning for random access in blocks, and so the byteRange start and end are known and only a minimal number of blocks are read. Also, the index itself is very compact. The BAM structure however is apparently such that you don't know ahead of time how much stuff you'll be reading. You read a little, get some more info, then read a little more. Luckily many of the items are grouped sequentially so that keeping the connection open and reading a little more next time is a good bet for BAM. -Galt Galt Barber wrote: > Hi, Tim! > > This was a change added to UDC cache layer > back in November. > > As you have noted, in the current > implementation the end of the byterange is > not being used. > > I believe that this had something to do with trying to > improve FTP performance, > trying to keep a connection open to read successive > sequential blocks in one go. > > You are right that it is actually asking the server for more data than > it will actually read, simply closing the connection when it has > received enough data. > But it doesn't appear to be causing any harm. > > So, this is working as intended. > I will double-check with the developer about this question. > Thanks for reporting it. > > -Galt > >> -------- Original Message -------- >> Subject: Open-ended Range in GET header requests for bigWig files >> Date: Tue, 4 May 2010 14:38:08 -0500 >> From: Charnecki, Timothy A <[email protected]> >> To: [email protected] <[email protected]> >> >> Hello, >> >> I noticed something that appears to have changed When viewing bigWig >> files in the browser I am seeing GET requests to my server that have >> headers containing byte ranges that don't have the end byte defined. I >> understand that this is allowed by HTTP, but understand that the server >> will serve the start byte through the end of the file. I can see >> requests with header "Range => bytes=0-". This causes the server to >> attempt to serve the entire file. I know I used to see requests with >> much smaller request ranges such as "Range => bytes=0-8191". My data is >> displaying without any problems in the browser, I'm just wondering if >> these partial range requests are intentional on your end. >> >> Thanks, >> Tim Charnecki >> > > _______________________________________________ > Genome maillist - [email protected] > https://lists.soe.ucsc.edu/mailman/listinfo/genome _______________________________________________ Genome maillist - [email protected] https://lists.soe.ucsc.edu/mailman/listinfo/genome
