Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
Awesome, thanks!
On Tue, May 31, 2016 at 4:11 PM, Michael Lawrence
wrote:
> Sure, done.
>
> On Tue, May 31, 2016 at 11:18 AM, Leonardo Collado Torres
> wrote:
>> Hi Michael,
>>
>> Thanks!
>>
>> Actually, it looks like there are a few more quick changes I need you
>> to do. Simply at
>> https://github.com/Bioconductor-mirror/rtracklayer/blob/917973eb7e9f16bbcd6f6e4b9452f9e40d9a1e94/R/bigWig.R
>> replace path.expand() with expandPath(). I'm not sure this applies to
>> all current path.expand() calls, but at least it does for
>> https://github.com/Bioconductor-mirror/rtracklayer/blob/917973eb7e9f16bbcd6f6e4b9452f9e40d9a1e94/R/bigWig.R#L20
>>
>> Best,
>> Leo
>>
>>
>>
>>
>>> library(recount); system.time( regions <- expressed_regions('SRP009615',
>>> 'chrY', cutoff = 5L) )
>> 2016-05-31 14:11:52 loadCoverage: loading BigWig file
>> http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw
>> Error in seqinfo(con) : UCSC library operation failed
>> In addition: Warning message:
>> In seqinfo(con) :
>> Couldn't open http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw
>> Timing stopped at: 0.068 0.009 0.817
>>> traceback()
>> 14: .Call(BWGFile_seqlengths, path.expand(path(x)))
>> 13: seqinfo(con)
>> 12: seqinfo(con)
>> 11: .local(con, format, text, ...)
>> 10: import(file, selection = range, as = "RleList")
>> 9: import(file, selection = range, as = "RleList")
>> 8: FUN(X[[i]], ...)
>> 7: lapply(as.list(X), FUN = FUN, ...)
>> 6: lapply(as.list(X), FUN = FUN, ...)
>> 5: lapply(bList, .loadCoverageBigWig, range = which, chr = chr,
>>verbose = verbose)
>> 4: lapply(bList, .loadCoverageBigWig, range = which, chr = chr,
>>verbose = verbose)
>> 3: loadCoverage(files = meanFile, chr = chr, chrlen = chrlen)
>> 2: expressed_regions("SRP009615", "chrY", cutoff = 5L)
>> 1: system.time(regions <- expressed_regions("SRP009615", "chrY",
>>cutoff = 5L))
>>> options(width = 120); devtools::session_info()
>> Session info
>> ---
>> setting value
>> version R version 3.3.0 RC (2016-05-01 r70572)
>> system x86_64, darwin13.4.0
>> ui AQUA
>> language (EN)
>> collate en_US.UTF-8
>> tz America/New_York
>> date 2016-05-31
>>
>> Packages
>> ---
>> package * version date source
>> acepack1.3-3.3 2014-11-24 CRAN (R 3.3.0)
>> AnnotationDbi 1.35.3 2016-05-27 Bioconductor
>> Biobase2.33.0 2016-05-05 Bioconductor
>> BiocGenerics * 0.19.0 2016-05-05 Bioconductor
>> BiocParallel 1.7.22016-05-20 Bioconductor
>> biomaRt2.29.2 2016-05-30 Bioconductor
>> Biostrings 2.41.1 2016-05-27 Bioconductor
>> bitops 1.0-62013-08-17 CRAN (R 3.3.0)
>> BSgenome 1.41.0 2016-05-05 Bioconductor
>> bumphunter 1.13.0 2016-05-05 Bioconductor
>> chron 2.3-47 2015-06-24 CRAN (R 3.3.0)
>> cluster2.0.42016-04-18 CRAN (R 3.3.0)
>> codetools 0.2-14 2015-07-15 CRAN (R 3.3.0)
>> colorspace 1.2-62015-03-11 CRAN (R 3.3.0)
>> data.table 1.9.62015-09-19 CRAN (R 3.3.0)
>> DBI0.4-12016-05-08 CRAN (R 3.3.0)
>> derfinder* 1.7.52016-05-20 Bioconductor
>> derfinderHelper1.7.32016-05-20 Bioconductor
>> devtools 1.11.1 2016-04-21 CRAN (R 3.3.0)
>> digest 0.6.92016-01-08 CRAN (R 3.3.0)
>> doRNG 1.6 2014-03-07 CRAN (R 3.3.0)
>> foreach1.4.32015-10-13 CRAN (R 3.3.0)
>> foreign0.8-66 2015-08-19 CRAN (R 3.3.0)
>> Formula1.2-12015-04-07 CRAN (R 3.3.0)
>> GenomeInfoDb * 1.9.12016-05-13 Bioconductor
>> GenomicAlignments 1.9.02016-05-05 Bioconductor
>> GenomicFeatures1.25.12 2016-05-21 Bioconductor
>> GenomicFiles 1.9.72016-05-27 Bioconductor
>> GenomicRanges* 1.25.0 2016-05-05 Bioconductor
>> ggplot22.1.02016-03-01 CRAN (R 3.3.0)
>> gridExtra 2.2.12016-02-29 CRAN (R 3.3.0)
>> gtable 0.2.02016-02-26 CRAN (R 3.3.0)
>> Hmisc 3.17-4 2016-05-02 CRAN (R 3.3.0)
>> IRanges * 2.7.12016-05-27 Bioconductor
>> iterators 1.0.82015-10-13 CRAN (R 3.3.0)
>> lattice0.20-33 2015-07-14 CRAN (R 3.3.0)
>> latticeExtra 0.6-28 2016-02-09 CRAN (R 3.3.0)
>> locfit 1.5-9.1 2013-04-20 CRAN (R 3.3.0)
>> magrittr 1.5 2014-11-22 CRAN (R 3.3.0)
>> Matrix 1.2-62016-05-02 CRAN (R 3.3.0)
>> matrixStats0.50.2 2016-04-24 CRAN (R 3.3.0
Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
Sure, done.
On Tue, May 31, 2016 at 11:18 AM, Leonardo Collado Torres
wrote:
> Hi Michael,
>
> Thanks!
>
> Actually, it looks like there are a few more quick changes I need you
> to do. Simply at
> https://github.com/Bioconductor-mirror/rtracklayer/blob/917973eb7e9f16bbcd6f6e4b9452f9e40d9a1e94/R/bigWig.R
> replace path.expand() with expandPath(). I'm not sure this applies to
> all current path.expand() calls, but at least it does for
> https://github.com/Bioconductor-mirror/rtracklayer/blob/917973eb7e9f16bbcd6f6e4b9452f9e40d9a1e94/R/bigWig.R#L20
>
> Best,
> Leo
>
>
>
>
>> library(recount); system.time( regions <- expressed_regions('SRP009615',
>> 'chrY', cutoff = 5L) )
> 2016-05-31 14:11:52 loadCoverage: loading BigWig file
> http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw
> Error in seqinfo(con) : UCSC library operation failed
> In addition: Warning message:
> In seqinfo(con) :
> Couldn't open http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw
> Timing stopped at: 0.068 0.009 0.817
>> traceback()
> 14: .Call(BWGFile_seqlengths, path.expand(path(x)))
> 13: seqinfo(con)
> 12: seqinfo(con)
> 11: .local(con, format, text, ...)
> 10: import(file, selection = range, as = "RleList")
> 9: import(file, selection = range, as = "RleList")
> 8: FUN(X[[i]], ...)
> 7: lapply(as.list(X), FUN = FUN, ...)
> 6: lapply(as.list(X), FUN = FUN, ...)
> 5: lapply(bList, .loadCoverageBigWig, range = which, chr = chr,
>verbose = verbose)
> 4: lapply(bList, .loadCoverageBigWig, range = which, chr = chr,
>verbose = verbose)
> 3: loadCoverage(files = meanFile, chr = chr, chrlen = chrlen)
> 2: expressed_regions("SRP009615", "chrY", cutoff = 5L)
> 1: system.time(regions <- expressed_regions("SRP009615", "chrY",
>cutoff = 5L))
>> options(width = 120); devtools::session_info()
> Session info
> ---
> setting value
> version R version 3.3.0 RC (2016-05-01 r70572)
> system x86_64, darwin13.4.0
> ui AQUA
> language (EN)
> collate en_US.UTF-8
> tz America/New_York
> date 2016-05-31
>
> Packages
> ---
> package * version date source
> acepack1.3-3.3 2014-11-24 CRAN (R 3.3.0)
> AnnotationDbi 1.35.3 2016-05-27 Bioconductor
> Biobase2.33.0 2016-05-05 Bioconductor
> BiocGenerics * 0.19.0 2016-05-05 Bioconductor
> BiocParallel 1.7.22016-05-20 Bioconductor
> biomaRt2.29.2 2016-05-30 Bioconductor
> Biostrings 2.41.1 2016-05-27 Bioconductor
> bitops 1.0-62013-08-17 CRAN (R 3.3.0)
> BSgenome 1.41.0 2016-05-05 Bioconductor
> bumphunter 1.13.0 2016-05-05 Bioconductor
> chron 2.3-47 2015-06-24 CRAN (R 3.3.0)
> cluster2.0.42016-04-18 CRAN (R 3.3.0)
> codetools 0.2-14 2015-07-15 CRAN (R 3.3.0)
> colorspace 1.2-62015-03-11 CRAN (R 3.3.0)
> data.table 1.9.62015-09-19 CRAN (R 3.3.0)
> DBI0.4-12016-05-08 CRAN (R 3.3.0)
> derfinder* 1.7.52016-05-20 Bioconductor
> derfinderHelper1.7.32016-05-20 Bioconductor
> devtools 1.11.1 2016-04-21 CRAN (R 3.3.0)
> digest 0.6.92016-01-08 CRAN (R 3.3.0)
> doRNG 1.6 2014-03-07 CRAN (R 3.3.0)
> foreach1.4.32015-10-13 CRAN (R 3.3.0)
> foreign0.8-66 2015-08-19 CRAN (R 3.3.0)
> Formula1.2-12015-04-07 CRAN (R 3.3.0)
> GenomeInfoDb * 1.9.12016-05-13 Bioconductor
> GenomicAlignments 1.9.02016-05-05 Bioconductor
> GenomicFeatures1.25.12 2016-05-21 Bioconductor
> GenomicFiles 1.9.72016-05-27 Bioconductor
> GenomicRanges* 1.25.0 2016-05-05 Bioconductor
> ggplot22.1.02016-03-01 CRAN (R 3.3.0)
> gridExtra 2.2.12016-02-29 CRAN (R 3.3.0)
> gtable 0.2.02016-02-26 CRAN (R 3.3.0)
> Hmisc 3.17-4 2016-05-02 CRAN (R 3.3.0)
> IRanges * 2.7.12016-05-27 Bioconductor
> iterators 1.0.82015-10-13 CRAN (R 3.3.0)
> lattice0.20-33 2015-07-14 CRAN (R 3.3.0)
> latticeExtra 0.6-28 2016-02-09 CRAN (R 3.3.0)
> locfit 1.5-9.1 2013-04-20 CRAN (R 3.3.0)
> magrittr 1.5 2014-11-22 CRAN (R 3.3.0)
> Matrix 1.2-62016-05-02 CRAN (R 3.3.0)
> matrixStats0.50.2 2016-04-24 CRAN (R 3.3.0)
> memoise1.0.02016-01-29 CRAN (R 3.3.0)
> munsell0.4.32016-02-13 CRAN (R 3.3.0)
> nnet 7.3-12 2016-02-02 CRAN (R 3.3.0
Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
Hi Michael,
Thanks!
Actually, it looks like there are a few more quick changes I need you
to do. Simply at
https://github.com/Bioconductor-mirror/rtracklayer/blob/917973eb7e9f16bbcd6f6e4b9452f9e40d9a1e94/R/bigWig.R
replace path.expand() with expandPath(). I'm not sure this applies to
all current path.expand() calls, but at least it does for
https://github.com/Bioconductor-mirror/rtracklayer/blob/917973eb7e9f16bbcd6f6e4b9452f9e40d9a1e94/R/bigWig.R#L20
Best,
Leo
> library(recount); system.time( regions <- expressed_regions('SRP009615',
> 'chrY', cutoff = 5L) )
2016-05-31 14:11:52 loadCoverage: loading BigWig file
http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw
Error in seqinfo(con) : UCSC library operation failed
In addition: Warning message:
In seqinfo(con) :
Couldn't open http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw
Timing stopped at: 0.068 0.009 0.817
> traceback()
14: .Call(BWGFile_seqlengths, path.expand(path(x)))
13: seqinfo(con)
12: seqinfo(con)
11: .local(con, format, text, ...)
10: import(file, selection = range, as = "RleList")
9: import(file, selection = range, as = "RleList")
8: FUN(X[[i]], ...)
7: lapply(as.list(X), FUN = FUN, ...)
6: lapply(as.list(X), FUN = FUN, ...)
5: lapply(bList, .loadCoverageBigWig, range = which, chr = chr,
verbose = verbose)
4: lapply(bList, .loadCoverageBigWig, range = which, chr = chr,
verbose = verbose)
3: loadCoverage(files = meanFile, chr = chr, chrlen = chrlen)
2: expressed_regions("SRP009615", "chrY", cutoff = 5L)
1: system.time(regions <- expressed_regions("SRP009615", "chrY",
cutoff = 5L))
> options(width = 120); devtools::session_info()
Session info
---
setting value
version R version 3.3.0 RC (2016-05-01 r70572)
system x86_64, darwin13.4.0
ui AQUA
language (EN)
collate en_US.UTF-8
tz America/New_York
date 2016-05-31
Packages
---
package * version date source
acepack1.3-3.3 2014-11-24 CRAN (R 3.3.0)
AnnotationDbi 1.35.3 2016-05-27 Bioconductor
Biobase2.33.0 2016-05-05 Bioconductor
BiocGenerics * 0.19.0 2016-05-05 Bioconductor
BiocParallel 1.7.22016-05-20 Bioconductor
biomaRt2.29.2 2016-05-30 Bioconductor
Biostrings 2.41.1 2016-05-27 Bioconductor
bitops 1.0-62013-08-17 CRAN (R 3.3.0)
BSgenome 1.41.0 2016-05-05 Bioconductor
bumphunter 1.13.0 2016-05-05 Bioconductor
chron 2.3-47 2015-06-24 CRAN (R 3.3.0)
cluster2.0.42016-04-18 CRAN (R 3.3.0)
codetools 0.2-14 2015-07-15 CRAN (R 3.3.0)
colorspace 1.2-62015-03-11 CRAN (R 3.3.0)
data.table 1.9.62015-09-19 CRAN (R 3.3.0)
DBI0.4-12016-05-08 CRAN (R 3.3.0)
derfinder* 1.7.52016-05-20 Bioconductor
derfinderHelper1.7.32016-05-20 Bioconductor
devtools 1.11.1 2016-04-21 CRAN (R 3.3.0)
digest 0.6.92016-01-08 CRAN (R 3.3.0)
doRNG 1.6 2014-03-07 CRAN (R 3.3.0)
foreach1.4.32015-10-13 CRAN (R 3.3.0)
foreign0.8-66 2015-08-19 CRAN (R 3.3.0)
Formula1.2-12015-04-07 CRAN (R 3.3.0)
GenomeInfoDb * 1.9.12016-05-13 Bioconductor
GenomicAlignments 1.9.02016-05-05 Bioconductor
GenomicFeatures1.25.12 2016-05-21 Bioconductor
GenomicFiles 1.9.72016-05-27 Bioconductor
GenomicRanges* 1.25.0 2016-05-05 Bioconductor
ggplot22.1.02016-03-01 CRAN (R 3.3.0)
gridExtra 2.2.12016-02-29 CRAN (R 3.3.0)
gtable 0.2.02016-02-26 CRAN (R 3.3.0)
Hmisc 3.17-4 2016-05-02 CRAN (R 3.3.0)
IRanges * 2.7.12016-05-27 Bioconductor
iterators 1.0.82015-10-13 CRAN (R 3.3.0)
lattice0.20-33 2015-07-14 CRAN (R 3.3.0)
latticeExtra 0.6-28 2016-02-09 CRAN (R 3.3.0)
locfit 1.5-9.1 2013-04-20 CRAN (R 3.3.0)
magrittr 1.5 2014-11-22 CRAN (R 3.3.0)
Matrix 1.2-62016-05-02 CRAN (R 3.3.0)
matrixStats0.50.2 2016-04-24 CRAN (R 3.3.0)
memoise1.0.02016-01-29 CRAN (R 3.3.0)
munsell0.4.32016-02-13 CRAN (R 3.3.0)
nnet 7.3-12 2016-02-02 CRAN (R 3.3.0)
pkgmaker 0.22 2014-05-14 CRAN (R 3.3.0)
plyr 1.8.32015-06-12 CRAN (R 3.3.0)
qvalue 2.5.22016-05-20 Bioconductor
RColorBrewer 1.1-22014-12-07 CRAN (R 3.3.0)
Rcpp 0.12.5 2016
Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
Thanks for pointing out that buglet. Fixed.
On Tue, May 31, 2016 at 10:55 AM, Leonardo Collado Torres
wrote:
> Hi Michael,
>
> We tried getting things to work with Amazon Cloud Drive (see Abhi's
> efforts at https://github.com/nellore/duffel/commits/master). But we
> now have the data hosted elsewhere where the links work properly.
>
> I just noted a small mistake on rtracklayer:::expandPath(). See:
>
>> startsWith('http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw',
>> 'http||ftp')
> [1] FALSE
>> startsWith('http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw',
>> 'http')
> [1] TRUE
>
>
> The fix is simple. At
> https://github.com/Bioconductor-mirror/rtracklayer/blob/c4b842bc4daa4b9db26cb86f3284cf8cf5c32ebd/R/web.R#L62-L66,
> change it to:
>
> expandPath <- function(x) {
> if (startsWith(x, "http") | startsWith(x, "ftp"))
> expandURL(x)
> else path.expand(x)
> }
>
> Best,
> Leo
>
> On Thu, May 5, 2016 at 8:10 PM, Michael Lawrence
> wrote:
>> I checked in something that tries to find openssl automatically on the Mac.
>>
>> It looks like AWS is for some reason returning 404 for the HEAD command that
>> the UCSC library uses the get info about the file like the content size.
>> Same thing happens when I play around in Firefox's developer tools. The
>> error response header claims a JSON content type, but no JSON is actually
>> sent, so there is no further description of the error. I think this is a bug
>> in Amazon.
>>
>> Seems like for now you'll need to download the file first.
>>
>> Michael
>>
>> On Thu, May 5, 2016 at 2:46 PM, Leonardo Collado Torres
>> wrote:
>>>
>>> Hi Michael,
>>>
>>> I forgot about pkg-util (just did a fresh BioC 3.3 install). I assumed
>>> the OS X binary would work out of the box.
>>>
>>> Anyhow, I installed rtracklayer (release) manually and got another
>>> error (slightly different message now).
>>>
>>>
>>>
>>>
>>> $ svn co
>>> https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_3/madman/Rpacks/rtracklayer
>>> $ R CMD INSTALL rtracklayer
>>> Loading required package: colorout
>>> * installing to library
>>> ‘/Library/Frameworks/R.framework/Versions/3.3release/Resources/library’
>>> * installing *source* package ‘rtracklayer’ ...
>>> checking for pkg-config... /usr/local/bin/pkg-config
>>> checking pkg-config is at least version 0.9.0... yes
>>> checking for OPENSSL... yes
>>> ## more output
>>>
>>> $ R
>>> > library('rtracklayer')
>>> > unshorten_url <- function(uri) {
>>> + require('RCurl')
>>> + opts <- list(
>>> + followlocation = TRUE, # resolve redirects
>>> + ssl.verifyhost = FALSE, # suppress certain SSL errors
>>> + ssl.verifypeer = FALSE,
>>> + nobody = TRUE, # perform HEAD request
>>> + verbose = FALSE
>>> + )
>>> + curlhandle <- getCurlHandle(.opts = opts)
>>> + getURL(uri, curl = curlhandle)
>>> + info <- getCurlInfo(curlhandle)
>>> + rm(curlhandle) # release the curlhandle!
>>> + info$effective.url
>>> + }
>>> > url <-
>>> > unshorten_url('http://duffel.rail.bio/recount/DRP000366/bw/DRR000897.bw')
>>> Loading required package: RCurl
>>> Loading required package: bitops
>>> > url
>>> [1]
>>> "https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5";
>>> > x <- import.bw(url, as = 'RleList')
>>> Error in seqinfo(ranges) : UCSC library operation failed
>>> In addition: Warning message:
>>> In seqinfo(ranges) :
>>> Couldn't open
>>>
>>> https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
>>> > x <-
>>> > import.bw('http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5')
>>> Error in seqinfo(ranges) : UCSC library operation failed
>>> In addition: Warning messages:
>>> 1: In seqinfo(ranges) :
>>> TCP non-blocking connect() to content-na.drive.amazonaws.com
>>> timed-out in select() after 1 milliseconds - Cancelling!
>>> 2: In seqinfo(ranges) :
>>> Couldn't open
>>>
>>> http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
>>> > ## Reproducibility info
>>> > message(Sys.time())
>>> 2016-05-05 17:38:30
>>> > options(width = 120)
>>> > devtools::session_info()
>>> Session info
>>> ---
>>> setting value
>>> version R version 3.3.0 RC (2016-05-01 r70572)
>>> system x86_64, darwin13.4.0
>>> ui X11
>>> language (EN)
>>> collate en_US.UTF-8
>>> tz America/New_York
>>> date 2016-05-05
>>>
>>> Packages
>>> ---
>>> package * version date source
>>> Biobase2.32.0 2016-05-04 Bioconductor
>>> BiocGenerics * 0.18.0 2016-05-04 Bioconductor
>>> BiocParallel 1.6.02016-0
Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
Hi Michael,
We tried getting things to work with Amazon Cloud Drive (see Abhi's
efforts at https://github.com/nellore/duffel/commits/master). But we
now have the data hosted elsewhere where the links work properly.
I just noted a small mistake on rtracklayer:::expandPath(). See:
> startsWith('http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw',
> 'http||ftp')
[1] FALSE
> startsWith('http://duffel.rail.bio/recount/SRP009615/bw/mean_SRP009615.bw',
> 'http')
[1] TRUE
The fix is simple. At
https://github.com/Bioconductor-mirror/rtracklayer/blob/c4b842bc4daa4b9db26cb86f3284cf8cf5c32ebd/R/web.R#L62-L66,
change it to:
expandPath <- function(x) {
if (startsWith(x, "http") | startsWith(x, "ftp"))
expandURL(x)
else path.expand(x)
}
Best,
Leo
On Thu, May 5, 2016 at 8:10 PM, Michael Lawrence
wrote:
> I checked in something that tries to find openssl automatically on the Mac.
>
> It looks like AWS is for some reason returning 404 for the HEAD command that
> the UCSC library uses the get info about the file like the content size.
> Same thing happens when I play around in Firefox's developer tools. The
> error response header claims a JSON content type, but no JSON is actually
> sent, so there is no further description of the error. I think this is a bug
> in Amazon.
>
> Seems like for now you'll need to download the file first.
>
> Michael
>
> On Thu, May 5, 2016 at 2:46 PM, Leonardo Collado Torres
> wrote:
>>
>> Hi Michael,
>>
>> I forgot about pkg-util (just did a fresh BioC 3.3 install). I assumed
>> the OS X binary would work out of the box.
>>
>> Anyhow, I installed rtracklayer (release) manually and got another
>> error (slightly different message now).
>>
>>
>>
>>
>> $ svn co
>> https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_3/madman/Rpacks/rtracklayer
>> $ R CMD INSTALL rtracklayer
>> Loading required package: colorout
>> * installing to library
>> ‘/Library/Frameworks/R.framework/Versions/3.3release/Resources/library’
>> * installing *source* package ‘rtracklayer’ ...
>> checking for pkg-config... /usr/local/bin/pkg-config
>> checking pkg-config is at least version 0.9.0... yes
>> checking for OPENSSL... yes
>> ## more output
>>
>> $ R
>> > library('rtracklayer')
>> > unshorten_url <- function(uri) {
>> + require('RCurl')
>> + opts <- list(
>> + followlocation = TRUE, # resolve redirects
>> + ssl.verifyhost = FALSE, # suppress certain SSL errors
>> + ssl.verifypeer = FALSE,
>> + nobody = TRUE, # perform HEAD request
>> + verbose = FALSE
>> + )
>> + curlhandle <- getCurlHandle(.opts = opts)
>> + getURL(uri, curl = curlhandle)
>> + info <- getCurlInfo(curlhandle)
>> + rm(curlhandle) # release the curlhandle!
>> + info$effective.url
>> + }
>> > url <-
>> > unshorten_url('http://duffel.rail.bio/recount/DRP000366/bw/DRR000897.bw')
>> Loading required package: RCurl
>> Loading required package: bitops
>> > url
>> [1]
>> "https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5";
>> > x <- import.bw(url, as = 'RleList')
>> Error in seqinfo(ranges) : UCSC library operation failed
>> In addition: Warning message:
>> In seqinfo(ranges) :
>> Couldn't open
>>
>> https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
>> > x <-
>> > import.bw('http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5')
>> Error in seqinfo(ranges) : UCSC library operation failed
>> In addition: Warning messages:
>> 1: In seqinfo(ranges) :
>> TCP non-blocking connect() to content-na.drive.amazonaws.com
>> timed-out in select() after 1 milliseconds - Cancelling!
>> 2: In seqinfo(ranges) :
>> Couldn't open
>>
>> http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
>> > ## Reproducibility info
>> > message(Sys.time())
>> 2016-05-05 17:38:30
>> > options(width = 120)
>> > devtools::session_info()
>> Session info
>> ---
>> setting value
>> version R version 3.3.0 RC (2016-05-01 r70572)
>> system x86_64, darwin13.4.0
>> ui X11
>> language (EN)
>> collate en_US.UTF-8
>> tz America/New_York
>> date 2016-05-05
>>
>> Packages
>> ---
>> package * version date source
>> Biobase2.32.0 2016-05-04 Bioconductor
>> BiocGenerics * 0.18.0 2016-05-04 Bioconductor
>> BiocParallel 1.6.02016-05-04 Bioconductor
>> Biostrings 2.40.0 2016-05-04 Bioconductor
>> bitops * 1.0-62013-08-17 CRAN (R 3.3.0)
>> colorout * 1.1-22016-05-05 Github
>> (jalvesaq/colorout@6538970)
>> devtools 1.11.1 201
Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
I checked in something that tries to find openssl automatically on the Mac.
It looks like AWS is for some reason returning 404 for the HEAD command
that the UCSC library uses the get info about the file like the content
size. Same thing happens when I play around in Firefox's developer tools.
The error response header claims a JSON content type, but no JSON is
actually sent, so there is no further description of the error. I think
this is a bug in Amazon.
Seems like for now you'll need to download the file first.
Michael
On Thu, May 5, 2016 at 2:46 PM, Leonardo Collado Torres
wrote:
> Hi Michael,
>
> I forgot about pkg-util (just did a fresh BioC 3.3 install). I assumed
> the OS X binary would work out of the box.
>
> Anyhow, I installed rtracklayer (release) manually and got another
> error (slightly different message now).
>
>
>
>
> $ svn co
> https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_3/madman/Rpacks/rtracklayer
> $ R CMD INSTALL rtracklayer
> Loading required package: colorout
> * installing to library
> ‘/Library/Frameworks/R.framework/Versions/3.3release/Resources/library’
> * installing *source* package ‘rtracklayer’ ...
> checking for pkg-config... /usr/local/bin/pkg-config
> checking pkg-config is at least version 0.9.0... yes
> checking for OPENSSL... yes
> ## more output
>
> $ R
> > library('rtracklayer')
> > unshorten_url <- function(uri) {
> + require('RCurl')
> + opts <- list(
> + followlocation = TRUE, # resolve redirects
> + ssl.verifyhost = FALSE, # suppress certain SSL errors
> + ssl.verifypeer = FALSE,
> + nobody = TRUE, # perform HEAD request
> + verbose = FALSE
> + )
> + curlhandle <- getCurlHandle(.opts = opts)
> + getURL(uri, curl = curlhandle)
> + info <- getCurlInfo(curlhandle)
> + rm(curlhandle) # release the curlhandle!
> + info$effective.url
> + }
> > url <- unshorten_url('
> http://duffel.rail.bio/recount/DRP000366/bw/DRR000897.bw')
> Loading required package: RCurl
> Loading required package: bitops
> > url
> [1] "
> https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
> "
> > x <- import.bw(url, as = 'RleList')
> Error in seqinfo(ranges) : UCSC library operation failed
> In addition: Warning message:
> In seqinfo(ranges) :
> Couldn't open
>
> https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
> > x <- import.bw('
> http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
> ')
> Error in seqinfo(ranges) : UCSC library operation failed
> In addition: Warning messages:
> 1: In seqinfo(ranges) :
> TCP non-blocking connect() to content-na.drive.amazonaws.com
> timed-out in select() after 1 milliseconds - Cancelling!
> 2: In seqinfo(ranges) :
> Couldn't open
>
> http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
> > ## Reproducibility info
> > message(Sys.time())
> 2016-05-05 17:38:30
> > options(width = 120)
> > devtools::session_info()
> Session info
> ---
> setting value
> version R version 3.3.0 RC (2016-05-01 r70572)
> system x86_64, darwin13.4.0
> ui X11
> language (EN)
> collate en_US.UTF-8
> tz America/New_York
> date 2016-05-05
>
> Packages
> ---
> package * version date source
> Biobase2.32.0 2016-05-04 Bioconductor
> BiocGenerics * 0.18.0 2016-05-04 Bioconductor
> BiocParallel 1.6.02016-05-04 Bioconductor
> Biostrings 2.40.0 2016-05-04 Bioconductor
> bitops * 1.0-62013-08-17 CRAN (R 3.3.0)
> colorout * 1.1-22016-05-05 Github
> (jalvesaq/colorout@6538970)
> devtools 1.11.1 2016-04-21 CRAN (R 3.3.0)
> digest 0.6.92016-01-08 CRAN (R 3.3.0)
> GenomeInfoDb * 1.8.02016-05-04 Bioconductor
> GenomicAlignments 1.8.02016-05-04 Bioconductor
> GenomicRanges* 1.24.0 2016-05-04 Bioconductor
> IRanges * 2.6.02016-05-04 Bioconductor
> memoise1.0.02016-01-29 CRAN (R 3.3.0)
> RCurl* 1.95-4.8 2016-03-01 CRAN (R 3.3.0)
> Rsamtools 1.24.0 2016-05-04 Bioconductor
> rtracklayer * 1.32.0 2016-05-05 Bioconductor
> S4Vectors* 0.10.0 2016-05-04 Bioconductor
> SummarizedExperiment 1.2.02016-05-04 Bioconductor
> withr 1.0.12016-02-04 CRAN (R 3.3.0)
> XML3.98-1.4 2016-03-01 CRAN (R 3.3.0)
> XVector0.12.0 2016-05-04 Bioconductor
> zlibbioc 1.18.0 2016-05-04 Bioconductor
> >
>
> On Thu, Ma
Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
Hi Michael,
I forgot about pkg-util (just did a fresh BioC 3.3 install). I assumed
the OS X binary would work out of the box.
Anyhow, I installed rtracklayer (release) manually and got another
error (slightly different message now).
$ svn co
https://hedgehog.fhcrc.org/bioconductor/branches/RELEASE_3_3/madman/Rpacks/rtracklayer
$ R CMD INSTALL rtracklayer
Loading required package: colorout
* installing to library
‘/Library/Frameworks/R.framework/Versions/3.3release/Resources/library’
* installing *source* package ‘rtracklayer’ ...
checking for pkg-config... /usr/local/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for OPENSSL... yes
## more output
$ R
> library('rtracklayer')
> unshorten_url <- function(uri) {
+ require('RCurl')
+ opts <- list(
+ followlocation = TRUE, # resolve redirects
+ ssl.verifyhost = FALSE, # suppress certain SSL errors
+ ssl.verifypeer = FALSE,
+ nobody = TRUE, # perform HEAD request
+ verbose = FALSE
+ )
+ curlhandle <- getCurlHandle(.opts = opts)
+ getURL(uri, curl = curlhandle)
+ info <- getCurlInfo(curlhandle)
+ rm(curlhandle) # release the curlhandle!
+ info$effective.url
+ }
> url <-
> unshorten_url('http://duffel.rail.bio/recount/DRP000366/bw/DRR000897.bw')
Loading required package: RCurl
Loading required package: bitops
> url
[1]
"https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5";
> x <- import.bw(url, as = 'RleList')
Error in seqinfo(ranges) : UCSC library operation failed
In addition: Warning message:
In seqinfo(ranges) :
Couldn't open
https://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
> x <-
> import.bw('http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5')
Error in seqinfo(ranges) : UCSC library operation failed
In addition: Warning messages:
1: In seqinfo(ranges) :
TCP non-blocking connect() to content-na.drive.amazonaws.com
timed-out in select() after 1 milliseconds - Cancelling!
2: In seqinfo(ranges) :
Couldn't open
http://content-na.drive.amazonaws.com/cdproxy/templink/usTQCr2pAaI3tTps4AFQuz1H9kmm23EDYy39SQ3ke5EuFiZq5
> ## Reproducibility info
> message(Sys.time())
2016-05-05 17:38:30
> options(width = 120)
> devtools::session_info()
Session info
---
setting value
version R version 3.3.0 RC (2016-05-01 r70572)
system x86_64, darwin13.4.0
ui X11
language (EN)
collate en_US.UTF-8
tz America/New_York
date 2016-05-05
Packages
---
package * version date source
Biobase2.32.0 2016-05-04 Bioconductor
BiocGenerics * 0.18.0 2016-05-04 Bioconductor
BiocParallel 1.6.02016-05-04 Bioconductor
Biostrings 2.40.0 2016-05-04 Bioconductor
bitops * 1.0-62013-08-17 CRAN (R 3.3.0)
colorout * 1.1-22016-05-05 Github (jalvesaq/colorout@6538970)
devtools 1.11.1 2016-04-21 CRAN (R 3.3.0)
digest 0.6.92016-01-08 CRAN (R 3.3.0)
GenomeInfoDb * 1.8.02016-05-04 Bioconductor
GenomicAlignments 1.8.02016-05-04 Bioconductor
GenomicRanges* 1.24.0 2016-05-04 Bioconductor
IRanges * 2.6.02016-05-04 Bioconductor
memoise1.0.02016-01-29 CRAN (R 3.3.0)
RCurl* 1.95-4.8 2016-03-01 CRAN (R 3.3.0)
Rsamtools 1.24.0 2016-05-04 Bioconductor
rtracklayer * 1.32.0 2016-05-05 Bioconductor
S4Vectors* 0.10.0 2016-05-04 Bioconductor
SummarizedExperiment 1.2.02016-05-04 Bioconductor
withr 1.0.12016-02-04 CRAN (R 3.3.0)
XML3.98-1.4 2016-03-01 CRAN (R 3.3.0)
XVector0.12.0 2016-05-04 Bioconductor
zlibbioc 1.18.0 2016-05-04 Bioconductor
>
On Thu, May 5, 2016 at 5:24 PM, Michael Lawrence
wrote:
> The URL redirection is something I can try to add. For the other error, you
> need to get openssl installed and made visible to pkg-config, so that
> rtracklayer finds it during its build process.
>
> Michael
>
> On Thu, May 5, 2016 at 2:01 PM, Leonardo Collado Torres
> wrote:
>>
>> Hi Michael,
>>
>> I have a use case that is similar to
>> https://support.bioconductor.org/p/81267/#82142 and looks to me like
>> it might need some changes in rtracklayer to work. That's why I'm
>> posting it here this time.
>>
>> Basically, I'm trying to use rtracklayer to import a bigwig file over
>> the web which is in a different type of url than before. Using
>> utils::download.file() with the defaults doesn't work, I have to use
>> method = 'curl' and extra =
Re: [Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
The URL redirection is something I can try to add. For the other error, you need to get openssl installed and made visible to pkg-config, so that rtracklayer finds it during its build process. Michael On Thu, May 5, 2016 at 2:01 PM, Leonardo Collado Torres wrote: > Hi Michael, > > I have a use case that is similar to > https://support.bioconductor.org/p/81267/#82142 and looks to me like > it might need some changes in rtracklayer to work. That's why I'm > posting it here this time. > > Basically, I'm trying to use rtracklayer to import a bigwig file over > the web which is in a different type of url than before. Using > utils::download.file() with the defaults doesn't work, I have to use > method = 'curl' and extra = '-L'. > > More specifically, the original url > http://duffel.rail.bio/recount/DRP000366/bw/DRR000897.bw has an > effective url > https://content-na.drive.amazonaws.com/cdproxy/templink/i_aQAPZJkJ9d9lN1NO5DJJtlbpvAdgbNuc1SkqSTHFouFiZq5 > > Now, using the second url with utils::download.file() and default > methods also doesn't work. It does on the browser though. > > > As you can see, downloading the file doesn't work out of the box. > Which I guess that it's not surprising that using rtracklayer I get > errors like: > > In seqinfo(ranges) : > No openssl available in netConnectHttps for > content-na.drive.amazonaws.com : 443 > > You can find further details (code and log file) at > https://gist.github.com/lcolladotor/c500dd79d49aed1ef33ade5417111453 > > Thanks, > Leo > [[alternative HTML version deleted]] ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
[Bioc-devel] Issue importing bigwig files with rtracklayer from Amazon Cloud Drive
Hi Michael, I have a use case that is similar to https://support.bioconductor.org/p/81267/#82142 and looks to me like it might need some changes in rtracklayer to work. That's why I'm posting it here this time. Basically, I'm trying to use rtracklayer to import a bigwig file over the web which is in a different type of url than before. Using utils::download.file() with the defaults doesn't work, I have to use method = 'curl' and extra = '-L'. More specifically, the original url http://duffel.rail.bio/recount/DRP000366/bw/DRR000897.bw has an effective url https://content-na.drive.amazonaws.com/cdproxy/templink/i_aQAPZJkJ9d9lN1NO5DJJtlbpvAdgbNuc1SkqSTHFouFiZq5 Now, using the second url with utils::download.file() and default methods also doesn't work. It does on the browser though. As you can see, downloading the file doesn't work out of the box. Which I guess that it's not surprising that using rtracklayer I get errors like: In seqinfo(ranges) : No openssl available in netConnectHttps for content-na.drive.amazonaws.com : 443 You can find further details (code and log file) at https://gist.github.com/lcolladotor/c500dd79d49aed1ef33ade5417111453 Thanks, Leo ___ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
