Re: [R] GAM with the negative binomial distribution: why do predictions no match with original values?

[Marine Regis]

Thanks a lot for your answers.
Peter: sorry, here is the missing information:

  *   I use the function gam() of the package �mgcv�
  *   Yes, the output changes when I use offset(log_trap_eff) instead of 
offset=log_trap_eff. By using offset(log_trap_eff), the output is more coherent 
with the observed values. Here are the new predictions:
> summary(mod$fit)
Min. 1st Qu.  MedianMean 3rd Qu.Max.
10.01   68.14   85.71   83.16  101.00  130.20


  *   I have tried to create a reproductive example to show the difference 
between offset(log_trap_eff) and offset=log_trap_eff.
nb_unique <- rnegbin(58, mu=82, theta=13.446)
x <- runif(58,min=-465300,max=435200)
prop_forest <- runif(58,min=0,max=1)
log_trap_eff <- runif(58,min=4,max=6)

With offset=log_trap_eff:

mod1 <- gam(nb_unique ~ s(x,prop_forest), offset=log_trap_eff, 
family=nb(theta=NULL, link="log"), method = "REML", select = TRUE)

> mod1Pred <- predict.gam(mod1, se.fit=TRUE, type="response")

> summary(mod1Pred$fit)

   Min. 1st Qu.  MedianMean 3rd Qu.Max.

 0.5852  0.5852  0.5852  0.5852  0.5852  0.5852


With offset(log_trap_eff):
mod2 <- gam(nb_unique ~ s(x,prop_forest) + offset(log_trap_eff), 
family=nb(theta=NULL, link="log"), method = "REML", select = TRUE)

> mod2Pred <- predict.gam(mod2, se.fit=TRUE, type="response")

> summary(mod2Pred$fit)

   Min. 1st Qu.  MedianMean 3rd Qu.Max.

  32.03   61.18   97.20  112.20  165.00  226.00



Value range of observed data:

> summary(nb_unique)

   Min. 1st Qu.  MedianMean 3rd Qu.Max.

  43.00   67.00   81.00   84.16   92.75  153.00


  *   By using fitted(mod), I obtain NULL.
I am a novice in GAMs. So, I don�t know why the results are different between 
models with offset=argument and offset().
Thanks a lot for your help.
Have a nice day
Marine




De : peter dalgaard 
Envoy� : mardi 22 novembre 2016 23:52
� : Bert Gunter
Cc : Marine Regis; r-help@r-project.org
Objet : Re: [R] GAM with the negative binomial distribution: why do predictions 
no match with original values?


> On 22 Nov 2016, at 23:07 , Bert Gunter  wrote:
>
> Define "very different."  Sounds like a subjective opinion to me, for
> which I have no response. Apparently others are similarly flummoxed.
> Of course they would not in general be identical.

Er? I don't see much reason to disagree that a range 0.10-0.18 is different 
from 17-147.

However, other bits of information are missing: We don't know which gam() 
function is being used (to my knowledge there is one in package gam but also 
one in mgcv). We don't have the data, so we cannot reproduce and try to find 
the root of the problem.

Offhand, it looks like the predict.gam() function is misbehaving, which could 
have something to do with the offset term and/or the nb dispersion parameter. 
On a hunch, does anything change if you use

nb_unique ~ s(x,prop_forest) + offset(log_trap_eff)

instead of the offset= argument? And, by the way, does fitted(mod,...) change 
anything?

-pd

>
> Cheers,
> Bert
>
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along
> and sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Tue, Nov 22, 2016 at 1:29 PM, Marine Regis  wrote:
>> Hello,
>>
>>> From capture data, I would like to assess the effect of longitudinal 
>>> changes in proportion of forests on abundance of skunks. To test this, I 
>>> built this GAM where the dependent variable is the number of unique skunks 
>>> and the independent variables are the X coordinates of the centroids of 
>>> trapping sites (called "X" in the GAM) and the proportion of forests within 
>>> the trapping sites (called "prop_forest" in the GAM):
>>
>>mod <- gam(nb_unique ~ s(x,prop_forest), offset=log_trap_eff, 
>> family=nb(theta=NULL, link="log"), data=succ_capt_skunk, method = "REML", 
>> select = TRUE)
>>summary(mod)
>>
>>Family: Negative Binomial(13.446)
>>Link function: log
>>
>>Formula:
>>nb_unique ~ s(x, prop_forest)
>>
>>Parametric coefficients:
>>Estimate Std. Error z value Pr(>|z|)
>>(Intercept) -2.020950.03896  -51.87   <2e-16 ***
>>---
>>Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>
>>Approximate significance of smooth terms:
>>   edf Ref.df Chi.sq  p-value
>>s(x,prop_forest) 3.182 29  17.76 0.000102 ***
>>---
>>Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>
>>R-sq.(adj) =   0.37   Deviance explained =   49%
>>-REML = 268.61  Scale est. = 1 n = 58
>>
>>
>> I built a GAM  for the negative binomial family. When I use the function 
>> `predict.gam`, the predictions of capture success from the GAM and the 
>> values of capture success from original data are very different. What is the 
>> reason for differences occur?
>>
>> **With GAM:**
>>

Re: [R] GAM with the negative binomial distribution: why do predictions no match with original values?

[peter dalgaard]

> On 22 Nov 2016, at 23:07 , Bert Gunter  wrote:
> 
> Define "very different."  Sounds like a subjective opinion to me, for
> which I have no response. Apparently others are similarly flummoxed.
> Of course they would not in general be identical.

Er? I don't see much reason to disagree that a range 0.10-0.18 is different 
from 17-147.

However, other bits of information are missing: We don't know which gam() 
function is being used (to my knowledge there is one in package gam but also 
one in mgcv). We don't have the data, so we cannot reproduce and try to find 
the root of the problem. 

Offhand, it looks like the predict.gam() function is misbehaving, which could 
have something to do with the offset term and/or the nb dispersion parameter. 
On a hunch, does anything change if you use

nb_unique ~ s(x,prop_forest) + offset(log_trap_eff)

instead of the offset= argument? And, by the way, does fitted(mod,...) change 
anything?

-pd 

> 
> Cheers,
> Bert
> 
> 
> Bert Gunter
> 
> "The trouble with having an open mind is that people keep coming along
> and sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
> 
> 
> On Tue, Nov 22, 2016 at 1:29 PM, Marine Regis  wrote:
>> Hello,
>> 
>>> From capture data, I would like to assess the effect of longitudinal 
>>> changes in proportion of forests on abundance of skunks. To test this, I 
>>> built this GAM where the dependent variable is the number of unique skunks 
>>> and the independent variables are the X coordinates of the centroids of 
>>> trapping sites (called "X" in the GAM) and the proportion of forests within 
>>> the trapping sites (called "prop_forest" in the GAM):
>> 
>>mod <- gam(nb_unique ~ s(x,prop_forest), offset=log_trap_eff, 
>> family=nb(theta=NULL, link="log"), data=succ_capt_skunk, method = "REML", 
>> select = TRUE)
>>summary(mod)
>> 
>>Family: Negative Binomial(13.446)
>>Link function: log
>> 
>>Formula:
>>nb_unique ~ s(x, prop_forest)
>> 
>>Parametric coefficients:
>>Estimate Std. Error z value Pr(>|z|)
>>(Intercept) -2.020950.03896  -51.87   <2e-16 ***
>>---
>>Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>> 
>>Approximate significance of smooth terms:
>>   edf Ref.df Chi.sq  p-value
>>s(x,prop_forest) 3.182 29  17.76 0.000102 ***
>>---
>>Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>> 
>>R-sq.(adj) =   0.37   Deviance explained =   49%
>>-REML = 268.61  Scale est. = 1 n = 58
>> 
>> 
>> I built a GAM  for the negative binomial family. When I use the function 
>> `predict.gam`, the predictions of capture success from the GAM and the 
>> values of capture success from original data are very different. What is the 
>> reason for differences occur?
>> 
>> **With GAM:**
>> 
>>modPred <- predict.gam(mod, se.fit=TRUE,type="response")
>>summary(modPred$fit)
>>   Min. 1st Qu.  MedianMean 3rd Qu.Max.
>> 0.1026  0.1187  0.1333  0.1338  0.1419  0.1795
>> 
>> **With original data:**
>> 
>>summary(succ_capt_skunk$nb_unique)
>>   Min. 1st Qu.  MedianMean 3rd Qu.Max.
>>  17.00   59.00   82.00   81.83  106.80  147.00
>> 
>> The question has already been posted on Cross validated 
>> (http://stats.stackexchange.com/questions/247347/gam-with-the-negative-binomial-distribution-why-do-predictions-no-match-with-or)
>>  without success.
>> 
>> Thanks a lot for your time.
>> Have a nice day
>> Marine
>> 
>> 
>>[[alternative HTML version deleted]]
>> 
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd@cbs.dk  Priv: pda...@gmail.com

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] GAM with the negative binomial distribution: why do predictions no match with original values?

[Cade, Brian]

Well part of the issue is that the negative binomial estimates are for
means and they can differ a fair bit from the raw counts, but I'm also
guessing that part of the issue is that the offset may not be accounted for
with the predict.gam() function.

Brian

Brian S. Cade, PhD

U. S. Geological Survey
Fort Collins Science Center
2150 Centre Ave., Bldg. C
Fort Collins, CO  80526-8818

email:  ca...@usgs.gov 
tel:  970 226-9326


On Tue, Nov 22, 2016 at 2:29 PM, Marine Regis 
wrote:

> Hello,
>
> >From capture data, I would like to assess the effect of longitudinal
> changes in proportion of forests on abundance of skunks. To test this, I
> built this GAM where the dependent variable is the number of unique skunks
> and the independent variables are the X coordinates of the centroids of
> trapping sites (called "X" in the GAM) and the proportion of forests within
> the trapping sites (called "prop_forest" in the GAM):
>
> mod <- gam(nb_unique ~ s(x,prop_forest), offset=log_trap_eff,
> family=nb(theta=NULL, link="log"), data=succ_capt_skunk, method = "REML",
> select = TRUE)
> summary(mod)
>
> Family: Negative Binomial(13.446)
> Link function: log
>
> Formula:
> nb_unique ~ s(x, prop_forest)
>
> Parametric coefficients:
> Estimate Std. Error z value Pr(>|z|)
> (Intercept) -2.020950.03896  -51.87   <2e-16 ***
> ---
> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>
> Approximate significance of smooth terms:
>edf Ref.df Chi.sq  p-value
> s(x,prop_forest) 3.182 29  17.76 0.000102 ***
> ---
> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>
> R-sq.(adj) =   0.37   Deviance explained =   49%
> -REML = 268.61  Scale est. = 1 n = 58
>
>
> I built a GAM  for the negative binomial family. When I use the function
> `predict.gam`, the predictions of capture success from the GAM and the
> values of capture success from original data are very different. What is
> the reason for differences occur?
>
> **With GAM:**
>
> modPred <- predict.gam(mod, se.fit=TRUE,type="response")
> summary(modPred$fit)
>Min. 1st Qu.  MedianMean 3rd Qu.Max.
>  0.1026  0.1187  0.1333  0.1338  0.1419  0.1795
>
>  **With original data:**
>
> summary(succ_capt_skunk$nb_unique)
>Min. 1st Qu.  MedianMean 3rd Qu.Max.
>   17.00   59.00   82.00   81.83  106.80  147.00
>
> The question has already been posted on Cross validated (
> http://stats.stackexchange.com/questions/247347/gam-with-
> the-negative-binomial-distribution-why-do-predictions-no-match-with-or)
> without success.
>
> Thanks a lot for your time.
> Have a nice day
> Marine
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] GAM with the negative binomial distribution: why do predictions no match with original values?

[David Winsemius]

> On Nov 22, 2016, at 1:29 PM, Marine Regis  wrote:
> 
> Hello,
> 
>> From capture data, I would like to assess the effect of longitudinal changes 
>> in proportion of forests on abundance of skunks. To test this, I built this 
>> GAM where the dependent variable is the number of unique skunks and the 
>> independent variables are the X coordinates of the centroids of trapping 
>> sites (called "X" in the GAM) and the proportion of forests within the 
>> trapping sites (called "prop_forest" in the GAM):
> 
>mod <- gam(nb_unique ~ s(x,prop_forest), offset=log_trap_eff, 
> family=nb(theta=NULL, link="log"), data=succ_capt_skunk, method = "REML", 
> select = TRUE)
>summary(mod)
> 
>Family: Negative Binomial(13.446)
>Link function: log
> 
>Formula:
>nb_unique ~ s(x, prop_forest)
> 
>Parametric coefficients:
>Estimate Std. Error z value Pr(>|z|)
>(Intercept) -2.020950.03896  -51.87   <2e-16 ***
>---
>Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> 
>Approximate significance of smooth terms:
>   edf Ref.df Chi.sq  p-value
>s(x,prop_forest) 3.182 29  17.76 0.000102 ***
>---
>Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> 
>R-sq.(adj) =   0.37   Deviance explained =   49%
>-REML = 268.61  Scale est. = 1 n = 58
> 
> 
> I built a GAM  for the negative binomial family. When I use the function 
> `predict.gam`, the predictions of capture success from the GAM and the values 
> of capture success from original data are very different. What is the reason 
> for differences occur?

You have an offset that is not described. And `gam` suppresses the Intercept. 
These would seem to be likely sources of confusion. For the best answers either 
on Rhelp or on CrossValidated.com you should be offering a working example. 
It's not our responsibility to build these for you.

I found that others had included offsets and then had questions about 
prediction. I haven't reviewed these candidates but perhaps you can find one in 
this modest listing that comes up from the MarkMail search engine:

http://markmail.org/search/?q=list%3Aorg.r-project.r-help+mgcv+gam+offset+predict

library(mgcv) 
x<-seq(0,10,length=100) 
y<-x^2+rnorm(100) 
m1<-gam(y~s(x,k=10,bs='cs')) 
m2<-gam(y~s(x,k=10,bs='cs'), offset= rep(10,100) ) 
x1<-seq(0,10,0.1) 
y1<-predict(m1,newdata=list(x=x1)) 
y2<-predict(m2,newdata=list(x=x1))

plot(x,y,ylim=c(0,100)) 
lines(x1,y1,lwd=4,col='red') 
lines(x1,y2,lwd=4,col='blue')


-- 
David.


> 
> **With GAM:**
> 
>modPred <- predict.gam(mod, se.fit=TRUE,type="response")
>summary(modPred$fit)
>   Min. 1st Qu.  MedianMean 3rd Qu.Max.
> 0.1026  0.1187  0.1333  0.1338  0.1419  0.1795
> 
> **With original data:**
> 
>summary(succ_capt_skunk$nb_unique)
>   Min. 1st Qu.  MedianMean 3rd Qu.Max.
>  17.00   59.00   82.00   81.83  106.80  147.00
> 
> The question has already been posted on Cross validated 
> (http://stats.stackexchange.com/questions/247347/gam-with-the-negative-binomial-distribution-why-do-predictions-no-match-with-or)
>  without success.
> 
> Thanks a lot for your time.
> Have a nice day
> Marine
> 
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] GAM with the negative binomial distribution: why do predictions no match with original values?

[Bert Gunter]

Define "very different."  Sounds like a subjective opinion to me, for
which I have no response. Apparently others are similarly flummoxed.
Of course they would not in general be identical.

Cheers,
Bert


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Nov 22, 2016 at 1:29 PM, Marine Regis  wrote:
> Hello,
>
> >From capture data, I would like to assess the effect of longitudinal changes 
> >in proportion of forests on abundance of skunks. To test this, I built this 
> >GAM where the dependent variable is the number of unique skunks and the 
> >independent variables are the X coordinates of the centroids of trapping 
> >sites (called "X" in the GAM) and the proportion of forests within the 
> >trapping sites (called "prop_forest" in the GAM):
>
> mod <- gam(nb_unique ~ s(x,prop_forest), offset=log_trap_eff, 
> family=nb(theta=NULL, link="log"), data=succ_capt_skunk, method = "REML", 
> select = TRUE)
> summary(mod)
>
> Family: Negative Binomial(13.446)
> Link function: log
>
> Formula:
> nb_unique ~ s(x, prop_forest)
>
> Parametric coefficients:
> Estimate Std. Error z value Pr(>|z|)
> (Intercept) -2.020950.03896  -51.87   <2e-16 ***
> ---
> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>
> Approximate significance of smooth terms:
>edf Ref.df Chi.sq  p-value
> s(x,prop_forest) 3.182 29  17.76 0.000102 ***
> ---
> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>
> R-sq.(adj) =   0.37   Deviance explained =   49%
> -REML = 268.61  Scale est. = 1 n = 58
>
>
> I built a GAM  for the negative binomial family. When I use the function 
> `predict.gam`, the predictions of capture success from the GAM and the values 
> of capture success from original data are very different. What is the reason 
> for differences occur?
>
> **With GAM:**
>
> modPred <- predict.gam(mod, se.fit=TRUE,type="response")
> summary(modPred$fit)
>Min. 1st Qu.  MedianMean 3rd Qu.Max.
>  0.1026  0.1187  0.1333  0.1338  0.1419  0.1795
>
>  **With original data:**
>
> summary(succ_capt_skunk$nb_unique)
>Min. 1st Qu.  MedianMean 3rd Qu.Max.
>   17.00   59.00   82.00   81.83  106.80  147.00
>
> The question has already been posted on Cross validated 
> (http://stats.stackexchange.com/questions/247347/gam-with-the-negative-binomial-distribution-why-do-predictions-no-match-with-or)
>  without success.
>
> Thanks a lot for your time.
> Have a nice day
> Marine
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R-es] ACCESS 2007 2010: acceder a tablas

[Carlos Ortega]

Hola,

¿Has probado con alguna de estas opciones?:

http://www.statmethods.net/input/dbinterface.html

http://rprogramming.net/connect-to-ms-access-in-r/

Saludos,
Carlos Ortega
www.qualityexcellence.es

El 22 de noviembre de 2016, 17:01, Antonio Punzón 
escribió:

> Hola,
> Necesitaría acceder a las base de datos En versiones 2007 o superior.
> Hasta ahora lo que hacía era guardarlas en ACCESS 2003 y acceder a ellas
> de R a 32 bits
>
> Pero eso no me permite trabajr de forma continuada con los datos
> actualizados.
> Hay alguna forma de acceder a las tablas de ACCESS 2007 o superior desde R
> a 64 bits?
>
> Muchas gracias
>
> [[alternative HTML version deleted]]
>
>
> ___
> R-help-es mailing list
> R-help-es@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-help-es
>



-- 
Saludos,
Carlos Ortega
www.qualityexcellence.es

[[alternative HTML version deleted]]

___
R-help-es mailing list
R-help-es@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-help-es

[R] GAM with the negative binomial distribution: why do predictions no match with original values?

[Marine Regis]

Hello,

>From capture data, I would like to assess the effect of longitudinal changes 
>in proportion of forests on abundance of skunks. To test this, I built this 
>GAM where the dependent variable is the number of unique skunks and the 
>independent variables are the X coordinates of the centroids of trapping sites 
>(called "X" in the GAM) and the proportion of forests within the trapping 
>sites (called "prop_forest" in the GAM):

mod <- gam(nb_unique ~ s(x,prop_forest), offset=log_trap_eff, 
family=nb(theta=NULL, link="log"), data=succ_capt_skunk, method = "REML", 
select = TRUE)
summary(mod)

Family: Negative Binomial(13.446)
Link function: log

Formula:
nb_unique ~ s(x, prop_forest)

Parametric coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.020950.03896  -51.87   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Approximate significance of smooth terms:
   edf Ref.df Chi.sq  p-value
s(x,prop_forest) 3.182 29  17.76 0.000102 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

R-sq.(adj) =   0.37   Deviance explained =   49%
-REML = 268.61  Scale est. = 1 n = 58


I built a GAM  for the negative binomial family. When I use the function 
`predict.gam`, the predictions of capture success from the GAM and the values 
of capture success from original data are very different. What is the reason 
for differences occur?

**With GAM:**

modPred <- predict.gam(mod, se.fit=TRUE,type="response")
summary(modPred$fit)
   Min. 1st Qu.  MedianMean 3rd Qu.Max.
 0.1026  0.1187  0.1333  0.1338  0.1419  0.1795

 **With original data:**

summary(succ_capt_skunk$nb_unique)
   Min. 1st Qu.  MedianMean 3rd Qu.Max.
  17.00   59.00   82.00   81.83  106.80  147.00

The question has already been posted on Cross validated 
(http://stats.stackexchange.com/questions/247347/gam-with-the-negative-binomial-distribution-why-do-predictions-no-match-with-or)
 without success.

Thanks a lot for your time.
Have a nice day
Marine


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R-es] CHARACTER A NUMERIC

[javier.ruben.marcuzzi]

mitent i esborri el missatge de qualsevol ordinador. En el cas que
> aquest missatge vagi a ser contestat per la mateixa via, ha de saber-se que
> la seva resposta podria ser coneguda per tercers a l'entrar a la xarxa. Per
> això, si el missatge inclou contrasenyes, números de targetes de crèdit o
> qualsevol altra informació que vostè consideri confidencial, seria més
> segur contestar per una altra via i cancel·lar la seva transmissió.
> L'Ajuntament de Mataró i els seus organismes dependents no poden assumir la
> responsabilitat derivada del fet de què terceres persones puguin arribar a
> conèixer el contingut d'aquest missatge durant la seva transmissió.
>
> La presente información se envía únicamente a la persona a la que va
> dirigida y puede contener información privilegiada o de carácter
> confidencial. Cualquier modificación, retransmisión, difusión u otro uso de
> esta información por persones o entidades diferentes a la persona a la que
> va dirigida está prohibida. Si usted la ha recibido por error, por favor
> contacte con el remitente y borre el mensaje. En el caso de que este
> mensaje vaya a ser contestado por la misma vía, ha de saberse que su
> respuesta podría ser conocida por terceros al entrar en la red. Por este
> motivo, si el mensaje incluye contraseñas, números de tarjetas de crédito o
> cualquier otra información que considere confidencial, sería más seguro
> contestar por otra vía y cancelar su transmisión. El Ayuntamiento de Mataró
> y sus organismos dependientes no pueden asumir la responsabilidad derivada
> del hecho de que terceras personas puedan llegar a conocer el contenido de
> este mensaje durante su transmisión.
>
>
>  próxima parte 
> Se ha borrado un adjunto en formato HTML...
> URL: <https://stat.ethz.ch/pipermail/r-help-es/
> attachments/20161122/78872107/attachment-0001.html>
>
> --
>
> Message: 3
> Date: Tue, 22 Nov 2016 17:01:54 +0100
> From: Antonio Punzón <antonio.pun...@st.ieo.es>
> To: <r-help-es@r-project.org>
> Subject: [R-es] ACCESS 2007 2010: acceder a tablas
> Message-ID:
> <f62aee6406f87340a7d7829ee67f371001c...@servidor2003.st.ieo.es>
> Content-Type: text/plain; charset="UTF-8"
>
> Hola,
> Necesitar?acceder a las base de datos En versiones 2007 o superior.
> Hasta ahora lo que hac?era guardarlas en ACCESS 2003 y acceder a ellas de
> R a 32 bits
>
> Pero eso no me permite trabajr de forma continuada con los datos
> actualizados.
> Hay alguna forma de acceder a las tablas de ACCESS 2007 o superior desde R
> a 64 bits?
>
> Muchas gracias
>
> [[alternative HTML version deleted]]
>
>
>
> --
>
> Message: 4
> Date: Tue, 22 Nov 2016 13:24:05 -0300
> From: <javier.ruben.marcu...@gmail.com>
> To: Antonio Punzón <antonio.pun...@st.ieo.es>,
> "r-help-es@r-project.org" <r-help-es@r-project.org>
> Subject: Re: [R-es] ACCESS 2007 2010: acceder a tablas
> Message-ID: <58347125.a527c80a.fadc0.7...@mx.google.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Estimado Antonio Punzón
>
> En mi caso lo resolví hace algunos años, hoy no uso más Access. Mi forma
> fue descargar sql server, importar, y listo. Lógicamente la potencia de
> base de datos es mayor como su forma de trabajar con R, y su costo puede
> ser gratuito a grandes cantidades, pero si usted tiene los datos en Access
> la versión menor que es gratis para todos debería ser suficiente.
>
> Javier Rubén Marcuzzi
>
> De: Antonio Punzón
> Enviado: martes, 22 de noviembre de 2016 13:04
> Para: r-help-es@r-project.org
> Asunto: [R-es] ACCESS 2007 2010: acceder a tablas
>
> Hola,
> Necesitar?a acceder a las base de datos En versiones 2007 o superior.
> Hasta ahora lo que hac?a era guardarlas en ACCESS 2003 y acceder a ellas
> de R a 32 bits
>
> Pero eso no me permite trabajr de forma continuada con los datos
> actualizados.
> Hay alguna forma de acceder a las tablas de ACCESS 2007 o superior desde R
> a 64 bits?
>
> Muchas gracias
>
> [[alternative HTML version deleted]]
>
>
>
> [[alternative HTML version deleted]]
>
>
>
> --
>
> Message: 5
> Date: Tue, 22 Nov 2016 16:00:56 -0300
> From: <javier.ruben.marcu...@gmail.com>
> To: Santiago Repetto <santiagopnss...@gmail.com>,
> "r-help-es@r-project.org" <r-help-es@r-project.org>
> Subject: Re: [R-es] charater a numeric
> Message-ID: <583495e9.b12ec80a.2c938.a...@mx.google.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Estimado Santiago Repet

Re: [R] Memory problem

[Henrik Bengtsson]

On Windows 32-bit I think (it's been a while) you can push it to 3 GB but
to go beyond you need to run R  on 64-bit Windows (same rule for all
software not just R). I'm pretty sure this is already documented in the R
documentation.

Henrik

On Nov 22, 2016 19:49, "Ista Zahn"  wrote:

Not conveniently. Memory is cheap, you should buy more.

Best,
Ista

On Nov 22, 2016 12:19 PM, "Partha Sinha"  wrote:

>  I am using R 3.3.2 on win 7, 32 bit with 2gb Ram. Is it possible to use
> more than 2 Gb data set ?
>
> Regards
> Partha
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory problem

[Jeff Newmiller]

Ah, you also need to use a 64-bit operating system. Depending on the age of 
your hardware this may also mean you need a new computer. 

There are ways to process data on disk for certain algorithms, but you will be 
glad to leave them behind once the opportunity arises, so you might as well do 
so now. 
-- 
Sent from my phone. Please excuse my brevity.

On November 22, 2016 10:47:29 AM PST, Ista Zahn  wrote:
>Not conveniently. Memory is cheap, you should buy more.
>
>Best,
>Ista
>
>On Nov 22, 2016 12:19 PM, "Partha Sinha"  wrote:
>
>>  I am using R 3.3.2 on win 7, 32 bit with 2gb Ram. Is it possible to
>use
>> more than 2 Gb data set ?
>>
>> Regards
>> Partha
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/
>> posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>   [[alternative HTML version deleted]]
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R-es] charater a numeric

[Carlos J. Gil Bellosta]

Tu problema es la coma. Cámbiala a punto y luego, as.numeric.

El 22/11/2016 8:08 p. m.,  escribió:

> Estimado Santiago Repetto
>
> Sus datos están mal.
>
> "#¡NULO!"
>
> En R es NULL, pero al importar o escribir los datos, el mismo R debería
> colocarlos en nulo, también usted puede colocar valores en nulo, pero creo
> que no es el caso.
>
> Luego con as.numeric(datos…) debería funcionar.
>
> Javier Rubén Marcuzzi
>
> De: Santiago Repetto
> Enviado: martes, 22 de noviembre de 2016 12:07
> Para: r-help-es@r-project.org
> Asunto: [R-es] charater a numeric
>
> Hola!!
> Soy novato en esto del R. Consulto por que ya estoy agotando los recursos
> para resolver un problema con tutoriales, consultas que se encuentran en
> google, etc. Si bien el problema parece ser simple no le encuentro la
> vuelta.
>
> Tengo un data.frame que armé importando un csv. Algunas de las columnas son
> mayoritariamente compuestas por valores números pero class character ya que
> al contener valores no numéricos ("#¡NULO!" por ejemplo) los importa como
> character. Quiero convertirlas justamente a un vector numeric (o mejor aun
> dentro del mismo data.frame que sean numeric) para poder operar con ellas
> (los valores que no sean números deberían quedar como NA).
>
> Probé con
> <-as.numeric(as.character(SS))
> pero me devuelve valores NA.
>
> También probé con
> SS1<- type.convert(SS, na.strings = "NA", as.is = FALSE, dec = ",",
> numerals = c("allow.loss", "warn.loss", "no.loss")) y luego con as.numeric
> pero me devuelve los números de ordenación de niveles del factor (creo)
>
> Muy agradecido de antemano por cualquier sugerencia, comando o solución (o
> si estoy cometiendo algún problema lógico en la propia formulación del
> problema).
> Este sería mas o menos el contenido de mis datos (como vector).
> Saludos!
> Santiago
>
> > SS
>  [1] "137155,00" "134714,00" "136184,00" "#¡NULO!"   "139683,00" "13827,00"
>  [7] "139450,00" "#¡NULO!"   "134462,00" "136058,00" "132818,00" "30020,00"
> [13] "29283,00"  "33885,00"  "30885,00"  "34390,00"  "#¡NULO!"   "35104,00"
> [19] "32154,00"  "39554,00"  "36246,00"  "7226,00"   "40595,00"  "36674,00"
> [25] "46877,00"  "45278,00"  "47273,00"  "44755,00"  "42184,00"  "38457,00"
> [31] "41808,00"  "39004,00"  "#¡NULO!"   "#¡NULO!"   "#¡NULO!"   "#¡NULO!"
> [37] "2351,00"   "2982,00"   "2080,00"   "2331,00"   "2562,00"   "2479,00"
>
> > data.class(SS)
> [1] "character"
>
> > as.numeric(as.character(SS))
>  [1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
> NA NA
> [26] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
>
> [[alternative HTML version deleted]]
>
> ___
> R-help-es mailing list
> R-help-es@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-help-es
>
>
> [[alternative HTML version deleted]]
>
> ___
> R-help-es mailing list
> R-help-es@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-help-es

[[alternative HTML version deleted]]

___
R-help-es mailing list
R-help-es@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-help-es

Re: [R] Memory problem

[Ista Zahn]

Not conveniently. Memory is cheap, you should buy more.

Best,
Ista

On Nov 22, 2016 12:19 PM, "Partha Sinha"  wrote:

>  I am using R 3.3.2 on win 7, 32 bit with 2gb Ram. Is it possible to use
> more than 2 Gb data set ?
>
> Regards
> Partha
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] The code itself disappears after starting to execute the for loop

[Maram SAlem]

Thanks for helping Jim. 

I'm actually using the pbapply function together with the print function within 
a loop. In earlier versions, the progress bar and the output of the print 
function used to appear after each iteration of the loop. But with the 3.3.1. 
Version nothing appears, instead the console turns white and the cursor turns 
blue ( busy) and I know nothing about the progress of the running code.

I just want to see the bar and the output of the print function as I used to, 
any help?

Thanks in advance.
Maram Salem



Sent from my iPhone

> On Nov 3, 2016, at 8:30 PM, jim holtman  wrote:
> 
> A little more information would help.  How exactly are out creating the 
> output to the console?  Are you using 'print', 'cat' or something else?  Do 
> you have buffered output checked on the GUI (you probably don't want it 
> checked or you output will be delayed till the buffer is full -- this might 
> be the cause of your problem.
> 
> 
> Jim Holtman
> Data Munger Guru
>  
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
> 
>> On Thu, Nov 3, 2016 at 1:55 PM, Maram SAlem  
>> wrote:
>> Hi all,
>> 
>> I've a question concerning the R 3.3.1 version. I have a long code that I 
>> used to run on versions earlier to the 3.3.1 version, and when I copied the 
>> code to the R console, I can still see the code while the loop is executing 
>> , along with the output printed after each iteration of the loop.
>> 
>> Now, on the 3.3.1 version, after I copy the code to the console, it 
>> disappears and I only see the printed output of only one iteration at a 
>> time, that is, after the first iteration the printed output disappears ( 
>> though it's only 6 lines, just giving me some guidance, not a long output).
>> This is causing me some problems, so I don't know if there is a general 
>> option for R that enables me to still see the code and the output of all the 
>> iterations till the loop is over, as was the case with earlier R versions.
>> 
>> I didn't include the code as it's a long one.
>> 
>> Thanks a lot in advance,
>> 
>> Maram
>> 
>> 
>> Sent from my iPhone
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> 

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory problem

[Marcus Nunes]

Yes.

If you cannot read the dataset with the usual means, using functions like
read.table or read.csv, try the ff package: https://cran.r-
project.org/web/packages/ff/index.html.

Best,

On Tue, Nov 22, 2016 at 2:16 PM, Partha Sinha  wrote:

>  I am using R 3.3.2 on win 7, 32 bit with 2gb Ram. Is it possible to use
> more than 2 Gb data set ?
>
> Regards
> Partha
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Marcus Nunes
http://marcusnunes.me/

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory problem

[Bert Gunter]

Depends how you use it. e.g. it can be stored on disk and worked with
in pieces. Or some packages work with virtual memory, I believe.

However, it is certainly not possible to read it into R. In fact, you
probably won't be able to handle more (and maybe much less) than about
500 mb in R.

Cheers,
Bert
Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Nov 22, 2016 at 9:16 AM, Partha Sinha  wrote:
>  I am using R 3.3.2 on win 7, 32 bit with 2gb Ram. Is it possible to use
> more than 2 Gb data set ?
>
> Regards
> Partha
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Memory problem

[Partha Sinha]

 I am using R 3.3.2 on win 7, 32 bit with 2gb Ram. Is it possible to use
more than 2 Gb data set ?

Regards
Partha

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Getting the index of specific quantiles

[Matteo Richiardi]

Thanks Sarah (and all the others who replied) for your precious
suggestions! Matteo

On 22 November 2016 at 14:18, Sarah Goslee  wrote:
> Here's how to get one:
>
> x <- c(9,9,1,3,2,7,6,10,5,6)
>> which.min(abs(x - quantile(x, .25)))
> [1] 4
>
> And here's one of the various ways to get the entire set:
>
>> xq <- quantile(x)
>> sapply(xq, function(y)which.min(abs(x - y)))
>   0%  25%  50%  75% 100%
>34718
>
> Sarah
>
> On Tue, Nov 22, 2016 at 7:21 AM, Matteo Richiardi
>  wrote:
>> Dear R-users,
>> a very easy one for you, I guess. I need to extract the indexes of the
>> elements corresponding to different quantiles of a vector. When a
>> quantile is an interpolation between two adjacent values, I need the
>> index of the value which is closer (the lower value - or the higher
>> value for what matters - in case the quantile is exactly half way
>> through).
>>
>> This is an example.
>>
>>> x <- c(9,9,1,3,2,7,6,10,5,6)
>>> quantile(x)
>>   0%  25%  50%  75% 100%
>>  1.0  3.5  6.0  8.5 10.0
>>
>> What I need is a vector 'index' which looks like
>>
>>> index
>> 3   4   7   1   8
>>
>> Many thanks for your help ! Matteo
>>
> --
> Sarah Goslee
> http://www.functionaldiversity.org

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Getting the index of specific quantiles

[William Dunlap via R-help]

You might try something like
   x <- c(9,9,1,3,2,7,6,10,5,6)
   p <- (0:4)/4
   order(x) [ quantile(seq_along(x),p, type=1) ]
   # [1] 3 4 7 1 8
Selecting which value of 'type' works makes my head hurt.
You could also use 1+(p*(length(x)-1) as the index into order(x).


Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Tue, Nov 22, 2016 at 4:21 AM, Matteo Richiardi <
matteo.richia...@gmail.com> wrote:

> Dear R-users,
> a very easy one for you, I guess. I need to extract the indexes of the
> elements corresponding to different quantiles of a vector. When a
> quantile is an interpolation between two adjacent values, I need the
> index of the value which is closer (the lower value - or the higher
> value for what matters - in case the quantile is exactly half way
> through).
>
> This is an example.
>
> > x <- c(9,9,1,3,2,7,6,10,5,6)
> > quantile(x)
>   0%  25%  50%  75% 100%
>  1.0  3.5  6.0  8.5 10.0
>
> What I need is a vector 'index' which looks like
>
> > index
> 3   4   7   1   8
>
> Many thanks for your help ! Matteo
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Getting the index of specific quantiles

[David Winsemius]

> On Nov 22, 2016, at 4:21 AM, Matteo Richiardi  
> wrote:
> 
> Dear R-users,
> a very easy one for you, I guess. I need to extract the indexes of the
> elements corresponding to different quantiles of a vector. When a
> quantile is an interpolation between two adjacent values, I need the
> index of the value which is closer (the lower value - or the higher
> value for what matters - in case the quantile is exactly half way
> through).
> 
> This is an example.
> 
>> x <- c(9,9,1,3,2,7,6,10,5,6)
>> quantile(x)
>  0%  25%  50%  75% 100%
> 1.0  3.5  6.0  8.5 10.0
> 
> What I need is a vector 'index' which looks like
> 
>> index
> 3   4   7   1   8
> 

Try this (calculate distances with `outer` and then sweep the columns with 
`apply`)

> dqt <- outer(x, quantile(x), function(x,y) sqrt( (x-y)^2) )
> apply(dqt, 2, which.min)
  0%  25%  50%  75% 100% 
   34718 

-- 
David.
> Many thanks for your help ! Matteo
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Fast continious wavelet transformation (CWT) and plot?

[stvienna wiener]

Dear all,

This is my test code:

library(wmtsa)
sunspotsLong <- rep(sunspots, times=3000) ## try "times=30" (or 300)

sunspots.cwt <- wavCWT(sunspotsLong)
plot(sunspots.cwt, series=TRUE)

If you adapt times in the second line with "30", the code works. But 300 or
3000 not so much.

Are there more faster ways to calculate the CWT? Different packages that
are better for the continious wavelet transform?

My real time series has around 1 Million observations.


Stefan

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Getting the index of specific quantiles

[Sarah Goslee]

Here's how to get one:

x <- c(9,9,1,3,2,7,6,10,5,6)
> which.min(abs(x - quantile(x, .25)))
[1] 4

And here's one of the various ways to get the entire set:

> xq <- quantile(x)
> sapply(xq, function(y)which.min(abs(x - y)))
  0%  25%  50%  75% 100%
   34718

Sarah

On Tue, Nov 22, 2016 at 7:21 AM, Matteo Richiardi
 wrote:
> Dear R-users,
> a very easy one for you, I guess. I need to extract the indexes of the
> elements corresponding to different quantiles of a vector. When a
> quantile is an interpolation between two adjacent values, I need the
> index of the value which is closer (the lower value - or the higher
> value for what matters - in case the quantile is exactly half way
> through).
>
> This is an example.
>
>> x <- c(9,9,1,3,2,7,6,10,5,6)
>> quantile(x)
>   0%  25%  50%  75% 100%
>  1.0  3.5  6.0  8.5 10.0
>
> What I need is a vector 'index' which looks like
>
>> index
> 3   4   7   1   8
>
> Many thanks for your help ! Matteo
>
-- 
Sarah Goslee
http://www.functionaldiversity.org

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Breaking down a list into a table

[Ulrik Stervbo]

Hi Ferri,

It sounds like the function 'separate' from the tidyr package is what you
look for,

HTH
Ulrik

On Tue, 22 Nov 2016 at 14:49 Ferri Leberl  wrote:



Dear All,
I asked for support to deal with a hirarchy within a character separated
list.
I solved the problem crudely but effectively by

- Choosing for a TSV as input, where in columns that may contain several
(or as well no) items the items are separated via semicolon
- adding semicolons to the first row to grant that the first row has the
maximum number of semicolons of this column
- grasping the column(x<-myarray[,y], where y is some integer value) and
saving it as a TSV (with only one column)
- importing it again, defining it semicolumn-separated, with fill option

To all those who feel pain reading this: Is there a shortcut?
Thank you in advance.
Yours, Ferri

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Breaking down a list into a table

[Ferri Leberl]

Dear All,
I asked for support to deal with a hirarchy within a character separated list.
I solved the problem crudely but effectively by

- Choosing for a TSV as input, where in columns that may contain several (or as 
well no) items the items are separated via semicolon
- adding semicolons to the first row to grant that the first row has the 
maximum number of semicolons of this column
- grasping the column(x<-myarray[,y], where y is some integer value) and saving 
it as a TSV (with only one column)
- importing it again, defining it semicolumn-separated, with fill option

To all those who feel pain reading this: Is there a shortcut?
Thank you in advance.
Yours, Ferri

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Getting the index of specific quantiles

[Matteo Richiardi]

Dear R-users,
a very easy one for you, I guess. I need to extract the indexes of the
elements corresponding to different quantiles of a vector. When a
quantile is an interpolation between two adjacent values, I need the
index of the value which is closer (the lower value - or the higher
value for what matters - in case the quantile is exactly half way
through).

This is an example.

> x <- c(9,9,1,3,2,7,6,10,5,6)
> quantile(x)
  0%  25%  50%  75% 100%
 1.0  3.5  6.0  8.5 10.0

What I need is a vector 'index' which looks like

> index
3   4   7   1   8

Many thanks for your help ! Matteo

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] how to use vector of values to change row order of a heatmap

[PIKAL Petr]

Hi

Heatmap does not work with data frames (at least the version I have)

> b=as.data.frame(matrix(c(3,4,5,8,9,10,13,14,15,27,19,20),3,4))
> heatmap(b)
Error in heatmap(b) : 'x' must be a numeric matrix

With matrix, heatmap works as expected.

> b=matrix(c(3,4,5,8,9,10,13,14,15,27,19,20),3,4)
> heatmap(b)

However for row ordering you need to read Details of help page.

compare

> heatmap(b, Rowv=c(3,2,1))

with

> heatmap(b, Rowv=c(10,2,1))

If either is a vector (of ‘weights’) then the appropriate dendrogram is 
reordered according to the supplied values subject to the constraints imposed 
by the dendrogram, by reorder(dd, Rowv), in the row case. If either is
^
missing, as by default, then the ordering of the corresponding dendrogram is by 
the mean value of the rows/columns, i.e., in the case of rows, Rowv <- 
rowMeans(x, na.rm = na.rm). If either is NA, no reordering will be done for the 
corresponding side.

Cheers
Petr



> -Original Message-
> From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Fix Ace via
> R-help
> Sent: Monday, November 21, 2016 9:14 PM
> To: r-help@r-project.org
> Subject: [R] how to use vector of values to change row order of a heatmap
>
> Hello, there,
> R document for heatmap says that Rowv could be a vector of values to
> specify the row order. However, I couldn't figure out how to apply it. A
> simple example here:>
> b=as.data.frame(matrix(c(3,4,5,8,9,10,13,14,15,27,19,20),3,4))
> > b
>   V1 V2 V3 V4
> 1  3  8 13 27
> 2  4  9 14 19
> 3  5 10 15 20
> > row.names(b)=c("a","b","c")
> > b
>   V1 V2 V3 V4
> a  3  8 13 27
> b  4  9 14 19
> c  5 10 15 20
> > heatmap(as.matrix(b))
> What I got: "a" stays at the bottom of the heatmap.
>  Now I would like to put row "a" to the top row, how do I do that?I tried
> provide a vector of values (all the possible combination of 1,2,3) to Rowv,  
> "a"
> is always stay at the bottom Any input would be very helpful!
> Thanks.
> Ace
>
>   [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.


Tento e-mail a jakékoliv k němu připojené dokumenty jsou důvěrné a jsou určeny 
pouze jeho adresátům.
Jestliže jste obdržel(a) tento e-mail omylem, informujte laskavě neprodleně 
jeho odesílatele. Obsah tohoto emailu i s přílohami a jeho kopie vymažte ze 
svého systému.
Nejste-li zamýšleným adresátem tohoto emailu, nejste oprávněni tento email 
jakkoliv užívat, rozšiřovat, kopírovat či zveřejňovat.
Odesílatel e-mailu neodpovídá za eventuální škodu způsobenou modifikacemi či 
zpožděním přenosu e-mailu.

V případě, že je tento e-mail součástí obchodního jednání:
- vyhrazuje si odesílatel právo ukončit kdykoliv jednání o uzavření smlouvy, a 
to z jakéhokoliv důvodu i bez uvedení důvodu.
- a obsahuje-li nabídku, je adresát oprávněn nabídku bezodkladně přijmout; 
Odesílatel tohoto e-mailu (nabídky) vylučuje přijetí nabídky ze strany příjemce 
s dodatkem či odchylkou.
- trvá odesílatel na tom, že příslušná smlouva je uzavřena teprve výslovným 
dosažením shody na všech jejích náležitostech.
- odesílatel tohoto emailu informuje, že není oprávněn uzavírat za společnost 
žádné smlouvy s výjimkou případů, kdy k tomu byl písemně zmocněn nebo písemně 
pověřen a takové pověření nebo plná moc byly adresátovi tohoto emailu případně 
osobě, kterou adresát zastupuje, předloženy nebo jejich existence je adresátovi 
či osobě jím zastoupené známá.

This e-mail and any documents attached to it may be confidential and are 
intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender. 
Delete the contents of this e-mail with all attachments and its copies from 
your system.
If you are not the intended recipient of this e-mail, you are not authorized to 
use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by 
modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a 
contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately 
accept such offer; The sender of this e-mail (offer) excludes any acceptance of 
the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an 
express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into 
any contracts on behalf of 

Re: [R] Melt and compute Max, Mean, Min

[PIKAL Petr]

Hi

Well, thean you could try as I sugested.

.mean <- apply(temp[,-(1:3)],1, mean, na.rm=T)
.max <- apply(temp[,-(1:3)],1, max, na.rm=T)
.min <- apply(temp[,-(1:3)],1, min, na.rm=T)
temp2 <- data.frame(temp[,1:3], maxim = .max, minim = .min, aver = .mean)

You should construct a cycle, read the year file to temp, add above lines, add 
year and rbind temp2 with previous result(s).

You could do it also manually but if I remember correctly you have plenty of 
files.

Cheers
Petr

From: Miluji Sb [mailto:miluj...@gmail.com]
Sent: Monday, November 21, 2016 6:15 PM
To: PIKAL Petr 
Subject: Re: [R] Melt and compute Max, Mean, Min

Hello Petr,

Thank you so much for your reply. Apologies for the HTML posting, there's 
something wrong with my email editor. My goal is to compute the maximum, 
minimum, and mean for each observation by year and then merge them, so they 
look like the following:

lat | lon | year | max | min | mean

Thanks again.

Sincerely,

Milu

On Mon, Nov 21, 2016 at 6:57 AM, PIKAL Petr 
> wrote:
Hi

see in line

From: Miluji Sb [mailto:miluj...@gmail.com]
Sent: Friday, November 18, 2016 3:57 PM
To: PIKAL Petr >
Cc: r-help mailing list >
Subject: Re: [R] Melt and compute Max, Mean, Min

If I do:

as.data.frame(apply(df[,-(1:3)],1, mean, na.rm=T))

is it possible to sequentially name the variables as "mean_1960", "max_1960". 
"min_1960", "mean_1961", "max_1961". "min_1961", ...?

But here you have only mean. How do you want to add min or max?


On Fri, Nov 18, 2016 at 3:10 PM, Miluji Sb 
> wrote:
Dear Petr,

Thank you for the code, apologies though as I copied the wrong data, This is 
precipitation data and not temperature.

For the loop, could I do something like this?

filelist <- list.files(pattern=".csv")


Not exactly.

In this case I would use for cycle. It is quite easy to do something like:

for( i in 1:length(filelist)) {

temp<-read.csv(filelist[i]) #you need to read your file in R first

.mean <- apply(temp[,-(1:3)],1, mean, na.rm=T)
.max <- apply(temp[,-(1:3)],1, max, na.rm=T)
.min <- apply(temp[,-(1:3)],1, min, na.rm=T)

# now you can concatenate those results as you wish, name them or anything. It 
is difficult to suggest any direct code as you did not disclose what do you 
want to do with summaries further.

}

And BTW, please, do not post in HTML.



myDTs <- lapply(filelist, function(.file) {

apply(temp[,-(1:3)],1, mean, na.rm=T)

}

Thanks again!

Sincerely,

Milu


On Fri, Nov 18, 2016 at 2:46 PM, PIKAL Petr 
> wrote:
Hi

I am not completely sure what you want to do but

> apply(temp[,-(1:3)],1, mean, na.rm=T)
   12345
 NaN  NaN 2.159516 1.519914 1.514007
> apply(temp[,-(1:3)],1, max, na.rm=T)
   12345
-Inf -Inf 57.36528 39.45348 45.23904
Warning messages:
1: In FUN(newX[, i], ...) :
  no non-missing arguments to max; returning -Inf
2: In FUN(newX[, i], ...) :
  no non-missing arguments to max; returning -Inf
> apply(temp[,-(1:3)],1, min, na.rm=T)
  1   2   3   4   5
Inf Inf   0   0   0

gives you mentioned summary for each row. If you have duplicate rows you shall 
first aggregate them. However, it seems to me that your data are not correct. 
It is quite strange that for given lat/lon you have one day value 23 and the 
next day 0.

temp[1:5, 1:10]
  ISO3 lon lat day_1 day_2day_3 day_4 day_5 day_6day_7
1  CHL -69 -55NANA   NANANANA   NA
2  CHL -68 -55NANA   NANANANA   NA
3  CHL -72 -54 0 0  0.0 0 0 0  2.83824
4  -71 -54 0 0 23.37984 0 0 0 11.80116
5  CHL -70 -54 0 0  0.0 0 0 0  1.24956

If you want to process all your files you can do it in cycle. The function

list.files()

can be handy for that task.

Cheers
Petr

> -Original Message-
> From: R-help 
> [mailto:r-help-boun...@r-project.org] On 
> Behalf Of Miluji Sb
> Sent: Friday, November 18, 2016 1:49 PM
> To: r-help mailing list >
> Subject: [R] Melt and compute Max, Mean, Min
>
> Dear all,
>
> I have 51 years of data (1960 - 2010) in csv format, where each file 
> represents
> one year of data. Below is what each file looks like.
>
> These are temperature data by coordinates, my goal is to to compute max,
> min, and mean by year for each of the coordinates and construct a panel
> dataset. Any help will be appreciated, thank you!
>
> Sincerely,
>
> Milu
>
> temp <- dput(head(df,5))
> structure(list(ISO3 = structure(c(28L, 28L, 28L, NA, 28L), .Label = c("AFG",
> "AGO", "ALB", "ARE", "ARG", "ARM", "AUS", "AUT", 

Re: [R] using conditionals to select rows in data.table

[Martin Maechler]

Dear Carl,

this came through fine, as text only
... but then I did not see any question anymore.

Best regards,
Martin Maechler
(R core and mailing list "operator")

> Carl Sutton via R-help 
> on Tue, 22 Nov 2016 05:38:49 + writes:

> Hopefully the attached is text and not html.  I have not
> found a text option in firefox.  I have also been informed
> that a windows cut and paste is not truly text, which is
> the reason for the attachment from notepad++

> If there is any way I can improve the question, please
> inform of the problem and I will do what I can to correct
> it. Carl Sutton
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and
> more, see https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.