Re: [R] cbind giving NA's?
Note that this issue was raised on StackOverflow recently. http://stackoverflow.com/questions/7678090/xts-merge-odd-behaviour Here's the solution: index(a) - index(a) index(b) - index(b) merge(a,b) ZWD.UGX SCHB.Close 2010-03-31 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 28.30 2010-04-05 7.4040 28.38 2010-04-06 7.3317 28.21 2010-04-07 NA 28.31 2010-04-08 NA 28.47 Jeff's answer on StackOverflow explains why this works. Best, -- Joshua Ulrich | FOSS Trading: www.fosstrading.com On Fri, Aug 26, 2011 at 8:09 AM, R. Michael Weylandt michael.weyla...@gmail.com wrote: This seems to be the easiest way to handle the problem: a = xts(coredata(a), time(a)) b = xts(coredata(b), time(b)) merge(a,b) ZWD.UGX SCHB.Close 2010-03-31 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 28.30 2010-04-05 7.4040 28.38 2010-04-06 7.3317 28.21 2010-04-07 NA 28.31 2010-04-08 NA 28.47 I can't be sure where the problem was coming from, but if you look closely into the mumbo jumbo of the dput() code you'll see that the time indices don't actually match anywhere as POSIXct/POSIXt objects. Even though they map to the same day, something or other is keeping them as internally different and it's probably inherited from the data sources -- one set maps to times captured at 2300 EDT on my machine and the other to 0800 EDT. If you want, it might be interesting to track down the difference, but the fix I gave above, which basically pulls the time data out through the xts class and only keeps the date in creating a new xts that's exactly the same should handle it. In short, it's all black magic derived from R's various time/date mechanisms. If you don't really want to see whats under the hood, this will also work in this case. a = as.xts(a) b = as.xts(b) merge(a, b) But this can lead to some strange code if you don't take a moment to think about it is.xts(a) TRUE identical(a, as.xts(a)) FALSE Hope this helps and thanks for the fun problem! Michael Weylandt On Fri, Aug 26, 2011 at 8:08 AM, Petr PIKAL petr.pi...@precheza.cz wrote: I was rather too quick It has probably something to do with versions of zoo and xts after updating to zoo 1.7.4 and xts 0.8.2 I got with your examples merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 but when I did a-as.zoo(a) b-as.zoo(b) I got merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 7.6343 28.02 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 7.4040 28.30 2010-04-06 7.3317 28.38 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 which is probably what you want. Regards Petr Hi On 26 August 2011 03:37, R. Michael Weylandt michael.weyla...@gmail.com wrote: If you could, dput() them so we can see everything about them. You also might see if merge() gives you more expected behavior Ok... dput(a) structure(c(7.6343, 7.6343, 7.5458, 7.4532, 7.404, 7.3317), class = c(xts, zoo), .indexCLASS = Date, .indexTZ = , index = structure(c(1270105200, 1270191600, 1270278000, 1270364400, 1270450800, 1270537200), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, ZWD.UGX)) dput(b) structure(c(28.02, 28.3, 28.38, 28.21, 28.31, 28.47), .indexCLASS = Date, .indexTZ = , src = yahoo, updated = structure(1314356091.21457, class = c(POSIXct, POSIXt)), class = c(xts, zoo), index = structure(c(1270072800, 1270418400, 1270504800, 1270591200, 1270677600, 1270764000), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, SCHB.Close)) merge(a,b) merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 q() I get slightly different result xx-(merge(a,b)) xx ZWD.UGX SCHB.Close 1270072800 NA 28.02 1270105200 7.6343 NA 1270191600 7.6343 NA 1270278000 7.5458 NA
[R] cbind giving NA's?
I have two xts objects, call them a and b, and am trying to merge them... class(a) [1] xts zoo class(b) [1] xts zoo head(a) 2010-04-01 7.6343 2010-04-02 7.6343 2010-04-03 7.5458 2010-04-04 7.4532 2010-04-05 7.4040 2010-04-06 7.3317 head(b) 2010-04-01 568.80 2010-04-05 571.01 2010-04-06 568.22 2010-04-07 563.54 2010-04-08 567.49 2010-04-09 566.22 head(cbind(a,b)) 2010-04-01 568.80 NA 2010-04-01 NA 0.3998 2010-04-02 NA 0.3998 2010-04-03 NA 0.3999 2010-04-04 NA 0.3994 2010-04-05 571.01 NA q() I'd like the cbind'ed version to look like: 2010-04-01 568.800.3998 2010-04-02 NA 0.3998 2010-04-03 NA 0.3999 2010-04-04 NA 0.3994 2010-04-05 571.01 0.3991 What gives? Many thanks, in advance. -- H -- Sent from my mobile device Envoyait de mon telephone mobil __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] cbind giving NA's?
Hmm, that's quite puzzling. I don't know but I'd willing to guess the time/date stamps on a,b are more different than the output is leading us to believe. My experience is that there's always more to a time/date object to trick us up than one would expect. If you could, dput() them so we can see everything about them. You also might see if merge() gives you more expected behavior Michael Weylandt On Fri, Aug 26, 2011 at 6:05 AM, Hasan Diwan hasan.di...@gmail.com wrote: I have two xts objects, call them a and b, and am trying to merge them... class(a) [1] xts zoo class(b) [1] xts zoo head(a) 2010-04-01 7.6343 2010-04-02 7.6343 2010-04-03 7.5458 2010-04-04 7.4532 2010-04-05 7.4040 2010-04-06 7.3317 head(b) 2010-04-01 568.80 2010-04-05 571.01 2010-04-06 568.22 2010-04-07 563.54 2010-04-08 567.49 2010-04-09 566.22 head(cbind(a,b)) 2010-04-01 568.80 NA 2010-04-01 NA 0.3998 2010-04-02 NA 0.3998 2010-04-03 NA 0.3999 2010-04-04 NA 0.3994 2010-04-05 571.01 NA q() I'd like the cbind'ed version to look like: 2010-04-01 568.800.3998 2010-04-02 NA 0.3998 2010-04-03 NA 0.3999 2010-04-04 NA 0.3994 2010-04-05 571.01 0.3991 What gives? Many thanks, in advance. -- H -- Sent from my mobile device Envoyait de mon telephone mobil __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] cbind giving NA's?
On 26 August 2011 03:37, R. Michael Weylandt michael.weyla...@gmail.com wrote: If you could, dput() them so we can see everything about them. You also might see if merge() gives you more expected behavior Ok... dput(a) structure(c(7.6343, 7.6343, 7.5458, 7.4532, 7.404, 7.3317), class = c(xts, zoo), .indexCLASS = Date, .indexTZ = , index = structure(c(1270105200, 1270191600, 1270278000, 1270364400, 1270450800, 1270537200), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, ZWD.UGX)) dput(b) structure(c(28.02, 28.3, 28.38, 28.21, 28.31, 28.47), .indexCLASS = Date, .indexTZ = , src = yahoo, updated = structure(1314356091.21457, class = c(POSIXct, POSIXt)), class = c(xts, zoo), index = structure(c(1270072800, 1270418400, 1270504800, 1270591200, 1270677600, 1270764000), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, SCHB.Close)) merge(a,b) merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 q() So, no, merge doesn't work... -- Sent from my mobile device Envoyait de mon telephone mobil __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] cbind giving NA's?
Hi On 26 August 2011 03:37, R. Michael Weylandt michael.weyla...@gmail.com wrote: If you could, dput() them so we can see everything about them. You also might see if merge() gives you more expected behavior Ok... dput(a) structure(c(7.6343, 7.6343, 7.5458, 7.4532, 7.404, 7.3317), class = c(xts, zoo), .indexCLASS = Date, .indexTZ = , index = structure(c(1270105200, 1270191600, 1270278000, 1270364400, 1270450800, 1270537200), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, ZWD.UGX)) dput(b) structure(c(28.02, 28.3, 28.38, 28.21, 28.31, 28.47), .indexCLASS = Date, .indexTZ = , src = yahoo, updated = structure(1314356091.21457, class = c(POSIXct, POSIXt)), class = c(xts, zoo), index = structure(c(1270072800, 1270418400, 1270504800, 1270591200, 1270677600, 1270764000), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, SCHB.Close)) merge(a,b) merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 q() I get slightly different result xx-(merge(a,b)) xx ZWD.UGX SCHB.Close 1270072800 NA 28.02 1270105200 7.6343 NA 1270191600 7.6343 NA 1270278000 7.5458 NA 1270364400 7.4532 NA 1270418400 NA 28.30 1270450800 7.4040 NA 1270504800 NA 28.38 1270537200 7.3317 NA 1270591200 NA 28.21 1270677600 NA 28.31 1270764000 NA 28.47 but from what you want it seems to me that aggregate could be suitable second step. something like aggregate(xx, list(those dates you have but I don't), mean, na.rm=T) could do what you want. Regards Petr So, no, merge doesn't work... -- Sent from my mobile device Envoyait de mon telephone mobil __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] cbind giving NA's?
Hi: Try this: require('xts') merge.zoo(zoo(a), zoo(b), all = c(TRUE, TRUE)) ZWD.UGX SCHB.Close 2010-03-31 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 28.30 2010-04-05 7.4040 28.38 2010-04-06 7.3317 28.21 2010-04-07 NA 28.31 2010-04-08 NA 28.47 HTH, Dennis On Fri, Aug 26, 2011 at 3:58 AM, Hasan Diwan hasan.di...@gmail.com wrote: On 26 August 2011 03:37, R. Michael Weylandt michael.weyla...@gmail.com wrote: If you could, dput() them so we can see everything about them. You also might see if merge() gives you more expected behavior Ok... dput(a) structure(c(7.6343, 7.6343, 7.5458, 7.4532, 7.404, 7.3317), class = c(xts, zoo), .indexCLASS = Date, .indexTZ = , index = structure(c(1270105200, 1270191600, 1270278000, 1270364400, 1270450800, 1270537200), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, ZWD.UGX)) dput(b) structure(c(28.02, 28.3, 28.38, 28.21, 28.31, 28.47), .indexCLASS = Date, .indexTZ = , src = yahoo, updated = structure(1314356091.21457, class = c(POSIXct, POSIXt)), class = c(xts, zoo), index = structure(c(1270072800, 1270418400, 1270504800, 1270591200, 1270677600, 1270764000), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, SCHB.Close)) merge(a,b) merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 q() So, no, merge doesn't work... -- Sent from my mobile device Envoyait de mon telephone mobil __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] cbind giving NA's?
I was rather too quick It has probably something to do with versions of zoo and xts after updating to zoo 1.7.4 and xts 0.8.2 I got with your examples merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 but when I did a-as.zoo(a) b-as.zoo(b) I got merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 7.6343 28.02 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 7.4040 28.30 2010-04-06 7.3317 28.38 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 which is probably what you want. Regards Petr Hi On 26 August 2011 03:37, R. Michael Weylandt michael.weyla...@gmail.com wrote: If you could, dput() them so we can see everything about them. You also might see if merge() gives you more expected behavior Ok... dput(a) structure(c(7.6343, 7.6343, 7.5458, 7.4532, 7.404, 7.3317), class = c(xts, zoo), .indexCLASS = Date, .indexTZ = , index = structure(c(1270105200, 1270191600, 1270278000, 1270364400, 1270450800, 1270537200), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, ZWD.UGX)) dput(b) structure(c(28.02, 28.3, 28.38, 28.21, 28.31, 28.47), .indexCLASS = Date, .indexTZ = , src = yahoo, updated = structure(1314356091.21457, class = c(POSIXct, POSIXt)), class = c(xts, zoo), index = structure(c(1270072800, 1270418400, 1270504800, 1270591200, 1270677600, 1270764000), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, SCHB.Close)) merge(a,b) merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 q() I get slightly different result xx-(merge(a,b)) xx ZWD.UGX SCHB.Close 1270072800 NA 28.02 1270105200 7.6343 NA 1270191600 7.6343 NA 1270278000 7.5458 NA 1270364400 7.4532 NA 1270418400 NA 28.30 1270450800 7.4040 NA 1270504800 NA 28.38 1270537200 7.3317 NA 1270591200 NA 28.21 1270677600 NA 28.31 1270764000 NA 28.47 but from what you want it seems to me that aggregate could be suitable second step. something like aggregate(xx, list(those dates you have but I don't), mean, na.rm=T) could do what you want. Regards Petr So, no, merge doesn't work... -- Sent from my mobile device Envoyait de mon telephone mobil __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] cbind giving NA's?
This seems to be the easiest way to handle the problem: a = xts(coredata(a), time(a)) b = xts(coredata(b), time(b)) merge(a,b) ZWD.UGX SCHB.Close 2010-03-31 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 28.30 2010-04-05 7.4040 28.38 2010-04-06 7.3317 28.21 2010-04-07 NA 28.31 2010-04-08 NA 28.47 I can't be sure where the problem was coming from, but if you look closely into the mumbo jumbo of the dput() code you'll see that the time indices don't actually match anywhere as POSIXct/POSIXt objects. Even though they map to the same day, something or other is keeping them as internally different and it's probably inherited from the data sources -- one set maps to times captured at 2300 EDT on my machine and the other to 0800 EDT. If you want, it might be interesting to track down the difference, but the fix I gave above, which basically pulls the time data out through the xts class and only keeps the date in creating a new xts that's exactly the same should handle it. In short, it's all black magic derived from R's various time/date mechanisms. If you don't really want to see whats under the hood, this will also work in this case. a = as.xts(a) b = as.xts(b) merge(a, b) But this can lead to some strange code if you don't take a moment to think about it is.xts(a) TRUE identical(a, as.xts(a)) FALSE Hope this helps and thanks for the fun problem! Michael Weylandt On Fri, Aug 26, 2011 at 8:08 AM, Petr PIKAL petr.pi...@precheza.cz wrote: I was rather too quick It has probably something to do with versions of zoo and xts after updating to zoo 1.7.4 and xts 0.8.2 I got with your examples merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 but when I did a-as.zoo(a) b-as.zoo(b) I got merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 7.6343 28.02 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 7.4040 28.30 2010-04-06 7.3317 28.38 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 which is probably what you want. Regards Petr Hi On 26 August 2011 03:37, R. Michael Weylandt michael.weyla...@gmail.com wrote: If you could, dput() them so we can see everything about them. You also might see if merge() gives you more expected behavior Ok... dput(a) structure(c(7.6343, 7.6343, 7.5458, 7.4532, 7.404, 7.3317), class = c(xts, zoo), .indexCLASS = Date, .indexTZ = , index = structure(c(1270105200, 1270191600, 1270278000, 1270364400, 1270450800, 1270537200), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, ZWD.UGX)) dput(b) structure(c(28.02, 28.3, 28.38, 28.21, 28.31, 28.47), .indexCLASS = Date, .indexTZ = , src = yahoo, updated = structure(1314356091.21457, class = c(POSIXct, POSIXt)), class = c(xts, zoo), index = structure(c(1270072800, 1270418400, 1270504800, 1270591200, 1270677600, 1270764000), tzone = , tclass = Date), .Dim = c(6L, 1L), .Dimnames = list(NULL, SCHB.Close)) merge(a,b) merge(a,b) ZWD.UGX SCHB.Close 2010-04-01 NA 28.02 2010-04-01 7.6343 NA 2010-04-02 7.6343 NA 2010-04-03 7.5458 NA 2010-04-04 7.4532 NA 2010-04-05 NA 28.30 2010-04-05 7.4040 NA 2010-04-06 NA 28.38 2010-04-06 7.3317 NA 2010-04-07 NA 28.21 2010-04-08 NA 28.31 2010-04-09 NA 28.47 q() I get slightly different result xx-(merge(a,b)) xx ZWD.UGX SCHB.Close 1270072800 NA 28.02 1270105200 7.6343 NA 1270191600 7.6343 NA 1270278000 7.5458 NA 1270364400 7.4532 NA 1270418400 NA 28.30 1270450800 7.4040 NA 1270504800 NA 28.38 1270537200 7.3317 NA 1270591200 NA 28.21 1270677600 NA 28.31 1270764000 NA 28.47 but from what you want it seems to me that aggregate could be suitable second step. something like aggregate(xx, list(those dates you have but I don't), mean, na.rm=T) could do what you want. Regards Petr So, no, merge doesn't work... -- Sent from my mobile device Envoyait de mon telephone mobil __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do