Re: [R] Calculate daily means from 5-minute interval data
IMO assuming periodicity is a bad practice for this. Missing timestamps happen too, and there is no reason to build a broken analysis process. On August 29, 2021 7:09:01 PM PDT, Richard O'Keefe wrote: >Why would you need a package for this? >> samples.per.day <- 12*24 > >That's 12 5-minute intervals per hour and 24 hours per day. >Generate some fake data. > >> x <- rnorm(samples.per.day * 365) >> length(x) >[1] 105120 > >Reshape the fake data into a matrix where each row represents one >24-hour period. > >> m <- matrix(x, ncol=samples.per.day, byrow=TRUE) > >Now we can summarise the rows any way we want. >The basic tool here is ?apply. >?rowMeans is said to be faster than using apply to calculate means, >so we'll use that. There is no *rowSds so we have to use apply >for the standard deviation. I use ?head because I don't want to >post tens of thousands of meaningless numbers. > >> head(rowMeans(m)) >[1] -0.03510177 0.11817337 0.06725203 -0.03578195 -0.02448077 -0.03033692 >> head(apply(m, MARGIN=1, FUN=sd)) >[1] 1.0017718 0.9922920 1.0100550 0.9956810 1.0077477 0.9833144 > >Now whether this is a *sensible* way to summarise your flow data is a question >that a hydrologist would be better placed to answer. I would have started with >> plot(density(x)) >which I just did with some real river data (only a month of it, sigh). >Very long tail. >Even >> plot(density(log(r))) >shows a very long tail. Time to plot the data against time. Oh my! >All of the long tail came from a single event. >There's a period of low flow, then there's a big rainstorm and the >flow goes WAY up, then over about two days the flow subsides to a new >somewhat higher level. > >None of this is reflected in means or standard deviations. >This is *time series* data, and time series data of a fairly special kind. > >One thing that might be helpful with your data would simply be >> image(log(m)) >For my one month sample, the spike showed up very clearly that way. >Because right now, your first task is to get an idea of what the data >look like, and means-and-standard-deviations won't really do that. > >Oh heck, here's another reason to go with image(log(m)). >With image(m) I just see the one big spike. >With image(log(m)), I can see that little spikes often start in the >afternoon of one day and continue into the morning of the next. >From daily means, it looks like two unusual, but not very >unusual, days. From the image, it's clearly ONE rainfall event >that just happens to straddle a day boundary. > >This is all very basic stuff, which is really the point. You want to use >elementary tools to look at the data before you reach for fancy ones. > > >On Mon, 30 Aug 2021 at 03:09, Rich Shepard wrote: >> >> I have a year's hydraulic data (discharge, stage height, velocity, etc.) >> from a USGS monitoring gauge recording values every 5 minutes. The data >> files contain 90K-93K lines and plotting all these data would produce a >> solid block of color. >> >> What I want are the daily means and standard deviation from these data. >> >> As an occasional R user (depending on project needs) I've no idea what >> packages could be applied to these data frames. There likely are multiple >> paths to extracting these daily values so summary statistics can be >> calculated and plotted. I'd appreciate suggestions on where to start to >> learn how I can do this. >> >> TIA, >> >> Rich >> >> __ >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > >__ >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. -- Sent from my phone. Please excuse my brevity. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
Why would you need a package for this? > samples.per.day <- 12*24 That's 12 5-minute intervals per hour and 24 hours per day. Generate some fake data. > x <- rnorm(samples.per.day * 365) > length(x) [1] 105120 Reshape the fake data into a matrix where each row represents one 24-hour period. > m <- matrix(x, ncol=samples.per.day, byrow=TRUE) Now we can summarise the rows any way we want. The basic tool here is ?apply. ?rowMeans is said to be faster than using apply to calculate means, so we'll use that. There is no *rowSds so we have to use apply for the standard deviation. I use ?head because I don't want to post tens of thousands of meaningless numbers. > head(rowMeans(m)) [1] -0.03510177 0.11817337 0.06725203 -0.03578195 -0.02448077 -0.03033692 > head(apply(m, MARGIN=1, FUN=sd)) [1] 1.0017718 0.9922920 1.0100550 0.9956810 1.0077477 0.9833144 Now whether this is a *sensible* way to summarise your flow data is a question that a hydrologist would be better placed to answer. I would have started with > plot(density(x)) which I just did with some real river data (only a month of it, sigh). Very long tail. Even > plot(density(log(r))) shows a very long tail. Time to plot the data against time. Oh my! All of the long tail came from a single event. There's a period of low flow, then there's a big rainstorm and the flow goes WAY up, then over about two days the flow subsides to a new somewhat higher level. None of this is reflected in means or standard deviations. This is *time series* data, and time series data of a fairly special kind. One thing that might be helpful with your data would simply be > image(log(m)) For my one month sample, the spike showed up very clearly that way. Because right now, your first task is to get an idea of what the data look like, and means-and-standard-deviations won't really do that. Oh heck, here's another reason to go with image(log(m)). With image(m) I just see the one big spike. With image(log(m)), I can see that little spikes often start in the afternoon of one day and continue into the morning of the next. >From daily means, it looks like two unusual, but not very unusual, days. From the image, it's clearly ONE rainfall event that just happens to straddle a day boundary. This is all very basic stuff, which is really the point. You want to use elementary tools to look at the data before you reach for fancy ones. On Mon, 30 Aug 2021 at 03:09, Rich Shepard wrote: > > I have a year's hydraulic data (discharge, stage height, velocity, etc.) > from a USGS monitoring gauge recording values every 5 minutes. The data > files contain 90K-93K lines and plotting all these data would produce a > solid block of color. > > What I want are the daily means and standard deviation from these data. > > As an occasional R user (depending on project needs) I've no idea what > packages could be applied to these data frames. There likely are multiple > paths to extracting these daily values so summary statistics can be > calculated and plotted. I'd appreciate suggestions on where to start to > learn how I can do this. > > TIA, > > Rich > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] A glitch (???) in tools::texi2pf.
On Sat, 28 Aug 2021, Rolf Turner wrote: On Sat, 28 Aug 2021 12:49:04 +0300 Eric Berger wrote: As Achim wrote in point (2), Makefile is your friend. Well, all I can say is that Makefile is *not* my friend; I have never made its acquaintance and wouldn't know where to begin looking. Would it be possible for you to provide a template/prototype Makefile and give me some idea what to *do* with it? In a Makefile you can declare "rules" for producing certain "target" files from "prerequisite" files. The rules are only applied if any of the prerequisite files have changed since the target was last produced. In your case the .bib is a prerequisite for the target .bbl which in turn is a prerequisite (along with the .tex) for the .pdf. A nice introduction to GNU Make for data analysis workflows is this JSS paper by Peter Baker (http://petebaker.id.au/): https://doi.org/10.18637/jss.v094.c01 Best wishes, Z __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Upgraded to 4.1.1 assignment operator keys not working [RESOLVED]
On Sun, 29 Aug 2021, Rich Shepard wrote: But, ... help. Sigh. It helps to start ESS in emacs first. Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Upgraded to 4.1.1 assignment operator keys not working
Just upgraded to R-4.1.1-x86_64-1_SBo on Slackware-14.2/x86_64. While it's been a while since I last ran R trying 'Alt + -' to enter the assignment monitor is not working. Assuming the problem is with me I'm not finding what I'm doing incorrectly in all the docs I searched and I'm suitably embarrassed that I need to ask for help. But, ... help. Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
On Sun, 29 Aug 2021, Andrew Simmons wrote: I would suggest something like: Thanks, Andrew. Stay well, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
On Sun, 29 Aug 2021, Rui Barradas wrote: Hope this helps, Rui, Greatly! I'll study it carefully so I fully understand the process. Many thanks. Stay well, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
Hello, I forgot in my previous answer, sorry for the duplicated mails. The function in my previous mail has a na.rm argument, defaulting to FALSE, pass na.rm = TRUE to remove the NA's. agg <- aggregate(cfs ~ date, df1, fun, na.rm = TRUE) Or simply change the default. I prefer to set na.rm = FALSE because that's what R's functions do. And I will only be used to one default, with base R functions or my own code. Hope this helps, Rui Barradas Às 17:52 de 29/08/21, Rich Shepard escreveu: On Sun, 29 Aug 2021, Jeff Newmiller wrote: The general idea is to create a "grouping" column with repeated values for each day, and then to use aggregate to compute your combined results. The dplyr package's group_by/summarise functions can also do this, and there are also proponents of the data.table package which is high performance but tends to depend on altering data in-place unlike most other R data handling functions. Also pay attention to missing data... if you have any then you will need to consider whether you want the strictness of na.rm=FALSE or permissiveness of na.rm=TRUE for your aggregation functions. Jeff, Thank you. Yes, there are missing data as sometimes the equipment fails, or there's some other reason why some samples are missing. Grouping on each day is just what I need. I'll re-learn dplyr and take a look at data.table. Regards, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
On Sun, 29 Aug 2021, Jeff Newmiller wrote: You may find something useful on handling timestamp data here: https://jdnewmil.github.io/ Jeff, I'll certainly read those articles. Many thanks, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
Hello,
You have date and hour in two separate columns, so to compute daily
stats part of the work is already done. (Were they in the same column
you would have to extract the date only.)
# convert to class "Date"
df1$date <- as.Date(df1$date)
# function to compute the stats required
# it's important to note that all the stats
# are returned in a vector, see below
fun <- function(x, na.rm = FALSE){
c(mean_cfs = mean(x, na.rm = na.rm),
sd_cfs = sd(x, na.rm = na.rm))
}
# now this will put a *matrix* under cfs
# each row has the statistics computed
# by the function
agg <- aggregate(cfs ~ date, df1, fun)
str(agg)
#'data.frame': 1 obs. of 2 variables:
# $ date: Date, format: "2020-08-26"
# $ cfs : num [1, 1:2] 110400 16143
# ..- attr(*, "dimnames")=List of 2
# .. ..$ : NULL
# .. ..$ : chr [1:2] "mean_cfs" "sd_cfs"
# so now put everything in separate columns
agg <- cbind(agg[-ncol(agg)], agg[[ncol(agg)]])
str(agg)
#'data.frame': 1 obs. of 3 variables:
# $ date: Date, format: "2020-08-26"
# $ mean_cfs: num 110400
# $ sd_cfs : num 16143
Hope this helps,
Rui Barradas
Às 17:49 de 29/08/21, Rich Shepard escreveu:
On Sun, 29 Aug 2021, Eric Berger wrote:
Provide dummy data (e.g. 5-10 lines), say like the contents of a csv
file,
and calculate by hand what you'd like to see in the plot. (And describe
what the plot would look like.)
Eric,
Mea culpa! I extracted a set of sample data and forgot to include it in the
message. Here it is:
date,time,cfs
2020-08-26,09:30,136000
2020-08-26,09:35,126000
2020-08-26,09:40,13
2020-08-26,09:45,128000
2020-08-26,09:50,126000
2020-08-26,09:55,125000
2020-08-26,10:00,121000
2020-08-26,10:05,117000
2020-08-26,10:10,12
...
2020-08-26,23:10,108000
2020-08-26,23:15,96200
2020-08-26,23:20,86700
2020-08-26,23:25,103000
2020-08-26,23:30,103000
2020-08-26,23:35,99500
2020-08-26,23:40,85200
2020-08-26,23:45,103000
2020-08-26,23:50,95800
2020-08-26,23:55,88200
Rich
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
Hello,
I would suggest something like:
date <- seq(as.Date("2020-01-01"), as.Date("2020-12-31"), 1)
time <- sprintf("%02d:%02d", rep(0:23, each = 12), seq.int(0, 55, 5))
x <- data.frame(
date = rep(date, each = length(time)),
time = time
)
x$cfs <- stats::rnorm(nrow(x))
cols2aggregate <- "cfs" # add more as necessary
S <- split(x[cols2aggregate], x$date)
means <- do.call("rbind", lapply(S, colMeans, na.rm = TRUE))
sds <- do.call("rbind", lapply(S, function(xx) sapply(xx, sd, na.rm =
TRUE)))
On Sun, Aug 29, 2021 at 11:09 AM Rich Shepard
wrote:
> I have a year's hydraulic data (discharge, stage height, velocity, etc.)
> from a USGS monitoring gauge recording values every 5 minutes. The data
> files contain 90K-93K lines and plotting all these data would produce a
> solid block of color.
>
> What I want are the daily means and standard deviation from these data.
>
> As an occasional R user (depending on project needs) I've no idea what
> packages could be applied to these data frames. There likely are multiple
> paths to extracting these daily values so summary statistics can be
> calculated and plotted. I'd appreciate suggestions on where to start to
> learn how I can do this.
>
> TIA,
>
> Rich
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
On Sun, 29 Aug 2021, Rui Barradas wrote: I forgot in my previous answer, sorry for the duplicated mails. The function in my previous mail has a na.rm argument, defaulting to FALSE, pass na.rm = TRUE to remove the NA's. agg <- aggregate(cfs ~ date, df1, fun, na.rm = TRUE) Or simply change the default. I prefer to set na.rm = FALSE because that's what R's functions do. And I will only be used to one default, with base R functions or my own code. Hope this helps, Rui, Again, yes it does. Regards, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
You may find something useful on handling timestamp data here: https://jdnewmil.github.io/ On August 29, 2021 9:23:31 AM PDT, Jeff Newmiller wrote: >The general idea is to create a "grouping" column with repeated values for >each day, and then to use aggregate to compute your combined results. The >dplyr package's group_by/summarise functions can also do this, and there are >also proponents of the data.table package which is high performance but tends >to depend on altering data in-place unlike most other R data handling >functions. > >Also pay attention to missing data... if you have any then you will need to >consider whether you want the strictness of na.rm=FALSE or permissiveness of >na.rm=TRUE for your aggregation functions. > >On August 29, 2021 8:08:58 AM PDT, Rich Shepard >wrote: >>I have a year's hydraulic data (discharge, stage height, velocity, etc.) >>from a USGS monitoring gauge recording values every 5 minutes. The data >>files contain 90K-93K lines and plotting all these data would produce a >>solid block of color. >> >>What I want are the daily means and standard deviation from these data. >> >>As an occasional R user (depending on project needs) I've no idea what >>packages could be applied to these data frames. There likely are multiple >>paths to extracting these daily values so summary statistics can be >>calculated and plotted. I'd appreciate suggestions on where to start to >>learn how I can do this. >> >>TIA, >> >>Rich >> >>__ >>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >>https://stat.ethz.ch/mailman/listinfo/r-help >>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >>and provide commented, minimal, self-contained, reproducible code. > -- Sent from my phone. Please excuse my brevity. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
On Sun, 29 Aug 2021, Jeff Newmiller wrote: The general idea is to create a "grouping" column with repeated values for each day, and then to use aggregate to compute your combined results. The dplyr package's group_by/summarise functions can also do this, and there are also proponents of the data.table package which is high performance but tends to depend on altering data in-place unlike most other R data handling functions. Also pay attention to missing data... if you have any then you will need to consider whether you want the strictness of na.rm=FALSE or permissiveness of na.rm=TRUE for your aggregation functions. Jeff, Thank you. Yes, there are missing data as sometimes the equipment fails, or there's some other reason why some samples are missing. Grouping on each day is just what I need. I'll re-learn dplyr and take a look at data.table. Regards, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
On Sun, 29 Aug 2021, Eric Berger wrote: Provide dummy data (e.g. 5-10 lines), say like the contents of a csv file, and calculate by hand what you'd like to see in the plot. (And describe what the plot would look like.) Eric, Mea culpa! I extracted a set of sample data and forgot to include it in the message. Here it is: date,time,cfs 2020-08-26,09:30,136000 2020-08-26,09:35,126000 2020-08-26,09:40,13 2020-08-26,09:45,128000 2020-08-26,09:50,126000 2020-08-26,09:55,125000 2020-08-26,10:00,121000 2020-08-26,10:05,117000 2020-08-26,10:10,12 ... 2020-08-26,23:10,108000 2020-08-26,23:15,96200 2020-08-26,23:20,86700 2020-08-26,23:25,103000 2020-08-26,23:30,103000 2020-08-26,23:35,99500 2020-08-26,23:40,85200 2020-08-26,23:45,103000 2020-08-26,23:50,95800 2020-08-26,23:55,88200 Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
The general idea is to create a "grouping" column with repeated values for each day, and then to use aggregate to compute your combined results. The dplyr package's group_by/summarise functions can also do this, and there are also proponents of the data.table package which is high performance but tends to depend on altering data in-place unlike most other R data handling functions. Also pay attention to missing data... if you have any then you will need to consider whether you want the strictness of na.rm=FALSE or permissiveness of na.rm=TRUE for your aggregation functions. On August 29, 2021 8:08:58 AM PDT, Rich Shepard wrote: >I have a year's hydraulic data (discharge, stage height, velocity, etc.) >from a USGS monitoring gauge recording values every 5 minutes. The data >files contain 90K-93K lines and plotting all these data would produce a >solid block of color. > >What I want are the daily means and standard deviation from these data. > >As an occasional R user (depending on project needs) I've no idea what >packages could be applied to these data frames. There likely are multiple >paths to extracting these daily values so summary statistics can be >calculated and plotted. I'd appreciate suggestions on where to start to >learn how I can do this. > >TIA, > >Rich > >__ >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. -- Sent from my phone. Please excuse my brevity. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Calculate daily means from 5-minute interval data
Hi Rich, Your request is a bit open-ended but here's a suggestion that might help get you an answer. Provide dummy data (e.g. 5-10 lines), say like the contents of a csv file, and calculate by hand what you'd like to see in the plot. (And describe what the plot would look like.) It sounds like what you want could be done in a few lines of R code which would work both on the dummy data and the real data. HTH, Eric On Sun, Aug 29, 2021 at 6:09 PM Rich Shepard wrote: > I have a year's hydraulic data (discharge, stage height, velocity, etc.) > from a USGS monitoring gauge recording values every 5 minutes. The data > files contain 90K-93K lines and plotting all these data would produce a > solid block of color. > > What I want are the daily means and standard deviation from these data. > > As an occasional R user (depending on project needs) I've no idea what > packages could be applied to these data frames. There likely are multiple > paths to extracting these daily values so summary statistics can be > calculated and plotted. I'd appreciate suggestions on where to start to > learn how I can do this. > > TIA, > > Rich > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Calculate daily means from 5-minute interval data
I have a year's hydraulic data (discharge, stage height, velocity, etc.) from a USGS monitoring gauge recording values every 5 minutes. The data files contain 90K-93K lines and plotting all these data would produce a solid block of color. What I want are the daily means and standard deviation from these data. As an occasional R user (depending on project needs) I've no idea what packages could be applied to these data frames. There likely are multiple paths to extracting these daily values so summary statistics can be calculated and plotted. I'd appreciate suggestions on where to start to learn how I can do this. TIA, Rich __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Finding if numbers fall within a range
Thanks Jeff, It worked!! From: Jeff Newmiller Sent: Saturday 28 August 2021 16:32 To: r-help@r-project.org ; Eliza Botto ; r-help@r-project.org Subject: Re: [R] Finding if numbers fall within a range You messed up the dput somehow... but I think this works: m <- t(structure(c(1, 57, 59, 271, 279, 59, 179, 279, 278, 359, 52, 118, 178, 239, 334), .Dim = c(5L, 3L))) brk <- c( 0, 60, 120, 180, 240, 300 ) mc <- matrix( findInterval( m, brk ), ncol = ncol(m) ) m mc DBI <- apply( mc, 1, function(v) length( unique( v ) ) ) On August 28, 2021 8:57:09 AM PDT, Eliza Botto wrote: >Dear useRs, > >Is there a way in R to see if the numbers in a matrix-row fall within the >given range or not? For example, I have the following data; > >> dput(EB) > >structure(c(1, 57, 59, 271, 279, 59, 179, 279, 278, 359, 52, >118, 178, 239, 334), .Dim = c(3L, 5L)) > >The ranges for which these numbers are to be reserved are; > >0-60, 61-120, 121-180, 181-240, 241-300, 301-360 > >In row 1, the number "1", "57", and "59" fall 2ith the range 0-60. Whereas >"271" and "279" falls within the range 241-300. So in total 2 ranges are >covered and coding should execute 2 for row one. > >In row 2, the number "59" falls in the range 0-60, "179" falls in the range >121-180, "279" and "278" fall within the range 241-300, and 359 falls in the >range 301-360. In total 4 ranges are covered for row 2. > >In the like-wise pattern, 5 ranges are covered in row 3. > >So, what I want is a matrix that looks in the following > >> dput(EBI) > >structure(c(2, 4, 5), .Dim = c(3L, 1L)) > >Is there an easy way of doing it? > >I thank you all very much in advance > >EB > > [[alternative HTML version deleted]] > >__ >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code. -- Sent from my phone. Please excuse my brevity. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] coercion to an object...
dear Duncun,
Thanks a lot...
From: Duncan Murdoch
Sent: Sunday, August 29, 2021 5:49 PM
To: akshay kulkarni ; Enrico Schumann
Cc: R help Mailing list ; r-help-requ...@r-project.org
Subject: Re: [R] coercion to an object...
On 29/08/2021 7:52 a.m., akshay kulkarni wrote:
> dear Enrico,
> it works. Thanks a lot. I spent over an hour looking
> for this function in the web, but was bootless. Do you have any way to get
> functions where you are given what it has to do? The most common case is that
> you are given a list of functions, but they number to over 10. Any idea
> to find them without resorting to help from another R expert?
The sos package does a pretty good job, but your query needs to be
fairly narrow or the result list will be really long. I don't think it
would have found get() in the top hits given the words in your original
question. Some questions need intelligence to interpret, not just
pattern matching.
Other than asking on this list, there's stackoverflow.com to get humans
involved in the search.
Duncan Murdoch
>
> yours sincerely
> AKSHAY M KULKARNI
>
>
> From: Enrico Schumann
> Sent: Sunday, August 29, 2021 4:57 PM
> To: akshay kulkarni
> Cc: R help Mailing list ; r-help-requ...@r-project.org
>
> Subject: Re: [R] coercion to an object...
>
> On Sun, 29 Aug 2021, akshay kulkarni writes:
>
>> Dear members,
>> I think the following question is rudimentary,
>> but I couldn't find an answer in the Internet.
>>
>> Suppose there is an object A, and ls() lists it as "A". How do you convert
>> this character object to the object A. i.e I want a function f such that
>> class(f("A")) = class(A) (of course, class("A") = "character") . f should
>> just coerce the character to the object represented by it.
>
>
> Perhaps ?get is what you're looking for:
>
>A <- 42
>get("A")
>## [1] 42
>
>
>> Thank you,
>> Yours sincerely,
>> AKSHAY M KULKARNI
>>
>
> --
> Enrico Schumann
> Lucerne, Switzerland
> http://enricoschumann.net
>
> [https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif]
> Virus-free.
> www.avast.com
>
>[[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] coercion to an object...
On 29/08/2021 7:52 a.m., akshay kulkarni wrote:
dear Enrico,
it works. Thanks a lot. I spent over an hour looking
for this function in the web, but was bootless. Do you have any way to get
functions where you are given what it has to do? The most common case is that
you are given a list of functions, but they number to over 10. Any idea to
find them without resorting to help from another R expert?
The sos package does a pretty good job, but your query needs to be
fairly narrow or the result list will be really long. I don't think it
would have found get() in the top hits given the words in your original
question. Some questions need intelligence to interpret, not just
pattern matching.
Other than asking on this list, there's stackoverflow.com to get humans
involved in the search.
Duncan Murdoch
yours sincerely
AKSHAY M KULKARNI
From: Enrico Schumann
Sent: Sunday, August 29, 2021 4:57 PM
To: akshay kulkarni
Cc: R help Mailing list ; r-help-requ...@r-project.org
Subject: Re: [R] coercion to an object...
On Sun, 29 Aug 2021, akshay kulkarni writes:
Dear members,
I think the following question is rudimentary,
but I couldn't find an answer in the Internet.
Suppose there is an object A, and ls() lists it as "A". How do you convert this character object to the
object A. i.e I want a function f such that class(f("A")) = class(A) (of course, class("A") =
"character") . f should just coerce the character to the object represented by it.
Perhaps ?get is what you're looking for:
A <- 42
get("A")
## [1] 42
Thank you,
Yours sincerely,
AKSHAY M KULKARNI
--
Enrico Schumann
Lucerne, Switzerland
http://enricoschumann.net
[https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif]
Virus-free.
www.avast.com
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] ggplot geom_line problem
Thank you Jeff. This solves my problem.
On 2021-08-28 21:54, Jeff Newmiller wrote:
Maybe you will find that coord_cartesian( ylim=c(-30,30) ) works
better since it doesn't filter out data before rendering.
On August 28, 2021 6:45:11 PM PDT, p...@philipsmith.ca wrote:
I am preparing a time series plot using the ggplot function. It
includes
an area plot outlined at its edges with a line plot. For some reason
the
line plot, drawn with geom_line(), has some broken portions where the
line does not appear, although the filled geom_area() part of the plot
is shown. The problem is related to the last line of code because when
I
remove it, the problem disappears.
Why is this happening and what can I do about it? Thank you. Philip
library(tidyverse)
df <- dget("df.txt")
mnGr <- 345.05809
mnGDP <- 859.752763485
ggplot(df)+
geom_area(aes(x=Date,y=GDP*(mnGr/mnGDP)-Gr),
fill="red",alpha=0.3)+
geom_line(aes(x=Date,y=GDP*(mnGr/mnGDP)-Gr),
colour="black",size=1)+
scale_y_continuous(position="right",limits=c(-30,30))
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] coercion to an object...
dear Enrico,
it works. Thanks a lot. I spent over an hour looking for
this function in the web, but was bootless. Do you have any way to get
functions where you are given what it has to do? The most common case is that
you are given a list of functions, but they number to over 10. Any idea to
find them without resorting to help from another R expert?
yours sincerely
AKSHAY M KULKARNI
From: Enrico Schumann
Sent: Sunday, August 29, 2021 4:57 PM
To: akshay kulkarni
Cc: R help Mailing list ; r-help-requ...@r-project.org
Subject: Re: [R] coercion to an object...
On Sun, 29 Aug 2021, akshay kulkarni writes:
> Dear members,
> I think the following question is rudimentary,
> but I couldn't find an answer in the Internet.
>
> Suppose there is an object A, and ls() lists it as "A". How do you convert
> this character object to the object A. i.e I want a function f such that
> class(f("A")) = class(A) (of course, class("A") = "character") . f should
> just coerce the character to the object represented by it.
Perhaps ?get is what you're looking for:
A <- 42
get("A")
## [1] 42
> Thank you,
> Yours sincerely,
> AKSHAY M KULKARNI
>
--
Enrico Schumann
Lucerne, Switzerland
http://enricoschumann.net
[https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif]
Virus-free.
www.avast.com
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] coercion to an object...
On Sun, 29 Aug 2021, akshay kulkarni writes:
> Dear members,
> I think the following question is rudimentary,
> but I couldn't find an answer in the Internet.
>
> Suppose there is an object A, and ls() lists it as "A". How do you convert
> this character object to the object A. i.e I want a function f such that
> class(f("A")) = class(A) (of course, class("A") = "character") . f should
> just coerce the character to the object represented by it.
Perhaps ?get is what you're looking for:
A <- 42
get("A")
## [1] 42
> Thank you,
> Yours sincerely,
> AKSHAY M KULKARNI
>
--
Enrico Schumann
Lucerne, Switzerland
http://enricoschumann.net
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] coercion to an object...
Dear members,
I think the following question is rudimentary, but
I couldn't find an answer in the Internet.
Suppose there is an object A, and ls() lists it as "A". How do you convert this
character object to the object A. i.e I want a function f such that
class(f("A")) = class(A) (of course, class("A") = "character") . f should just
coerce the character to the object represented by it.
Thank you,
Yours sincerely,
AKSHAY M KULKARNI
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] A glitch (???) in tools::texi2pf.
It is a general "feature" of TeX that documents with tables of contents, indices, bibliographies, and so on, have to be "iterated to convergence". A couple of PhD theses came out of Stanford; the problem is in that which page one thing goes on depends on where other things went, which depends on where the first thing went... TeX users got accustomed to this, which meant that they also tolerated similar things in outboard tools like bibtex. You could of course run bibtex explicitly to recreate the .bbl file. Frankly, the suggestion of patching texi2pdf to delete the .bbl file(s) seems like the simplest way forward. On Sun, 29 Aug 2021 at 01:25, J C Nash wrote: > > I can understand Rolf's concern. Make is a tool that is very helpful, > but also not trivial to learn how to make work. If a good Makefile > has been set up, then things are easy, but I've generally found my > skills limited to fairly simple Makefiles. > > I would suggest that what is needed is a bit of modest collaboration > to set up a Makefile for dealing with this issue that has enough comments > so it is clear what Make will be doing. I suspect that in this case the > need is to remove the offending .bbl file and then run the tex*** operations. > Possibly the result is small enough to post in this thread. Anyone? > > Cheers, JN > > > > On 2021-08-28 9:14 a.m., Rolf Turner wrote: > > > > On Sat, 28 Aug 2021 12:49:04 +0300 > > Eric Berger wrote: > > > >> As Achim wrote in point (2), Makefile is your friend. > > > > Well, all I can say is that Makefile is *not* my friend; I have never > > made its acquaintance and wouldn't know where to begin looking. > > > > Would it be possible for you to provide a template/prototype Makefile > > and give me some idea what to *do* with it? > > > > cheers, > > > > Rolf Turner > > > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
