Re: [R] Please help(urgent) - How to simulate transactional data for reliability/survival analysis
I am trying to figure out the algorithm you are using to calculate event_episodes, event_status, and start_minutes. Where does the 129600 come from? Why is the start(minutes) 0 for the last row instead of 40? Mark R. Mark Sharp, Ph.D. msh...@txbiomed.org > On Jul 5, 2017, at 1:03 AM, Sunny Singha > wrote: > > Mark, > Below is the sampled simulated granular data format for pumps for > trial period of 3 months that I need to transform for survival > analysis: > 3 months = (60*24*90) minutes i.e 129600 minutes > > pump_id timingsevents vibration temprature flow > pump1 01-07-2017 00:00 03.44369.6 139.806 > pump1 01-07-2017 00:10 10.50145.27 140.028 > pump1 01-07-2017 00:20 02.03152.9 137.698 > pump1 01-07-2017 00:30 02.26760.12 139.054 > pump1 01-07-2017 00:40 12.26760.12 139.054 > pump1 01-07-2017 00:50 02.26760.12 139.054 > pump2 01-07-2017 00:00 03.44369.6 139.806 > pump2 01-07-2017 00:10 00.50145.27 140.028 > pump2 01-07-2017 00:20 02.03152.9 137.698 > pump2 01-07-2017 00:30 02.26760.12 139.054 > pump2 01-07-2017 00:40 12.26760.12 139.054 > pump2 01-07-2017 00:50 02.26760.12 139.054 > > The above data set records observations and timings where 'pumps' > experienced failure, tagged as '1' in column 'events'. > In the above granular dataset the pump1 experiences 2 "event episodes." > > Below is the desired transformed format. the covariates in this data > set will have the mean value: > pump_id event_episodes event_status start(minutes) > stop(minutes) > pump1 1 1 > 0 10 > pump1 2 1 > 10 40 > pump1 3 0 > 40 129600 > pump2 1 1 > 0 40 > pump2 2 0 > 0 129600 > . > . > > The 'start' and 'stop' columns are evaluated from the 'timings' > columns. I need help in performing such transformation in 'R'. > Please guide and help. > > Regards, > Sandeep > > On Wed, Jul 5, 2017 at 7:26 AM, Mark Sharp wrote: >> A small example data set that illustrates your question will be of great >> value to those trying to help. This appears to be a transformation that you >> are wanting to do (timestamp to units of time) so a data representing what >> you have (dput() is handy for this) and one representing what you want to >> have with any guidance regarding how to use the other columns in you data >> set (e.g., the event(0/1)). >> >> Mark >> R. Mark Sharp, Ph.D. >> msh...@txbiomed.org >> >> >> >> >> >>> On Jul 4, 2017, at 7:02 AM, Sunny Singha >>> wrote: >>> >>> Thanks Boris and Bret, >>> I was successful in simulating granular/transactional data. >>> Now I need some guidance to transform the same data in format acceptable >>> for survival analysis i.e below format: >>> >>> pump_id | event_episode_no. | event(0/1) | start | stop | time_to_dropout >>> >>> The challenge I'm experience is to generate the 'start' and 'stop' in units >>> of minutes/days from single column of 'Timestamp' which is >>> the column from transactional/granular data based on condition tagged in >>> separate column, 'event 0/1, (i.e event ). >>> >>> Please guide how to do such transformation in 'R'. >>> >>> Regards, >>> Sandeep >>> >>> >>> >>> On Wed, Jun 28, 2017 at 2:51 PM, Boris Steipe >>> wrote: >>> In principle what you need to do is the following: - break down the time you wish to simulate into intervals. - for each interval, and each failure mode, determine the probability of an event. Determining the probability is the fun part, where you make your domain knowledge explicit and include all the factors into your model: cumulative load, failure history, pressure, temperature, phase of the moon ... - once you have a probability of failure, use the runif() function to give you a uniformly distributed random number in [0, 1]. If the number is smaller than your failure probability, accept the failure event, and record it. - Repeat many times. Hope this helps. B. > On Jun 27, 2017, at 10:58 AM, sandeep Rana wrote: > > Hi friends, > I haven't done such a simulation before and any help would be greatly appreciated. I need your guidance. > > I need to simulate end to end data for Reliability/
Re: [R] Please help(urgent) - How to simulate transactional data for reliability/survival analysis
Strictly speaking, this is reliability, not survival, data, since failed pumps are apparently repaired and put back in service as new. Also, it is not clear from your data whether there is interval censoring: is the recorded "event" time (failure) the actual failure time -- so no censoring -- or the time at which the pump has been discovered to have failed, so that it is known to have failed in the interval since the last time it was recorded, but exactly when is unknown. Presumably there is also standard right censoring -- the pump is still running when the testing period concludes. Cheers, Bert Bert Gunter "The trouble with having an open mind is that people keep coming along and sticking things into it." -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) On Tue, Jul 4, 2017 at 11:03 PM, Sunny Singha wrote: > Mark, > Below is the sampled simulated granular data format for pumps for > trial period of 3 months that I need to transform for survival > analysis: > 3 months = (60*24*90) minutes i.e 129600 minutes > > pump_id timingsevents vibration temprature flow > pump1 01-07-2017 00:00 03.44369.6 139.806 > pump1 01-07-2017 00:10 10.50145.27 140.028 > pump1 01-07-2017 00:20 02.03152.9 137.698 > pump1 01-07-2017 00:30 02.26760.12 139.054 > pump1 01-07-2017 00:40 12.26760.12 139.054 > pump1 01-07-2017 00:50 02.26760.12 139.054 > pump2 01-07-2017 00:00 03.44369.6 139.806 > pump2 01-07-2017 00:10 00.50145.27 140.028 > pump2 01-07-2017 00:20 02.03152.9 137.698 > pump2 01-07-2017 00:30 02.26760.12 139.054 > pump2 01-07-2017 00:40 12.26760.12 139.054 > pump2 01-07-2017 00:50 02.26760.12 139.054 > > The above data set records observations and timings where 'pumps' > experienced failure, tagged as '1' in column 'events'. > In the above granular dataset the pump1 experiences 2 "event episodes." > > Below is the desired transformed format. the covariates in this data > set will have the mean value: > pump_id event_episodes event_status start(minutes) > stop(minutes) > pump1 1 1 > 0 10 > pump1 2 1 > 10 40 > pump1 3 0 > 40 129600 > pump2 1 1 > 0 40 > pump2 2 0 > 0 129600 > . > . > > The 'start' and 'stop' columns are evaluated from the 'timings' > columns. I need help in performing such transformation in 'R'. > Please guide and help. > > Regards, > Sandeep > > On Wed, Jul 5, 2017 at 7:26 AM, Mark Sharp wrote: >> A small example data set that illustrates your question will be of great >> value to those trying to help. This appears to be a transformation that you >> are wanting to do (timestamp to units of time) so a data representing what >> you have (dput() is handy for this) and one representing what you want to >> have with any guidance regarding how to use the other columns in you data >> set (e.g., the event(0/1)). >> >> Mark >> R. Mark Sharp, Ph.D. >> msh...@txbiomed.org >> >> >> >> >> >>> On Jul 4, 2017, at 7:02 AM, Sunny Singha >>> wrote: >>> >>> Thanks Boris and Bret, >>> I was successful in simulating granular/transactional data. >>> Now I need some guidance to transform the same data in format acceptable >>> for survival analysis i.e below format: >>> >>> pump_id | event_episode_no. | event(0/1) | start | stop | time_to_dropout >>> >>> The challenge I'm experience is to generate the 'start' and 'stop' in units >>> of minutes/days from single column of 'Timestamp' which is >>> the column from transactional/granular data based on condition tagged in >>> separate column, 'event 0/1, (i.e event ). >>> >>> Please guide how to do such transformation in 'R'. >>> >>> Regards, >>> Sandeep >>> >>> >>> >>> On Wed, Jun 28, 2017 at 2:51 PM, Boris Steipe >>> wrote: >>> In principle what you need to do is the following: - break down the time you wish to simulate into intervals. - for each interval, and each failure mode, determine the probability of an event. Determining the probability is the fun part, where you make your domain knowledge explicit and include all the factors into your model: cumulative load, failure history, pressure, temperature, phase of the moon ... - once you have a probability of failure, use the runif(
Re: [R] Please help(urgent) - How to simulate transactional data for reliability/survival analysis
Mark, Below is the sampled simulated granular data format for pumps for trial period of 3 months that I need to transform for survival analysis: 3 months = (60*24*90) minutes i.e 129600 minutes pump_id timingsevents vibration temprature flow pump1 01-07-2017 00:00 03.44369.6 139.806 pump1 01-07-2017 00:10 10.50145.27 140.028 pump1 01-07-2017 00:20 02.03152.9 137.698 pump1 01-07-2017 00:30 02.26760.12 139.054 pump1 01-07-2017 00:40 12.26760.12 139.054 pump1 01-07-2017 00:50 02.26760.12 139.054 pump2 01-07-2017 00:00 03.44369.6 139.806 pump2 01-07-2017 00:10 00.50145.27 140.028 pump2 01-07-2017 00:20 02.03152.9 137.698 pump2 01-07-2017 00:30 02.26760.12 139.054 pump2 01-07-2017 00:40 12.26760.12 139.054 pump2 01-07-2017 00:50 02.26760.12 139.054 The above data set records observations and timings where 'pumps' experienced failure, tagged as '1' in column 'events'. In the above granular dataset the pump1 experiences 2 "event episodes." Below is the desired transformed format. the covariates in this data set will have the mean value: pump_id event_episodes event_status start(minutes) stop(minutes) pump1 1 1 0 10 pump1 2 1 10 40 pump1 3 0 40 129600 pump2 1 1 0 40 pump2 2 0 0 129600 . . The 'start' and 'stop' columns are evaluated from the 'timings' columns. I need help in performing such transformation in 'R'. Please guide and help. Regards, Sandeep On Wed, Jul 5, 2017 at 7:26 AM, Mark Sharp wrote: > A small example data set that illustrates your question will be of great > value to those trying to help. This appears to be a transformation that you > are wanting to do (timestamp to units of time) so a data representing what > you have (dput() is handy for this) and one representing what you want to > have with any guidance regarding how to use the other columns in you data set > (e.g., the event(0/1)). > > Mark > R. Mark Sharp, Ph.D. > msh...@txbiomed.org > > > > > >> On Jul 4, 2017, at 7:02 AM, Sunny Singha >> wrote: >> >> Thanks Boris and Bret, >> I was successful in simulating granular/transactional data. >> Now I need some guidance to transform the same data in format acceptable >> for survival analysis i.e below format: >> >> pump_id | event_episode_no. | event(0/1) | start | stop | time_to_dropout >> >> The challenge I'm experience is to generate the 'start' and 'stop' in units >> of minutes/days from single column of 'Timestamp' which is >> the column from transactional/granular data based on condition tagged in >> separate column, 'event 0/1, (i.e event ). >> >> Please guide how to do such transformation in 'R'. >> >> Regards, >> Sandeep >> >> >> >> On Wed, Jun 28, 2017 at 2:51 PM, Boris Steipe >> wrote: >> >>> In principle what you need to do is the following: >>> >>> - break down the time you wish to simulate into intervals. >>> - for each interval, and each failure mode, determine the probability of >>> an event. >>> Determining the probability is the fun part, where you make your domain >>> knowledge explicit and include all the factors into your model: >>> cumulative load, >>> failure history, pressure, temperature, phase of the moon ... >>> - once you have a probability of failure, use the runif() function to >>> give you >>> a uniformly distributed random number in [0, 1]. If the number is >>> smaller than >>> your failure probability, accept the failure event, and record it. >>> - Repeat many times. >>> >>> Hope this helps. >>> B. >>> >>> >>> >>> On Jun 27, 2017, at 10:58 AM, sandeep Rana wrote: Hi friends, I haven't done such a simulation before and any help would be greatly >>> appreciated. I need your guidance. I need to simulate end to end data for Reliability/survival analysis of >>> a Pump ,with correlation in place, that is at 'Transactional level' or at >>> the granularity of time-minutes, where each observation is a reading >>> captured via Pump's sensors each minute. Once transactional data is prepared I Then need to summarise above data >>> for reliability/ survival analysis. To begin with below is the transactional data format that i want prepare: Pump-id| Timestamp | temp | vibration | suction
Re: [R] Please help(urgent) - How to simulate transactional data for reliability/survival analysis
A small example data set that illustrates your question will be of great value to those trying to help. This appears to be a transformation that you are wanting to do (timestamp to units of time) so a data representing what you have (dput() is handy for this) and one representing what you want to have with any guidance regarding how to use the other columns in you data set (e.g., the event(0/1)). Mark R. Mark Sharp, Ph.D. msh...@txbiomed.org > On Jul 4, 2017, at 7:02 AM, Sunny Singha > wrote: > > Thanks Boris and Bret, > I was successful in simulating granular/transactional data. > Now I need some guidance to transform the same data in format acceptable > for survival analysis i.e below format: > > pump_id | event_episode_no. | event(0/1) | start | stop | time_to_dropout > > The challenge I'm experience is to generate the 'start' and 'stop' in units > of minutes/days from single column of 'Timestamp' which is > the column from transactional/granular data based on condition tagged in > separate column, 'event 0/1, (i.e event ). > > Please guide how to do such transformation in 'R'. > > Regards, > Sandeep > > > > On Wed, Jun 28, 2017 at 2:51 PM, Boris Steipe > wrote: > >> In principle what you need to do is the following: >> >> - break down the time you wish to simulate into intervals. >> - for each interval, and each failure mode, determine the probability of >> an event. >> Determining the probability is the fun part, where you make your domain >> knowledge explicit and include all the factors into your model: >> cumulative load, >> failure history, pressure, temperature, phase of the moon ... >> - once you have a probability of failure, use the runif() function to >> give you >> a uniformly distributed random number in [0, 1]. If the number is >> smaller than >> your failure probability, accept the failure event, and record it. >> - Repeat many times. >> >> Hope this helps. >> B. >> >> >> >> >>> On Jun 27, 2017, at 10:58 AM, sandeep Rana wrote: >>> >>> Hi friends, >>> I haven't done such a simulation before and any help would be greatly >> appreciated. I need your guidance. >>> >>> I need to simulate end to end data for Reliability/survival analysis of >> a Pump ,with correlation in place, that is at 'Transactional level' or at >> the granularity of time-minutes, where each observation is a reading >> captured via Pump's sensors each minute. >>> Once transactional data is prepared I Then need to summarise above data >> for reliability/ survival analysis. >>> >>> To begin with below is the transactional data format that i want prepare: >>> Pump-id| Timestamp | temp | vibration | suction pressure| discharge >> pressure | Flow >>> >>> Above transactional data has to be prepared with below failure modes >>> Defects : >>> (1)Cavitation – very high in frequency but low impact >>> (2)Bearing Damage – very low in frequency but high impact >>> (3)Worn Shaft – medium frequency but medium impact >>> >>> I have used survsim package but that's not what I need here. >>> Please help and guide. >>> >>> Regards, >>> Sandeep >>> >>> __ >>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> PLEASE do read the posting guide http://www.R-project.org/ >> posting-guide.html >>> and provide commented, minimal, self-contained, reproducible code. >> >> __ >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/ >> posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. CONFIDENTIALITY NOTICE: This e-mail and any files and/or attachments transmitted, may contain privileged and confidential information and is intended solely for the exclusive use of the individual or entity to whom it is addressed. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or copying of this e-mail and/or attachments is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender stating that this transmission was misdirected; return the e-mail to sender; destroy all paper copies and delete all electronic copies from your system without disclosing its contents. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R
Re: [R] Please help(urgent) - How to simulate transactional data for reliability/survival analysis
Thanks Boris and Bret, I was successful in simulating granular/transactional data. Now I need some guidance to transform the same data in format acceptable for survival analysis i.e below format: pump_id | event_episode_no. | event(0/1) | start | stop | time_to_dropout The challenge I'm experience is to generate the 'start' and 'stop' in units of minutes/days from single column of 'Timestamp' which is the column from transactional/granular data based on condition tagged in separate column, 'event 0/1, (i.e event ). Please guide how to do such transformation in 'R'. Regards, Sandeep On Wed, Jun 28, 2017 at 2:51 PM, Boris Steipe wrote: > In principle what you need to do is the following: > > - break down the time you wish to simulate into intervals. > - for each interval, and each failure mode, determine the probability of > an event. >Determining the probability is the fun part, where you make your domain >knowledge explicit and include all the factors into your model: > cumulative load, >failure history, pressure, temperature, phase of the moon ... > - once you have a probability of failure, use the runif() function to > give you >a uniformly distributed random number in [0, 1]. If the number is > smaller than >your failure probability, accept the failure event, and record it. > - Repeat many times. > > Hope this helps. > B. > > > > > > On Jun 27, 2017, at 10:58 AM, sandeep Rana wrote: > > > > Hi friends, > > I haven't done such a simulation before and any help would be greatly > appreciated. I need your guidance. > > > > I need to simulate end to end data for Reliability/survival analysis of > a Pump ,with correlation in place, that is at 'Transactional level' or at > the granularity of time-minutes, where each observation is a reading > captured via Pump's sensors each minute. > > Once transactional data is prepared I Then need to summarise above data > for reliability/ survival analysis. > > > > To begin with below is the transactional data format that i want prepare: > > Pump-id| Timestamp | temp | vibration | suction pressure| discharge > pressure | Flow > > > > Above transactional data has to be prepared with below failure modes > > Defects : > > (1)Cavitation – very high in frequency but low impact > > (2)Bearing Damage – very low in frequency but high impact > > (3)Worn Shaft – medium frequency but medium impact > > > > I have used survsim package but that's not what I need here. > > Please help and guide. > > > > Regards, > > Sandeep > > > > __ > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide http://www.R-project.org/ > posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/ > posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Please help(urgent) - How to simulate transactional data for reliability/survival analysis
In principle what you need to do is the following: - break down the time you wish to simulate into intervals. - for each interval, and each failure mode, determine the probability of an event. Determining the probability is the fun part, where you make your domain knowledge explicit and include all the factors into your model: cumulative load, failure history, pressure, temperature, phase of the moon ... - once you have a probability of failure, use the runif() function to give you a uniformly distributed random number in [0, 1]. If the number is smaller than your failure probability, accept the failure event, and record it. - Repeat many times. Hope this helps. B. > On Jun 27, 2017, at 10:58 AM, sandeep Rana wrote: > > Hi friends, > I haven't done such a simulation before and any help would be greatly > appreciated. I need your guidance. > > I need to simulate end to end data for Reliability/survival analysis of a > Pump ,with correlation in place, that is at 'Transactional level' or at the > granularity of time-minutes, where each observation is a reading captured via > Pump's sensors each minute. > Once transactional data is prepared I Then need to summarise above data for > reliability/ survival analysis. > > To begin with below is the transactional data format that i want prepare: > Pump-id| Timestamp | temp | vibration | suction pressure| discharge pressure > | Flow > > Above transactional data has to be prepared with below failure modes > Defects : > (1)Cavitation – very high in frequency but low impact > (2)Bearing Damage – very low in frequency but high impact > (3)Worn Shaft – medium frequency but medium impact > > I have used survsim package but that's not what I need here. > Please help and guide. > > Regards, > Sandeep > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Please help(urgent) - How to simulate transactional data for reliability/survival analysis
I think you need to find a local consultant. Someone here might have a suggestion or two where to look (as I do below), but this list only provides help on R programming code, not statistical issues (see programming guide below for details). You might wish to have a look at the CRAN survival analysis task view to see if any packages might address your needs (but warning: it's mostly about medical applications): https://cran.r-project.org/web/views/Survival.html Cheers, Bert Bert Gunter "The trouble with having an open mind is that people keep coming along and sticking things into it." -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip ) On Tue, Jun 27, 2017 at 7:58 AM, sandeep Rana wrote: > Hi friends, > I haven't done such a simulation before and any help would be greatly > appreciated. I need your guidance. > > I need to simulate end to end data for Reliability/survival analysis of a > Pump ,with correlation in place, that is at 'Transactional level' or at the > granularity of time-minutes, where each observation is a reading captured via > Pump's sensors each minute. > Once transactional data is prepared I Then need to summarise above data for > reliability/ survival analysis. > > To begin with below is the transactional data format that i want prepare: > Pump-id| Timestamp | temp | vibration | suction pressure| discharge pressure > | Flow > > Above transactional data has to be prepared with below failure modes > Defects : > (1)Cavitation – very high in frequency but low impact > (2)Bearing Damage – very low in frequency but high impact > (3)Worn Shaft – medium frequency but medium impact > > I have used survsim package but that's not what I need here. > Please help and guide. > > Regards, > Sandeep > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
