Re: [ntp:questions] Selective peerstats collection
On 08/12/2012 06:23, Hal Murray wrote: [] What's the cost of a SD card? Pick a medium size one and run the experiment with full logging. You might get lucky or unlucky, but it will give you one data point. Anybody interested in this area should be sure to read: (bunnie's blog) On MicroSD Problems http://www.bunniestudios.com/blog/?p=918 The problem is that if it takes, say, 3 months to wear out the SD card, I don't really have that long to wait! My time costs a lot more than the SD card. Likely I will turn off statistics collection outside major NTP changes. Thanks for the micro-SD card blog - I recall reading that some time back, but it's still a great great investigation tale. -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Selective peerstats collection
Hal Murray wrote: In article k9q1fg$vd0$1...@dont-email.me, David Taylor david-tay...@blueyonder.co.uk.invalid writes: Thanks for your suggestions, Hal. At the moment, I don't have a handle on what to expect the lifetime of an SD card used as a Linux system disk, so therefore how much effort to put into finding a solution. What's the cost of a SD card? Pick a medium size one and run the experiment with full logging. You might get lucky or unlucky, but it will give you one data point. I presume this is being done as a hobbyist project, where the value put on human time is very subjective. Even commercially, you can have situations where SD card is part of hardware sold as part of some managed service, and where one is expected to send a technician out to replace it, making the cost more like £50-100. The real problem, though is there seems to be no reliable information on life times. It is fairly easy to find the raw lifetime of a cell (3000-5000 cycles) but the actual way that external writes are mapped into row writes tends to be a carefully guarded secret and any clues seem to be missing from packaging and marketing material. What I found out is a suggestion that more expensive, branded, controllers are likely to do wear levelling. However, there is was also a suggestion that all cards, do not update in place, except for some sort of index memory, which is more resilient than the main memory. They copy a row to a new location on each update. How wear levelling can improve on the basic copy operation was not clear and nor where things like the row size and when the rewrite actually took place (using a write back cache of one or more rows would reduce the actual number of physical updates, but what is the policy for forcing a write). All this sort of information seems to fall under the class of trade secrets that a buyer needs to know, to make intelligent buying and design decisions, but the release of which is vetoed by the vendor's marketing people, nervous of their IPR. In the case of the Pi, they are designed for an extremely price sensitive market, and good quality SD cards cost about 25% of the cost of the computer board. Cheap cards may well expire very quickly when used as a computer filesystem, rather than a still camera's film. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Selective peerstats collection
On 08/12/2012 10:25, Hal Murray wrote: In article k9uqvn$mf2$1...@dont-email.me, David Taylor david-tay...@blueyonder.co.uk.invalid writes: The problem is that if it takes, say, 3 months to wear out the SD card, I don't really have that long to wait! My time costs a lot more than the SD card. Likely I will turn off statistics collection outside major NTP changes. I think it will last years. I can't prove that. I might be wrong. I don't know how to evaluate your time. A low cost SD card is ballpark of $10. If your time is worth anything, you have already spent more than that discussing this. :) Just logging to the SD card is simple and doesn't waste time trying to figure out how to do something else. If it dies after 3 months, you go Oh shit and put the system back together. I'm assuming you backup the log files over to another system occasionally, say once a day. So if the SD disk dies, you lose at most a day of log files. Then you get to decide if swapping in a new SD card every now and then is cheaper than working out the alternatives. -- I have a small embedded type box generating lots of log files. It's on a UPS. I'd like to minimize the power it burns to maximize the time it can run on the UPS. Several years ago, I put all the log files into ramdisk so the disk could spin down and occasionally copied them to the real disk. After I got that stuff all debugged, I discovered that it didn't save enough power to be worth the effort. So I ripped out that stuff and went back to just logging to disk. I suppose I'm prepared to put the time in to understanding the issues involved, but not into getting half a dozen different brand of SD cards and seeing when they fail. Yes, having a 4 GB or 8 GB SD card which can be loaded from the most recent backup of the working SD card would be more than adequate in this particular case. As it happens, the log files are backed up daily with a scheduled task on the Windows PC and PKZip25 (yes, that old) accessing the data through a Samba share. Even then, I'm interested to know how well does NTP work, and that should be normally constant. Retaining the log files is in no way critical. However, what I would like to know is the answer to: What is causing the most write activity on the SD card, what effect does that have on its life, and how can such activity be minimised? But if I don't find that out it doesn't really matter. Thanks for the information on your own system. The nice thing about this NTP server is that it takes less than 4 watts ... so I'm definitely /not/ worrying about power consumption! G (But people are telling me that that are employing these devices remotely, e.g. on mountain tops, solar powered perhaps, so if they are going to fail after just a few weeks or months it would be helpful to be warned in advance.) -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Selective peerstats collection
On 06/12/2012 08:42, Hal Murray wrote: [] I don't know how to get selective logging. If I wanted to save wear on an SD card, I'd look into logging to RAMDisk and then filtering the copy that goes to real/SD disk. In terms of overall effort, it may be simpler to split the file system into 2 disks. Put the logging on one and replace it when it wears out. Modest size SD disks aren't very expensive. If you can only get 1 disk, take good backups so you can put the system back together quickly and easily. Thanks for your suggestions, Hal. At the moment, I don't have a handle on what to expect the lifetime of an SD card used as a Linux system disk, so therefore how much effort to put into finding a solution. Actually, for routine operation there's no need to log the statistics, but they are handy for performance measurement across configuration changes. My own idea was to compare the peerstats offsets of different stratum-1 servers as seen from a single stratum-1 server, to understand the different interrupt latency on FreeBSD/Intel-Atom, Linux/RaspberryPi, and Windows/Intel systems. Fortunately, it's quite easy to make a backup of the system SD disk using the Win32DiskImager program. -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Selective peerstats collection
On 02/12/2012 17:51, David Lord wrote: [] I use peer_summary (and loop_summary) from the ntpd distro. It may not be what you are looking for. It should also be easy to set a cron job to filter peerstats each day to extract entries for peers you want. This is from peer_summary on 192.168.59.67: Two stratum-1 on the lan, 192.168.59.61 from a Sure gps/pps usually with offset 3 usec, and 192.168.59.64 from a Conrad MSF receiver usually with offset 300 usec. peerstats.20110101 ident cnt mean rms max delay dist disp == 192.168.59.221120.1580.6241.5570.567 23.7639.854 192.168.59.61 900.5850.8121.7310.811 22.4829.296 192.168.59.64207 -0.4210.8533.1060.415 192.2597.576 192.168.59.210920.4240.5661.5670.630 20.6488.252 158.152.1.76 55 -0.0880.7682.052 30.318 46.053 16.456 130.159.196.117 52 -0.1240.7311.691 39.524 51.098 16.490 [] David Thanks for that, David. No, what I want is to /collect/ data only for selected peers, to reduce the file size and amount of I/O activity (the disk for the system might be an SD card). I can select post-collection with my own NTP Plotter program, which runs on Windows. http://www.satsignal.eu/software/net.htm#NTPplotter -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Selective peerstats collection
David Taylor wrote: On 02/12/2012 17:51, David Lord wrote: [] I use peer_summary (and loop_summary) from the ntpd distro. It may not be what you are looking for. It should also be easy to set a cron job to filter peerstats each day to extract entries for peers you want. This is from peer_summary on 192.168.59.67: Two stratum-1 on the lan, 192.168.59.61 from a Sure gps/pps usually with offset 3 usec, and 192.168.59.64 from a Conrad MSF receiver usually with offset 300 usec. peerstats.20110101 ident cnt mean rms max delay dist disp == 192.168.59.221120.1580.6241.5570.567 23.763 9.854 192.168.59.61 900.5850.8121.7310.811 22.482 9.296 192.168.59.64207 -0.4210.8533.1060.415 192.259 7.576 192.168.59.210920.4240.5661.5670.630 20.648 8.252 158.152.1.76 55 -0.0880.7682.052 30.318 46.053 16.456 130.159.196.117 52 -0.1240.7311.691 39.524 51.098 16.490 [] David Thanks for that, David. No, what I want is to /collect/ data only for selected peers, to reduce the file size and amount of I/O activity (the disk for the system might be an SD card). I can select post-collection with my own NTP Plotter program, which runs on Windows. http://www.satsignal.eu/software/net.htm#NTPplotter My peerstats grows but the summary is manageable. I have the option of logging to /tmpfs in memory but if your peerstats is that large that would be a problem. My daily peerstats would be under 1 MB for when I had about 6 pcs on the lan but is under 100 kB per system and I have enough ram to use /tmpfs if it was needed. Using peer_summary trims the logfile daily so just now peerstats for the day is 43076 whilst peer_summary 20120101-20121201 is 204606 bytes. When I started with NetBSD 1.5 the pcs each had 8 MB ram, and hdd were one with 245 MB and second with 450 MB so ram was tight but storage wasn't a problem. David ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
