[weewx-user] Re: RAM disc for web server data

[R A Lichtensteiger]

I wrote:


Killing three birds with one stone ... I have an init script
(attached) that:


OFFFS ... first post and I screw up by failing to attach the script


R
--
R A Lichtensteiger

   “Don’t make something unless it is both necessary and useful; but if it is
both necessary and useful, don’t hesitate to make it beautiful.”
  —Shaker philosophy

--
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/20190729164129.GC10844%40mta.tifosi.com.
#!/bin/sh 

# Startup script to sync to/from a ramdisk for WeeWX for Debian derivatives
#
### BEGIN INIT INFO
# Provides:  weewx-ramdisk
# Required-Start:$local_fs $remote_fs $syslog $time
# Required-Stop: $local_fs $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop:  0 1 6
# Short-Description: weewx weather system ramdisk
# Description:   Manages a ramdisk for the web output of weewx
### END INIT INFO

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="weewx weather system ramdisk"
NAME=weewx-ramdisk

TARGET="/var/www/weather"
BACKUP="/var/backups/weewx-webdir"
LOGFILE="/var/log/weewx/webdir-sync.log"

RSYNC_BIN="/usr/bin/rsync"

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Exit if the rsync package is not installed
[ -x "$RSYNC_BIN" ] || exit 0

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

if [ "$START_WEEWX" = no ]
then
  log_action_msg "Not starting $DESC, see /etc/default/$NAME"
  exit 0
fi

case "$1" in
  start)
echo "Copying files to ramdisk"
$RSYNC_BIN -av ${BACKUP}/ ${TARGET}/
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk restored from HD >> ${LOGFILE}
;;
  stop|sync|reload)
echo "Synching files from ramdisk to Harddisk"
echo [`date +"%Y-%m-%d %H:%M"`] Ramdisk backed up to HD >> ${LOGFILE}
$RSYNC_BIN -av --delete --recursive --force ${TARGET}/ ${BACKUP}/
;;
  *)
echo "Usage: /etc/init.d/$NAME {start|stop|sync|reload}"
exit 1
;;
esac

exit 0

[weewx-user] Re: RAM disc for web server data

[R A Lichtensteiger]

("Long time listener, first time caller")

vince wrote:


On Wednesday, July 24, 2019 at 6:55:42 AM UTC-7, James Muirhead wrote


As the web server data is written to the SD every 5 minutes, I felt it
would be appropriate to create a RAM disc for it. Should, in theory,
improve lifespan of the SD card.
[...]


One thing you might think about is stashing your NOAA files to disk
periodically (couple times a day) and restoring them, as they take a long
time to regenerate as you get more and more data.  The other stuff will
regenerate quickly on firstboot of course.


Like many, I stumbled onto the tmpfs ram drive solution a while ago.

Killing three birds with one stone ... I have an init script
(attached) that:

  o  Backs up the web directory on shutdown or when called with
 "sync" or "reload"
  o  Restores the web directory on startup

Cron entry runs the "sync" nightly, the other two modes are self
explanatory.

R

People using systemd are on their own
--
R A Lichtensteiger

Power corrupts. Absolute power corrupts absolutely. But it rocks
absolutely, too.

--
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/20190729163747.GA10844%40mta.tifosi.com.

Re: [weewx-user] Re: RAM disc for web server data

[vince]

On Wednesday, July 24, 2019 at 11:16:48 AM UTC-7, Thomas Keffer wrote:
>
> Weewx generates any missing NOAA files on every reporting cycle.
>
>
>
I did a little test (admittedly a bit contrived) in which I deleted the 
NOAA file for a month from earlier this year and weewx did 'not' generate 
the missing intermediate month file. 

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/814fceb1-81a2-4507-ab2b-2121c557866f%40googlegroups.com.

Re: [weewx-user] Re: RAM disc for web server data

[Thomas Keffer]

Weewx generates any missing NOAA files on every reporting cycle.

On Wed, Jul 24, 2019 at 1:23 PM vince  wrote:

> On Wednesday, July 24, 2019 at 8:06:37 AM UTC-7, James Muirhead wrote:
>>
>> Does it not regenerate all of the files on the webserver side every 5
>> minutes? Or does it only generate the ones it can't find? I got the
>> impression it regenerated everything.
>>
>>
>>
> The historical NOAA files are a weewx bootup thing only (ie, my previous
> 12 years of summaries, in my case).
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/76401500-fba3-424a-9d72-aba1a89d1679%40googlegroups.com
> 
> .
>
-- 
-tk

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEC9b3Y_teXGCzR5saoZ1TMTBox7AE26aREgGNrivWw3%2Bg%40mail.gmail.com.

[weewx-user] Re: RAM disc for web server data

[vince]

On Wednesday, July 24, 2019 at 8:06:37 AM UTC-7, James Muirhead wrote:
>
> Does it not regenerate all of the files on the webserver side every 5 
> minutes? Or does it only generate the ones it can't find? I got the 
> impression it regenerated everything.
>
>
>
The historical NOAA files are a weewx bootup thing only (ie, my previous 12 
years of summaries, in my case). 

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/76401500-fba3-424a-9d72-aba1a89d1679%40googlegroups.com.

[weewx-user] Re: RAM disc for web server data

[James Muirhead]

Does it not regenerate all of the files on the webserver side every 5 
minutes? Or does it only generate the ones it can't find? I got the 
impression it regenerated everything.

-James Muirhead

On Wednesday, 24 July 2019 15:09:30 UTC+1, vince wrote:
>
> On Wednesday, July 24, 2019 at 6:55:42 AM UTC-7, James Muirhead wrote
>>
>> As the web server data is written to the SD every 5 minutes, I felt it 
>> would be appropriate to create a RAM disc for it. Should, in theory, 
>> improve lifespan of the SD card.
>> [...]
>>
>  
>
>> Just means, that on boot it can take up to 5 minutes for the web server 
>> to be populated with files and will likely return a 403 error when you try 
>> to look at it from a browser.
>>
>> I also created a temporary splash page that just says "loading, please 
>> wait." which is loaded by a cron entry on @reboot.
>>
>>  

> *cp /home/pi/temp-splash.html /var/www/html/index.html*
>>
>>
> Yes - I think several folks (including me) have posted something similar 
> along the same lines.   I have one pi that I run basically 'everything' in 
> ramdisk, although that of course loses you the syslogs if you need them for 
> forensics after a crash or the like.
>
> One thing you might think about is stashing your NOAA files to disk 
> periodically (couple times a day) and restoring them, as they take a long 
> time to regenerate as you get more and more data.  The other stuff will 
> regenerate quickly on firstboot of course.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/a0c7400d-20fe-4375-86af-4d7108f6429a%40googlegroups.com.

[weewx-user] Re: RAM disc for web server data

[vince]

On Wednesday, July 24, 2019 at 6:55:42 AM UTC-7, James Muirhead wrote
>
> As the web server data is written to the SD every 5 minutes, I felt it 
> would be appropriate to create a RAM disc for it. Should, in theory, 
> improve lifespan of the SD card.
> [...]
>
 

> Just means, that on boot it can take up to 5 minutes for the web server to 
> be populated with files and will likely return a 403 error when you try to 
> look at it from a browser.
>
> I also created a temporary splash page that just says "loading, please 
> wait." which is loaded by a cron entry on @reboot.
>
> *cp /home/pi/temp-splash.html /var/www/html/index.html*
>
>
Yes - I think several folks (including me) have posted something similar 
along the same lines.   I have one pi that I run basically 'everything' in 
ramdisk, although that of course loses you the syslogs if you need them for 
forensics after a crash or the like.

One thing you might think about is stashing your NOAA files to disk 
periodically (couple times a day) and restoring them, as they take a long 
time to regenerate as you get more and more data.  The other stuff will 
regenerate quickly on firstboot of course.


-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/710ca3d7-2bd1-42fa-ba35-b1d48ddb2e02%40googlegroups.com.