(whimsy) branch master updated: Skip processing if no change to input since last update

[sebb]

This is an automated email from the ASF dual-hosted git repository.

sebb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/whimsy.git


The following commit(s) were added to refs/heads/master by this push:
     new f19cd6a4 Skip processing if no change to input since last update
f19cd6a4 is described below

commit f19cd6a4e138d47bfefad09de115bfd601991757
Author: Sebb <s...@apache.org>
AuthorDate: Tue Mar 12 14:02:22 2024 +0000

    Skip processing if no change to input since last update
---
 tools/parsemail.rb | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tools/parsemail.rb b/tools/parsemail.rb
index 915955f8..fa0adec6 100755
--- a/tools/parsemail.rb
+++ b/tools/parsemail.rb
@@ -25,6 +25,7 @@ require 'whimsy/asf/yaml'
 
 require 'mail'
 require 'date'
+require 'fileutils'
 
 module ParseMail
   MAIL_ROOT = '/srv/mail'
@@ -37,6 +38,16 @@ module ParseMail
   end
 
   def self.parse_dir(maildir, yamlfile)
+    # Has the directory changed since the last run?
+    # If not, don't reprocess
+    ytime = File.mtime(yamlfile)
+    dtime = File.mtime(maildir)
+    if ytime > dtime + 60 # Allow for yaml update window
+      log :INFO, "No change to #{maildir} (#{dtime}) since #{yamlfile} 
(#{ytime}), skipping"
+      return
+    else
+      log :INFO, "Timediff #{dtime - ytime}"
+    end
     data = Hash.new
 
     begin
@@ -74,7 +85,9 @@ module ParseMail
     log :INFO, "Found #{entries} files, with #{dupes} duplicates, giving 
#{data.size} new entries"
 
     if data.size == 0
-      log :INFO, "No new entries found"
+      log :INFO, "No new entries found, updating last date"
+      FileUtils.touch yamlfile # needed to skip processing next time
+      # Should not happen often, an updated dir should result in updating the 
yaml
     else
       # update the file with any new entries (this locks the file)
       YamlFile.update(yamlfile) do |yaml|