thanks for the explanation. On Mon, Sep 14, 2015 at 11:15 AM, Thomas Eckardt <thomas.ecka...@thockar.com > wrote:
> >but I guess ASSP is waiting for it to complete or something > > Yes, this is a configuration task - assp will (has to) wait for the result > of the script. > > >way for you to code ASSP to actually run the external > >calls externally as a separate thread > > This make no sense for several reasons. > > 1. scheduled starting an external program without wating for it, is the > same like using the system scheduler - no need to > 2. fork a new process in Perl will copy the complete code and data > instance - wasted resources and very BIG problems if any database is used > 3. having an extra thread - same like 1 > 4. calling the progam in any other thread - if the program stucks, the > thread stucks > > In any case - assp has absolutely no control to an externaly started > program. > See it the same way like clamd and freshclam are working together. > freshclam is running totaly independend, it only tells the clamd that it > has done something. clamd than reloads the pattern database. > > Thomas > > > > Von: K Post <nntp.p...@gmail.com> > An: ASSP development mailing list <assp-test@lists.sourceforge.net> > Datum: 14.09.2015 16:21 > Betreff: Re: [Assp-test] MainThread stuck occasionally calling > external script > > > > I'll change the workflow ASAP so that an externally scheduled task > generates the files and the files are included in the group definition > file. That's no problem and it's a good idea. > > Thinking bigger picture though: > Might there be a way for you to code ASSP to actually run the external > calls externally as a separate thread such that ASSP continues even if the > external program gets hung? I'm calling perl, so it's a separate perl > instance, but I guess ASSP is waiting for it to complete or something? > > On Mon, Sep 14, 2015 at 4:20 AM, Thomas Eckardt > <thomas.ecka...@thockar.com> > wrote: > > > You should change the logic to prevent the MainThread from doing > > DNS-queries in an external script. > > > > Read the help text at the bottom of the GUI: > > > > "Fields marked with at least one asterisk (*) accept .... > > .... > > # include filename > > where filename is the relative path (from C:/assp) to the included file > > like files/inc1.txt or inc1.txt > > (one file per line). The line will be internaly replaced by the contents > > of the included file!" > > > > solution: > > Run the script that builds the list (doing DNS queries) in a cron job > and > > store the result in a text file - lets say 'c:/assp/files/spfIPs.txt' > > > > In the 'Groups' file define the result file as included file > > > > [GROUP-CONSTANTCONTACT-IPS] > > #include files/spfIPs.txt > > > > ASSP constantly watches all defined files and all included files for > > updates. If the cron job changes the include file, assp will reload all > > groups. > > > > I found an issue for encrypted include-files. With the current release, > > one restart after making this changes may be required. An updated dev > > version will be released today. > > > > Thomas > > > > > > > > > > > > > > Von: K Post <nntp.p...@gmail.com> > > An: ASSP development mailing list <assp-test@lists.sourceforge.net> > > Datum: 13.09.2015 16:31 > > Betreff: [Assp-test] MainThread stuck occasionally calling > external > > script > > > > > > > > I wrote a scrpt to populate ASSP config files with IP address ranges > based > > on SPF values. For example, I call my GetDomainIPSfromSPF.pl script to > > get > > all of the IP's that ConstantContact.com says they can send from. It > > works > > well. > > > > HOWEVER, on rare occasion, maybe once every 2-3 weeks, MainThread gets > > stuck calling the script, always when I do it with the ConstantContact > > parameter. My guess is that there's something quirky with the > > ConstantContact SPF, I'm doing something wrong, or something is broken. > > That's on me to figure out, but the problem is that when the script > hangs, > > so does ASSP's SMTP processing. > > > > My Group Definitions config file has this entry: > > [GROUP-CONSTANTCONTACT-IPS] > > exec:c:/perl/bin/perl.exe "c:\scripts\GetDomainIPSfromSPF.pl" > > constantcontact.com > > > > That calls the perl script, telling it to get the ip's for > > constantcontact.com. The group looks fine upon visual inspection from > the > > gui. > > > > > > Here's the log from this morning's temporary SMTP outage. SMTP was > > unresponsive for 16 minutes according to my monitors, consistent with > the > > ASSP log. > > > > > > Sep-13-15 08:41:16 Saving config > > Sep-13-15 08:41:16 Info: no configuration changes detected - nothing to > > save - file c:/ASSP/assp.cfg is unchanged > > Sep-13-15 08:41:16 Adminupdate: [root x.x.x.9] file > > c:/ASSP/cfg/GroupDefinitions.cfg' for config 'Groups' was changed > > Sep-13-15 08:41:16 Option list file: c:/ASSP/cfg/GroupDefinitions.cfg' > > reloaded (Groups) with 43 records > > Sep-13-15 08:41:16 Info: loading groups from groupsfile > > c:/ASSP/cfg/GroupDefinitions.cfg > > Sep-13-15 08:41:16 Info: group GROUP-OUR-IPS loaded with 2 records > > Sep-13-15 08:41:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:41:55 Info: Name Server x.x.x.52: ResponseTime = 2 ms for > > sourceforge.net > > Sep-13-15 08:41:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > > sourceforge.net > > Sep-13-15 08:42:17 Info: notification message queued to sent to > > assp-not...@ourcharity.org > > *Sep-13-15 08:42:17 Error: MainThread stuck for 61 seconds after: > MainLoop > > start - last debug step was: ConfigMakeGroupRe - > exec:c:/perl/bin/perl.exe > > "c:\scripts\GetDomainIPSfromSPF.pl" constantcontact.com > > <http://constantcontact.com>!* > > Sep-13-15 08:42:20 Info: successful sent file > > c:/ASSP/messages/resendmail/n1000000000320.txt to x.x.x.35:25 > > (smtpDestination) > > Sep-13-15 08:42:28 Info: synchronizing all BerkeleyDB hashes to disk > > Sep-13-15 08:42:28 Info: compacting all BerkeleyDB hashes on disk > > Sep-13-15 08:42:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > > sourceforge.net > > Sep-13-15 08:42:55 Info: Name Server x.x.x.52: ResponseTime = 17 ms for > > sourceforge.net > > Sep-13-15 08:42:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > > sourceforge.net > > Sep-13-15 08:43:11 IPNumTries: cleaning cache finished: IP's before=1, > > deleted=1 > > Sep-13-15 08:43:11 SMTPdomainIP: cleaning cache finished: domain's > > before=9, deleted=0 > > Sep-13-15 08:43:11 SSLfailedCache: cleaning cache finished: IP's > before=1, > > deleted=0 > > Sep-13-15 08:43:11 SubjectFrequency: cleaning cache finished: subjects > > before=6, deleted=4 > > Sep-13-15 08:43:55 Info: Name Server x.x.x.51: ResponseTime = 18 ms for > > sourceforge.net > > Sep-13-15 08:43:55 Info: Name Server x.x.x.52: ResponseTime = 3 ms for > > sourceforge.net > > Sep-13-15 08:43:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:44:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:44:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:44:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:45:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:45:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:45:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:46:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:46:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:46:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:47:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > > sourceforge.net > > Sep-13-15 08:47:55 Info: Name Server x.x.x.52: ResponseTime = 19 ms for > > sourceforge.net > > Sep-13-15 08:47:55 Info: Name Server x.x.x.53: ResponseTime = 19 ms for > > sourceforge.net > > Sep-13-15 08:48:11 SMTPdomainIP: cleaning cache finished: domain's > > before=9, deleted=0 > > Sep-13-15 08:48:11 SSLfailedCache: cleaning cache finished: IP's > before=1, > > deleted=0 > > Sep-13-15 08:48:11 SubjectFrequency: cleaning cache finished: subjects > > before=2, deleted=2 > > Sep-13-15 08:48:55 Info: Name Server x.x.x.51: ResponseTime = 23 ms for > > sourceforge.net > > Sep-13-15 08:48:55 Info: Name Server x.x.x.52: ResponseTime = 6 ms for > > sourceforge.net > > Sep-13-15 08:48:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:49:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:49:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:49:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:50:55 Info: Name Server x.x.x.51: ResponseTime = 3 ms for > > sourceforge.net > > Sep-13-15 08:50:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:50:55 Info: Name Server x.x.x.53: ResponseTime = 4 ms for > > sourceforge.net > > Sep-13-15 08:51:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:51:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:51:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:52:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:52:55 Info: Name Server x.x.x.52: ResponseTime = 18 ms for > > sourceforge.net > > Sep-13-15 08:52:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:53:11 SMTPdomainIP: cleaning cache finished: domain's > > before=9, deleted=0 > > Sep-13-15 08:53:11 SSLfailedCache: cleaning cache finished: IP's > before=1, > > deleted=0 > > Sep-13-15 08:53:55 Info: Name Server x.x.x.51: ResponseTime = 10 ms for > > sourceforge.net > > Sep-13-15 08:53:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:53:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:54:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:54:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:54:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:55:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > > sourceforge.net > > Sep-13-15 08:55:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:55:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > > sourceforge.net > > Sep-13-15 08:56:55 Info: Name Server x.x.x.51: ResponseTime = 18 ms for > > sourceforge.net > > Sep-13-15 08:56:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:56:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:57:28 Info: synchronizing all BerkeleyDB hashes to disk > > Sep-13-15 08:57:28 Info: compacting all BerkeleyDB hashes on disk > > Sep-13-15 08:57:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > > sourceforge.net > > Sep-13-15 08:57:55 Info: Name Server x.x.x.52: ResponseTime = 3 ms for > > sourceforge.net > > Sep-13-15 08:57:55 Info: Name Server x.x.x.53: ResponseTime = 18 ms for > > sourceforge.net > > Sep-13-15 08:58:11 SMTPdomainIP: cleaning cache finished: domain's > > before=9, deleted=0 > > Sep-13-15 08:58:11 SSLfailedCache: cleaning cache finished: IP's > before=1, > > deleted=0 > > Sep-13-15 08:58:30 Info: group GROUP-CONSTANTCONTACT-IPS loaded 24 > > addresses via exec(line 1) > > Sep-13-15 08:58:30 Info: group GROUP-CONSTANTCONTACT-IPS loaded with 24 > > records > > Sep-13-15 08:58:31 Info: group GROUP-MANDRILL-IPS loaded 9 addresses via > > exec(line 1) > > Sep-13-15 08:58:31 Info: group GROUP-MANDRILL-IPS loaded with 9 records > > Sep-13-15 08:58:31 Info: group GROUP-MICROSOFT-IPS loaded 83 addresses > via > > exec(line 1) > > Sep-13-15 08:58:32 Info: group GROUP-MICROSOFT-IPS loaded 83 addresses > via > > exec(line 2) > > Sep-13-15 08:58:32 Info: group GROUP-MICROSOFT-IPS loaded with 166 > records > > Sep-13-15 08:58:32 Info: group GROUP-GOOGLE-IPS loaded 12 addresses via > > exec(line 1) > > Sep-13-15 08:58:32 Info: group GROUP-GOOGLE-IPS loaded with 12 records > > Sep-13-15 08:58:33 Info: group GROUP-AMAZON-IPS loaded 20 addresses via > > exec(line 1) > > Sep-13-15 08:58:33 Info: group GROUP-AMAZON-IPS loaded with 20 records > > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded 8 addresses via > > exec(line 1) > > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded 6 addresses via > > exec(line 2) > > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded with 14 records > > Sep-13-15 08:58:34 Info: group GROUP-HONEYPOT-EMAILS loaded with 12 > > records > > Sep-13-15 08:58:34 Option list file: > > c:/ASSP/cfg/SpamLoverSpecific/SpamLovers-NoDelaying.cfg' reloaded > > (delaySpamLovers) with 2 records > > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/ValidMailboxes.cfg' > > reloaded (LocalAddresses_Flat) with 214 records > > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/SpamOnlyAddresses.cfg' > > reloaded (spamaddresses) with 48 records > > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > > reloaded (noBlockingIPs) with 5 records > > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/LocalDomains.cfg' > > reloaded > > (localDomains) with 58 records > > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > > reloaded (noBlockingIPs) with 5 records > > Sep-13-15 08:58:34 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > > Sep-13-15 08:58:34 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > > records > > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' > reloaded > > (noDelay) with 9 records > > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > > reloaded (noRBL) with 6 records > > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' > reloaded > > (noDelay) with 9 records > > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > > reloaded (noRBL) with 6 records > > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > > reloaded (noBlockingIPs) with 5 records > > Sep-13-15 08:58:35 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > > Sep-13-15 08:58:35 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > > records > > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' > reloaded > > (noDelay) with 9 records > > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > > reloaded (noRBL) with 6 records > > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > > reloaded (noRBL) with 6 records > > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > > reloaded (noBlockingIPs) with 5 records > > Sep-13-15 08:58:35 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > > Sep-13-15 08:58:35 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > > records > > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' > reloaded > > (noDelay) with 9 records > > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > > reloaded (noRBL) with 6 records > > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > > reloaded (noBlockingIPs) with 5 records > > Sep-13-15 08:58:35 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > > Sep-13-15 08:58:35 Option list file: > > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > > records > > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' > reloaded > > (noDelay) with 9 records > > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > > reloaded (noRBL) with 6 records > > Sep-13-15 08:58:35 Info: MainThread has retured to normal state after > > stuck > > [[ email processing resumes here ]] > > > > So, is there some way to have ASSP continue on without MainThread > getting > > stuck if there's a problem with a script called when updating a group > > definition? > > > > > > Note "rutured" is spelled wrong above in the log. > > > > > > THANKS! > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > > Assp-test mailing list > > Assp-test@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > > > > > DISCLAIMER: > > ******************************************************* > > This email and any files transmitted with it may be confidential, > legally > > privileged and protected in law and are intended solely for the use of > the > > > > individual to whom it is addressed. > > This email was multiple times scanned for viruses. There should be no > > known virus in this email! > > ******************************************************* > > > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > > Assp-test mailing list > > Assp-test@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > ------------------------------------------------------------------------------ _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
