Thanks Thomas. It makes sense to me that the most handles to DBD error files are kept open. 10 smtp workers = 10 open handles to each DB. Opening and closing each time would create unnecessary overhead.
I never paid any attention to handles until my system started crashing. I think I found why it's crashing (invalid director for griplist), so maybe all of this is moot, but I'm noticing lots of rebuildDB error handles open.. I restarted ASSP either yesterday or the day before. I see about 50 open handles to \tmpDB\rebuildDB\DBD-error.txt Why so many after just 1-2 rebuilds? Will this keep growing? Should I care? Do you think there's something else wrong in my config? Is there a benefit to not closing handles that are opened during a rebuild, during the griplist delta download, etc? For many years, I've seen ASSP process memory usage grow the longer it's run. I just wrote this off as Windows being Windows, but maybe some of it is due to unnecessarily open handles?? I get that a handle needs to be open for each file that a thread is connected to. Why would you be seeing 1000+ threads on your Windows box Just temporarily open threads to message corpus files or something on a high volume server? For #2/#3, where I had a non-existent path for griplist (obviously my error), I think ASSP should still check for that, not because I should be allowed to make stupid mistakes like that, but because it appears that such a mistake can cause difficult to diagnose horrible system behavior and *delayed* ASSP complete crashes. ASSP thought it had a griplist, I thought I had a griplist, but there was no file and ASSP would keep trying to write there without logging even a warning. I believe that the system crashes that I was setting when the bad TLS would come in was caused because of this invalid griplist path. It seems like ASSP would open a *new *handle to the \tmpDB\Griplist\BDB-error.txt file each time a bad SMTP server would try an early TLS session (which could be very frequently) and it wouldn't close them. I saw over 1300 open handles to the griplist/dbd-error.txt file, and that's when ASSP wasn't getting bombarded with TLS early talkers. The BDB-error.txt file was empty, so nothing was being saved, but a handle was open and stayed open. I suspect that once I'd get slammed by bad TLS early talkers, ASSP would open many multiples more, exceeding the 10,000 (?) thread limit or some other limit, causing errors like: error: Worker_2 accept to client failed IO::Socket::INET=GLOB(0x82c3f108) (timeout: 2 s) : Too many open files error: unable to close Socket IO::Socket::INET=GLOB(0xd872058) - - Bad file descriptor And that was game over for ASSP. It would do the bad file descriptor / Too many open files errors for a few minutes and then shut down. ASSP would run fine for around a day, would get hit with the bad TLS stuff, and then die. At least I think that's what was happening.... For the griplist. here's a suggested addition to the GUI so that griplist functionality is documented: Griplist Database (griplist) <-- Instead of GreyIPlist Database (griplist) The griplist holds IP scores based on notspam / spam messages stored in the ASSP corpus [ the last X days worth of messages? or is it the entire corpus? Is it also updated in real time as messages come in??] An IP's griplist score is used in a formula to either positively or negatively impact the message/ip score for incoming mail. See the gripValencePB for more explanation. By default, ASSP shares the local griplist to Sourceforge (this can be disabled by checking noGriplistUpload). This uploaded griplist is analyzed by a process running at Sourceforge and then used to populate a new griplist based on the submitted griplist data from all participating ASSP servers. This combined griplist is downloaded (again by default, this can be disabled using noGriplistDownload), replacing the locally created griplist with the data from all ASSP servers. While the griplist was originally called "Grey-IP-List" in ASSP, that was changed in 2007 (???), the griplist should not be confused with Greylisting. Griplist has nothing to do with greylisting/delaying. [may also include something to explain the difference between griplist and penaltybox?] On Mon, Oct 11, 2021 at 3:58 AM Thomas Eckardt <thomas.ecka...@thockar.com> wrote: > 1) - this will be fixed > > 2) - not default settings needs to be checked by the admin > > 3) - BDB-error.txt files are used to record BerkeleyDB errors - a handle > is keeped opened by each thread for each BDB-environment (db) as long as > the database-env is opened by the thread > most times such handles are opened until the thread is stopped (assp > ends) > > 4) > a) yes > > b) yes > > c) > Scoring is done using gripValencePB and the resulting message/ip - score > is calculated as follows: > > if the grip value is < 0.3 : -int(((0.3 - grip value) / 0.3) * > gripValencePB) > if the grip value is > 0.7 : int(((grip value - 0.7) / 0.3) * > gripValencePB) > grip values between 0.3 and 0.7 are ignored. > > > windows handles (IMHO): > > system-max handles : ~ 2**24 (>16.000.000) > process/thread max handles : ~ 10.000 (configurable in the registry - > hex(2710)) > > use Testlimit.exe / Testlimit64.exe -h > > How ever - the c-library used by the process sets the handle (and other) > limits! If a valid but not-default-system c-lib is found in the PATH, it > will be used (with there internal limit settings). > > assp never uses more than 2000 handles (typical less than 1000) on any > system watched by me > most handles (~ 1.000.000) are used permanently by mysqld on windows > > > Thomas > > > > Von: "K Post" <nntp.p...@gmail.com> > An: "ASSP development mailing list" < > assp-test@lists.sourceforge.net> > Datum: 09.10.2021 16:21 > Betreff: [Assp-test] Multiple topics: Griplist vs GripList, stuck > open handles to dbd-error on windows, more. > ------------------------------ > > > > Several related items here: > > 1. Bug? Rebuild process still uploading griplist, even if disabled, > due possibly to case error in code. > 2. ASSP not checking for valid griplist, if an invalid folder name is > entered > 3. On windows, Rebuild process > leaving \tmpDB\rebuildDB\BDB-error.txt handle open (wider issue with > DBD-error.txt files getting stuck open on Windows?) > 4. Griplist clarification request > > > 1) I've got noGriplistUpload and noGriplistDownload both checked in the > GUI. But I noticed that ,at the end of my rebuild log, it's still doing > the upload > > Uploading Griplist via Direct Connection > > > *rebuildspamdb.pm* <http://rebuildspamdb.pm/> has > return if $main::noGripListUpload; > before the upload happens, but I think there's a case mistake there. > > I believe it should be Griplist, with a lower case L. > > > 2) In trying to figure out my ASSP on Windows crashes due to too many open > files, I've started using Sysinternals Handle utility to look at open > handles. Yesterday I saw 1300+ handles open > to d:\assp\tmpDB\Griplist\BDB-error.txt > > I believe that had something to do with me having an invalid griplist > database name, "d/griplist" but the d folder didn't exist on my new server > config!! My fault there for sure, not sure how d/ prefixed it, but it > appears ASSP didn't check to see that the folder existed. Shame on me for > missing my config file, but it might be good for ASSP to warn or create the > folder if it doesn't exist. > > I cleared out the griplist database file to test and restarted. I don't > see any more open griplist\dbd-error.txt handles, even though I'm getting > many many bad SMTP servers connecting with early TLS. Good. I'll probably > put the griplist database back (obviously with a valid filename!!) See > #3.... > > > 3) Windows (both 2012 and 2019) might not be closing Berkely error files > correctly in general. > > When the new version started logging the new SSL errors, I >believe< > that's when it started trying to access the non-existent "d/" folder config > error in m assp.cfg. Every time an early TLS line was caught, BerkeleyDB > would keep the error file open. Strangely, the error file was 0kb. > > Now that I've cleared out the griplist entry, I don't get those > /tmpdb/griplist/dbd-error.txt open handles. However, after my rebuild over > night, I see and open handle to: > tmpDB\rebuildDB\BDB-error.txt > about 9+ hours after rebuild. > > Is that normal? Might Windows not be closing handles correctly in general? > > 4) While I'm at it, could some clarification be provided as to the > function of the griplist? (and please correct me if anything I say here is > incorrect!!) I've searched like crazy over the last couple of days, but > can't find the answers I'm looking for. > > The griplist is an ip scoring database correct? I know it was originally > called the grey-ip-list, 15+ years ago, but then greylisting became common > language for delaying, so the original grayiplist started being called the > "griplist" to avoid confusion. > > Note: the gui still says: > GreyIPlist Database (griplist) > The file with the current Grey-IP-List database -- make this blank if you > don't use it. > If I'm understanding griplist correctly, I think the gui should be > reworded. Maybe some explanation added too?? > > There's also the optional upload and download concept of the griplist. > This appears to send the local griplist to sourceforge, it gets processed > by whatever you've got running on the backend, and then I can (also > optionally) download another griplist which is based on all ASSP user > data. If you don't upload, you can't download, and that's fair - you need > to contribute to benefit from the group. > > The charity that I work for has a pretty poorly thought out privacy policy > that requires me to jump through all kinds of hoops when sharing >any< > information. It's frustrating for sure. That means I cannot upload our > griplist without petitioning an internal committee. I'm thinking I want to > do that, but need to fully understand the griplist first. > > - If we don't upload/download, the griplist is stlil maintained > locally, just only with my data right? > - If I get approval to share the ip data and download the griplist, > that downloaded griplist is merged with my local griplist? > - And in the end, how is a match on the griplist scored? Let's say an > IP is on the griplist as being a really bad IP. What score does a message > get? Is that configurable? > > > As always, thank you > Ken > > > > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test >
_______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
