Re: [sqlite] Mailing list shutting down...

[Jeffrey Schiller]

Check out:

   http://c.qyv.net/mailmansrc/info/a6250b7ba075294c

This is a patch to mailman version 2.1.20 (the version deployed on
sqlite.org). It adds a three day waiting period between subscription
attempts. When applying this patch also be sure to edit Defaults.py (which
is generated at installation time from Defaults.py.in) to include the new
variable for controlling the subscription window.

You may also want to set SUBSCRIBE_FORM_SECRET to a secret string. This
will enable verification that a submitted form is both from the site and
filled out after 5 seconds (to thwart bots).

Enjoy.

-Jeff

On Thu, Jun 14, 2018 at 11:02 PM Jeffrey Schiller <
jeffrey.schil...@gmail.com> wrote:

> I am so offering...
>
> -Jeff
>
>
> On Wed, Jun 13, 2018 at 12:42 PM Richard Hipp  wrote:
>
>> On 6/13/18, Michael Tiernan  wrote:
>> > May I respectfully suggest to everyone that offering solutions, while
>> > valuable and helpful, may not be as valuable as the offer of assistance
>> > to our listmaster.
>>
>> +1  :-)
>>
>> --
>> D. Richard Hipp
>> d...@sqlite.org
>> ___
>> sqlite-users mailing list
>> sqlite-users@mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Jeffrey Schiller]

I am so offering...

-Jeff

On Wed, Jun 13, 2018 at 12:42 PM Richard Hipp  wrote:

> On 6/13/18, Michael Tiernan  wrote:
> > May I respectfully suggest to everyone that offering solutions, while
> > valuable and helpful, may not be as valuable as the offer of assistance
> > to our listmaster.
>
> +1  :-)
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Jeffrey Schiller]

The latest version of Mailman does implement a Captcha (via reCaptcha).
Also, if you set mm_cfg.SUBSCRIBE_FORM_SECRET to a secret value, Mailman
will insist that the subscription form be submitted after a slight delay,
which defaults to 5 seconds. This features exists in the version currently
used by the mailing list (2.1.20).

I have downloaded the source code (version 2.1.20) and am looking into
adding code to limit the rate of subscriptions for a given e-mail address
to a configurable value. Something like 1 to 2 days should do the trick.

-Jeff

On Wed, Jun 13, 2018 at 12:15 PM Richard Hipp  wrote:

> On 6/13/18, Brian Curley  wrote:
> > Doesn't the Fossil site already have a Capcha interface built into it
> that
> > could be adopted to enforce additional authentication around
> subscriptions?
>
> There are no captchas built into GNU MailMan.  You enter your email
> address to subscribe and you get a confirmation email.  Click on a
> link in the confirmation email.  Then your subscription goes to
> moderation.  After the moderator approves, you are signed up.
>
> The above system works fine to keep nefarious actors out of the subscriber
> list.
>
> But that is not the problem.  The problem is that the bad guys don't
> care about getting onto the subscriber list.  They just want to
> generate as many bogus confirmation emails as they can, to harass the
> people who are receiving the confirmation emails.
>
> The obvious solution in GNU Mailman would be to only allow a single
> confirmation email to go out per email address.  After that one email,
> the corresponding email address is never allowed to sign up again.
>
> This simple fix is complicated by several factors:
>
> (1) Nobody seems to want to own the GNU MailMan software.  It is not
> well maintained as far as I can see.
>
> (2) MailMan does not seem to use a database other than the filesystem
> and perhaps Python Pickle files, at least not that I have found, so
> recording extra information such as who has previously requested a
> subscription involves major structural changes to the code.
>
> (3) MailMan itself seems to be a collection of scripts that must be
> all installed in multiple well-known directories.  It is difficult to
> identify what files are part of the MailMan implementation and what
> files are not, making maintenance error-prone for people (like me) who
> are unfamiliar with where to find all the pieces.
>
> (4) There is a GNU MailMan mailing list, but in my past interactions,
> there was nobody there who was willing to help with spam problems.
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Ian Zimmerman]

On 2018-06-14 17:17, Vincenzo Campanella wrote:

> uses googlegroups.com, that then works exactly as a mailing list
> (their mail address is wx-us...@googlegroups.com); perhaps this can
> give you an alternative idea...

It works fine as a mailing list _if_ most (better: all) posters use
it as such.  When you get large number of posts from the Web interface
it gets ugly, with broken threads and such all over.

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Vincenzo Campanella]

Il 13.06.2018 15:28, Olivier Mascia ha scritto:

Le 13 juin 2018 à 13:22, Richard Hipp  a écrit :

Unfortunately, I'm going to need to shut down this mailing list due to
robot harassment.
...
I have already suspended new subscriptions.  Existing subscribers will
be able to continue using this list until I come up with a replacement
(or a fix to the current problem) but no new subscribers will be
accepted.

I don't have experience with GNU MailMan, but isn't there some facility to protect the 
subscription request page using some Googlesque "I'm not a Robot!" CAPTCHA, or 
anything like if GNU MailMan does not want to offer people to have whatever business with 
Google for any reason?

This, plus a black-listing mechanism which would warn admins (once!) when the 
same non-member subscription request has happened let's say twice, without user 
confirmation, and simply denies new requests for that same email until admins 
either validate the subscription or reset it.

Might complicate the work of robots enough to render the game uninteresting.

That's a personal preference, but I value mailing lists and appreciate much 
less web-based forums.
WxWidgets (a library for writing GUI with C++ or Python) uses 
googlegroups.com, that then works exactly as a mailing list (their mail 
address is wx-us...@googlegroups.com); perhaps this can give you an 
alternative idea...

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[George]

On Wed, 13 Jun 2018 07:22:20 -0400
Richard Hipp  wrote:

> Unfortunately, I'm going to need to shut down this mailing list due to
> robot harassment.  I am working to come up with a fix or an
> alternative now.  Your suggestions are welcomed.
> 
> This mailing list has operated for many years using GNU MailMan.
> Unfortunately, that software is not able to cope with modern robot
> spammers, even with the latest updates.  And the source code for
> MailMan is sufficiently opaque that I am unable to work on it.
> 
> The most recent problem is that robots are visiting the subscription
> page and entering innocent user's email addresses and names.  This
> causes a confirmation email to be sent to that user.  If it were just
> single confirmation email that the user could ignore, that would be
> fine.  But apparently MailMan sends one email for each subscription
> request.  The robots have figured this out and are putting in hundreds
> of subscription requests for the same individual, apparently to harass
> them.
> 
> I have already suspended new subscriptions.  Existing subscribers will
> be able to continue using this list until I come up with a replacement
> (or a fix to the current problem) but no new subscribers will be
> accepted.
> 

Hello,

Sounds like a nasty problem, sorry to hear.

The usual algorithm and way of dealing with this is to throttle the
nasty one's while degrading slightly the service for legitimate
subscribers. I have in mind spamd from OpenBSD when I am giving this
example here. Sounds like the code of MailMan is difficult to change is
this the repo:

https://savannah.gnu.org/cvs/?group=mailman

How many pieces of the infrastructure are under your control? What is
the pattern of undesired activity. If you can log the request for
registration and pick out the addresses you can block further emails
from being send, put a max etc.. If you control a firewall the only way
to really control stuff you can block throttle the access for the
offending IP's or networks while the problem appears. 

Such things are best dealt with initially by collecting data and
modifying the middle layer control algorithm or its meta data until you
have a good average. I am not someone who deals with this problem on a
daily basis so please take this with a grain of salt. 

I think the issue here is that mailman talks to your mail server and
then it is actually you sending the emails to the victims. Depending on
the mail server you may be able to limit the requests, block them
completely or parse or classify them as spam. The best solution if you
are not able to modify mailman and don't wish to mess with mail servers
is to write a capture service that speaks only the basics of the mail
protocol and can count the number of to headers with the same address
per second, minute whatever and only after that validation forward it to
the legitimate mail sending server. There must be a place where you
configure your mail server in mailman just plop your capture
service there and the mail server can be configured or will not care if
the mail takes another hop before arriving at 25.

There are certainly thousand ways to do this but since you're running
MailMan I assume you wish a clean and simple solution ... heck look at
SQLite ... so I think the capture one might be the best no matter what
the mailman like package and mail server. You are in control ultimately.

HTH,
Regards,
George

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Damon]

On 6/13/18 6:52 PM, Bob Friesenhahn wrote:
> On Wed, 13 Jun 2018, Jeffrey Schiller wrote:
>
>> Would limiting subscription requests to one per day help. I'm
>> familiar with
>> the Mailman code, having modified it for use at MIT, and can code the
>> necessary changes. I suspect only one file would need to be changed.
>
> The problem is knowing what "one" means.  The subscription request is
> likely submitted via http/https into the web form and using a bogus
> email subscription address (of the "victim").  A botnet is able to
> submit these requests from hundreds of IP addresses.
>
> If mailman supports subscription requests via SMTP email (I don't
> remember that it does), then the problem is worse.
>
> If only one new subscription is allowed on the list per day, then
> there is a trivial DOS (no new valid subscriptions are possible) as
> soon as the one daily subscription has been consumed.
>
> Bob

Mailman does allow for email subscriptions, which has the same risks of
spoofing.

Where One Subscription limits could help is that it should be possible
for Mailman to allow there to be only one pending subscription for a
given email address (and these by default expire after 3 days), so if
the botnet is spamming the subscription address, the victim gets just
one email every 3 days.

It should also be possible to log these IP addresses and excessive
requests could trigger fail2ban to block that IP address for a while.

-- 
Richard Damon

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[J Decker]

On Wed, Jun 13, 2018 at 5:17 PM Simon Slavin  wrote:

> 13 Jun 2018, at 11:52pm, Bob Friesenhahn 
> wrote:
>
> > The problem is knowing what "one" means.  The subscription request is
> likely submitted via http/https into the web form and using a bogus email
> subscription address (of the "victim").  A botnet is able to submit these
> requests from hundreds of IP addresses.
>
>
And Hooray for TOR
https://www.dan.me.uk/tornodes


> First you accept only one request per IP address for every twentyfour
> hours.  You might as well just wipe your address list at midnight rather
> than do the tricky programming to implement a rolling 12 hour window.
>
> Second you have the form page generate a random number every time it shows
> the form.  The submission has to include the number sent to that IP
> address, and it has to be done at least five seconds after the number was
> generated.  This ties up that bot (though not the whole botnet) for five
> seconds.  One assumes that humans take more than 5 seconds to type their
> password twice and hit 'submit' so they won't even notice the difference.
> People who copy-and-paste their email address into the 'verify' field
> deserve what they get.
>
> Third you accept only one request per email address per week.
>
> The second of the above defeats a lot of bots.  They submit the request
> without ever downloading the form in the first place.
>
> For all the above you need two tables of data and some python
> programming.  Unfortunately I don't know Python.
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Simon Slavin]

13 Jun 2018, at 11:52pm, Bob Friesenhahn  wrote:

> The problem is knowing what "one" means.  The subscription request is likely 
> submitted via http/https into the web form and using a bogus email 
> subscription address (of the "victim").  A botnet is able to submit these 
> requests from hundreds of IP addresses.

First you accept only one request per IP address for every twentyfour hours.  
You might as well just wipe your address list at midnight rather than do the 
tricky programming to implement a rolling 12 hour window.

Second you have the form page generate a random number every time it shows the 
form.  The submission has to include the number sent to that IP address, and it 
has to be done at least five seconds after the number was generated.  This ties 
up that bot (though not the whole botnet) for five seconds.  One assumes that 
humans take more than 5 seconds to type their password twice and hit 'submit' 
so they won't even notice the difference.  People who copy-and-paste their 
email address into the 'verify' field deserve what they get.

Third you accept only one request per email address per week.

The second of the above defeats a lot of bots.  They submit the request without 
ever downloading the form in the first place.

For all the above you need two tables of data and some python programming.  
Unfortunately I don't know Python.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Bob Friesenhahn]

On Wed, 13 Jun 2018, Jeffrey Schiller wrote:


Would limiting subscription requests to one per day help. I'm familiar with
the Mailman code, having modified it for use at MIT, and can code the
necessary changes. I suspect only one file would need to be changed.


The problem is knowing what "one" means.  The subscription request is 
likely submitted via http/https into the web form and using a bogus 
email subscription address (of the "victim").  A botnet is able to 
submit these requests from hundreds of IP addresses.


If mailman supports subscription requests via SMTP email (I don't 
remember that it does), then the problem is worse.


If only one new subscription is allowed on the list per day, then 
there is a trivial DOS (no new valid subscriptions are possible) as 
soon as the one daily subscription has been consumed.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Jeffrey Schiller]

Would limiting subscription requests to one per day help. I'm familiar with
the Mailman code, having modified it for use at MIT, and can code the
necessary changes. I suspect only one file would need to be changed.

-Jeff

On Wed, Jun 13, 2018 at 6:16 PM Gary R. Schmidt  wrote:

> On 13/06/2018 21:42, Gary R. Schmidt wrote:
> > On 13/06/2018 21:22, Richard Hipp wrote:
> >> Unfortunately, I'm going to need to shut down this mailing list due to
> >> robot harassment.  I am working to come up with a fix or an
> >> alternative now.  Your suggestions are welcomed.
> >>
> >> This mailing list has operated for many years using GNU MailMan.
> >> Unfortunately, that software is not able to cope with modern robot
> >> spammers, even with the latest updates.  And the source code for
> >> MailMan is sufficiently opaque that I am unable to work on it.
> >>
> >> The most recent problem is that robots are visiting the subscription
> >> page and entering innocent user's email addresses and names.  This
> >> causes a confirmation email to be sent to that user.  If it were just
> >> single confirmation email that the user could ignore, that would be
> >> fine.  But apparently MailMan sends one email for each subscription
> >> request.  The robots have figured this out and are putting in hundreds
> >> of subscription requests for the same individual, apparently to harass
> >> them.
> >>
> >> I have already suspended new subscriptions.  Existing subscribers will
> >> be able to continue using this list until I come up with a replacement
> >> (or a fix to the current problem) but no new subscribers will be
> >> accepted.
> >>
> > This is an increasing problem, and has been discussed on the Mailman
> > mailing list recently, you should join them and see what mitigation
> > strategies are available.
> >
> One is here: https://github.com/noabospam/abospam
>
> Cheers,
> GaryB-)
>
>
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Gary R. Schmidt]

On 13/06/2018 21:42, Gary R. Schmidt wrote:

On 13/06/2018 21:22, Richard Hipp wrote:

Unfortunately, I'm going to need to shut down this mailing list due to
robot harassment.  I am working to come up with a fix or an
alternative now.  Your suggestions are welcomed.

This mailing list has operated for many years using GNU MailMan.
Unfortunately, that software is not able to cope with modern robot
spammers, even with the latest updates.  And the source code for
MailMan is sufficiently opaque that I am unable to work on it.

The most recent problem is that robots are visiting the subscription
page and entering innocent user's email addresses and names.  This
causes a confirmation email to be sent to that user.  If it were just
single confirmation email that the user could ignore, that would be
fine.  But apparently MailMan sends one email for each subscription
request.  The robots have figured this out and are putting in hundreds
of subscription requests for the same individual, apparently to harass
them.

I have already suspended new subscriptions.  Existing subscribers will
be able to continue using this list until I come up with a replacement
(or a fix to the current problem) but no new subscribers will be
accepted.

This is an increasing problem, and has been discussed on the Mailman 
mailing list recently, you should join them and see what mitigation 
strategies are available.



One is here: https://github.com/noabospam/abospam

Cheers,
GaryB-)


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Gary R. Schmidt]

On 14/06/2018 03:37, John Long wrote:

On Wed, 2018-06-13 at 21:42 +1000, Gary R. Schmidt wrote:



This is an increasing problem, and has been discussed on the Mailman
mailing list recently, you should join them and see what mitigation
strategies are available.


Well I'm sure he would like to, but subscriptions have probably been
suspended because of the attacks ;)


They haven't shut down access to the mailman lists, why would they do that?

If you're trying to be funny, don't give up your day job!

Cheers,
GaryB-)

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

On 6/13/18, Florian Weimer  wrote:
> if the bots are
> actually targeting innocent users, for most recipients, they can just
> use multiple aliases of the form ,
> , and so on.

I have a accumulated a good assortment of robot counter-measures over
the years, a few of which are outlined here:
https://www.fossil-scm.org/fossil/doc/trunk/www/antibot.wiki

The key thing is that MailMan is opaque (to me).  I cannot add these
defenses to MailMan.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Florian Weimer]

* Richard Hipp:

> On 6/13/18, Brian Curley  wrote:
>> Doesn't the Fossil site already have a Capcha interface built into it that
>> could be adopted to enforce additional authentication around subscriptions?
>
> There are no captchas built into GNU MailMan.  You enter your email
> address to subscribe and you get a confirmation email.  Click on a
> link in the confirmation email.  Then your subscription goes to
> moderation.  After the moderator approves, you are signed up.

Some largish operators use CAPTCHAs:

  

But wouldn't any replacement that allows email notification have
exactly the same signup issue?  Sure, you might only have one pending
signup request per email instead per list, but if the bots are
actually targeting innocent users, for most recipients, they can just
use multiple aliases of the form ,
, and so on.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Alessandro Marzocchi]

Using policyd one should be able to limit the rate at which one account
(subscribe confirmation) is able to send messages to a particular address,
or the overall number of mails sent to an address.

https://wiki.policyd.org/quotas

On Wed, Jun 13, 2018, 9:36 PM Richard Hipp  wrote:

> On 6/13/18, Alessandro Marzocchi  wrote:
> > Do you have control over postfix server? If so maybe adding a policy to
> the
> > account used for subscription confirmation may work. I dont have a PC
> > available at the moment but in the case i may check.
>
> I don't see how that could possibly help.  Please enlighten me if I am
> overlooking something.
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[rob . sqlite]

Richard,

We use Discourse (as a user) to get support for the Mail in a Box 
system. MIAB use Discourse for their support systems. I also think 
Discourse is used for the Ionic support pages as well as they have very 
similar looking interfaces. Until now I paid little attention to them.


I was going to say that I wasn't too impressed with it as a system, then 
I thought again and realised that it actually works pretty well and 
doesn't get in your way too much. That's a pretty good compliment as the 
software isn't in your face all the time telling you how nice it is, 
anybody used Slack recently :) We use it quite a lot and in hindsight it 
works well.


We've never spun a Discourse system up, but I have some spare time this 
evening and might just put one on a VMWare ESXI server and see how it 
looks.


I may be older than Dr Hipp as I can recall running Unix on a 64KB (yep 
KB) box in the 80's, so am very familiar with maximising resources, but 
I go the other way now and and run dedicated (but small and self 
contained) boxes that are very focused and don't try to cram as much 
into a single box/instance/VM as possible. I don't care about the fact 
I'm running 30 small Linux boxes on my single ESXI server as I can spin 
them up and most of the time they don't do anything.


I accept the issues over maintenance though, but I have a standard set 
of instructions I follow to harden the boxes and restrict logins with 
things like fail2ban. From start to finish I can have a hardened Ubuntu 
box up in around 20-30 minutes.  Very happy to share these instructions 
as somebody may say they are rubbish and can provide better hardening 
instructions.


I'm UK based, but happy to help, setting this sort of stuff up is 
something I can do and have regularly done (but NOT for Discourse), 
anyway I'm better at this than SQL :)


It's currently 20:30 UK time, can help, other people have helped me 
enough on this forum, so I feel I can contribute something back.


Thoughts on what needs to be done:

1. Setup the VMware instance correctly based on the Discourse info.

2. Provide some sort of access via ssh, passwords or whatever.

3. Details of IP addresses.

4. Firewall configure, Its not clear if these VM's are behind other 
firewalls and what the access rights are, e.g. you have https.


5. What's the SSL situation. We've just moved from RapidSSL to 
LetsEncrypt as a) They are free b) They self renew c) They weren't going 
to be blacklisted by Google as they were really Symantec certificates.


6. Does the installation need root access?

7. Postfix information, e.g. is it a satellite, a relay etc etc. One 
wrong move here and we get the IP address and domain name banned. Did 
that for our domain whilst setting up MIAB.


8. Installation of Discourse.

9. How do multiple people work together on the same box? Slack? Skype? 
Shouting loudly


10. Documenting the build?

11. How to test the build? Testers needed and a test plan needs to be 
put together. SQLite has an excellent reputation, this shouldn't sully 
it.


12. Profit?

Just my 2p worth,

Rob

On 13 Jun 2018, at 19:59, Richard Hipp wrote:


Cross-posted to the fossil-users mailing list since www.fossil-scm.org
and www.sqlite.org are the same machine and both mailing lists are
impacted by the current problem.

On 6/13/18, Luiz Américo  wrote:

How about using https://www.discourse.org/ ?

Open source projects can use for free


Thanks for the pointer, Luiz.

Discourse is moving the right direction, I think.  To install it, one
downloads a docker container and runs it on some Linux VM someplace.
(They recommend Digital Ocean, which is where I www3.sqlite.org is
hosted already.)  It's a self-contained package with minimal
dependencies that just works.  And it uses SQLite!  My kind of
software!

Here are my remaining points of heartburn with Discourse:

(1) The installation guide recommends using an external email service,
and they even recommend four appropriate services.  I clicked through
to each one, having never heard of any of them before.  All four are
pushing email marketing for companies sending 10 million or more
emails per month.  It seems to me that aggressive email marketing is
the root cause of my problem in the first place, so I am somewhat
reluctant to engage a marketing firm to help with the solution.
Fortunately, Discourse also allows one to use a self-hosting Postfix
installation, which is what we are currently running on sqlite.org.

(2) Discourse seems to want to run on a machine all by itself.  (It is
written in Rails and has its own webserver.)  I suppose I could spin
up yet another VM to do that.  But I learned this craft in an age
where machines were big and expensive and the goal was to cram as many
services as you could fit onto a single machine and IP address, and so
spinning up a separate machine with its own domain name just to manage
the mailing list seems wasteful, somehow.  And, that means there is
one more machine that I have 

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

On 6/13/18, Simon Slavin  wrote:
>
>
> On 13 Jun 2018, at 3:02pm, Simon Slavin  wrote:
>
>> you might try the solution described in the lower part of
>>
>> 
>>
>> and also the measure recommended here:
>>
>> 
>
> Did you get a chance to try these ?  One is a one-line fix.  The other is
> adding a few lines.  They can both be done with a text editor and the pages
> tell you which files to edit.  If I understand the problem you reported they
> would both fix it.

I did just try these.  It causes MailMan to fail with an error.
Probably the MailMan installation is messed up somehow.

I am currently looking at locating and saving off the subscriber
lists, then deleting and reinstalling MailMan and bulk subscribing
everybody

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

On 6/13/18, Alessandro Marzocchi  wrote:
> Do you have control over postfix server? If so maybe adding a policy to the
> account used for subscription confirmation may work. I dont have a PC
> available at the moment but in the case i may check.

I don't see how that could possibly help.  Please enlighten me if I am
overlooking something.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Alessandro Marzocchi]

Do you have control over postfix server? If so maybe adding a policy to the
account used for subscription confirmation may work. I dont have a PC
available at the moment but in the case i may check.

On Wed, Jun 13, 2018, 9:16 PM Chris Brody  wrote:

> On Wed, Jun 13, 2018 at 3:00 PM Richard Hipp  wrote:
> >
> > Cross-posted to the fossil-users mailing list since www.fossil-scm.org
>
> +1
>
> > Even so, Discourse does seem like considering.  Does anybody else have
> > any experience with Discourse, good or bad?
>
> SQLCipher switched over to Discourse for the discussion forum at:
> https://discuss.zetetic.net/c/sqlcipher
>
> Seems to work pretty well for the user community. I really like having
> a choice of social login, using Twitter myself.
>
> I cannot argue with you about the "heartburn", looks like a bear to setup.
>
> > Are there any volunteers willing to call me on skype and help set this
> up?
>
> I have very limited experience with the software stack involved, would
> be happy to teach myself in the process in case better qualified help
> is not forthcoming.
>
> On Wed, Jun 13, 2018 at 3:03 PM Simon Slavin  wrote:
> > [...]
> > > 
> > > [...]
> > > 
> >
> > Did you get a chance to try these ?
>
> Both sound like nice short-term solutions, seem to admit that the bots
> are bound to catch up someday:)
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Chris Brody]

On Wed, Jun 13, 2018 at 3:00 PM Richard Hipp  wrote:
>
> Cross-posted to the fossil-users mailing list since www.fossil-scm.org

+1

> Even so, Discourse does seem like considering.  Does anybody else have
> any experience with Discourse, good or bad?

SQLCipher switched over to Discourse for the discussion forum at:
https://discuss.zetetic.net/c/sqlcipher

Seems to work pretty well for the user community. I really like having
a choice of social login, using Twitter myself.

I cannot argue with you about the "heartburn", looks like a bear to setup.

> Are there any volunteers willing to call me on skype and help set this up?

I have very limited experience with the software stack involved, would
be happy to teach myself in the process in case better qualified help
is not forthcoming.

On Wed, Jun 13, 2018 at 3:03 PM Simon Slavin  wrote:
> [...]
> > 
> > [...]
> > 
>
> Did you get a chance to try these ?

Both sound like nice short-term solutions, seem to admit that the bots
are bound to catch up someday:)
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Simon Slavin]

On 13 Jun 2018, at 3:02pm, Simon Slavin  wrote:

> you might try the solution described in the lower part of
> 
> 
> 
> and also the measure recommended here:
> 
> 

Did you get a chance to try these ?  One is a one-line fix.  The other is 
adding a few lines.  They can both be done with a text editor and the pages 
tell you which files to edit.  If I understand the problem you reported they 
would both fix it.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

Cross-posted to the fossil-users mailing list since www.fossil-scm.org
and www.sqlite.org are the same machine and both mailing lists are
impacted by the current problem.

On 6/13/18, Luiz Américo  wrote:
> How about using https://www.discourse.org/ ?
>
> Open source projects can use for free

Thanks for the pointer, Luiz.

Discourse is moving the right direction, I think.  To install it, one
downloads a docker container and runs it on some Linux VM someplace.
(They recommend Digital Ocean, which is where I www3.sqlite.org is
hosted already.)  It's a self-contained package with minimal
dependencies that just works.  And it uses SQLite!  My kind of
software!

Here are my remaining points of heartburn with Discourse:

(1) The installation guide recommends using an external email service,
and they even recommend four appropriate services.  I clicked through
to each one, having never heard of any of them before.  All four are
pushing email marketing for companies sending 10 million or more
emails per month.  It seems to me that aggressive email marketing is
the root cause of my problem in the first place, so I am somewhat
reluctant to engage a marketing firm to help with the solution.
Fortunately, Discourse also allows one to use a self-hosting Postfix
installation, which is what we are currently running on sqlite.org.

(2) Discourse seems to want to run on a machine all by itself.  (It is
written in Rails and has its own webserver.)  I suppose I could spin
up yet another VM to do that.  But I learned this craft in an age
where machines were big and expensive and the goal was to cram as many
services as you could fit onto a single machine and IP address, and so
spinning up a separate machine with its own domain name just to manage
the mailing list seems wasteful, somehow.  And, that means there is
one more machine that I have to keep track of and manage and defend
from attacks, etc.

(Possible remedy to 2):  The main SQLite server (www.sqlite.org)
actually owns 3 IP addresses, only 2 of which are currently in use.  I
suppose I could run Discourse on that 3rd unused IP address.  But that
will end up being a non-standard setup

(3) The installation guide says that Discourse takes between 2 and 8
minutes to boot up.  Seriously?

Even so, Discourse does seem like considering.  Does anybody else have
any experience with Discourse, good or bad?

Are there any volunteers willing to call me on skype and help set this up?

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Luiz Américo]

How about using https://www.discourse.org/ ?

Open source projects can use for free

Luiz

Em qua, 13 de jun de 2018 14:37, John Long  escreveu:

> On Wed, 2018-06-13 at 21:42 +1000, Gary R. Schmidt wrote:
> >
> >
> > This is an increasing problem, and has been discussed on the Mailman
> > mailing list recently, you should join them and see what mitigation
> > strategies are available.
>
> Well I'm sure he would like to, but subscriptions have probably been
> suspended because of the attacks ;)
>
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[John Long]

On Wed, 2018-06-13 at 21:42 +1000, Gary R. Schmidt wrote:
> 
> 
> This is an increasing problem, and has been discussed on the Mailman 
> mailing list recently, you should join them and see what mitigation 
> strategies are available.

Well I'm sure he would like to, but subscriptions have probably been
suspended because of the attacks ;)

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

On 6/13/18, Michael Tiernan  wrote:
> May I respectfully suggest to everyone that offering solutions, while
> valuable and helpful, may not be as valuable as the offer of assistance
> to our listmaster.

+1  :-)

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Michael Tiernan]

May I respectfully suggest to everyone that offering solutions, while 
valuable and helpful, may not be as valuable as the offer of assistance 
to our listmaster.

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[José María Mateos]

On Wed, Jun 13, 2018 at 12:14:46PM -0400, Richard Hipp wrote:
> (1) Nobody seems to want to own the GNU MailMan software.  It is not
> well maintained as far as I can see.

I'm not an expert, but how does Sympa handle this? I remember a few 
years ago a lot of people were moving their Mailman systems to Sympa. It 
seems to be properly maintained too (latest release was April 19th).

Cheers,

-- 
José María (Chema) Mateos
https://rinzewind.org/blog-es || https://rinzewind.org/blog-en
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

On 6/13/18, Brian Curley  wrote:
> Doesn't the Fossil site already have a Capcha interface built into it that
> could be adopted to enforce additional authentication around subscriptions?

There are no captchas built into GNU MailMan.  You enter your email
address to subscribe and you get a confirmation email.  Click on a
link in the confirmation email.  Then your subscription goes to
moderation.  After the moderator approves, you are signed up.

The above system works fine to keep nefarious actors out of the subscriber list.

But that is not the problem.  The problem is that the bad guys don't
care about getting onto the subscriber list.  They just want to
generate as many bogus confirmation emails as they can, to harass the
people who are receiving the confirmation emails.

The obvious solution in GNU Mailman would be to only allow a single
confirmation email to go out per email address.  After that one email,
the corresponding email address is never allowed to sign up again.

This simple fix is complicated by several factors:

(1) Nobody seems to want to own the GNU MailMan software.  It is not
well maintained as far as I can see.

(2) MailMan does not seem to use a database other than the filesystem
and perhaps Python Pickle files, at least not that I have found, so
recording extra information such as who has previously requested a
subscription involves major structural changes to the code.

(3) MailMan itself seems to be a collection of scripts that must be
all installed in multiple well-known directories.  It is difficult to
identify what files are part of the MailMan implementation and what
files are not, making maintenance error-prone for people (like me) who
are unfamiliar with where to find all the pieces.

(4) There is a GNU MailMan mailing list, but in my past interactions,
there was nobody there who was willing to help with spam problems.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Brian Curley]

Doesn't the Fossil site already have a Capcha interface built into it that
could be adopted to enforce additional authentication around subscriptions?
Or a 2-step, email confirmation-type option, maybe? If they're robots
causing the problem, then they wouldn't be able to mive beyond the initial
attempt.

I signed up so long ago that I forget what the process involves.

Regards.

Brian P Curley


On Wed, Jun 13, 2018, 11:46 AM Richard Hipp  wrote:

> On 6/13/18, Chris Brody  wrote:
> > On Wed, Jun 13, 2018 at 10:44 AM jungle Boogie 
> > wrote:
> >> [...]
> >> http://spamassassin.apache.org/
> >
> > Maybe just add SpamAssassin to the existing GNU MailMan setup?
> >
> > http://www.jamesh.id.au/articles/mailman-spamassassin/
>
> That solves a different problem from the one we are having.
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

On 6/13/18, Chris Brody  wrote:
> On Wed, Jun 13, 2018 at 10:44 AM jungle Boogie 
> wrote:
>> [...]
>> http://spamassassin.apache.org/
>
> Maybe just add SpamAssassin to the existing GNU MailMan setup?
>
> http://www.jamesh.id.au/articles/mailman-spamassassin/

That solves a different problem from the one we are having.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Chris Brody]

On Wed, Jun 13, 2018 at 10:44 AM jungle Boogie  wrote:
> [...]
> http://spamassassin.apache.org/

Maybe just add SpamAssassin to the existing GNU MailMan setup?

http://www.jamesh.id.au/articles/mailman-spamassassin/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[jungle Boogie]

On 13 June 2018 at 04:22, Richard Hipp  wrote:
> Unfortunately, I'm going to need to shut down this mailing list due to
> robot harassment.  I am working to come up with a fix or an
> alternative now.  Your suggestions are welcomed.
>


OpenBSD uses Majordomo for their mailing lists:
https://en.wikipedia.org/wiki/Majordomo_(software)

However, on the page below they indicate how they fight spam - with
spamd and SpamAssassin.
https://www.openbsd.org/mail.html

https://man.openbsd.org/spamd
http://spamassassin.apache.org/

Maybe those can give you an idea on how to fight the spam submitted to
your subscribers.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Simon Slavin]

On 13 Jun 2018, at 12:22pm, Richard Hipp  wrote:

> The most recent problem is that robots are visiting the subscription
> page and entering innocent user's email addresses and names.

I'm surprised the server lasted this long.  That problem has been around since 
2010.  I don't know what you've already tried, or what type of bot is abusing 
the list, but you might try the solution described in the lower part of



and also the measure recommended here:



However, newer bots which work around these may have developed since those were 
invented.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Olivier Mascia]

> Le 13 juin 2018 à 13:22, Richard Hipp  a écrit :
> 
> Unfortunately, I'm going to need to shut down this mailing list due to
> robot harassment.
> ...
> I have already suspended new subscriptions.  Existing subscribers will
> be able to continue using this list until I come up with a replacement
> (or a fix to the current problem) but no new subscribers will be
> accepted.

I don't have experience with GNU MailMan, but isn't there some facility to 
protect the subscription request page using some Googlesque "I'm not a Robot!" 
CAPTCHA, or anything like if GNU MailMan does not want to offer people to have 
whatever business with Google for any reason?

This, plus a black-listing mechanism which would warn admins (once!) when the 
same non-member subscription request has happened let's say twice, without user 
confirmation, and simply denies new requests for that same email until admins 
either validate the subscription or reset it.

Might complicate the work of robots enough to render the game uninteresting.

That's a personal preference, but I value mailing lists and appreciate much 
less web-based forums.
-- 
Best Regards, Meilleures salutations, Met vriendelijke groeten,
Olivier Mascia


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Richard Hipp]

On 6/13/18, Tim Streater  wrote:
> Personally I'd be loath to see this list moved to a web page, for
> instance.

We invite you to submit working code that implements your desired solution.  :-)


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Tim Streater]

On 13 Jun 2018, at 12:22, Richard Hipp  wrote:

> Unfortunately, I'm going to need to shut down this mailing list due to
> robot harassment.  I am working to come up with a fix or an
> alternative now.  Your suggestions are welcomed.

Perhaps another subscription mechanism is needed, if that is their attack 
vector. Personally I'd be loath to see this list moved to a web page, for 
instance.


-- 
Cheers  --  Tim
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Mailing list shutting down...

[Gary R. Schmidt]

On 13/06/2018 21:22, Richard Hipp wrote:

Unfortunately, I'm going to need to shut down this mailing list due to
robot harassment.  I am working to come up with a fix or an
alternative now.  Your suggestions are welcomed.

This mailing list has operated for many years using GNU MailMan.
Unfortunately, that software is not able to cope with modern robot
spammers, even with the latest updates.  And the source code for
MailMan is sufficiently opaque that I am unable to work on it.

The most recent problem is that robots are visiting the subscription
page and entering innocent user's email addresses and names.  This
causes a confirmation email to be sent to that user.  If it were just
single confirmation email that the user could ignore, that would be
fine.  But apparently MailMan sends one email for each subscription
request.  The robots have figured this out and are putting in hundreds
of subscription requests for the same individual, apparently to harass
them.

I have already suspended new subscriptions.  Existing subscribers will
be able to continue using this list until I come up with a replacement
(or a fix to the current problem) but no new subscribers will be
accepted.

This is an increasing problem, and has been discussed on the Mailman 
mailing list recently, you should join them and see what mitigation 
strategies are available.


Cheers,
GaryB-)
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Mailing list shutting down...

[Richard Hipp]

Unfortunately, I'm going to need to shut down this mailing list due to
robot harassment.  I am working to come up with a fix or an
alternative now.  Your suggestions are welcomed.

This mailing list has operated for many years using GNU MailMan.
Unfortunately, that software is not able to cope with modern robot
spammers, even with the latest updates.  And the source code for
MailMan is sufficiently opaque that I am unable to work on it.

The most recent problem is that robots are visiting the subscription
page and entering innocent user's email addresses and names.  This
causes a confirmation email to be sent to that user.  If it were just
single confirmation email that the user could ignore, that would be
fine.  But apparently MailMan sends one email for each subscription
request.  The robots have figured this out and are putting in hundreds
of subscription requests for the same individual, apparently to harass
them.

I have already suspended new subscriptions.  Existing subscribers will
be able to continue using this list until I come up with a replacement
(or a fix to the current problem) but no new subscribers will be
accepted.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users