Re: [fpc-other] [fpc-pascal] The end of Gmane

[Mark Morgan Lloyd]

Lukasz Sokol wrote:

On 30/07/16 20:04, Mark Morgan Lloyd wrote:

Lukasz Sokol wrote:



I'm sure that if one dug there would be plenty of prewritten mail
distribution programs, probably in Perl on unix (and if somebody
doesn't like that, he might as well stop reading here).


there was even a pre-cursor to (probably) every known program like that, 
fetchmail...
up to the interested reader to find what was it written in ;) and by who.
(as in, I remember fetchmail from ~15-20 years ago, it had quite /specific/ 
approach to config files)


Fetchmail's OK if you can rely on an ISP running a mail server (MTA, 
Sendmail or equivalent) which will accept all IDs for a given domain in 
a compatible format. Some usages aren't compatible, and I've had to 
write a local equivalent.



The key to managing anything like this is usually to use a different
email ID for each mailing list, e.g. something like
fred.fpc-ot...@telegraphy.co.uk for this one. Then either make sure
you deal with an ISP who is prepared to accept either all mail for
your domain or a significant number of predefined names (i.e. rather
than just a single one) or set up your own email server
(Sendmail+Cyrus or whatever). Then use Thunderbird etc. rules to
distribute traffic to folders by recipient.


Yes. However it requires that filter/sorter be running somewhere all the
time, to sort the messages. So you end up having a 24/7 computer in
the house, which isn't to everybody's taste... or hosting it to run in a 
vm somewhere for free/non-free. And, well if you happen to think, that

an DD/OpenWRT'd home ADSL modem(*) with some storage is enough, try to think
of a good answer to 'Why do you need that hard drive attached to the modem all 
the time?'
question, in 2 minutes ... ;)

(*) (of which workable ones are not as easy to come by by the way either)


This is basically stuff I'm working on at the moment, to the detriment 
of other things like testing FPC on outrageous platforms. Let's consider 
the two sides of the problem separately.


First, if you want to receive all possible emails (i.e. with user names 
mis-spelt etc.) you'll usually have to set up your own server somewhere. 
But these days a Raspberry Pi can use a 64Gb SDCard instead of a 
conventional hard disc, and that is more than adequate to buffer email 
traffic. In our case we've got Sendmail as an MTA, greylisting, and 
Cyrus as storage... there's a few problems in there related to 
duplicated messages but it works. Client systems need to know what user 
IDs they're collecting mail for, and periodically anything uncollected 
will need to be reviewed.


Second, connection etc. A dedicated email server will usually require a 
routable IP4 address (i.e. not 192.168.x.x etc.) and these have been 
increasingly difficult to get over the last few years. Routable IP6 
addresses are easier, but ADSL services that support IP6 are still 
comparatively rare.


A solution to this is to use a non-routable IP address allocated by DHCP 
and served over ADSL or 3G/4G mobile. Over that you set up an L2TP 
tunnel, and at your end you have a mail server listening on a routable 
IP6 address (and IP4 if you can get one). You possibly also have a very 
low-priority IP4 server run by your ISP, but this will need very heavy 
spam filtering since it will get a lot of undesirable stuff.


L2TP in this mode is still rare, we're beta-testing it with our ISP but 
since they also manufacture the Firebrick appliances used by a fair 
number of small ISPs expect it to be rolled out gradually. 
http://aa.net.uk/broadband-l2tp.html


We've historically used Draytek router/modems connected via multiple 
carriers to AAISP (link above). Over the years we've seen reliability 
gradually improve as carriers have gradually introduced proper failover 
when e.g. taking a core router out overnight (we run 24x7. They don't). 
We're currently experimenting with diversified connections, and one of 
the things I've got on my plate is to integrate an inference engine with 
our firewall to handle traffic control (i.e. choosing which connection 
to use, irrespective of routing).


The inference engine is from NASA, and written in Perl. Sorry.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-other maillist  -  fpc-other@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-other

Re: [fpc-other] [fpc-pascal] The end of Gmane

[Lukasz Sokol]

On 30/07/16 20:04, Mark Morgan Lloyd wrote:
> Lukasz Sokol wrote:

> I'm sure that if one dug there would be plenty of prewritten mail
> distribution programs, probably in Perl on unix (and if somebody
> doesn't like that, he might as well stop reading here).
> 

there was even a pre-cursor to (probably) every known program like that, 
fetchmail...
up to the interested reader to find what was it written in ;) and by who.
(as in, I remember fetchmail from ~15-20 years ago, it had quite /specific/ 
approach to config files)

> The key to managing anything like this is usually to use a different
> email ID for each mailing list, e.g. something like
> fred.fpc-ot...@telegraphy.co.uk for this one. Then either make sure
> you deal with an ISP who is prepared to accept either all mail for
> your domain or a significant number of predefined names (i.e. rather
> than just a single one) or set up your own email server
> (Sendmail+Cyrus or whatever). Then use Thunderbird etc. rules to
> distribute traffic to folders by recipient.

Yes. However it requires that filter/sorter be running somewhere all the
time, to sort the messages. So you end up having a 24/7 computer in
the house, which isn't to everybody's taste... or hosting it to run in a 
vm somewhere for free/non-free. And, well if you happen to think, that
an DD/OpenWRT'd home ADSL modem(*) with some storage is enough, try to think
of a good answer to 'Why do you need that hard drive attached to the modem all 
the time?'
question, in 2 minutes ... ;)

(*) (of which workable ones are not as easy to come by by the way either)

> 
> Otherwise it's not particularly difficult to pipe messages from
> Sendmail into a discussion group server, although I'd suggest that
> anybody who chooses INN for this use v1 rather than anything later.
> It's also not too difficult to redirect anything sent locally to an
> external mailing list, but there is a number of "gotchas" that one
> has to be aware of lest one sets up a loop.
> 

Or for the matter, if the project happens to run a mailing list, to set up 
their own mail2news gateway with a nntp server of their own.
(however it increases maintenance which nobody ever has time for...)
(and you still might need to deal with takedown notices, as Gmane admin had to)

-L.

___
fpc-other maillist  -  fpc-other@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-other

Re: [fpc-other] [fpc-pascal] The end of Gmane

[Mark Morgan Lloyd]

Lukasz Sokol wrote:


And Thunderbird can search the nntp archive on the server too;
something that is quite much more efficient than searching in mailing list 
archives.

Also it enabled you to be able to read the mailing list, without going through 
subscription process too;
(I've no problem to subscribe if I want to post somewhere but for just reading, 
gmane nntp was infinitely more convenient
than e.g. the mailman web interface before HyperKitty :J )

I wish you also could have seen the thread.gmane.org web interface, and the way 
it searched
however they've offlined it already.

Gmane will be sorely missed if the nntp gateway ever happens to go away...


I'm sure that if one dug there would be plenty of prewritten mail 
distribution programs, probably in Perl on unix (and if somebody doesn't 
like that, he might as well stop reading here).


The key to managing anything like this is usually to use a different 
email ID for each mailing list, e.g. something like 
fred.fpc-ot...@telegraphy.co.uk for this one. Then either make sure you 
deal with an ISP who is prepared to accept either all mail for your 
domain or a significant number of predefined names (i.e. rather than 
just a single one) or set up your own email server (Sendmail+Cyrus or 
whatever). Then use Thunderbird etc. rules to distribute traffic to 
folders by recipient.


Otherwise it's not particularly difficult to pipe messages from Sendmail 
into a discussion group server, although I'd suggest that anybody who 
chooses INN for this use v1 rather than anything later. It's also not 
too difficult to redirect anything sent locally to an external mailing 
list, but there is a number of "gotchas" that one has to be aware of 
lest one sets up a loop.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-other maillist  -  fpc-other@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-other

Re: [fpc-other] [fpc-pascal] The end of Gmane

[Lukasz Sokol]

On 29/07/16 09:48, Sven Barth wrote:
> Am 29.07.2016 10:02 schrieb "Bo Berglund"  >:
>> 
>> On Fri, 29 Jul 2016 01:38:16 +0530, Vasudev Ram 
>> mailto:vasudev...@gmail.com>> wrote:
>> 
>>> Hi list,
>>> 
>>> Sorry if this has been mentioned earlier here already, or if it
>>> is considered off-topic (and if so, will not do it again). But I
>>> think it is relevant and of interest to people reading this list
>>> via Gmane:
>>> 
>>> The End of Gmane? (ingebrigtsen.no )
>>> 
>>> https://news.ycombinator.com/item?id=12180547
>> 
>> OH NO!!! I have used the GMane SMTP->NNTP service since at least 10
>> years. It is my main channel into development communities using
>> mail lists and is an invaluable resource.
>> 
>> I cannot even begin to understand how I would be able to solve the 
>> programming problems I encounter without this resource available.
>> 
>> Mail lists is no alternative because there is no structure and I
>> would be flooded with email hiding the normal mail use. NNTP is
>> king in these regards.
> 
> I only use the mailing lists and I don't see your problems at all. 
> I've set up my google mail account in such a way that all mails from
> a specific list get the list's name as label attached and that they
> don't go through the normal mail entry. In Thunderbird these labels
> appear as IMAP folders. So no clutter here. Additionally I use
> threaded view and as long as no one('s client) messes up the list-ID
> that also works without problems. How do you think Gmane manages to
> keep track of the structure? It also uses information contained in
> the mails...
> 
> Regards, Sven
> 

I started using Gmane long ago, because back then neither Google or Yahoo,
IIRC, neither of them had the email rules feature (like, filter by sender 
address
and move to a folder by name of the mailing list. )
Some free and non-free email providers still might not have that feature, even 
in this day and age...

Also email providers can have quite tough quota rules, for message size and/or 
count.

Thunderbird, well yes, it does download same data from NNTP server as you would 
via IMAP.
(and keeps it on your disk)
But you're in control here too - it only downloads whole article IFF you wish 
to read it (click its header) or
otherwise request it to be downloaded while syncing the newsgroup.
(IMAP synchronization of folders is not on demand: you either sync whole 
messages (can not just only download headers),
 or have the 'keep the messages for this account on this computer' checkbox 
ticked, so you keep nothing locally)

And Thunderbird can search the nntp archive on the server too;
something that is quite much more efficient than searching in mailing list 
archives.

Also it enabled you to be able to read the mailing list, without going through 
subscription process too;
(I've no problem to subscribe if I want to post somewhere but for just reading, 
gmane nntp was infinitely more convenient
than e.g. the mailman web interface before HyperKitty :J )

I wish you also could have seen the thread.gmane.org web interface, and the way 
it searched
however they've offlined it already.

Gmane will be sorely missed if the nntp gateway ever happens to go away...

-L.

___
fpc-other maillist  -  fpc-other@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-other

Re: [fpc-other] [fpc-pascal] The end of Gmane

[Sven Barth]

Am 29.07.2016 10:02 schrieb "Bo Berglund" :
>
> On Fri, 29 Jul 2016 01:38:16 +0530, Vasudev Ram
>  wrote:
>
> >Hi list,
> >
> >Sorry if this has been mentioned earlier here already, or if it is
> >considered off-topic (and if so, will not do it again). But I think it
> >is relevant and of interest to people reading this list via Gmane:
> >
> >The End of Gmane? (ingebrigtsen.no)
> >
> >https://news.ycombinator.com/item?id=12180547
>
> OH NO!!!
> I have used the GMane SMTP->NNTP service since at least 10 years.
> It is my main channel into development communities using mail lists
> and is an invaluable resource.
>
> I cannot even begin to understand how I would be able to solve the
> programming problems I encounter without this resource available.
>
> Mail lists is no alternative because there is no structure and I would
> be flooded with email hiding the normal mail use. NNTP is king in
> these regards.

I only use the mailing lists and I don't see your problems at all.
I've set up my google mail account in such a way that all mails from a
specific list get the list's name as label attached and that they don't go
through the normal mail entry. In Thunderbird these labels appear as IMAP
folders. So no clutter here.
Additionally I use threaded view and as long as no one('s client) messes up
the list-ID that also works without problems. How do you think Gmane
manages to keep track of the structure? It also uses information contained
in the mails...

Regards,
Sven
___
fpc-other maillist  -  fpc-other@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-other