Re: [mailop] harassment/death threat detection/filtering/prosecution
Google has attempted to make a comment moderation engine using ML, available at https://www.perspectiveapi.com/#/home I don't know how well it works at the moment, it's typically been a very hard thing to do, since the amount of "data" in a comment tends to be small, and there is a lot of jargon and such in terms of context that can make different usage very complicated. My searches to remember that came up with some other alternatives in the same sort of space. As for "preferring to see them", that makes more sense when you receive very few of them. It's more complicated if your every post on say twitter generates dozens to hundreds of them. Most of them at that point are not "real", just offensive, and it seems unlikely that even humans would be able to tell the difference or deal with them at scale. One could imagine an automated system that collects the information in each and tries to group them by likely perps, using a scoring system over time to highlight the worse offenders. In a shared system, that could go even farther. In David Brin's _Earth_, one of the protagonists had an aside where they described a system that would scan their large email inbox and create a sentiment report highlighting with examples. I'm sure there are various systems like that probably tuned for brands and such on social media, and maybe even some for support desks. This use case is kind of a special case of that. Another alternative for such a system of detection is to have other people help go through the detritus for them, to relieve some of the dread that must come with constantly being bombarded with the cesspool. Anyways, I don't doubt that such crap isn't limited to social media, and that some makes its way to email... but I don't think there's been much work on it at that level. Brandon On Wed, Jul 15, 2020 at 2:15 PM Chris via mailop wrote: > On 2020-07-15 16:09, Jaroslaw Rafa via mailop wrote: > > Dnia 15.07.2020 o godz. 12:59:39 Grant Taylor via mailop pisze: > >> On 7/12/20 11:28 AM, Paul Ebersman via mailop wrote: > >>> But in these tense times, lots of PoC/non-white/non-cis get hate > >>> emails, death threats, etc. > > >> Death threats seem like the warrant more effort to stop than spam. > > > If someone would send death threats to me, I'd rather would like to > receive > > them, so that I can know whether I'm actually in danger and optionally > > report to the police or whatever... > > > > Which is what I was about to say too. In fact, by being really good at > blocking harrassment/death threats, you become partly liable if > something does happen. Because you hid the threat from the victim, and > potentially increased their risk of harm from something they didn't know > about. > > I had to deal with some of these in a large corporate environment. Each > time our "reaction" to the threatening (or sometimes just crackpot) > emails was carefully considered between me (filter options) and > Corporate Security before deployment. Eg: > > - Do we reject/bounce, or silently ignore? Usually the latter. > - Do we forward captures to CorpSec, not the user? > - Law enforcement? Legal C? > > Etc. > > These also have to be carefully differentiated from some of the > wide-spread stuff like the "I have a contract on you, pay me, and I > won't kill you" extortion soan/scams. Those should just be nuked. > > I would generally not recommend filtering on "threatening content" > per-se, but on spam, however you usually do it - like DNSBLs or > spamassassin or whatever... Then make it a point of user education to > tell people what to do if they get threatening email that gets past the > general spam filtering. Which may be LE, may be your own corporate, may > be custom filtering, etc. This approach will generally tend to weed out > the normal broadcast bullshit, and leave behind the stuff the victim/you > really need to know about. > > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] harassment/death threat detection/filtering/prosecution
On 2020-07-15 16:09, Jaroslaw Rafa via mailop wrote: Dnia 15.07.2020 o godz. 12:59:39 Grant Taylor via mailop pisze: On 7/12/20 11:28 AM, Paul Ebersman via mailop wrote: But in these tense times, lots of PoC/non-white/non-cis get hate emails, death threats, etc. Death threats seem like the warrant more effort to stop than spam. If someone would send death threats to me, I'd rather would like to receive them, so that I can know whether I'm actually in danger and optionally report to the police or whatever... Which is what I was about to say too. In fact, by being really good at blocking harrassment/death threats, you become partly liable if something does happen. Because you hid the threat from the victim, and potentially increased their risk of harm from something they didn't know about. I had to deal with some of these in a large corporate environment. Each time our "reaction" to the threatening (or sometimes just crackpot) emails was carefully considered between me (filter options) and Corporate Security before deployment. Eg: - Do we reject/bounce, or silently ignore? Usually the latter. - Do we forward captures to CorpSec, not the user? - Law enforcement? Legal C? Etc. These also have to be carefully differentiated from some of the wide-spread stuff like the "I have a contract on you, pay me, and I won't kill you" extortion soan/scams. Those should just be nuked. I would generally not recommend filtering on "threatening content" per-se, but on spam, however you usually do it - like DNSBLs or spamassassin or whatever... Then make it a point of user education to tell people what to do if they get threatening email that gets past the general spam filtering. Which may be LE, may be your own corporate, may be custom filtering, etc. This approach will generally tend to weed out the normal broadcast bullshit, and leave behind the stuff the victim/you really need to know about. ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] harassment/death threat detection/filtering/prosecution
Dnia 15.07.2020 o godz. 12:59:39 Grant Taylor via mailop pisze: > On 7/12/20 11:28 AM, Paul Ebersman via mailop wrote: > >But in these tense times, lots of PoC/non-white/non-cis get hate > >emails, death threats, etc. > > Death threats seem like the warrant more effort to stop than spam. If someone would send death threats to me, I'd rather would like to receive them, so that I can know whether I'm actually in danger and optionally report to the police or whatever... -- Regards, Jaroslaw Rafa r...@rafa.eu.org -- "In a million years, when kids go to school, they're gonna know: once there was a Hushpuppy, and she lived with her daddy in the Bathtub." ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] harassment/death threat detection/filtering/prosecution
On 7/12/20 11:28 AM, Paul Ebersman via mailop wrote: But in these tense times, lots of PoC/non-white/non-cis get hate emails, death threats, etc. Death threats seem like the warrant more effort to stop than spam. Is there software out there like spamassassin et al for detecting these kinds of threats? Does anyone have pointers on web pages/instructions for how to configure common mail readers/services so users can self-filter? I would think that it should be possible to re-use the typical tools but to have them look for different words / phrases. Custom SpamAssassin rules come to mind. Finally, I know that there are avenues for tracking illegal activity and ways to work with FBI, Interpol, etc. Any of these teams specialize in or at least familiar with how to prosecute hate crimes perpetrated via email threats? I have no idea how to go about doing this. But I would encourage exactly this. Perhaps an official police report in your jurisdiction and / or the jurisdiction of the business entity that sent the message to you / your customer, followed up by a (canned) cease and desist letter from an attorney. It's probably a loosing battle. But I feel like hate based threats are a windmill that's worth tilting at. Dare I say it, perhaps it's time for a new Real-time Block List of sources of threats / hate, be it IP and / or domain name and / or content. -- Grant. . . . unix || die smime.p7s Description: S/MIME Cryptographic Signature ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] harassment/death threat detection/filtering/prosecution
It’s both a tricky and a ongoing problem. Many of the folks doing the harassing have access to a host of tools that make it difficult, if not impossible, to block them. Blocking by content is fraught with false positives and often the targets are activists who have legitimate reasons to be discussing hate mail and sensitive topics. Much like some anti-spam lists have discovered filters will often block discussions of spam samples. I think it’s a valuable topic to discuss. One thing I would suggest is reaching out to the groups who are actively working in the anti-harassment space. They probably have been through many of the early iterations of “how do we protect people who come to us for help.” While I don’t have a lot of experience here, the first person I’d talk to is Jayne Hitchcock and then chase down the places she recommended. There are also a number of women’s groups who work directly with victims of harassment, that have visible contact information. I’m less familiar with the anti-racist and anti-anti-semitic groups but I know they have presences online, too. Even a few 30 minute phone calls will give you a crash course in what they’ve already tried and help create a much better policy. laura > On 15 Jul 2020, at 10:38, Hans-Martin Mosner via mailop > wrote: > > This is a tricky problem. I would guess that some traditional spam-rejection > mechanisms might be applicable, but there are grey areas: > Rejection of mails from anonymous origins may help, but on one hand there are > some (sent via hacked ordinary mailboxes) which can't technically be > distinguished from non-anonymous communication, and on the other hand people > who are under threat of harassment might want to use anonymizing services > themselves to communicate. > Content filtering requires training samples, and there is probably not enough > data to train a Bayesian filter as in SpamAssassin or more modern sentiment > analysis neural networks. The false positive and false negative rates might > be unacceptably high, requiring manual checks. > In addition, these tools are not helping in identifying perpetrators. I would > suspect that they will in most cases use techniques which make them > untraceable. Given that trust in LEAs is damaged after the many racist police > incidents in the USA or the recent hatemail incidents based on information > retrieved through police computer systems in Germany, I would also suspect > that some victims would be reluctant to hope that police authorities will > track down perpetrators. > > Some groups on both ends of the political spectrum therefore prefer to track > down and attack peeceived "enemies" themselves. This is obviously not a > solution, even when it's well-intentioned. > > Cheers, > Hans-Martin > > Am 15. Juli 2020 10:40:05 vorm. schrieb Paul Ebersman via mailop > : > >> We have all sorts of tools for detecting, filtering, documenting spam >> and processes for trying to get spammers off the net (and occasionally >> even prosecuted). >> >> But in these tense times, lots of PoC/non-white/non-cis get hate emails, >> death threats, etc. >> >> Is there software out there like spamassassin et al for detecting these >> kinds of threats? Does anyone have pointers on web pages/instructions >> for how to configure common mail readers/services so users can >> self-filter? >> >> Finally, I know that there are avenues for tracking illegal activity and >> ways to work with FBI, Interpol, etc. Any of these teams specialize in >> or at least familiar with how to prosecute hate crimes perpetrated via >> email threats? >> >> >> ___ >> mailop mailing list >> mailop@mailop.org >> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop > > > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop -- Having an Email Crisis? We can help! 800 823-9674 Laura Atkins Word to the Wise la...@wordtothewise.com (650) 437-0741 Email Delivery Blog: https://wordtothewise.com/blog ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
[mailop] harassment/death threat detection/filtering/prosecution
We have all sorts of tools for detecting, filtering, documenting spam and processes for trying to get spammers off the net (and occasionally even prosecuted). But in these tense times, lots of PoC/non-white/non-cis get hate emails, death threats, etc. Is there software out there like spamassassin et al for detecting these kinds of threats? Does anyone have pointers on web pages/instructions for how to configure common mail readers/services so users can self-filter? Finally, I know that there are avenues for tracking illegal activity and ways to work with FBI, Interpol, etc. Any of these teams specialize in or at least familiar with how to prosecute hate crimes perpetrated via email threats? ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
