I was checking out your recent post here: https://syfershock.com/users/syfershock and noticed:

"It looks like the spam in sci.crypt has almost stopped. Knock on wood!"

I've actually been attacking the spam for about a week now with extensive filtering mods, so what you're seeing is the result of my work, not a lack of spam in sci.crypt (or comp.lang.c or rec.motorcycles, etc.)

I've been trying hard to filter without entirely blocking google groups, and it's been difficult, but kind of fun (challenging) also.

Check out another server and you can see that the spam is still rolling in at a rapid pace (sometimes more than one per second). paganini.bofh.team has an unfiltered feed that shows what is really there.

Happy spam avoidance month!

--
Retro Guy

On Sun, 17 Sep 2023 14:13:01 +0000
retro.guy@rocksolidbbs.com (Retro Guy) wrote:

> I was checking out your recent post here:
> https://syfershock.com/users/syfershock and noticed:
>
> "It looks like the spam in sci.crypt has almost stopped. Knock on
> wood!"
>
> I've actually been attacking the spam for about a week now with
> extensive filtering mods, so what you're seeing is the result of my
> work, not a lack of spam in sci.crypt (or comp.lang.c or
> rec.motorcycles, etc.)

Yeah, nobody else seems to care. Time will tell, if it already hasn't.

In the meantime I've moved to alt.sources.crypto for my postings on the
subject of cryptography. I am still watching sci.crypt but my original
posts will go to the other group.
> I've been trying hard to filter without entirely blocking google
> groups, and it's been difficult, but kind of fun (challenging) also.

The good fight ... that which does not kill you ... only makes you ...
stranger ...
> Check out another server and you can see that the spam is still
> rolling in at a rapid pace (sometimes more than one per second).
> paganini.bofh.team has an unfiltered feed that shows what is really
> there.

Yeah, and they switched to posting a lot of foreign language posts.
These ogres hate humanity.
> Happy spam avoidance month!

Your efforts are valiant and appreciated. I hope others catch the bug.

--
3883@sugar.bug | web: sybershock.com | news: alt.sources.crypto

Syber Shock wrote:

> On Sun, 17 Sep 2023 14:13:01 +0000
> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

>> I was checking out your recent post here:
>> https://syfershock.com/users/syfershock and noticed:
>>
>> "It looks like the spam in sci.crypt has almost stopped. Knock on
>> wood!"
>>
>> I've actually been attacking the spam for about a week now with
>> extensive filtering mods, so what you're seeing is the result of my
>> work, not a lack of spam in sci.crypt (or comp.lang.c or
>> rec.motorcycles, etc.)

> Yeah, nobody else seems to care. Time will tell, if it already hasn't.

I just moved my spamassassin server to a different machine. The very short
downtime allowed some spam in, lol (just seconds). Also, it will need to
learn (autolearn), so might not be quite as good as previous for a short
time.

It should be as before in a short time.

--
Retro Guy

X-Spam-Flag: YES
X-Spam-Level: **********************
X-Spam-Status: Yes, score=22.3 required=5.0 tests= ... autolearn=spam

That looks good (autolearn is working)

--
Retro Guy

Retro Guy <retro.guy@rocksolidbbs.com> wrote:
> X-Spam-Flag: YES
> X-Spam-Level: **********************
> X-Spam-Status: Yes, score=22.3 required=5.0 tests= ... autolearn=spam
>
> That looks good (autolearn is working)

How did you plug spam filter to INN?

--
Puzzle

Puzzle wrote:

> Retro Guy <retro.guy@rocksolidbbs.com> wrote:
>> X-Spam-Flag: YES
>> X-Spam-Level: **********************
>> X-Spam-Status: Yes, score=22.3 required=5.0 tests= ... autolearn=spam
>>
>> That looks good (autolearn is working)

> How did you plug spam filter to INN?

I wrote a script quite a while ago that I call from cleanfeed to pass
messages to spamassassin.

I pass the message, then delete the message file if clean, keep the file
if spam. Then, cleanfeed checks if the message file exists. If it does,
it's spam and dropped.

--
Retro Guy

Retro Guy wrote:

> Syber Shock wrote:

>> On Sun, 17 Sep 2023 14:13:01 +0000
>> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

>>> I was checking out your recent post here:
>>> https://syfershock.com/users/syfershock and noticed:
>>>
>>> "It looks like the spam in sci.crypt has almost stopped. Knock on
>>> wood!"
>>>
>>> I've actually been attacking the spam for about a week now with
>>> extensive filtering mods, so what you're seeing is the result of my
>>> work, not a lack of spam in sci.crypt (or comp.lang.c or
>>> rec.motorcycles, etc.)

>> Yeah, nobody else seems to care. Time will tell, if it already hasn't.

> I just moved my spamassassin server to a different machine. The very short
> downtime allowed some spam in, lol (just seconds). Also, it will need to
> learn (autolearn), so might not be quite as good as previous for a short
> time.

> It should be as before in a short time.

Seems to be working normally. I also completely cleaned up my spamassassin
local.cf, making it much easier to manage.

If you find a group that needs attention, please let me know. I can't monitor
all groups and hopefully the filtering is handling groups I've never even
heard of.

The only groups where google groups is completely filters are comp.lang.python
and alt.comp.freeware. All other groups are depending on filtering by other
than the fact a message may be from google groups.

--
Retro Guy

Retro Guy wrote:

> Retro Guy wrote:

>> I just moved my spamassassin server to a different machine. The very short
>> downtime allowed some spam in, lol (just seconds). Also, it will need to
>> learn (autolearn), so might not be quite as good as previous for a short
>> time.

>> It should be as before in a short time.

> Seems to be working normally. I also completely cleaned up my spamassassin
> local.cf, making it much easier to manage.

> If you find a group that needs attention, please let me know. I can't monitor
> all groups and hopefully the filtering is handling groups I've never even
> heard of.

> The only groups where google groups is completely filters are comp.lang.python
> and alt.comp.freeware. All other groups are depending on filtering by other
> than the fact a message may be from google groups.

I'm using patterns instead of blocking users or servers individually. Blocking
users is just whac-a-mole and would drive you insane eventually.

The pattern matching seems to work best, as it seems to be blocking spam from
many groups as no particular group is specified (generally) for each rule.

Monitoring some groups I've never heard of, but apparently were being heavily
spammed, and I can see the spam has stopped there also.

I have no real way to monitor non English groups very well so if there is an
issue there, maybe someone else wants to pick that up and we can collaborate
or whatever might work.

--
Retro Guy

On Wed, 4 Oct 2023 13:47:43 +0000
retro.guy@rocksolidbbs.com (Retro Guy) wrote:

> I have no real way to monitor non English groups very well so if
> there is an issue there, maybe someone else wants to pick that up and
> we can collaborate or whatever might work.

This is a sticking point for me. I want to peer with some servers that
carry foreign language feeds but I know no effective way to filter spam
and abuse and warez in languages I don't understand. I think the
solution lies in asking the sysops that feed foreign language groups to
share their language-specific spam filtering strategies and configs in
an abandoned newsgroup or on their websites.

--
__|__|__|__ 3883@sugar.bug
__|__|__|__ https://sybershock.com
__|__|__|__ news://alt.sources.crypto
| | | #CipherTag #WaffleTag #Cryptologer

On Wed, 4 Oct 2023 12:16:00 -0500
Syber Shock <admin@sybershock.com> wrote:

> On Wed, 4 Oct 2023 13:47:43 +0000
> retro.guy@rocksolidbbs.com (Retro Guy) wrote:
>
> > I have no real way to monitor non English groups very well so if
> > there is an issue there, maybe someone else wants to pick that up and
> > we can collaborate or whatever might work.
>
> This is a sticking point for me. I want to peer with some servers that
> carry foreign language feeds but I know no effective way to filter spam
> and abuse and warez in languages I don't understand. I think the
> solution lies in asking the sysops that feed foreign language groups to
> share their language-specific spam filtering strategies and configs in
> an abandoned newsgroup or on their websites.

I would love to see some peers take on the spam problems for specific
language hierarchies, and we can share information. I, as you, can't read
those groups easily so can't really work on the spam. My servers peer and
provide access to most all text groups, not just English. fr.* de.* etc.
are all available, but I can't monitor them.

I've been doing some thinking about this. I'll post again when my brain is
ready to release more ideas.

--
Retro Guy

On Wed, 4 Oct 2023 14:00:57 -0700
Retro Guy <retroguy@novabbs.com> wrote:

> On Wed, 4 Oct 2023 12:16:00 -0500
> Syber Shock <admin@sybershock.com> wrote:
>
> > On Wed, 4 Oct 2023 13:47:43 +0000
> > retro.guy@rocksolidbbs.com (Retro Guy) wrote:
> >
> > > I have no real way to monitor non English groups very well so if
> > > there is an issue there, maybe someone else wants to pick that up and
> > > we can collaborate or whatever might work.
> >
> > This is a sticking point for me. I want to peer with some servers that
> > carry foreign language feeds but I know no effective way to filter spam
> > and abuse and warez in languages I don't understand. I think the
> > solution lies in asking the sysops that feed foreign language groups to
> > share their language-specific spam filtering strategies and configs in
> > an abandoned newsgroup or on their websites.
>
> I would love to see some peers take on the spam problems for specific
> language hierarchies, and we can share information. I, as you, can't read
> those groups easily so can't really work on the spam. My servers peer and
> provide access to most all text groups, not just English. fr.* de.* etc.
> are all available, but I can't monitor them.
>
> I've been doing some thinking about this. I'll post again when my brain is
> ready to release more ideas.

I've been monitoring number of messages caught by spamassassin, and also
group names that are showing up in captured spam.

Yesterday there seemed to be more than the usual number of messages sent from
peers. Like a 32% increase. I then checked the spam numbers and find that on a
normal day, on average about 30% of all messages are caught as spam, but yesterday
it was 53%! So, 53% of all messages received were identified as spam. Hopefully,
the false positive count is small (I do try to monitor for that)

The group names are logged for me to review, and I find spam is captured in quite
a few groups, and most of these groups get a lot of spam. This would also strengthen
the thought that there are not many false positives, as these aren't just random group
names, but the group names are repeated quite a bit.

I believe that pattern filtering is the way to go. User filtering is a losing battle,
and NoCeM for massive amounts of spam is too time consuming, and requires you to find
every group that's being spammed. The method I'm using doesn't require me to know what
groups are spammed, that's all handled by the filters.

My thinking on coordination of assistance, is that I'm willing to provide spamassassin
access (whitelist ip address for a server) to already established providers that peer
with my main peering server (i2pn.org). I can provide the cleanfeed code and php script
to automatically poll spamassassin from cleanfeed.

Then I would ask that a provider with other than English language skills, that is interested
in providing regex patters for spamassassin to me, to provide such and I can add them to
the filtering. We would of course, need to agree on what basis we're filtering.

Below is a short list of groups as they show up in my logging output from captured spam.
You can see the same group names repeat quite a bit. This is just a few lines from the log.

On average, a spam is caught approximately every 20 seconds.

nl.religie
uk.sci.weather
alt.ph.uk
microsoft.public.excel
sci.logic
soc.culture.punjab
microsoft.public.excel.programming
comp.lang.c
sci.crypt
microsoft.public.excel
microsoft.public.excel.programming
microsoft.public.de.excel
comp.lang.c
bit.listserv.ibm-main
soc.culture.punjab
comp.lang.c
comp.lang.c
comp.lang.c
comp.lang.c
microsoft.public.de.excel
microsoft.public.excel.programming
soc.culture.punjab
microsoft.public.excel.programming
microsoft.public.excel.programming
sci.crypt
sci.crypt
soc.culture.punjab
microsoft.public.excel.programming
uk.sport.cricket
comp.lang.c
comp.lang.c
microsoft.public.excel.programming
soc.culture.punjab

--
Retro Guy

Retro Guy wrote:

> On Wed, 4 Oct 2023 14:00:57 -0700
> Retro Guy <retroguy@novabbs.com> wrote:

>> On Wed, 4 Oct 2023 12:16:00 -0500
>> Syber Shock <admin@sybershock.com> wrote:
>>
>> > On Wed, 4 Oct 2023 13:47:43 +0000
>> > retro.guy@rocksolidbbs.com (Retro Guy) wrote:
>> >
>> > > I have no real way to monitor non English groups very well so if
>> > > there is an issue there, maybe someone else wants to pick that up and
>> > > we can collaborate or whatever might work.
>> >
>> > This is a sticking point for me. I want to peer with some servers that
>> > carry foreign language feeds but I know no effective way to filter spam
>> > and abuse and warez in languages I don't understand. I think the
>> > solution lies in asking the sysops that feed foreign language groups to
>> > share their language-specific spam filtering strategies and configs in
>> > an abandoned newsgroup or on their websites.
>>
>> I would love to see some peers take on the spam problems for specific
>> language hierarchies, and we can share information. I, as you, can't read
>> those groups easily so can't really work on the spam. My servers peer and
>> provide access to most all text groups, not just English. fr.* de.* etc.
>> are all available, but I can't monitor them.
>>
>> I've been doing some thinking about this. I'll post again when my brain is
>> ready to release more ideas.

> I've been monitoring number of messages caught by spamassassin, and also
> group names that are showing up in captured spam.

> snip...

Today I've started polling spamassassin on my peering servers. Before today,
all filtering (other than cleanfeed) was done on my nnrp servers only (servers
available to newsreaders). I'm now filtering at the source, the servers receiving
articles from peers, so we also won't be passing these articles (spam) to other
nntp servers.

--
Retro Guy

Retro Guy wrote:

> Retro Guy wrote:

>> On Wed, 4 Oct 2023 14:00:57 -0700
>> Retro Guy <retroguy@novabbs.com> wrote:

>>> On Wed, 4 Oct 2023 12:16:00 -0500
>>> Syber Shock <admin@sybershock.com> wrote:
>>>
>>> > On Wed, 4 Oct 2023 13:47:43 +0000
>>> > retro.guy@rocksolidbbs.com (Retro Guy) wrote:
>>> >
>>> > > I have no real way to monitor non English groups very well so if
>>> > > there is an issue there, maybe someone else wants to pick that up and
>>> > > we can collaborate or whatever might work.
>>> >
>>> > This is a sticking point for me. I want to peer with some servers that
>>> > carry foreign language feeds but I know no effective way to filter spam
>>> > and abuse and warez in languages I don't understand. I think the
>>> > solution lies in asking the sysops that feed foreign language groups to
>>> > share their language-specific spam filtering strategies and configs in
>>> > an abandoned newsgroup or on their websites.
>>>
>>> I would love to see some peers take on the spam problems for specific
>>> language hierarchies, and we can share information. I, as you, can't read
>>> those groups easily so can't really work on the spam. My servers peer and
>>> provide access to most all text groups, not just English. fr.* de.* etc.
>>> are all available, but I can't monitor them.
>>>
>>> I've been doing some thinking about this. I'll post again when my brain is
>>> ready to release more ideas.

>> I've been monitoring number of messages caught by spamassassin, and also
>> group names that are showing up in captured spam.

>> snip...

> Today I've started polling spamassassin on my peering servers. Before today,
> all filtering (other than cleanfeed) was done on my nnrp servers only (servers
> available to newsreaders). I'm now filtering at the source, the servers receiving
> articles from peers, so we also won't be passing these articles (spam) to other
> nntp servers.

Been monitoring the conversation at eternal-september, and it looks like Ray is
doing some great work filtering spam! He is also using spamassassin, and is not
just blanket blocking google groups. It takes more work to do it this way. I know,
it's the same way I'm doing it, but I think it's worth the effort, as there really
are some who use GG for actual useful posts.

It's nice to see. Maybe some more can get involved, but at least for now,
news.eternal-september.org and news.i2pn2.org are a good place to get a cleaner
(less spam) Usenet.

Now he and I can both try to work on false positives, which is another part of
the job.

Again, glad to see Ray taking on the challenge also!

--
Retro Guy

Retro Guy wrote:

> Retro Guy wrote:

>> Retro Guy wrote:

>>> On Wed, 4 Oct 2023 14:00:57 -0700
>>> Retro Guy <retroguy@novabbs.com> wrote:

>>>> On Wed, 4 Oct 2023 12:16:00 -0500
>>>> Syber Shock <admin@sybershock.com> wrote:
>>>>
>>>> > On Wed, 4 Oct 2023 13:47:43 +0000
>>>> > retro.guy@rocksolidbbs.com (Retro Guy) wrote:
>>>> >
>>>> > > I have no real way to monitor non English groups very well so if
>>>> > > there is an issue there, maybe someone else wants to pick that up and
>>>> > > we can collaborate or whatever might work.
>>>> >
>>>> > This is a sticking point for me. I want to peer with some servers that
>>>> > carry foreign language feeds but I know no effective way to filter spam
>>>> > and abuse and warez in languages I don't understand. I think the
>>>> > solution lies in asking the sysops that feed foreign language groups to
>>>> > share their language-specific spam filtering strategies and configs in
>>>> > an abandoned newsgroup or on their websites.
>>>>
>>>> I would love to see some peers take on the spam problems for specific
>>>> language hierarchies, and we can share information. I, as you, can't read
>>>> those groups easily so can't really work on the spam. My servers peer and
>>>> provide access to most all text groups, not just English. fr.* de.* etc.
>>>> are all available, but I can't monitor them.
>>>>
>>>> I've been doing some thinking about this. I'll post again when my brain is
>>>> ready to release more ideas.

>>> I've been monitoring number of messages caught by spamassassin, and also
>>> group names that are showing up in captured spam.

>>> snip...

>> Today I've started polling spamassassin on my peering servers. Before today,
>> all filtering (other than cleanfeed) was done on my nnrp servers only (servers
>> available to newsreaders). I'm now filtering at the source, the servers receiving
>> articles from peers, so we also won't be passing these articles (spam) to other
>> nntp servers.

> Been monitoring the conversation at eternal-september, and it looks like Ray is
> doing some great work filtering spam! He is also using spamassassin, and is not
> just blanket blocking google groups. It takes more work to do it this way. I know,
> it's the same way I'm doing it, but I think it's worth the effort, as there really
> are some who use GG for actual useful posts.

> It's nice to see. Maybe some more can get involved, but at least for now,
> news.eternal-september.org and news.i2pn2.org are a good place to get a cleaner
> (less spam) Usenet.

> Now he and I can both try to work on false positives, which is another part of
> the job.

As the filter seems to be working quite well, I'm addressing false positives now.
This requires more human interaction than just filtering, but not bad.

What I'm doing is I export all spam to a .mbox file, which I can review very
quickly in mutt. I can review 1,000 messages in just a couple of minutes, if that.

Any false positive I find (not many), I can address what rules triggered, and
make changes as necessary.

--
Retro Guy

On Sat, 7 Oct 2023 16:10:47 +0000
retro.guy@rocksolidbbs.com (Retro Guy) wrote:

<snip>

> As the filter seems to be working quite well, I'm addressing false
> positives now. This requires more human interaction than just
> filtering, but not bad.
>
> What I'm doing is I export all spam to a .mbox file, which I can
> review very quickly in mutt. I can review 1,000 messages in just a
> couple of minutes, if that.
>
> Any false positive I find (not many), I can address what rules
> triggered, and make changes as necessary.

I think I have a solution to this spam kerfuffle.

Every posting host should employ at a minimum user-level
authentication. Users can still hide their IP via Tor, but enforcing
authentication enables fine-grained spam and abuse controls.

Then each host should have local messaging for every user. If any host
publishes a nocem or cancel, a copy of that will be forwarded to the
posting host and deposited in the user's inbox.

Then if there was a false positive, the user can contact the canceling
party, and the cancel can be revoked manually.

This allows spammer accounts to be quickly identified and locked out
and resolves false positives in a straightforward scheme.

The user messaging can be in the form of a mailbox, a login portal, or
even a private newsgroup accessible only by that user. Reply-to would
automatically go to the author of the cancel message.

There are a few ways to skin this cat, but this is the gist.

I would prefer a posting host message solution for cancellation notices
if possible. Having the user subscribe to their own private newsgroup
for such messages would be more orthogonal to a plain newsreader.

--
__|__|__|__ 3883@sugar.bug
__|__|__|__ https://sybershock.com
__|__|__|__ news://alt.sources.crypto
| | | #CipherTag #WaffleTag #Cryptologer

Syber Shock wrote:

> On Sat, 7 Oct 2023 16:10:47 +0000
> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

> <snip>

>> As the filter seems to be working quite well, I'm addressing false
>> positives now. This requires more human interaction than just
>> filtering, but not bad.
>>
>> What I'm doing is I export all spam to a .mbox file, which I can
>> review very quickly in mutt. I can review 1,000 messages in just a
>> couple of minutes, if that.
>>
>> Any false positive I find (not many), I can address what rules
>> triggered, and make changes as necessary.

> I think I have a solution to this spam kerfuffle.

> Every posting host should employ at a minimum user-level
> authentication. Users can still hide their IP via Tor, but enforcing
> authentication enables fine-grained spam and abuse controls.

I think we're keeping it reasonably private right now. Here's the
Injection-Info for your post:

Injection-Info: novabbs.org;
logging-data="2005073"; mail-complaints-to="usenet@novabbs.org";
posting-account="TzG3Hl99Aa0Fgb506WreKRgRTO2mG9+aGjVDifyfNqo";

Other than the name of the host running inn, there is no personally
identifiable info there, but you could still be blocked by 'posting-account',
which is obfuscated.

> Then each host should have local messaging for every user. If any host
> publishes a nocem or cancel, a copy of that will be forwarded to the
> posting host and deposited in the user's inbox.
>
> Then if there was a false positive, the user can contact the canceling
> party, and the cancel can be revoked manually.

> This allows spammer accounts to be quickly identified and locked out
> and resolves false positives in a straightforward scheme.

This is a great idea, but I would be surprised if admins would set this
up. Unless a majority do, it wouldn't accomplish what it's meant to do.

On a positive note, I just checked the last 5,500 spam messages (only took
a few minutes), and found zero false positives.

--
Retro Guy