Rocksolid Light

News from da outaworlds

mail  files  register  groups  login

Message-ID:  

BOFH excuse #267: The UPS is on strike.


rocksolid / Rocksolid Nodes Help / Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only

SubjectAuthor
* Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyByrl Raze Buckbriar
+* Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyRetro Guy
|`* Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyRetro Guy
| `- Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyByrl Raze Buckbriar
`* Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyRetro Guy
 +* Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyRetro Guy
 |`* Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyByrl Raze Buckbriar
 | `* Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyRetro Guy
 |  `* Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyRetro Guy
 |   `- Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyRetro Guy
 `- Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode OnlyByrl Raze Buckbriar

1
Subject: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Byrl Raze Buckbriar
Newsgroups: rocksolid.nodes.help
Organization: OCTADE
Date: Mon, 14 Oct 2024 06:45 UTC
Attachments: unnamed (application/pgp-signature)
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: news0@octade.net (Byrl Raze Buckbriar)
Newsgroups: rocksolid.nodes.help
Subject: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 06:45:28 +0000
Organization: OCTADE
Message-ID: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net>
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature";
micalg="PGP-SHA256";
boundary="Signature=_Mon__14_Oct_2024_01_45_08_-0500_RO56doJKs9Vc3ioE"
Injection-Info: i2pn2.org;
logging-data="1959610"; mail-complaints-to="usenet@i2pn2.org";
posting-account="OI6weT9bZt7saiRF3+XsJJTIMUPjpyh/q6ChV6DtztU";
User-Agent: OCTADE
X-Newsreader: OCTADE
X-Spam-Checker-Version: SpamAssassin 4.0.0
View all headers

I am configuring and testing a Rocksolid Light peer to pull and archive 91 Usenet newsgroups, not counting the rocksolid groups.

The goal is to suck the same list of newsgroups from several remote peers without making any attempt to push new articles to those peers that are wide open. Push should only happen with the I2PN2 server. When I can ensure reliably against pushing to selected remotes I want to add several more sections for more remotes.

Three identical sections in the menu.conf file appear like so:

```
# Add your sections below:
rocksolid:1:1
I2PN2:1:1
BLUEWORLDHOSTING:1:1
MIXMIN:1:1
```

The bottom three sections have the exact same list of Big8 and ALT newsgroups in the same sorted order in their respective 'groups.txt' files. The top 'rocksolid' entry is pulling rocksolid groups only from I2PN2.

Only 'rocksolid' and 'I2PN2' are configured with a username and password for remote authentication. The remainder of these are not configured to authenticate. The remote username and remote password fields are left blank in their respective <section>.inc.php files.

After running the cron several times I began observing the log files and file generation in <spool> directory and population of articles to the spool visible via both web portal and newsreader software.

I am concerned about several things:

1. I do not want the rslight install to attempt to push articles to the wide open peers. I want to ensure this does not happen at all.

2. I can't tell if this is happening under the hood. How might I check?

3. I only want articles forwarded to peers for which a non-null, remote username and password exist in the <section>.inc.php file. For instance, I would like to add an open server that allows unauthenticated posting, yet I want to ensure that this server will NOT try to forward to that remote. Even though posting is possible, I want only to pull, never to push.

4. The newsreader software (Sylpheed) shows three identical listings of each newsgroup in the group list. How may I resolve this to show only one copy of the group?

5. I am concerned about pushing duplicates or unfiltered garbage from one server to another and I want to ensure that there is a way to selectively enforce suck mode only for certain remote servers (the wide open ones).

I have now disabled the bottom two peers in menu.conf until I can sort this out.

I thought about setting up a shield or transit peer, but I would like to see if it is possible to make do with just one Rocksolid install for this job.

Advice on forming a reliable method to check and verify these issues will be very helpful.

--
Byrl Raze Buckbriar . OCTADE . < https://octade.net >
Hacker Hotline . voice & SMS . (781) OCT-AGON
KeyOxide . < https://keyoxide.org/keyoxide0@octade.net >

Attachments: unnamed (application/pgp-signature)
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Retro Guy
Newsgroups: rocksolid.nodes.help
Organization: Rocksolid Light
Date: Mon, 14 Oct 2024 09:17 UTC
References: 1
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: retroguy@novabbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 09:17:39 +0000
Organization: Rocksolid Light
Message-ID: <b3998ca7a937e5a56ebd3e7332d3e16c@www.novabbs.org>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
logging-data="1976181"; mail-complaints-to="usenet@i2pn2.org";
posting-account="HNchV9coSOli+kkOHciCZlGs2VaRn7Emh38jWV1k174";
User-Agent: Rocksolid Light
X-Rslight-Site: $2y$10$FtHM//ZCpHcF0Jx6xzzkwOC43LhCm1xUH40PfHWJuotrgoMOmq23a
X-Rslight-Posting-User: 28547bffeec402cc2e544e2f8e28cc9329f66434
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf ?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B H0=hZt'[%)4kG|
X-Spam-Checker-Version: SpamAssassin 4.0.0
View all headers

On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:

> I am configuring and testing a Rocksolid Light peer to pull and archive
> 91 Usenet newsgroups, not counting the rocksolid groups.
>
> The goal is to suck the same list of newsgroups from several remote
> peers without making any attempt to push new articles to those peers
> that are wide open. Push should only happen with the I2PN2 server. When
> I can ensure reliably against pushing to selected remotes I want to add
> several more sections for more remotes.
>
> Three identical sections in the menu.conf file appear like so:
>
<snip>
>
> After running the cron several times I began observing the log files and
> file generation in <spool> directory and population of articles to the
> spool visible via both web portal and newsreader software.
>
> I am concerned about several things:
>
> 1. I do not want the rslight install to attempt to push articles to the
> wide open peers. I want to ensure this does not happen at all.

In the config file (rslight.inc.php) you can disable posting by setting:
'readonly' => '1',

For each section, you can have it's own config file:
rocksolid.inc.php
section2.inc.php
etc.

In this config file you can set all the settings from rocksolid.inc.php
and either change them or let them remain. See example.inc.php in
config_dir.

***BUT*** Having the same group names in multiple sections can cause
issues. It is still possible that rslight will try to send via one of
the other sections, as it searches sections when you post, then moves
message to 'outgoing' dir in that section ($spooldir/outgoing). I will
look at the code today to see how to avoid this.

It should not try to send if readonly is set, but the group in multiple
sections will defeat this. It should be reasonably simple to fix in the
code.

> 2. I can't tell if this is happening under the hood. How might I check?
>
> 3. I only want articles forwarded to peers for which a non-null, remote
> username and password exist in the <section>.inc.php file. For instance,
> I would like to add an open server that allows unauthenticated posting,
> yet I want to ensure that this server will NOT try to forward to that
> remote. Even though posting is possible, I want only to pull, never to
> push.

See above about 'readonly'. We can fix this.

> 4. The newsreader software (Sylpheed) shows three identical listings of
> each newsgroup in the group list. How may I resolve this to show only
> one copy of the group?

Strange, I'll try to duplicate this on a test site so we can fix this
issue.

Again, it's not standard to have the same groups in multiple sections,
but we should make it possible.

> I thought about setting up a shield or transit peer, but I would like to
> see if it is possible to make do with just one Rocksolid install for
> this job.

We should be able to do this in rslight.

> Advice on forming a reliable method to check and verify these issues
> will be very helpful.

I'll be messing with this on a test site today.

--
Retro Guy

Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Retro Guy
Newsgroups: rocksolid.nodes.help
Organization: Rocksolid Light
Date: Mon, 14 Oct 2024 09:55 UTC
References: 1 2
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: retroguy@novabbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 09:55:00 +0000
Organization: Rocksolid Light
Message-ID: <a3b1fd6fa70db9e39669b83e1edf19ab@www.novabbs.org>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net> <b3998ca7a937e5a56ebd3e7332d3e16c@www.novabbs.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
logging-data="1980179"; mail-complaints-to="usenet@i2pn2.org";
posting-account="HNchV9coSOli+kkOHciCZlGs2VaRn7Emh38jWV1k174";
User-Agent: Rocksolid Light
X-Rslight-Site: $2y$10$okkIfVJltjhCEPFFa8wkg.iW3SODuBfpHiwOyEVE9fMJouhzneoc2
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf ?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B H0=hZt'[%)4kG|
X-Rslight-Posting-User: 28547bffeec402cc2e544e2f8e28cc9329f66434
View all headers

On Mon, 14 Oct 2024 9:17:39 +0000, Retro Guy wrote:

> On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
>
>> I am configuring and testing a Rocksolid Light peer to pull and archive
>> 91 Usenet newsgroups, not counting the rocksolid groups.
>>
>> The goal is to suck the same list of newsgroups from several remote
>> peers without making any attempt to push new articles to those peers
>> that are wide open. Push should only happen with the I2PN2 server. When
>> I can ensure reliably against pushing to selected remotes I want to add
>> several more sections for more remotes.
>>
>> Three identical sections in the menu.conf file appear like so:
>>
> <snip>
>>
>> After running the cron several times I began observing the log files and
>> file generation in <spool> directory and population of articles to the
>> spool visible via both web portal and newsreader software.
>>
>> I am concerned about several things:
>>
>> 1. I do not want the rslight install to attempt to push articles to the
>> wide open peers. I want to ensure this does not happen at all.
>
> In the config file (rslight.inc.php) you can disable posting by setting:
> 'readonly' => '1',
>
> For each section, you can have it's own config file:
> rocksolid.inc.php
> section2.inc.php
> etc.
>
> In this config file you can set all the settings from rocksolid.inc.php
> and either change them or let them remain. See example.inc.php in
> config_dir.
>
> ***BUT*** Having the same group names in multiple sections can cause
> issues. It is still possible that rslight will try to send via one of
> the other sections, as it searches sections when you post, then moves
> message to 'outgoing' dir in that section ($spooldir/outgoing). I will
> look at the code today to see how to avoid this.
>
> It should not try to send if readonly is set, but the group in multiple
> sections will defeat this. It should be reasonably simple to fix in the
> code.

Ok, I see the first issue here.

When rslight receives a request for a group, it checks to find the
section for the group. This is done to handle moving groups from one
section to another by admin, without breaking links. So, if you move
'rocksolid.nodes' from 'rocksolid' to 'otherstuff' and a browser
requests it in 'rocksolid', rslight will automatically redirect them to
'otherstuff'.

This causes an issue when the group is in multiple sections.

So a question. Is the purpose of having the group in multiple sections
simply to pull from multiple servers? If that is the case, set the extra
sections in menu.conf to:

extrasection:0:1 instead of extrasection:1:1

This will remove it from the menu on the site, and remove it from
rslight searching what section includes a group for display.

Will this work for this one particular issue? I'm going to look into the
duplicate listing of groups now and try to duplicate that.

--
Retro Guy

Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Retro Guy
Newsgroups: rocksolid.nodes.help
Organization: Rocksolid Light
Date: Mon, 14 Oct 2024 10:38 UTC
References: 1
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: retroguy@novabbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 10:38:42 +0000
Organization: Rocksolid Light
Message-ID: <4a29d60d6c23db7a96d9be51580c6232@www.novabbs.org>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
logging-data="1985487"; mail-complaints-to="usenet@i2pn2.org";
posting-account="HNchV9coSOli+kkOHciCZlGs2VaRn7Emh38jWV1k174";
User-Agent: Rocksolid Light
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Rslight-Posting-User: 28547bffeec402cc2e544e2f8e28cc9329f66434
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf ?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B H0=hZt'[%)4kG|
X-Rslight-Site: $2y$10$J38yh3g7kHixt868NFpDt.xE0jvPRg0Wc70jKrXczhVWB/Pa2y45S
View all headers

On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:

> I am configuring and testing a Rocksolid Light peer to pull and archive
> 91 Usenet newsgroups, not counting the rocksolid groups.

> 4. The newsreader software (Sylpheed) shows three identical listings of
> each newsgroup in the group list. How may I resolve this to show only
> one copy of the group?

This identical listing issue should be fixed now. The fix is in cron.php
and has been pushed to devel (to gitlab).

--
Retro Guy

Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Retro Guy
Newsgroups: rocksolid.nodes.help
Organization: Rocksolid Light
Date: Mon, 14 Oct 2024 12:19 UTC
References: 1 2
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: retroguy@novabbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 12:19:53 +0000
Organization: Rocksolid Light
Message-ID: <ea2ac41badbf9daa6835a4dd51cf1704@www.novabbs.org>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net> <4a29d60d6c23db7a96d9be51580c6232@www.novabbs.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
logging-data="1996666"; mail-complaints-to="usenet@i2pn2.org";
posting-account="HNchV9coSOli+kkOHciCZlGs2VaRn7Emh38jWV1k174";
User-Agent: Rocksolid Light
X-Rslight-Posting-User: 28547bffeec402cc2e544e2f8e28cc9329f66434
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Rslight-Site: $2y$10$ikm5aynWgRd8Cn6/BI.UXO5bkjT0D6my.4v6DGE4WNjWJh1b2z/N.
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf ?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B H0=hZt'[%)4kG|
View all headers

On Mon, 14 Oct 2024 10:38:42 +0000, Retro Guy wrote:

> On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
>
>> I am configuring and testing a Rocksolid Light peer to pull and archive
>> 91 Usenet newsgroups, not counting the rocksolid groups.
>
>> 4. The newsreader software (Sylpheed) shows three identical listings of
>> each newsgroup in the group list. How may I resolve this to show only
>> one copy of the group?
>
> This identical listing issue should be fixed now. The fix is in cron.php
> and has been pushed to devel (to gitlab).

This seems fine now in Sylpheed (or should be any newsreader).

I now have checking in TWO places to make sure to NOT send if 'readonly'
is '1' for a section. Once when you try to post, and another time if for
some reason it gets past that, it will check again when trying to
actually feed it to the remote server.

This is logged in debug.log. I am still cleaning up reply codes so the
newsreader has a useful reply for it's logs (and debugging). So far I
have been unable to feed an article to a remote site if the section is
readonly.

If there are several sections with the same group, only a NON-readonly
section will allow it to be sent to remote. If there is no NON-readonly
section containing the group, posting will fail.

--
Retro Guy

Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Byrl Raze Buckbriar
Newsgroups: rocksolid.nodes.help
Organization: OCTADE
Date: Mon, 14 Oct 2024 19:54 UTC
References: 1 2
Attachments: unnamed (application/pgp-signature)
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: news0@octade.net (Byrl Raze Buckbriar)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 19:54:26 +0000
Organization: OCTADE
Message-ID: <4123cc2af76ff726fe8b71d8380b218e$1@octade.net>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net>
<4a29d60d6c23db7a96d9be51580c6232@www.novabbs.org>
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature";
micalg="PGP-SHA256";
boundary="Signature=_Mon__14_Oct_2024_14_48_28_-0500_sFvrG.sILvq3nCYL"
Injection-Info: i2pn2.org;
logging-data="2048094"; mail-complaints-to="usenet@i2pn2.org";
posting-account="OI6weT9bZt7saiRF3+XsJJTIMUPjpyh/q6ChV6DtztU";
User-Agent: OCTADE
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Newsreader: OCTADE
View all headers

On Mon, 14 Oct 2024 10:38:42 +0000
Retro Guy <retroguy@novabbs.com> wrote:

> On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
>
> > I am configuring and testing a Rocksolid Light peer to pull and archive
> > 91 Usenet newsgroups, not counting the rocksolid groups.
>
> > 4. The newsreader software (Sylpheed) shows three identical listings of
> > each newsgroup in the group list. How may I resolve this to show only
> > one copy of the group?
>
> This identical listing issue should be fixed now. The fix is in cron.php
> and has been pushed to devel (to gitlab).

Thank you. I will test later today and report back.

--
Byrl Raze Buckbriar . OCTADE . < https://octade.net >
Hacker Hotline . voice & SMS . (781) OCT-AGON
KeyOxide . < https://keyoxide.org/keyoxide0@octade.net >

Attachments: unnamed (application/pgp-signature)
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Byrl Raze Buckbriar
Newsgroups: rocksolid.nodes.help
Organization: OCTADE
Date: Mon, 14 Oct 2024 19:54 UTC
References: 1 2 3
Attachments: unnamed (application/pgp-signature)
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: news0@octade.net (Byrl Raze Buckbriar)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 19:54:31 +0000
Organization: OCTADE
Message-ID: <6fd35a2c2b23b8310f052480dc828f6e$1@octade.net>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net>
<b3998ca7a937e5a56ebd3e7332d3e16c@www.novabbs.org>
<a3b1fd6fa70db9e39669b83e1edf19ab@www.novabbs.org>
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature";
micalg="PGP-SHA256";
boundary="Signature=_Mon__14_Oct_2024_14_54_25_-0500_xNq=f4wLbsl6v.zw"
Injection-Info: i2pn2.org;
logging-data="2048094"; mail-complaints-to="usenet@i2pn2.org";
posting-account="OI6weT9bZt7saiRF3+XsJJTIMUPjpyh/q6ChV6DtztU";
User-Agent: OCTADE
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Newsreader: OCTADE
View all headers

On Mon, 14 Oct 2024 09:55:00 +0000
Retro Guy <retroguy@novabbs.com> wrote:

> On Mon, 14 Oct 2024 9:17:39 +0000, Retro Guy wrote:
>
> > On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
> >
> >> I am configuring and testing a Rocksolid Light peer to pull and archive
> >> 91 Usenet newsgroups, not counting the rocksolid groups.
> >>
> >> The goal is to suck the same list of newsgroups from several remote
> >> peers without making any attempt to push new articles to those peers
> >> that are wide open. Push should only happen with the I2PN2 server. When
> >> I can ensure reliably against pushing to selected remotes I want to add
> >> several more sections for more remotes.
> >>
> >> Three identical sections in the menu.conf file appear like so:
> >>
> > <snip>
> >>
> >> After running the cron several times I began observing the log files and
> >> file generation in <spool> directory and population of articles to the
> >> spool visible via both web portal and newsreader software.
> >>
> >> I am concerned about several things:
> >>
> >> 1. I do not want the rslight install to attempt to push articles to the
> >> wide open peers. I want to ensure this does not happen at all.
> >
> > In the config file (rslight.inc.php) you can disable posting by setting:
> > 'readonly' => '1',
> >
> > For each section, you can have it's own config file:
> > rocksolid.inc.php
> > section2.inc.php
> > etc.
> >
> > In this config file you can set all the settings from rocksolid.inc.php
> > and either change them or let them remain. See example.inc.php in
> > config_dir.
> >
> > ***BUT*** Having the same group names in multiple sections can cause
> > issues. It is still possible that rslight will try to send via one of
> > the other sections, as it searches sections when you post, then moves
> > message to 'outgoing' dir in that section ($spooldir/outgoing). I will
> > look at the code today to see how to avoid this.
> >
> > It should not try to send if readonly is set, but the group in multiple
> > sections will defeat this. It should be reasonably simple to fix in the
> > code.
>
> Ok, I see the first issue here.
>
> When rslight receives a request for a group, it checks to find the
> section for the group. This is done to handle moving groups from one
> section to another by admin, without breaking links. So, if you move
> 'rocksolid.nodes' from 'rocksolid' to 'otherstuff' and a browser
> requests it in 'rocksolid', rslight will automatically redirect them to
> 'otherstuff'.
>
> This causes an issue when the group is in multiple sections.
>
> So a question. Is the purpose of having the group in multiple sections
> simply to pull from multiple servers? If that is the case, set the extra
> sections in menu.conf to:

Correct. The purpose is just to pull from multiple sources so that messages missing or deleted from one peer might be possibly recovered from another peer. This way there is more probability of getting all messages for the archive.

>
> extrasection:0:1 instead of extrasection:1:1
>
> This will remove it from the menu on the site, and remove it from
> rslight searching what section includes a group for display.

Makes sense.

>
> Will this work for this one particular issue? I'm going to look into the
> duplicate listing of groups now and try to duplicate that.

Correct. There is no need to display all the sections. I will alter the config to reflect this. I'm not going to run cron yet to test though, because I am cautious that it might flood a bunch of necro posts to the wrong server(s).

--
Byrl Raze Buckbriar . OCTADE . < https://octade.net >
Hacker Hotline . voice & SMS . (781) OCT-AGON
KeyOxide . < https://keyoxide.org/keyoxide0@octade.net >

Attachments: unnamed (application/pgp-signature)
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Byrl Raze Buckbriar
Newsgroups: rocksolid.nodes.help
Organization: OCTADE
Date: Mon, 14 Oct 2024 19:54 UTC
References: 1 2 3
Attachments: unnamed (application/pgp-signature)
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: news0@octade.net (Byrl Raze Buckbriar)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 19:54:28 +0000
Organization: OCTADE
Message-ID: <7c2f96de7e94edc8c71ea42c933e47e2$1@octade.net>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net>
<4a29d60d6c23db7a96d9be51580c6232@www.novabbs.org>
<ea2ac41badbf9daa6835a4dd51cf1704@www.novabbs.org>
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature";
micalg="PGP-SHA256";
boundary="Signature=_Mon__14_Oct_2024_14_53_18_-0500_Kz+U=7EOsyqU07ML"
Injection-Info: i2pn2.org;
logging-data="2048094"; mail-complaints-to="usenet@i2pn2.org";
posting-account="OI6weT9bZt7saiRF3+XsJJTIMUPjpyh/q6ChV6DtztU";
User-Agent: OCTADE
X-Newsreader: OCTADE
X-Spam-Checker-Version: SpamAssassin 4.0.0
View all headers

On Mon, 14 Oct 2024 12:19:53 +0000
Retro Guy <retroguy@novabbs.com> wrote:

> On Mon, 14 Oct 2024 10:38:42 +0000, Retro Guy wrote:
>
> > On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
> >
> >> I am configuring and testing a Rocksolid Light peer to pull and archive
> >> 91 Usenet newsgroups, not counting the rocksolid groups.
> >
> >> 4. The newsreader software (Sylpheed) shows three identical listings of
> >> each newsgroup in the group list. How may I resolve this to show only
> >> one copy of the group?
> >
> > This identical listing issue should be fixed now. The fix is in cron.php
> > and has been pushed to devel (to gitlab).
>
> This seems fine now in Sylpheed (or should be any newsreader).
>
> I now have checking in TWO places to make sure to NOT send if 'readonly'
> is '1' for a section. Once when you try to post, and another time if for
> some reason it gets past that, it will check again when trying to
> actually feed it to the remote server.
>
> This is logged in debug.log. I am still cleaning up reply codes so the
> newsreader has a useful reply for it's logs (and debugging). So far I
> have been unable to feed an article to a remote site if the section is
> readonly.
>
> If there are several sections with the same group, only a NON-readonly
> section will allow it to be sent to remote. If there is no NON-readonly
> section containing the group, posting will fail.

That sounds right. I plan to check it out this evening. It looks like you merged 'devel' branch. Does that mean the changes are all in 'master' now?

--
Byrl Raze Buckbriar . OCTADE . < https://octade.net >
Hacker Hotline . voice & SMS . (781) OCT-AGON
KeyOxide . < https://keyoxide.org/keyoxide0@octade.net >

Attachments: unnamed (application/pgp-signature)
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Retro Guy
Newsgroups: rocksolid.nodes.help
Organization: Rocksolid Light
Date: Mon, 14 Oct 2024 22:24 UTC
References: 1 2 3 4
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: retroguy@novabbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Mon, 14 Oct 2024 15:24:52 -0700
Organization: Rocksolid Light
Message-ID: <235e17a78bd7a00d437facd5eb09626d$1@novabbs.org>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net> <4a29d60d6c23db7a96d9be51580c6232@www.novabbs.org> <ea2ac41badbf9daa6835a4dd51cf1704@www.novabbs.org> <7c2f96de7e94edc8c71ea42c933e47e2$1@octade.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Injection-Info: i2pn2.org;
logging-data="2062204"; mail-complaints-to="usenet@i2pn2.org";
posting-account="PGd4t4cXnWwgUWG9VtTiCsm47oOWbHLcTr4rYoM0Edo";
User-Agent: 40tude_Dialog/2.0.15.41
X-Spam-Checker-Version: SpamAssassin 4.0.0
View all headers

On Mon, 14 Oct 2024 19:54:28 +0000, Byrl Raze Buckbriar wrote:

> On Mon, 14 Oct 2024 12:19:53 +0000
> Retro Guy <retroguy@novabbs.com> wrote:
>
>> On Mon, 14 Oct 2024 10:38:42 +0000, Retro Guy wrote:
>>
>>> On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
>>>
>>>> I am configuring and testing a Rocksolid Light peer to pull and archive
>>>> 91 Usenet newsgroups, not counting the rocksolid groups.
>>>
>>>> 4. The newsreader software (Sylpheed) shows three identical listings of
>>>> each newsgroup in the group list. How may I resolve this to show only
>>>> one copy of the group?
>>>
>>> This identical listing issue should be fixed now. The fix is in cron.php
>>> and has been pushed to devel (to gitlab).
>>
>> This seems fine now in Sylpheed (or should be any newsreader).
>>
>> I now have checking in TWO places to make sure to NOT send if 'readonly'
>> is '1' for a section. Once when you try to post, and another time if for
>> some reason it gets past that, it will check again when trying to
>> actually feed it to the remote server.
>>
>> This is logged in debug.log. I am still cleaning up reply codes so the
>> newsreader has a useful reply for it's logs (and debugging). So far I
>> have been unable to feed an article to a remote site if the section is
>> readonly.
>>
>> If there are several sections with the same group, only a NON-readonly
>> section will allow it to be sent to remote. If there is no NON-readonly
>> section containing the group, posting will fail.
>
> That sounds right. I plan to check it out this evening. It looks like you merged 'devel' branch. Does that mean the changes are all in 'master' now?

No, they are still in devel, which is pushed to gitlab. I'm still working
on making sure it doesn't try to push articles to remote when it should
not. It's a bit confusing mixing 'readonly', don't display in header,
remote_server = '', etc. I'm trying to be careful to check the right things
and not break something. I'm concerned that 'readonly' is not the
appropriate thing to check, it's specifically whether to push to remote or
not. I'll need some more time to work on that part.

But the duplicate listings should be fixed in devel at this time.

Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Retro Guy
Newsgroups: rocksolid.nodes.help
Organization: Rocksolid Light
Date: Tue, 15 Oct 2024 12:17 UTC
References: 1 2 3 4 5
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: retroguy@novabbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Tue, 15 Oct 2024 12:17:13 +0000
Organization: Rocksolid Light
Message-ID: <5993f81a81ef152e887ae36d845b86f3@www.novabbs.org>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net> <4a29d60d6c23db7a96d9be51580c6232@www.novabbs.org> <ea2ac41badbf9daa6835a4dd51cf1704@www.novabbs.org> <7c2f96de7e94edc8c71ea42c933e47e2$1@octade.net> <235e17a78bd7a00d437facd5eb09626d$1@novabbs.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
logging-data="2138284"; mail-complaints-to="usenet@i2pn2.org";
posting-account="HNchV9coSOli+kkOHciCZlGs2VaRn7Emh38jWV1k174";
User-Agent: Rocksolid Light
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf ?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B H0=hZt'[%)4kG|
X-Rslight-Posting-User: 28547bffeec402cc2e544e2f8e28cc9329f66434
X-Rslight-Site: $2y$10$TJnjjlFlzCMx1lwFFvKyLeVEYuvwnrvFoT4PMxUdaGTEQB6BshCjC
X-Spam-Checker-Version: SpamAssassin 4.0.0
View all headers

On Mon, 14 Oct 2024 22:24:52 +0000, Retro Guy wrote:

> On Mon, 14 Oct 2024 19:54:28 +0000, Byrl Raze Buckbriar wrote:
>
>> On Mon, 14 Oct 2024 12:19:53 +0000
>> Retro Guy <retroguy@novabbs.com> wrote:
>>
>>> On Mon, 14 Oct 2024 10:38:42 +0000, Retro Guy wrote:
>>>
>>>> On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
>>>>
>>>>> I am configuring and testing a Rocksolid Light peer to pull and archive
>>>>> 91 Usenet newsgroups, not counting the rocksolid groups.
>>>>
>>>>> 4. The newsreader software (Sylpheed) shows three identical listings of
>>>>> each newsgroup in the group list. How may I resolve this to show only
>>>>> one copy of the group?
>>>>
>>>> This identical listing issue should be fixed now. The fix is in cron.php
>>>> and has been pushed to devel (to gitlab).
>>>
>>> This seems fine now in Sylpheed (or should be any newsreader).
>>>
>>> I now have checking in TWO places to make sure to NOT send if 'readonly'
>>> is '1' for a section. Once when you try to post, and another time if for
>>> some reason it gets past that, it will check again when trying to
>>> actually feed it to the remote server.
>>>
>>> This is logged in debug.log. I am still cleaning up reply codes so the
>>> newsreader has a useful reply for it's logs (and debugging). So far I
>>> have been unable to feed an article to a remote site if the section is
>>> readonly.
>>>
>>> If there are several sections with the same group, only a NON-readonly
>>> section will allow it to be sent to remote. If there is no NON-readonly
>>> section containing the group, posting will fail.
>>
>> That sounds right. I plan to check it out this evening. It looks like
>> you merged 'devel' branch. Does that mean the changes are all in
>> 'master' now?
>
> No, they are still in devel, which is pushed to gitlab. I'm still
> working
> on making sure it doesn't try to push articles to remote when it should
> not. It's a bit confusing mixing 'readonly', don't display in header,
> remote_server = '', etc. I'm trying to be careful to check the right
> things
> and not break something. I'm concerned that 'readonly' is not the
> appropriate thing to check, it's specifically whether to push to remote
> or
> not. I'll need some more time to work on that part.
>
> But the duplicate listings should be fixed in devel at this time.

Ok, I think I have it and I'm testing today on a test site.

Disabling "remote push" will be handled in overrides.inc.php:
// Disable pushing articles to remote server for these sections
// (replace 'testsection...' with your choice of sections)
'disable_remote_push' => array("testsectionone", "testsectiontwo"),

This should make it simple to toggle on or off, and not send stuff by
mistake because an admin forgets to toggle something like 'readonly',
etc. Plus, you may not want the section to be readonly.

Two things will disable pushing to remote:
1. Section found in overrides.inc.php (as above)
2. 'remote_server' => '',

This code is in send.php, so at the last point (where pushing actually
happens as spoolnews.php does not send articles).

For now, I'm leaving the articles that might possibly appear in
<spooldir>/<section>/outgoing (articles to be pushed) in that dir so we
can see what's happening. There probably WILL be articles there, but
that's fine. We just don't want them sent (once sent, they disappear
from that dir). Blocking a push is logged to debug.log.

The dir is pruned after a few days old articles. I'll remove them
immediately in a future release.

I hope to push this later today, unless I find a big issue. I'll post
back here to let you know.

--
Retro Guy

Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
From: Retro Guy
Newsgroups: rocksolid.nodes.help
Organization: Rocksolid Light
Date: Tue, 15 Oct 2024 13:21 UTC
References: 1 2 3 4 5 6
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!news.mixmin.net!news.neodome.net!rocksolid2!i2pn2.org!.POSTED!not-for-mail
From: retroguy@novabbs.com (Retro Guy)
Newsgroups: rocksolid.nodes.help
Subject: Re: Triplicate Entries in Newsgroups List - Ensuring Suck Mode Only
Date: Tue, 15 Oct 2024 13:21:28 +0000
Organization: Rocksolid Light
Message-ID: <0b7ff276264567d6dae6690fcb940be4@www.novabbs.org>
References: <6b4076f800aacef6ebb97e69aabd14d9$1@octade.net> <4a29d60d6c23db7a96d9be51580c6232@www.novabbs.org> <ea2ac41badbf9daa6835a4dd51cf1704@www.novabbs.org> <7c2f96de7e94edc8c71ea42c933e47e2$1@octade.net> <235e17a78bd7a00d437facd5eb09626d$1@novabbs.org> <5993f81a81ef152e887ae36d845b86f3@www.novabbs.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
logging-data="2145333"; mail-complaints-to="usenet@i2pn2.org";
posting-account="HNchV9coSOli+kkOHciCZlGs2VaRn7Emh38jWV1k174";
User-Agent: Rocksolid Light
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf ?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B H0=hZt'[%)4kG|
X-Rslight-Posting-User: 28547bffeec402cc2e544e2f8e28cc9329f66434
X-Rslight-Site: $2y$10$7fX.Oeb4AoHHI.Iy3W90F.Ilyl/1tiFgbBBJ7f8SCtVtgjobvCv26
View all headers

On Tue, 15 Oct 2024 12:17:13 +0000, Retro Guy wrote:

> On Mon, 14 Oct 2024 22:24:52 +0000, Retro Guy wrote:
>
>> On Mon, 14 Oct 2024 19:54:28 +0000, Byrl Raze Buckbriar wrote:
>>
>>> On Mon, 14 Oct 2024 12:19:53 +0000
>>> Retro Guy <retroguy@novabbs.com> wrote:
>>>
>>>> On Mon, 14 Oct 2024 10:38:42 +0000, Retro Guy wrote:
>>>>
>>>>> On Mon, 14 Oct 2024 6:45:28 +0000, Byrl Raze Buckbriar wrote:
>>>>>
>>>>>> I am configuring and testing a Rocksolid Light peer to pull and archive
>>>>>> 91 Usenet newsgroups, not counting the rocksolid groups.
>>>>>
>>>>>> 4. The newsreader software (Sylpheed) shows three identical listings of
>>>>>> each newsgroup in the group list. How may I resolve this to show only
>>>>>> one copy of the group?
>>>>>
>>>>> This identical listing issue should be fixed now. The fix is in cron.php
>>>>> and has been pushed to devel (to gitlab).
>>>>
>>>> This seems fine now in Sylpheed (or should be any newsreader).
>>>>
>>>> I now have checking in TWO places to make sure to NOT send if 'readonly'
>>>> is '1' for a section. Once when you try to post, and another time if for
>>>> some reason it gets past that, it will check again when trying to
>>>> actually feed it to the remote server.
>>>>
>>>> This is logged in debug.log. I am still cleaning up reply codes so the
>>>> newsreader has a useful reply for it's logs (and debugging). So far I
>>>> have been unable to feed an article to a remote site if the section is
>>>> readonly.
>>>>
>>>> If there are several sections with the same group, only a NON-readonly
>>>> section will allow it to be sent to remote. If there is no NON-readonly
>>>> section containing the group, posting will fail.
>>>
>>> That sounds right. I plan to check it out this evening. It looks like
>>> you merged 'devel' branch. Does that mean the changes are all in
>>> 'master' now?
>>
>> No, they are still in devel, which is pushed to gitlab. I'm still
>> working
>> on making sure it doesn't try to push articles to remote when it should
>> not. It's a bit confusing mixing 'readonly', don't display in header,
>> remote_server = '', etc. I'm trying to be careful to check the right
>> things
>> and not break something. I'm concerned that 'readonly' is not the
>> appropriate thing to check, it's specifically whether to push to remote
>> or
>> not. I'll need some more time to work on that part.
>>
>> But the duplicate listings should be fixed in devel at this time.
>
> Ok, I think I have it and I'm testing today on a test site.
>
> Disabling "remote push" will be handled in overrides.inc.php:
> // Disable pushing articles to remote server for these sections
> // (replace 'testsection...' with your choice of sections)
> 'disable_remote_push' => array("testsectionone", "testsectiontwo"),
>
> This should make it simple to toggle on or off, and not send stuff by
> mistake because an admin forgets to toggle something like 'readonly',
> etc. Plus, you may not want the section to be readonly.
>
> Two things will disable pushing to remote:
> 1. Section found in overrides.inc.php (as above)
> 2. 'remote_server' => '',
>
> This code is in send.php, so at the last point (where pushing actually
> happens as spoolnews.php does not send articles).
>
> For now, I'm leaving the articles that might possibly appear in
> <spooldir>/<section>/outgoing (articles to be pushed) in that dir so we
> can see what's happening. There probably WILL be articles there, but
> that's fine. We just don't want them sent (once sent, they disappear
> from that dir). Blocking a push is logged to debug.log.
>
> The dir is pruned after a few days old articles. I'll remove them
> immediately in a future release.
>
> I hope to push this later today, unless I find a big issue. I'll post
> back here to let you know.

This is now pushed to gitlab (in devel). I'm running it live on
news.novabbs.org and of course on a test site.

There are some new log entries in debug.log that show what config file
is being used for each section:

Oct 15 13:10:56 Using /etc/rslight/rslight.inc.php for spoolnews
Oct 15 13:10:56 Using /etc/rslight/rslight.inc.php for rocksolid
Oct 15 13:10:57 Using /etc/rslight/rslight.inc.php for french
Oct 15 13:10:59 Using /etc/rslight/dupe.inc.php for dupe
Oct 15 13:10:59 Remote disabled for dupe (no remote server)
Oct 15 13:10:59 Using /etc/rslight/other2.inc.php for other2
Oct 15 13:10:59 Remote disabled for other2 (no remote server)

When overrides is used, it is logged to debug.log also.

--
Retro Guy

1

rocksolid light 0.9.8
clearnet tor