Something with inventory comparison is making peer syncronization very
slow. It was taking almost 15 minutes to complete every run of
cron.php. My php is not very savvy but here's what I've gathered so far.

spoolnews.php had these variables:

if(!isset($maxarticles_per_run)) {
$maxarticles_per_run = 30;
} if(!isset($maxfirstrequest)) {
$maxfirstrequest = 3000;
}

It seems that every time cron.php runs the $maxfirstrequest value
is read and stat compared that many messages backward on the remote
peer. So for every newsgroup synced, 1000 stats and message-id
comparisons being performed, not just the first time, but every time
cron.php runs. This was happening regardless of the size of
$maxarticles_per_run. At least that is how it looks in the logs.

So if 1000 newsgroups are already syncronized then 1 million message
lookups will be performed even though there may be no new messages on
either peer. At least that's how it seems.

It also has the effect of making sync extremely slow even when there
are no new messages on the remote peer, and all messages have been
synchronized for months.

I reduced $maxfirstrequest to 1 and now there is a 25x speedup each
time cron.php runs.

Is there a workaround to get it just to check the head count value and
work back from newest and stop when the first inventory collision is
found? Or maybe should I try to create a function for it?

--
3883@sugar.bug | web: sybershock.com | news: alt.sources.crypto

Syber Shock wrote:

> Something with inventory comparison is making peer syncronization very
> slow. It was taking almost 15 minutes to complete every run of
> cron.php. My php is not very savvy but here's what I've gathered so far.

> spoolnews.php had these variables:

> if(!isset($maxarticles_per_run)) {
> $maxarticles_per_run = 30;
> }
> if(!isset($maxfirstrequest)) {
> $maxfirstrequest = 3000;
> }

This is what's in the file. It looks modified (just the numbers) but that
is what's in spoolnews.php.

> It seems that every time cron.php runs the $maxfirstrequest value
> is read and stat compared that many messages backward on the remote
> peer. So for every newsgroup synced, 1000 stats and message-id

I believe that cron.php is timing out. I think 15 minutes was the value
hardcoded to timeout. If that's the case, the last article retrieved
is not being recorded, so it's starting over every time.

> comparisons being performed, not just the first time, but every time
> cron.php runs. This was happening regardless of the size of
> $maxarticles_per_run. At least that is how it looks in the logs.

> So if 1000 newsgroups are already syncronized then 1 million message
> lookups will be performed even though there may be no new messages on
> either peer. At least that's how it seems.

> It also has the effect of making sync extremely slow even when there
> are no new messages on the remote peer, and all messages have been
> synchronized for months.

> I reduced $maxfirstrequest to 1 and now there is a 25x speedup each
> time cron.php runs.

Maybe it's not timing out?

Are you running through a slow network, or clearnet? If it's I2P, I'm
not terribly surprised if all the STAT requests are slowing it down to
an extreme level.

> Is there a workaround to get it just to check the head count value and
> work back from newest and stop when the first inventory collision is
> found? Or maybe should I try to create a function for it?

spoolnews.php has not used STAT for quite a while now. This has been
switched to XOVER. So it's one request for all messages it wants, not
one per request (for header info).

Then, duplicate checking is done from the overview received.

I do sync one rslight over I2P, and it works fine, but I'm running the
latest code. There have been a lot of changes since php7 code.

--
Retro Guy