David Jones:

> If this were a simple time series, one mainstream approach
> from "time-series analysis" would be to present a spectral
> analysis of a detrended and prefiltered version of the
> complete timeseries, to try to highlight any remaining
> periodicities.

The Miller data /are/ a time series in a way, with the
readings as uniform as the rotation of the device. Would it
be possible to analyse it using the SigSpec algorithm:

https://en.wikipedia.org/wiki/SigSpec

using the eponymous program:

https://arxiv.org/pdf/1006.5081.pdf
--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

Anton Shepelev wrote:

> David Jones:
>
> > If this were a simple time series, one mainstream approach
> > from "time-series analysis" would be to present a spectral
> > analysis of a detrended and prefiltered version of the
> > complete timeseries, to try to highlight any remaining
> > periodicities.
>
> The Miller data are a time series in a way,

They are only a time-series because they have been manipulated in the
form of a time-series. You should not remove real structure in the form
of groups of data unless you can sure that doing so
(a) does not remove or mask effects you are looking
(b) does not introduce effects of the kind you are looking for.

> with the
> readings as uniform as the rotation of the device. Would it
> be possible to analyse it using the SigSpec algorithm:
>
> https://en.wikipedia.org/wiki/SigSpec
>
> using the eponymous program:
>
> https://arxiv.org/pdf/1006.5081.pdf

There may be better/more-capable packages available from time-series
analysis specialists. But you should be aware that any statistical
tests would depend on the validity of the usual assumptions which would
need to be given serious consideration, If you were planning on doing
something depending in a simple way on FFTs you would need to consider
that there is an inherent assumption that the series being analysed is
a good representative of a stationary process (in terms of the length
of the series being analysed). Loosely speaking, can you imagine in a
general way how the observed time-series would have behaved before and
after the period supposedly observed. The "time-series" in the 2006
paper seems to show a distinct change in behaviour part way through.

One might consider a logical way forward that doesn't place heavy
reliance on assumptions would be to show that the apparent peak in the
FFT, such as shown in the 2006 paper, is or is not removed when any
explanatory effects are removed, perhaps leaving this to be judged on
an informal basis. Even if this can be done, you could still be left
with the problem that you are looking for an effect whose cause is
indistinguishable from the effects of other causes, as previously
identified in other literature.

On 3/18/23 4:41 PM, Anton Shepelev wrote:
> The Miller data /are/ a time series in a way, with the readings as
> uniform as the rotation of the device. [...]

Each run can be considered to be a (regular) time series ONLY if it has
no omitted adjustment turns. From the dozen or so of those that I have
seen [#], I can say that you are HIGHLY unlikely to find any significant
1/2-turn or 1-turn "signal" in them. The ones I remember were simply a
monotonic drift totaling less than 1 fringe during 20 turns -- this is a
self-fulfilling prophecy, because if it drifted much more than that, he
would have made an adjustment. (My memory 17 years later is hazy.)

[#] When I was at CWRU to give a physics colloquium on
this, Prof. Fickinger and I spent a few hours in the
archives scanning Miller's data sheets for runs with no
adjustments. Our only selection criteria were 20 turns
with no adjustments (some runs have fewer turns, a very
few have more). I don't remember if we looked at every
sheet or had to quit before that; I believe we looked
at more than half. IIRC we found several dozen -- that
was 17 years ago and my memory is hazy. I have copies
somewhere, but have no interest in typing them in.

Interestingly, Dayton Miller was the first head of the physics
department at what is now CWRU. He designed their physics building, and
it has several pillars intended to support interferometers, which go all
the way down to bedrock and never connect to the rest of the building
(for vibration isolation). Also: the archives' exhibit on Dayton Miller
was primarily about his extensive collection of flutes; they had to dig
to find his interferometer data sheets, which were kept loose in about a
dozen folders.

Tom Roberts

David Jones:

> > The Miller data are a time series in a way,
>
> They are only a time-series because they have been
> manipulated in the form of a time-series.

Not at all: each run, consisting of 20 consequtive turns
with perhaps a few "adjustment" turns, spans about 20
minutes and represents and single measurement of the aether-
drift. The manipulation comprises reinstating the
adjustments and unrolling the 20 turns of 16 observasions
into a sequence 320 observations.

> You should not remove real structure in the form of groups
> of data

No such structure was removed. The periodicity of individual
turns is preserved in the observatsion indices and time
markings. The data of a "run" is physically a time-series.

> unless you can sure that doing so
> (a) does not remove or mask effects you are looking
> (b) does not introduce effects of the kind you are
> looking for.

I am sure the serialisation in question does neither.

> > with the
> > readings as uniform as the rotation of the device. Would it
> > be possible to analyse it using the SigSpec algorithm:
> > https://en.wikipedia.org/wiki/SigSpec
> > using the eponymous program:
> > https://arxiv.org/pdf/1006.5081.pdf
>
> There may be better/more-capable packages available from
> time-series analysis specialists.

There may be, but SigSpec seems one of the very best, and
specifically designed to detect significant spectral
components in time series. It will no doubt find significant
high-magnitude and low-frequency components in the Miller
signal, but we are interested in whether full- and half-
period components are prominent above the others, and by how
much. "Multisine" analysis (like SigSpec) seems more
unbiased in this case than the standard Fourier, with its
fixed set of harmonics.

> But you should be aware that any statistical tests would
> depend on the validity of the usual assumptions which
> would need to be given serious consideration, If you were
> planning on doing something depending in a simple way on
> FFTs you would need to consider that there is an inherent
> assumption that the series being analysed is a good
> representative of a stationary process (in terms of the
> length of the series being analysed).

The signal sought is stationary within each run, the noise
is also stationary, whereas the instrumental drift is
probably not.

> Loosely speaking, can you imagine in a general way how the
> observed time-series would have behaved before and after
> the period supposedly observed. But if we assume the
> instrumental drift to be free of any periodicity in turn,
> we may discrard spectral components whose frequencies are
> not multiples of 1/turn.

I can imagine that about the hypothetical signal and noise,
but not about the instrumental drift. The assumption,
however, that it is of lower frequency than the signal, may
help to separate one from the other.

> The "time-series" in the 2006 paper seems to show a
> distinct change in behaviour part way through.

It does.

> One might consider a logical way forward that doesn't
> place heavy reliance on assumptions would be to show that
> the apparent peak in the FFT, such as shown in the 2006
> paper, is or is not removed when any explanatory effects
> are removed, perhaps leaving this to be judged on an
> informal basis.

I thought that the spectral-significance (SigSpec) measure
was made to answer such questions.

> Even if this can be done, you could still be left with the
> problem that you are looking for an effect whose cause is
> indistinguishable from the effects of other causes, as
> previously identified in other literature.

Yes, the instrumental error itself could be periodic, but
then it would be present with similar parameters in all
"runs", which is not the case. Mr. Roberts made the same
assumtion -- that the instrumental error is not periodic in
a turn.

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

Anton Shepelev wrote:

> David Jones:
>
> > > The Miller data are a time series in a way,
> >
> > They are only a time-series because they have been
> > manipulated in the form of a time-series.
>
> Not at all: each run, consisting of 20 consequtive turns
> with perhaps a few "adjustment" turns, spans about 20
> minutes and represents and single measurement of the aether-
> drift. The manipulation comprises reinstating the
> adjustments and unrolling the 20 turns of 16 observasions
> into a sequence 320 observations.
>
> > You should not remove real structure in the form of groups
> > of data
>
> No such structure was removed. The periodicity of individual
> turns is preserved in the observatsion indices and time
> markings. The data of a "run" is physically a time-series.
>
> > unless you can sure that doing so
> > (a) does not remove or mask effects you are looking
> > (b) does not introduce effects of the kind you are
> > looking for.
>
> I am sure the serialisation in question does neither.

The question will be: will anyone else be sure?

I think I see a common approach between you and Prof. Roberts:
"I think I see a problem, this is what I think will solve the problem,
this is what I have done, therefore I have solved the problem"

>
> > > with the
> > > readings as uniform as the rotation of the device. Would it
> > > be possible to analyse it using the SigSpec algorithm:
> > > https://en.wikipedia.org/wiki/SigSpec
> > > using the eponymous program:
> > > https://arxiv.org/pdf/1006.5081.pdf
> >
> > There may be better/more-capable packages available from
> > time-series analysis specialists.
>
> There may be, but SigSpec seems one of the very best, and
> specifically designed to detect significant spectral
> components in time series. It will no doubt find significant
> high-magnitude and low-frequency components in the Miller
> signal, but we are interested in whether full- and half-
> period components are prominent above the others, and by how
> much. "Multisine" analysis (like SigSpec) seems more
> unbiased in this case than the standard Fourier, with its
> fixed set of harmonics.
>
> > But you should be aware that any statistical tests would
> > depend on the validity of the usual assumptions which
> > would need to be given serious consideration, If you were
> > planning on doing something depending in a simple way on
> > FFTs you would need to consider that there is an inherent
> > assumption that the series being analysed is a good
> > representative of a stationary process (in terms of the
> > length of the series being analysed).
>
> The signal sought is stationary within each run, the noise
> is also stationary, whereas the instrumental drift is
> probably not.
>
> > Loosely speaking, can you imagine in a general way how the
> > observed time-series would have behaved before and after
> > the period supposedly observed. But if we assume the
> > instrumental drift to be free of any periodicity in turn,
> > we may discrard spectral components whose frequencies are
> > not multiples of 1/turn.
>
> I can imagine that about the hypothetical signal and noise,
> but not about the instrumental drift. The assumption,
> however, that it is of lower frequency than the signal, may
> help to separate one from the other.
>
> > The "time-series" in the 2006 paper seems to show a
> > distinct change in behaviour part way through.
>
> It does.
>
> > One might consider a logical way forward that doesn't
> > place heavy reliance on assumptions would be to show that
> > the apparent peak in the FFT, such as shown in the 2006
> > paper, is or is not removed when any explanatory effects
> > are removed, perhaps leaving this to be judged on an
> > informal basis.
>
> I thought that the spectral-significance (SigSpec) measure
> was made to answer such questions.

You will need to get someone competent to check all the assumptions
involved.

>
> > Even if this can be done, you could still be left with the
> > problem that you are looking for an effect whose cause is
> > indistinguishable from the effects of other causes, as
> > previously identified in other literature.
>
> Yes, the instrumental error itself could be periodic, but
> then it would be present with similar parameters in all
> "runs", which is not the case. Mr. Roberts made the same
> assumtion -- that the instrumental error is not periodic in
> a turn.

But it is not just "instrumental errors" that need to be thought, it is
all the possible explanations put forward by people like Shankland.

David Jones wrote:

> Anton Shepelev wrote:
>
> > > One might consider a logical way forward that doesn't
> > > place heavy reliance on assumptions would be to show that
> > > the apparent peak in the FFT, such as shown in the 2006
> > > paper, is or is not removed when any explanatory effects
> > > are removed, perhaps leaving this to be judged on an
> > > informal basis.
> >
> > I thought that the spectral-significance (SigSpec) measure
> > was made to answer such questions.
>
> You will need to get someone competent to check all the assumptions
> involved.
>

Let me expand on that. It seems that the "statistical tests" are based
on asymptotic properties/results that are only valid if there is a
stationary process to be analysed. You agreed that the observed series
looks non-stationary. So the basic results cannot be used. However the
package might contain something to allow some version to be applied.

You may be hoping that a spectral analysis package will provide all
your answers, but recall that results of the FFT are just a
sophisticated version of regression analysis, and you may be better off
looking to that for a way to proceed.... provided that you don't apply
the parts of the theory of regression that are not valid here.

David Jones to Anton Shepelev:
> > David Jones to Anton Shepelev:
> >
> > > > The data of a "run" is physically a time-series.
> > >
> > > unless you can sure that doing so
> > > (a) does not remove or mask effects you are looking
> > > (b) does not introduce effects of the kind you are
> > > looking for.
> >
> > I am sure the serialisation in question does neither.
>
> The question will be: will anyone else be sure?

Indeed, but it is hard to prove the absense of either loss
you mention.

If you, or anybody else, think that the serialisation of the
turns of a run may introduce some distorition or make the
signal otherwise less noticeable, then please share you
specific concerns, that we may discuss whether they are
justified.

For my part, I can only repeat the each "run" represents
twenty or more consequent turns of the interferometer within
a space of 15-20 minutes. It contains 20*16+1=321
observations made over twenty "observation turns",
occasionally interrupted by "adjustment turns", during which
no observations were recorded. The data, therefore, is a
physical time series with gaps. You can view them in this
form in the seq_t directory in this archive:

http://freeshell.de/~antonius/file_host/RobertsMillerData.7z

Since the signal we seek is half-periodic in a turn,
adjustment turns do not disrupt it (in any way that I can
think of).

> I think I see a common approach between you and Prof.
> Roberts: "I think I see a problem, this is what I think
> will solve the problem, this is what I have done,
> therefore I have solved the problem"

Please note, that I initiated discussion of the statistical
analysis of the Miller experiemnts in this group,
specifically because I needed your help and advice as expert
statisticians. Mr. Roberts, on the other hand, professes no
such desire...

> > I thought that the spectral-significance (SigSpec)
> > measure was made to answer such questions.
>
> You will need to get someone competent to check all the
> assumptions involved.

Can you help me first to identify those assumtptions? That
the signal saught is stationary and periodic in a half-turn
is a fact. Noise is not periodic. The instrumental dirft may
be assumed to be aperiodic from looking at the measurements,
but a specific phycial or statistical justification is
welcome. The key point is to determine whether it may pose
as signal or not.

> Let me expand on that. It seems that the "statistical
> tests" are based on asymptotic properties/results that are
> only valid if there is a stationary process to be
> analysed. You agreed that the observed series looks non-
> stationary. So the basic results cannot be used. However
> the package might contain something to allow some version
> to be applied.

How does one determine whether the instrumental drift is a
stationary process? What do you think can make that process
non-stationary? The dominance of the basic linear drift
during the entire run seems to indicate that it is
statuionary within the period of the run. After consulting
the definitiona of a stationary process, I retract my
previous statement to the contrary.

The SigSpec program performs a multisine analysys of a time
series, finding its most significant spectral components (in
no way limited to multiples of a fundamental frequency),
their respective significance, and the residual data. This
should work as well if the singal is stationary and the
error is not.

> You may be hoping that a spectral analysis package will
> provide all your answers, but recall that results of the
> FFT are just a sophisticated version of regression
> analysis, and you may be better off looking to that for a
> way to proceed.... provided that you don't apply the parts
> of the theory of regression that are not valid here.

With FFT, we know our basis beforehand. With multisine, we
do not, which makes it less "prejudiced" to what is sought.
If a significant half-period component appear in multisine,
it will indicate much more than such a component in the FFT,
where it is mathematicaly bound to appear, as Mr. Roberts
correctly observes. Thank you for the advice about
regression. I will think how I can apply it to the data in a
way different from that of Mr. Roberts. Basically,j

> But it is not just "instrumental errors" that need to be
> thought, it is all the possible explanations put forward
> by people like Shankland.

Yes, and that is another direction of research. Can such a
termperature gradient in the room be imagined as to produce
a half-period effect? Can this situation occur in reality?
Is it compaible with the thermometer indications during the
experiment? Is the themal inerita of the insulated
interferometer arms sufficient to suppress that effect to a
magnitude much lower than that of the observed signal?
Unfortunately, I have not been able to read Shankland's
original, and am only acquainted with it through the
criticism of James DeMeo:

http://www.orgonelab.org/miller.htm

See there "see: Shankland Team's 1955 Critique of Miller,"
which is quite interesting, e.g.:

If the periodic effects observed by Miller were the
product of temperature variations, as was claimed by
Shankland and Joos, then why would that variation
systematically point to the same set of azimuth
coordinates along the celestial sidereal clock, but not
to any single terrestrial coordinate linked to civil
time? Miller repeatedly asked this question of his
critics, who had no answer for it. The Shankland team
likewise evaded the question.

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

Tom Roberts:

> > The Miller data /are/ a time series in a way, with the
> > readings as uniform as the rotation of the device. [...]
>
> Each run can be considered to be a (regular) time series
> ONLY if it has no omitted adjustment turns.

If regular means gapless. In my pre-processing, I
incorporated the adjustments as you did but instead of
cutting the adjustment turns out I counted them, producing
time series with gaps in readings. Thankfully, there are
methods and even software to analyse such series.

> From the dozen or so of those that I have seen [#], I can
> say that you are HIGHLY unlikely to find any significant
> 1/2-turn or 1-turn "signal" in them.

You published set contains several runs marked "steady",
"very steady", and "perfectly steady" in your
transcriptions. They are not, however, literally monotonic,
but often predominantly so.

> The ones I remember were simply a monotonic drift totaling
> less than 1 fringe during 20 turns this is a self-
> fulfilling prophecy, because if it drifted much more than
> that, he would have made an adjustment. (My memory 17
> years later is hazy.)

Of course, yet there are "perfect" Mt. Wilson runs with no
adjustments and a considerable (in comparison to others)
half-periodic signal, present before any comb-filtering.

> When I was at CWRU to give a physics colloquium on this,
> Prof. Fickinger and I spent a few hours in the archives
> scanning Miller's data sheets for runs with no
> adjustments. Our only selection criteria were 20 turns
> with no adjustments (some runs have fewer turns, a very
> few have more). I don't remember if we looked at every
> sheet or had to quit before that; I believe we looked at
> more than half. IIRC we found several dozen -- that was 17
> years ago and my memory is hazy. I have copies somewhere,
> but have no interest in typing them in.

Will you share all your scans so that I can type them in?

> Interestingly, Dayton Miller was the first head of the
> physics department at what is now CWRU. He designed their
> physics building, and it has several pillars intended to
> support interferometers,

Why more than one? One might think he was erecting a temple
ofthe Ether Wind :-)

> which go all the way down to bedrock and never connect to
> the rest of the building (for vibration isolation).

Similar to some of my loudspeaker designs isolating the
driver from the front baffle of the enclosure.

> Also: the archives' exhibit on Dayton Miller was primarily
> about his extensive collection of flutes;

Does it include his works on acoustics, and particularly the
Phonodeik?

> they had to dig to find his interferometer data sheets,
> which were kept loose in about a dozen folders.

I believe they came from Shankland, after a long period of
being unaccounted and missing. Have you an idea how I can
order scans of those sheets wihtout actually coming to CWRU?
If all hope for a positive result is to be abanandoned, I
will do it after a diligent study of the available data,
meaning some 300 or so sheets from the Mt. Wilson
experiments.

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

Tom Roberts <tjoberts137@sbcglobal.net> wrote:

[-]
> Interestingly, Dayton Miller was the first head of the physics
> department at what is now CWRU. He designed their physics building, and
> it has several pillars intended to support interferometers, which go all
> the way down to bedrock and never connect to the rest of the building
> (for vibration isolation). Also: the archives' exhibit on Dayton Miller
> was primarily about his extensive collection of flutes; they had to dig
> to find his interferometer data sheets, which were kept loose in about a
> dozen folders.

That sounds like he no longer believed in it himself,

Jan

Anton Shepelev <anton.txt@g{oogle}mail.com> wrote:

> Tom Roberts:
[-]
> > Interestingly, Dayton Miller was the first head of the
> > physics department at what is now CWRU. He designed their
> > physics building, and it has several pillars intended to
> > support interferometers,
>
> Why more than one? One might think he was erecting a temple
> ofthe Ether Wind :-)

Of course not. There are a great many other physics experiments
that require a stable and vibration-free foundation.
(clocks, mechanical galvanometers, spectroscopes, etc)

Any prudent designer would foresee the need for stable foundations,

Jan

J. J. Lodder to Anton Shepelev:
> > Tom Roberts:
> >
> > > Interestingly, Dayton Miller was the first head of the
> > > physics department at what is now CWRU. He designed
> > > their physics building, and it has several pillars
> > > intended to support interferometers,
> >
> > Why more than one? One might think he was erecting a
> > temple ofthe Ether Wind :-)
>
> Of course not. There are a great many other physics
> experiments that require a stable and vibration-free
> foundation. (clocks, mechanical galvanometers,
> spectroscopes, etc)

But they are not interferometers.

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

J. J. Lodder to Tom Roberts:

> > Also: the archives' exhibit on Dayton Miller was
> > primarily about his extensive collection of flutes; they
> > had to dig to find his interferometer data sheets, which
> > were kept loose in about a dozen folders.
>
> That sounds like he no longer believed in it himself,

Loose sheets probably means they are not bound (cf. Pascal's
Pensees), not they are in disarray. Miller entrusted his
records to Shankland to "analyse them or burn them."
Prompted in 2002 by antirelativist speculations about
Shankland's destuction of them, CWRU staff made an
intensive search, finding the Miller archive in the Physics
Department.

I have no evidence whatsoever of Miller's disappointment in
the idea of either wind. The last publication, made in 1940
and a year before his death, is a short recap of his ether-
drift experiments and their positive results.

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

Anton Shepelev wrote:

> David Jones to Anton Shepelev:
> > > David Jones to Anton Shepelev:
> > >
> > > > > The data of a "run" is physically a time-series.
> > > >
> > > > unless you can sure that doing so
> > > > (a) does not remove or mask effects you are looking
> > > > (b) does not introduce effects of the kind you are
> > > > looking for.
> > >
> > > I am sure the serialisation in question does neither.
> >
> > The question will be: will anyone else be sure?
>
> Indeed, but it is hard to prove the absense of either loss
> you mention.
>
> If you, or anybody else, think that the serialisation of the
> turns of a run may introduce some distorition or make the
> signal otherwise less noticeable, then please share you
> specific concerns, that we may discuss whether they are
> justified.
>
> For my part, I can only repeat the each "run" represents
> twenty or more consequent turns of the interferometer within
> a space of 15-20 minutes. It contains 20*16+1=321
> observations made over twenty "observation turns",
> occasionally interrupted by "adjustment turns", during which
> no observations were recorded. The data, therefore, is a
> physical time series with gaps. You can view them in this
> form in the seq_t directory in this archive:
>
> http://freeshell.de/~antonius/file_host/RobertsMillerData.7z
>
> Since the signal we seek is half-periodic in a turn,
> adjustment turns do not disrupt it (in any way that I can
> think of).
>
> > I think I see a common approach between you and Prof.
> > Roberts: "I think I see a problem, this is what I think
> > will solve the problem, this is what I have done,
> > therefore I have solved the problem"
>
> Please note, that I initiated discussion of the statistical
> analysis of the Miller experiemnts in this group,
> specifically because I needed your help and advice as expert
> statisticians. Mr. Roberts, on the other hand, professes no
> such desire...
>
> > > I thought that the spectral-significance (SigSpec)
> > > measure was made to answer such questions.
> >
> > You will need to get someone competent to check all the
> > assumptions involved.
>
> Can you help me first to identify those assumtptions? That
> the signal saught is stationary and periodic in a half-turn
> is a fact. Noise is not periodic. The instrumental dirft may
> be assumed to be aperiodic from looking at the measurements,
> but a specific phycial or statistical justification is
> welcome. The key point is to determine whether it may pose
> as signal or not.
>
> > Let me expand on that. It seems that the "statistical
> > tests" are based on asymptotic properties/results that are
> > only valid if there is a stationary process to be
> > analysed. You agreed that the observed series looks non-
> > stationary. So the basic results cannot be used. However
> > the package might contain something to allow some version
> > to be applied.
>
> How does one determine whether the instrumental drift is a
> stationary process? What do you think can make that process
> non-stationary? The dominance of the basic linear drift
> during the entire run seems to indicate that it is
> statuionary within the period of the run. After consulting
> the definitiona of a stationary process, I retract my
> previous statement to the contrary.
>
> The SigSpec program performs a multisine analysys of a time
> series, finding its most significant spectral components (in
> no way limited to multiples of a fundamental frequency),
> their respective significance, and the residual data. This
> should work as well if the singal is stationary and the
> error is not.
>
> > You may be hoping that a spectral analysis package will
> > provide all your answers, but recall that results of the
> > FFT are just a sophisticated version of regression
> > analysis, and you may be better off looking to that for a
> > way to proceed.... provided that you don't apply the parts
> > of the theory of regression that are not valid here.
>
> With FFT, we know our basis beforehand. With multisine, we
> do not, which makes it less "prejudiced" to what is sought.
> If a significant half-period component appear in multisine,
> it will indicate much more than such a component in the FFT,
> where it is mathematicaly bound to appear, as Mr. Roberts
> correctly observes. Thank you for the advice about
> regression. I will think how I can apply it to the data in a
> way different from that of Mr. Roberts. Basically,j
>

<snip>

Obviously we can’t hope to deal with the whole of statistical theory
here. But we can look, in some simple cases, at the effects of dealing
or not dealing with pre-analysis data-manipulations within the data
analysis.

Even the most basic statistics work relates to dealing with
within-analysis manipulations. For example the usual formula for the
estimated variance contains the divisor (n-1) instead of the divisor n,
and this can be considered to be an adjustment to take account of the
fact that you subtract-off the sample mean within the analysis.
Similarly, in regression, the sum-of-squares is divided by (n-p) to
take account of fitting a total of p parameters. In both cases the
adjustment is made to get an unbiased estimate of the variance.

So, let’s consider some pre-analysis data manipulations. Let’s assume
you have two pairs of observations (X1,X2) and (Y1,Y2), with
statistical independence within and between pairs. Let the theoretical
mean of each observation in the first pair be M1, and let the
theoretical mean of each observation in the second pair be M2. Suppose
it is assumed the theoretical variance for each of the four
observations is the same, and consider two cases where this is either
known to be 1, or else it needs to be estimated. Then consider four
versions of analyses with different pre-analysis manipulations as
follows.

(a) Separate analysis. Here the data being analysed consists of the two
pairs (X1,X2) and (Y1,Y2). Then the sample-means with each pair,
provide unbiased estimates of the two values M1 and M2, and the
theoretical variance of each estimate is 1/2 if the variance of the
observations is assumed known at 1. If the variance of observations is
unknown, one could get and use two different estimates of that variance
from the sample variance applied within each pair. Each such estimate
would be unbiased.

(b) Separate analysis, but pooled. This is the same as for (a), above,
except that the variance of the observations is estimated by the
average the sampling variances from the two pairs. The theoretical
variances of the means remain the same as in (a), but one gets better
estimates of those variances. This is achieved by making use of an
assumed structure across the pairs (that the variances are the same).

(c) Subtraction of means. To yield a special case of what might be done
for longer series, suppose that a single dataset of 4 values
(Q1,Q2,Q3,Q4) is constructed from the two pairs by subtracting the two
sample means, giving
Q1=(X1-X2)/2, Q2=(X2-X1)/2, Q3=(Y1-Y2)/2, Q4=(Y2-Y1)/2
Obviously doing this prevents any estimation of the means M1 and M2.
Applying the usual formula to get a sample variance from (Q1,Q2,Q3,Q4)
gives an estimate that has a mean value of 2/3 when the true
observation variance is known to be 1. To get a good (unbiased)
estimate you have to know the structure of the pre-analysis data
manipulation that yielded the data-to-be-analysed (Q1,Q2,Q3,Q4). In
fact this turns out to be the pooled sample variance from the original
pairs as in (b). Thus, not all is necessarily lost in doing
pre-analysis data-manipulations, provided that the actual analysis
takes account of those manipulations.

(d) Joining of data. To emulate the data-joining of the paper and of
your proposed analysis, we can consider dealing with a revised dataset
(Z1,Z2,Z3), where
Z1=X1, Z2=X2, Z3=Y2+X2-Y1
Then the mean of each value is M1, and it clear that M1 can be
estimated but not M2. One might use the sample mean of (Z1,Z2,Z3) to
estimate M1: this estimate has a theoretical variance of 7/9. Thus this
estimate is worse than the sample mean of just (Z1,Z2), which is the
same as the sample mean of (X1,X2), whose variance is 1/2. The usual
sample variance obtained from (Z1,Z2,Z3) has an expected value of 5/3
when the theoretical observation variance is 1. If the usual sample
variance obtained from (Z1,Z2,Z3) is used to estimate the variance of
the sample mean of (Z1,Z2,Z3), this would have an expected value of 5/9
rather than the true variance of this sample mean which is 7/9. So
here, if one ignores the way in which (Z1,Z2,Z3) were obtained and just
uses the usual sample estimates, we get an estimate for M1 which is
worse (in terms of variance) than what might have been obtained by just
using one the one sample pair (X1,X2). Moreover the usual formula would
give estimated variances which are biased in either case of trying to
estimate the observation variance or the variance of the sample mean.
One might consider other estimates here, derived from (Z1,Z2,Z3), but
whether or not one looked for optimal estimates this would involve
taking into account the structure by which the dataset was created. To
summarise, poor performance will arise from any attempt to analyse the
constructed dataset without taking into account the details of how it
was constructed. In this example, the data-manipulation throws away any
ability to estimate an important property (M2) of one part the original
dataset whereas retaining all the original data and the structure
therein allows everything to be estimated.

On 3/20/23 6:07 AM, Anton Shepelev wrote:
> Have you an idea how I can order scans of those sheets wihtout
> actually coming to CWRU?

Send email to the archives, archives@case.edu. Ask for an inventory of
Miller's data sheets. They used to be willing to copy them for a nominal
fee, if you told them precisely which sheets to copy (e.g. all sheets
from Mt. Wilson).

When Prof. Fickinger and I visited the archives in 2006, they extended
him (+me) the ability to handle the sheets and flag individual ones for
copying. We had to be careful and preserve the order in the folders.
They snail-mailed paper copies to me. Today you can probably
request that they scan them to PDF and send them to you via email.

Tom Roberts

Thank you for the answer, David Jones:

> Obviously we can't hope to deal with the whole of
> statistical theory here. But we can look, in some simple
> cases, at the effects of dealing or not dealing with pre-
> analysis data-manipulations within the data analysis.
>
> Even the most basic statistics work relates to dealing
> with within-analysis manipulations.

Understood.

> For example the usual formula for the estimated variance
> contains the divisor (n-1) instead of the divisor n, and
> this can be considered to be an adjustment to take account
> of the fact that you subtract-off the sample mean within
> the analysis.

In my understanding, this relates to the summands (and
degrees of freedom) being one fewer than the elements in the
sample.

> Similarly, in regression, the sum-of-squares is divided by
> (n-p) to take account of fitting a total of p parameters.
> In both cases the adjustment is made to get an unbiased
> estimate of the variance.

Understood.

> So, let's consider some pre-analysis data manipulations.
> Let's assume you have two pairs of observations (X1,X2)
> and (Y1,Y2), with statistical independence within and
> between pairs. Let the theoretical mean of each
> observation in the first pair be M1, and let the
> theoretical mean of each observation in the second pair be
> M2. Suppose it is assumed the theoretical variance for
> each of the four observations is the same, and consider
> two cases where this is either known to be 1, or else it
> needs to be estimated. Then consider four versions of
> analyses with different pre-analysis manipulations as
> follows.
>
> (a) Separate analysis. Here the data being analysed
> consists of the two pairs (X1,X2) and (Y1,Y2). Then the
> sample-means with each pair, provide unbiased estimates of
> the two values M1 and M2, and the theoretical variance of
> each estimate is 1/2 if the variance of the observations
> is assumed known at 1. If the variance of observations is
> unknown, one could get and use two different estimates of
> that variance from the sample variance applied within each
> pair. Each such estimate would be unbiased.

OK.

> (b) Separate analysis, but pooled. This is the same as for
> (a), above, except that the variance of the observations
> is estimated by the average the sampling variances from
> the two pairs. The theoretical variances of the means
> remain the same as in (a), but one gets better estimates
> of those variances. This is achieved by making use of an
> assumed structure across the pairs (that the variances are
> the same).

In this case, we calculate the means separately, but then
pool the samples together to calculate their common
variance.

> (c) Subtraction of means. To yield a special case of what
> might be done for longer series, suppose that a single
> dataset of 4 values (Q1,Q2,Q3,Q4) is constructed from the
> two pairs by subtracting the two sample means, giving
> Q1=(X1-X2)/2, Q2=(X2-X1)/2, Q3=(Y1-Y2)/2, Q4=(Y2-Y1)/2

I don't like the method of this subtaction, because it
produces a reduandant dataset: Q1=-Q2 and Q3=-Q4. Since the
size of the dataset is equal to the total size of the
original datasets, half the information has been lost.

> Obviously doing this prevents any estimation of the means
> M1 and M2. Applying the usual formula to get a sample
> variance from (Q1,Q2,Q3,Q4) gives an estimate that has a
> mean value of 2/3 when the true observation variance is
> known to be 1.

Indeed, but this estimate belongs to a very different
sample. The difference of random variables is distributed
quite unlike either variable.

> To get a good (unbiased) estimate you have to know the
> structure of the pre-analysis data manipulation that
> yielded the data-to-be-analysed (Q1,Q2,Q3,Q4).

Yes.

> In fact this turns out to be the pooled sample variance
> from the original pairs as in (b).

Hmmmm. I don't see why, but will take it for granted now.
Will check later.

> Thus, not all is necessarily lost in doing pre-analysis
> data-manipulations, provided that the actual analysis
> takes account of those manipulations.

Thou shalt know thy data.

> (d) Joining of data. To emulate the data-joining of the
> paper and of your proposed analysis, we can consider
> dealing with a revised dataset (Z1,Z2,Z3), where
> Z1=X1, Z2=X2, Z3=Y2+X2-Y1

If the (X1,X2) represent the first turn and (Y1,Y2) the
second turn, then the revised dataset according to the paper
is: (X1, X2, Y1, Y2). Later Tom Roberts constucts error-
differences:

Ed1[0 ] = 0; Ed1[1 ] = Y1-X1
Ed2[1/2] = 0; Ed2[3/2] = Y2-X2

and fits their initial levels b1 and b2 to make the error
function E

E = b1 * Ed1 + b2 * Ed2

as smooth as possible in terms of L2 between adjacent values
weighted by the inverse errorbar. His calculation of the
errorbars is another story.

I did not propose subract the subsequences.

> Then the mean of each value is M1, and it clear that M1
> can be estimated but not M2. One might use the sample mean
> of (Z1,Z2,Z3) to estimate M1: this estimate has a
> theoretical variance of 7/9. Thus this estimate is worse
> than the sample mean of just (Z1,Z2), which is the same as
> the sample mean of (X1,X2), whose variance is 1/2. The
> usual sample variance obtained from (Z1,Z2,Z3) has an
> expected value of 5/3 when the theoretical observation
> variance is 1. If the usual sample variance obtained from
> (Z1,Z2,Z3) is used to estimate the variance of the sample
> mean of (Z1,Z2,Z3), this would have an expected value of
> 5/9 rather than the true variance of this sample mean
> which is 7/9. So here, if one ignores the way in which
> (Z1,Z2,Z3) were obtained and just uses the usual sample
> estimates, we get an estimate for M1 which is worse (in
> terms of variance) than what might have been obtained by
> just using one the one sample pair (X1,X2). Moreover the
> usual formula would give estimated variances which are
> biased in either case of trying to estimate the
> observation variance or the variance of the sample mean.

Yes, because the new data is transformed from the original,
it has a different distribution and, generally, different
moments.

> One might consider other estimates here, derived from
> (Z1,Z2,Z3), but whether or not one looked for optimal
> estimates this would involve taking into account the
> structure by which the dataset was created. To summarise,
> poor performance will arise from any attempt to analyse
> the constructed dataset without taking into account the
> details of how it was constructed.

That is /as if/ that dataset were the original,
untransformed, sample, which it is not.

> In this example, the data-manipulation throws away any
> ability to estimate an important property (M2) of one part
> the original dataset whereas retaining all the original
> data and the structure therein allows everything to be
> estimated.

Yes.

> So my conclusion is that you should not try to merge
> groups of data into one supposedly-continuous time-series
> as you don't have to do so.

No, I should not. But in the experiment in question[1], the
data is truly a continuous times series, arranged in a two-
dimensional table for presentation. Therefore, when I (and
Mr. Roberts) merge it back into a single time series, I
commit no fallacy nor transform the data in a statistically
significant manner. Of course, I should not join the
sequences of individual turns if they did not come from the
same pool, and were not produced in the same uninterruted
sequence of observations.

> It is possible to do a combined analysis of all groups
> within joining them.

Without?

> Since there is just one pre-specified frequency there is
> no need to do a spectral analysis.

Perhaps not, but I think it one of valid approaches (if not
the best): a comparison of the various spectral components
may provide insights into whether the one component in
question is genuine signal or part of the noise and
systematic device drift.

> But, if you really wanted to do a spectral analysis
> combining all groups without joining them together, this
> is certainly possible ... you just have to understand the
> meaning of the quantities produced in the analysis of a
> single series.

So, you propose to amend my analysis by performing 20
separate spectral analyses?
____________________
1. https://freeshell.de//~antonius/file_host/Miller-EtherDrift-1933.pdf

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

Anton Shepelev wrote:

> Thank you for the answer, David Jones:
>
>
>
> > But, if you really wanted to do a spectral analysis
> > combining all groups without joining them together, this
> > is certainly possible ... you just have to understand the
> > meaning of the quantities produced in the analysis of a
> > single series.
>
> So, you propose to amend my analysis by performing 20
> separate spectral analyses?
> ____________________

No, I suggest you do a single combined spoectral analysis, or that you
do a sine-curve regression, by not pretending you have a single
time-series. You have to understand that an ordinary spectral analysis
is just a special case of regression.

A usual time-series analysis would proceed on the basis that the
relevant "times" are equally spaced and that the position/index within
a single array can be used as a time, instead of having the time
specified explicitly. Then when the time is needed, for example to use
in a sine or cosine function, it is immediately available, rather than
deriving it from the position in the array, So, as a starting position
you would have a dataset that closely corresponds to the actual results
of the experiment. Thus, in instances where the "joining" approach
would replace two values with one, the two original values would be
kept separately, and there would be no related adjustments.
Additionally where there were actually gaps in the original
observations, this can be included. Similarly for incomplete sequences.

Of course this means that the usual time-series packages would not be
useable and, specifically, not the FFT. BUt the series for your problem
are not very long, so no strong need for the FFT.

If you take the view that one of your objectives is to leave a set of
data that are available for others to re-analyse, then it would be good
to include as much explicit information as possible, without
pre-judging what analysis might be done.

To analyse the data, you need to have a statstical model for the
original observations. Such a model explicitly represents what the
modeller thinks explains the variation in the observations. Possibly
this would be represented as an ordinary regression model, but a mixed
fixed- and random-effects model might be considered. THere are
reasonably standard procedures linked to regression that allow checks
to be made on the various assumptions and to look-out for unexplained
effects.

To turn this into a "spectral analysis", YOu would just need to do a
sequence of regression analyses, where each one would have a sine- and
cosine-pair of a single frequency, and where the corresponding value in
the "spectrogram" would be the sum of the square of the regression
coefficients of the sine and cosine terms. It may be possible to
justify reducing the compuations here, by first do a single regression
with no sinusoid at all, and then doing the spectral analysis on the
residuals.

I Suggest you find an experienced statistician to undertake much of
this. At the very least you need someone with the time and ability to
think, and someone who will not try to force the analysis to fit within
the compass of some existing computer package.