Shannon did some experiments to determine the entropy in English texts.
A later
work done by Cover and King [1] gave an estimate of 1.34 bits per
letter. This
implies that, if the letters are coded into 5 bits, one needs to
appropriately
combine 4 text files in order to obtain bit sequences of full entropy, since
4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32)
the coded
values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of
the
text files.

There are plenty of other schemes for obtaining high quality pseudo-random
sequences in practice, e.g. AES in counter mode. However our scheme seems to
be much simpler both in the underlying logic (understandability) and in
implementation and is thus a viable alternative that one could use/need
under
circumstances.

The software, TEXTCOMBINE-SP, is available at http://mok-kong-shen.de

M. K. Shen
-------------------------------------------------------------------------------

[1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the
Entropy of
English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.

You are repeating yourself. Do you think that if you say it three times
(as with the Bellman) it will suddenly become worthwhile?
As I have said, this is a horrible scheme. text has many long range
correlations (from charater pairs to paragraphs, etc) , which would mess up the random stream. (make it non-random).
Bad idea.
And you might want to look at what Shannon and others actually did.

Note that even if the letters really were completely random, your method
of combining them would make the output non-random.

On 2017-07-11, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
>
> Shannon did some experiments to determine the entropy in English texts.
> A later
> work done by Cover and King [1] gave an estimate of 1.34 bits per
> letter. This
> implies that, if the letters are coded into 5 bits, one needs to
> appropriately
> combine 4 text files in order to obtain bit sequences of full entropy, since
> 4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32)
> the coded
> values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of
> the
> text files.
>
> There are plenty of other schemes for obtaining high quality pseudo-random
> sequences in practice, e.g. AES in counter mode. However our scheme seems to
> be much simpler both in the underlying logic (understandability) and in
> implementation and is thus a viable alternative that one could use/need
> under
> circumstances.
>
> The software, TEXTCOMBINE-SP, is available at http://mok-kong-shen.de
>
> M. K. Shen
> -------------------------------------------------------------------------------
>
> [1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the
> Entropy of
> English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.

Am 11.07.2017 um 22:01 schrieb William Unruh:
> You are repeating yourself. Do you think that if you say it three times
> (as with the Bellman) it will suddenly become worthwhile?
> As I have said, this is a horrible scheme. text has many long range
> correlations (from charater pairs to paragraphs, etc) , which would mess up the random stream. (make it non-random).
> Bad idea.
> And you might want to look at what Shannon and others actually did.
>
> Note that even if the letters really were completely random, your method
> of combining them would make the output non-random.

See the reference I gave of the paper about entropy and the test
statistic of Maurer's test.

(I had answered your post in another group and wonder why you didn't
answer there and switched to this one. I post to diverse groups because
the readers of different groups are not the same. Understand?)

M. K. Shen

>
> On 2017-07-11, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
>>
>> Shannon did some experiments to determine the entropy in English texts.
>> A later
>> work done by Cover and King [1] gave an estimate of 1.34 bits per
>> letter. This
>> implies that, if the letters are coded into 5 bits, one needs to
>> appropriately
>> combine 4 text files in order to obtain bit sequences of full entropy, since
>> 4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32)
>> the coded
>> values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of
>> the
>> text files.
>>
>> There are plenty of other schemes for obtaining high quality pseudo-random
>> sequences in practice, e.g. AES in counter mode. However our scheme seems to
>> be much simpler both in the underlying logic (understandability) and in
>> implementation and is thus a viable alternative that one could use/need
>> under
>> circumstances.
>>
>> The software, TEXTCOMBINE-SP, is available at http://mok-kong-shen.de
>>
>> M. K. Shen
>> -------------------------------------------------------------------------------
>>
>> [1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the
>> Entropy of
>> English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.

On 12.07.17 17:37, Mok-Kong Shen wrote:
> Am 11.07.2017 um 22:01 schrieb William Unruh:
>> You are repeating yourself. Do you think that if you say it three times
>> (as with the Bellman) it will suddenly become worthwhile?
>> As I have said, this is a horrible scheme. text has many long range
>> correlations (from charater pairs to paragraphs, etc) , which would
>> mess up the random stream. (make it non-random).
>> Bad idea.
>> And you might want to look at what Shannon and others actually did.
>>
>> Note that even if the letters really were completely random, your method
>> of combining them would make the output non-random.
>
> See the reference I gave of the paper about entropy and the test
> statistic of Maurer's test.
>
Did it ever occur to you that verifying randomness only by the Maurer
test is not sufficient?

What about ENT, test for bias or, much more important, Pierre L'Ecuyer's
TestU01 suite?

In the past I have already demonstrated that one of your PRNG's, namely
PERMPOLYPRNG which passed the Maurer test, is massively flawed.

> (I had answered your post in another group and wonder why you didn't
> answer there and switched to this one. I post to diverse groups because
> the readers of different groups are not the same. Understand?)
>
> M. K. Shen
>
>>
>> On 2017-07-11, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
>>>
>>> Shannon did some experiments to determine the entropy in English texts.
>>> A later
>>> work done by Cover and King [1] gave an estimate of 1.34 bits per
>>> letter. This
>>> implies that, if the letters are coded into 5 bits, one needs to
>>> appropriately
>>> combine 4 text files in order to obtain bit sequences of full
>>> entropy, since
>>> 4*1.34 = 5.36 > 5. The method used in our software is to sum (mod 32)
>>> the coded
>>> values of a-z (mapped to 0-25) as 5 bits of the corresponding letters of
>>> the
>>> text files.
>>>
>>> There are plenty of other schemes for obtaining high quality
>>> pseudo-random
>>> sequences in practice, e.g. AES in counter mode. However our scheme
>>> seems to
>>> be much simpler both in the underlying logic (understandability) and in
>>> implementation and is thus a viable alternative that one could use/need
>>> under
>>> circumstances.
>>>
>>> The software, TEXTCOMBINE-SP, is available at http://mok-kong-shen.de
>>>
>>> M. K. Shen
>>> -------------------------------------------------------------------------------
>>>
>>>
>>> [1] T. M. Cover, R. C. King, A Convergent Gambling Estimate of the
>>> Entropy of
>>> English, IEEE Trans. Inf. Theory, vol. 24, 1978, pp. 413-421.
>

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

Am 12.07.2017 um 18:44 schrieb Karl.Frank:
> On 12.07.17 17:37, Mok-Kong Shen wrote:
>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>> You are repeating yourself. Do you think that if you say it three times
>>> (as with the Bellman) it will suddenly become worthwhile?
>>> As I have said, this is a horrible scheme. text has many long range
>>> correlations (from charater pairs to paragraphs, etc) , which would
>>> mess up the random stream. (make it non-random).
>>> Bad idea.
>>> And you might want to look at what Shannon and others actually did.
>>>
>>> Note that even if the letters really were completely random, your method
>>> of combining them would make the output non-random.
>>
>> See the reference I gave of the paper about entropy and the test
>> statistic of Maurer's test.
>>
> Did it ever occur to you that verifying randomness only by the Maurer
> test is not sufficient?
>
> What about ENT, test for bias or, much more important, Pierre L'Ecuyer's
> TestU01 suite?
>
> In the past I have already demonstrated that one of your PRNG's, namely
> PERMPOLYPRNG which passed the Maurer test, is massively flawed.

The unfortunate situation with PRN generation in general is that there
are lots of different tests. I don't have expertise in such and use just
one test for the sake of convenience. Further, my targeted users, the
common people who need security protection of their personal
communications, have only very limited volumes, so cryptanalytical risks
associated with assumptions of large volumes of encrypted materials are
not intimidating for them IMHO.

M. K. Shen

On 12.07.17 22:44, Mok-Kong Shen wrote:
> Am 12.07.2017 um 18:44 schrieb Karl.Frank:
>> On 12.07.17 17:37, Mok-Kong Shen wrote:
>>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>>> You are repeating yourself. Do you think that if you say it three times
>>>> (as with the Bellman) it will suddenly become worthwhile?
>>>> As I have said, this is a horrible scheme. text has many long range
>>>> correlations (from charater pairs to paragraphs, etc) , which would
>>>> mess up the random stream. (make it non-random).
>>>> Bad idea.
>>>> And you might want to look at what Shannon and others actually did.
>>>>
>>>> Note that even if the letters really were completely random, your
>>>> method
>>>> of combining them would make the output non-random.
>>>
>>> See the reference I gave of the paper about entropy and the test
>>> statistic of Maurer's test.
>>>
>> Did it ever occur to you that verifying randomness only by the Maurer
>> test is not sufficient?
>>
>> What about ENT, test for bias or, much more important, Pierre L'Ecuyer's
>> TestU01 suite?
>>
>> In the past I have already demonstrated that one of your PRNG's, namely
>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
>
> The unfortunate situation with PRN generation in general is that there
> are lots of different tests. I don't have expertise in such and use just
> one test for the sake of convenience. Further, my targeted users, the
> common people who need security protection of their personal
> communications, have only very limited volumes, so cryptanalytical risks
> associated with assumptions of large volumes of encrypted materials are
> not intimidating for them IMHO.
>
> M. K. Shen

It is not an unfortunate situation. In contrast these test suites are of
great help in revealing weak or even totally useless PRNG's for the
common people - let alone in terms of cryptography.

So now you're promoting another cryptographic scheme for the obfuscation
of your little sisters diary? - as Bruce Schneier called it once.

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

Am 13.07.2017 um 02:15 schrieb Karl.Frank:
> On 12.07.17 22:44, Mok-Kong Shen wrote:
>> Am 12.07.2017 um 18:44 schrieb Karl.Frank:
>>> On 12.07.17 17:37, Mok-Kong Shen wrote:
>>>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>>>> You are repeating yourself. Do you think that if you say it three
>>>>> times
>>>>> (as with the Bellman) it will suddenly become worthwhile?
>>>>> As I have said, this is a horrible scheme. text has many long range
>>>>> correlations (from charater pairs to paragraphs, etc) , which would
>>>>> mess up the random stream. (make it non-random).
>>>>> Bad idea.
>>>>> And you might want to look at what Shannon and others actually did.
>>>>>
>>>>> Note that even if the letters really were completely random, your
>>>>> method
>>>>> of combining them would make the output non-random.
>>>>
>>>> See the reference I gave of the paper about entropy and the test
>>>> statistic of Maurer's test.
>>>>
>>> Did it ever occur to you that verifying randomness only by the Maurer
>>> test is not sufficient?
>>>
>>> What about ENT, test for bias or, much more important, Pierre L'Ecuyer's
>>> TestU01 suite?
>>>
>>> In the past I have already demonstrated that one of your PRNG's, namely
>>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
>>
>> The unfortunate situation with PRN generation in general is that there
>> are lots of different tests. I don't have expertise in such and use just
>> one test for the sake of convenience. Further, my targeted users, the
>> common people who need security protection of their personal
>> communications, have only very limited volumes, so cryptanalytical risks
>> associated with assumptions of large volumes of encrypted materials are
>> not intimidating for them IMHO.
>>
>> M. K. Shen
>
> It is not an unfortunate situation. In contrast these test suites are of
> great help in revealing weak or even totally useless PRNG's for the
> common people - let alone in terms of cryptography.
>
> So now you're promoting another cryptographic scheme for the obfuscation
> of your little sisters diary? - as Bruce Schneier called it once.

If the volume of materials available for analyze is small, then
exploiting tiny biases would be more difficult, isn't it?

Views could indeed be entirely different. Schneier demanded also that
one who is interested in crypto should first be proficient in analyzing
the diverse classical schemes. If everyone follows that advice, I am
quite sure that a non-trivial percentage of persons currently in the
crypto groups would have been absent because they haven't yet been able
to finish the work that they are required to do.

M. K. Shen

On 13.07.17 08:36, Mok-Kong Shen wrote:
> Am 13.07.2017 um 02:15 schrieb Karl.Frank:
>> On 12.07.17 22:44, Mok-Kong Shen wrote:
>>> Am 12.07.2017 um 18:44 schrieb Karl.Frank:
>>>> On 12.07.17 17:37, Mok-Kong Shen wrote:
>>>>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>>>>> You are repeating yourself. Do you think that if you say it three
>>>>>> times
>>>>>> (as with the Bellman) it will suddenly become worthwhile?
>>>>>> As I have said, this is a horrible scheme. text has many long range
>>>>>> correlations (from charater pairs to paragraphs, etc) , which would
>>>>>> mess up the random stream. (make it non-random).
>>>>>> Bad idea.
>>>>>> And you might want to look at what Shannon and others actually did.
>>>>>>
>>>>>> Note that even if the letters really were completely random, your
>>>>>> method
>>>>>> of combining them would make the output non-random.
>>>>>
>>>>> See the reference I gave of the paper about entropy and the test
>>>>> statistic of Maurer's test.
>>>>>
>>>> Did it ever occur to you that verifying randomness only by the Maurer
>>>> test is not sufficient?
>>>>
>>>> What about ENT, test for bias or, much more important, Pierre
>>>> L'Ecuyer's
>>>> TestU01 suite?
>>>>
>>>> In the past I have already demonstrated that one of your PRNG's, namely
>>>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
>>>
>>> The unfortunate situation with PRN generation in general is that there
>>> are lots of different tests. I don't have expertise in such and use just
>>> one test for the sake of convenience. Further, my targeted users, the
>>> common people who need security protection of their personal
>>> communications, have only very limited volumes, so cryptanalytical risks
>>> associated with assumptions of large volumes of encrypted materials are
>>> not intimidating for them IMHO.
>>>
>>> M. K. Shen
>>
>> It is not an unfortunate situation. In contrast these test suites are of
>> great help in revealing weak or even totally useless PRNG's for the
>> common people - let alone in terms of cryptography.
>>
>> So now you're promoting another cryptographic scheme for the obfuscation
>> of your little sisters diary? - as Bruce Schneier called it once.
>
> If the volume of materials available for analyze is small, then
> exploiting tiny biases would be more difficult, isn't it?
>
Depending on how heavy the bias of the PRNG in question is. You might
notice that the most recent break of RC4 was basically managed with very
tiny ciphertexts.

> Views could indeed be entirely different. Schneier demanded also that
> one who is interested in crypto should first be proficient in analyzing
> the diverse classical schemes. If everyone follows that advice, I am
> quite sure that a non-trivial percentage of persons currently in the
> crypto groups would have been absent because they haven't yet been able
> to finish the work that they are required to do.
>
This would hold mostly true for you then.

> M. K. Shen
>

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

Am 13.07.2017 um 11:39 schrieb Karl.Frank:
> On 13.07.17 08:36, Mok-Kong Shen wrote:
>> Am 13.07.2017 um 02:15 schrieb Karl.Frank:
>>> On 12.07.17 22:44, Mok-Kong Shen wrote:
>>>> Am 12.07.2017 um 18:44 schrieb Karl.Frank:
>>>>> On 12.07.17 17:37, Mok-Kong Shen wrote:
>>>>>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>>>>>> You are repeating yourself. Do you think that if you say it three
>>>>>>> times
>>>>>>> (as with the Bellman) it will suddenly become worthwhile?
>>>>>>> As I have said, this is a horrible scheme. text has many long range
>>>>>>> correlations (from charater pairs to paragraphs, etc) , which would
>>>>>>> mess up the random stream. (make it non-random).
>>>>>>> Bad idea.
>>>>>>> And you might want to look at what Shannon and others actually did.
>>>>>>>
>>>>>>> Note that even if the letters really were completely random, your
>>>>>>> method
>>>>>>> of combining them would make the output non-random.
>>>>>>
>>>>>> See the reference I gave of the paper about entropy and the test
>>>>>> statistic of Maurer's test.
>>>>>>
>>>>> Did it ever occur to you that verifying randomness only by the Maurer
>>>>> test is not sufficient?
>>>>>
>>>>> What about ENT, test for bias or, much more important, Pierre
>>>>> L'Ecuyer's
>>>>> TestU01 suite?
>>>>>
>>>>> In the past I have already demonstrated that one of your PRNG's,
>>>>> namely
>>>>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
>>>>
>>>> The unfortunate situation with PRN generation in general is that there
>>>> are lots of different tests. I don't have expertise in such and use
>>>> just
>>>> one test for the sake of convenience. Further, my targeted users, the
>>>> common people who need security protection of their personal
>>>> communications, have only very limited volumes, so cryptanalytical
>>>> risks
>>>> associated with assumptions of large volumes of encrypted materials are
>>>> not intimidating for them IMHO.
>>>>
>>>> M. K. Shen
>>>
>>> It is not an unfortunate situation. In contrast these test suites are of
>>> great help in revealing weak or even totally useless PRNG's for the
>>> common people - let alone in terms of cryptography.
>>>
>>> So now you're promoting another cryptographic scheme for the obfuscation
>>> of your little sisters diary? - as Bruce Schneier called it once.
>>
>> If the volume of materials available for analyze is small, then
>> exploiting tiny biases would be more difficult, isn't it?
>>
> Depending on how heavy the bias of the PRNG in question is. You might
> notice that the most recent break of RC4 was basically managed with very
> tiny ciphertexts.

I don't yet know that recent break. How tiny? Could you give a
reference?

BTW, you wrote earlier:
"In the past I have already demonstrated that one of your PRNG's,
namely PERMPOLYPRNG which passed the Maurer test, is massively flawed."
The latest version of that software is 3.1, so at least its first
version was fairly unsatisfactory even to myself. However, all the
revisions were based on thoughts of myself, not of any other person. It
can certainly not be excluded that even the latest version may indeed
be "massively flawed". But where is your "demonstration"?? I just
checked and found that the thread in the group where PERMPOLYPRNG is
published is exceptionally short and all posts in it were from me and
not from any other person.

M. K. Shen
>
>
>> Views could indeed be entirely different. Schneier demanded also that
>> one who is interested in crypto should first be proficient in analyzing
>> the diverse classical schemes. If everyone follows that advice, I am
>> quite sure that a non-trivial percentage of persons currently in the
>> crypto groups would have been absent because they haven't yet been able
>> to finish the work that they are required to do.
>>
> This would hold mostly true for you then.
>
>
>> M. K. Shen
>>
>
>

On 13.07.17 13:29, Mok-Kong Shen wrote:
> Am 13.07.2017 um 11:39 schrieb Karl.Frank:
>> On 13.07.17 08:36, Mok-Kong Shen wrote:
>>> Am 13.07.2017 um 02:15 schrieb Karl.Frank:
>>>> On 12.07.17 22:44, Mok-Kong Shen wrote:
>>>>> Am 12.07.2017 um 18:44 schrieb Karl.Frank:
>>>>>> On 12.07.17 17:37, Mok-Kong Shen wrote:
>>>>>>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>>>>>>> You are repeating yourself. Do you think that if you say it three
>>>>>>>> times
>>>>>>>> (as with the Bellman) it will suddenly become worthwhile?
>>>>>>>> As I have said, this is a horrible scheme. text has many long range
>>>>>>>> correlations (from charater pairs to paragraphs, etc) , which would
>>>>>>>> mess up the random stream. (make it non-random).
>>>>>>>> Bad idea.
>>>>>>>> And you might want to look at what Shannon and others actually did.
>>>>>>>>
>>>>>>>> Note that even if the letters really were completely random, your
>>>>>>>> method
>>>>>>>> of combining them would make the output non-random.
>>>>>>>
>>>>>>> See the reference I gave of the paper about entropy and the test
>>>>>>> statistic of Maurer's test.
>>>>>>>
>>>>>> Did it ever occur to you that verifying randomness only by the Maurer
>>>>>> test is not sufficient?
>>>>>>
>>>>>> What about ENT, test for bias or, much more important, Pierre
>>>>>> L'Ecuyer's
>>>>>> TestU01 suite?
>>>>>>
>>>>>> In the past I have already demonstrated that one of your PRNG's,
>>>>>> namely
>>>>>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
>>>>>
>>>>> The unfortunate situation with PRN generation in general is that there
>>>>> are lots of different tests. I don't have expertise in such and use
>>>>> just
>>>>> one test for the sake of convenience. Further, my targeted users, the
>>>>> common people who need security protection of their personal
>>>>> communications, have only very limited volumes, so cryptanalytical
>>>>> risks
>>>>> associated with assumptions of large volumes of encrypted materials
>>>>> are
>>>>> not intimidating for them IMHO.
>>>>>
>>>>> M. K. Shen
>>>>
>>>> It is not an unfortunate situation. In contrast these test suites
>>>> are of
>>>> great help in revealing weak or even totally useless PRNG's for the
>>>> common people - let alone in terms of cryptography.
>>>>
>>>> So now you're promoting another cryptographic scheme for the
>>>> obfuscation
>>>> of your little sisters diary? - as Bruce Schneier called it once.
>>>
>>> If the volume of materials available for analyze is small, then
>>> exploiting tiny biases would be more difficult, isn't it?
>>>
>> Depending on how heavy the bias of the PRNG in question is. You might
>> notice that the most recent break of RC4 was basically managed with very
>> tiny ciphertexts.
>
> I don't yet know that recent break. How tiny? Could you give a
> reference?
>
Just tiny short 16-character cookies.

http://www.rc4nomore.com/

http://www.youtube.com/watch?v=d8MtmKrXlKQ

http://www.rc4nomore.com/vanhoef-usenix2015.pdf

> BTW, you wrote earlier:
> "In the past I have already demonstrated that one of your PRNG's,
> namely PERMPOLYPRNG which passed the Maurer test, is massively flawed."
> The latest version of that software is 3.1, so at least its first
> version was fairly unsatisfactory even to myself. However, all the
> revisions were based on thoughts of myself, not of any other person. It
> can certainly not be excluded that even the latest version may indeed
> be "massively flawed". But where is your "demonstration"?? I just
> checked and found that the thread in the group where PERMPOLYPRNG is
> published is exceptionally short and all posts in it were from me and
> not from any other person.
>
Over here are some visual results displaying the massive bias as well as
the source code and test tools used
http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/

Especially these two images are mostly interesting in regards of
displaying the bias

http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_1MB.bin_rnd.jpg

http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_4MB.bin_rnd.jpg

The original posting is over here
http://s13.zetaboards.com/Crypto/single/?p=8045140&t=7392575

> M. K. Shen
>>
>>
>>> Views could indeed be entirely different. Schneier demanded also that
>>> one who is interested in crypto should first be proficient in analyzing
>>> the diverse classical schemes. If everyone follows that advice, I am
>>> quite sure that a non-trivial percentage of persons currently in the
>>> crypto groups would have been absent because they haven't yet been able
>>> to finish the work that they are required to do.
>>>
>> This would hold mostly true for you then.
>>
>>
>>> M. K. Shen
>>>
>>
>>
>

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

Am 13.07.2017 um 14:08 schrieb Karl.Frank:
> On 13.07.17 13:29, Mok-Kong Shen wrote:
>> Am 13.07.2017 um 11:39 schrieb Karl.Frank:
>>> On 13.07.17 08:36, Mok-Kong Shen wrote:
>>>> Am 13.07.2017 um 02:15 schrieb Karl.Frank:
>>>>> On 12.07.17 22:44, Mok-Kong Shen wrote:
>>>>>> Am 12.07.2017 um 18:44 schrieb Karl.Frank:
>>>>>>> On 12.07.17 17:37, Mok-Kong Shen wrote:
>>>>>>>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>>>>>>>> You are repeating yourself. Do you think that if you say it three
>>>>>>>>> times
>>>>>>>>> (as with the Bellman) it will suddenly become worthwhile?
>>>>>>>>> As I have said, this is a horrible scheme. text has many long
>>>>>>>>> range
>>>>>>>>> correlations (from charater pairs to paragraphs, etc) , which
>>>>>>>>> would
>>>>>>>>> mess up the random stream. (make it non-random).
>>>>>>>>> Bad idea.
>>>>>>>>> And you might want to look at what Shannon and others actually
>>>>>>>>> did.
>>>>>>>>>
>>>>>>>>> Note that even if the letters really were completely random, your
>>>>>>>>> method
>>>>>>>>> of combining them would make the output non-random.
>>>>>>>>
>>>>>>>> See the reference I gave of the paper about entropy and the test
>>>>>>>> statistic of Maurer's test.
>>>>>>>>
>>>>>>> Did it ever occur to you that verifying randomness only by the
>>>>>>> Maurer
>>>>>>> test is not sufficient?
>>>>>>>
>>>>>>> What about ENT, test for bias or, much more important, Pierre
>>>>>>> L'Ecuyer's
>>>>>>> TestU01 suite?
>>>>>>>
>>>>>>> In the past I have already demonstrated that one of your PRNG's,
>>>>>>> namely
>>>>>>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
>>>>>>
>>>>>> The unfortunate situation with PRN generation in general is that
>>>>>> there
>>>>>> are lots of different tests. I don't have expertise in such and use
>>>>>> just
>>>>>> one test for the sake of convenience. Further, my targeted users, the
>>>>>> common people who need security protection of their personal
>>>>>> communications, have only very limited volumes, so cryptanalytical
>>>>>> risks
>>>>>> associated with assumptions of large volumes of encrypted materials
>>>>>> are
>>>>>> not intimidating for them IMHO.
>>>>>>
>>>>>> M. K. Shen
>>>>>
>>>>> It is not an unfortunate situation. In contrast these test suites
>>>>> are of
>>>>> great help in revealing weak or even totally useless PRNG's for the
>>>>> common people - let alone in terms of cryptography.
>>>>>
>>>>> So now you're promoting another cryptographic scheme for the
>>>>> obfuscation
>>>>> of your little sisters diary? - as Bruce Schneier called it once.
>>>>
>>>> If the volume of materials available for analyze is small, then
>>>> exploiting tiny biases would be more difficult, isn't it?
>>>>
>>> Depending on how heavy the bias of the PRNG in question is. You might
>>> notice that the most recent break of RC4 was basically managed with very
>>> tiny ciphertexts.
>>
>> I don't yet know that recent break. How tiny? Could you give a
>> reference?
>>
> Just tiny short 16-character cookies.
>
> http://www.rc4nomore.com/
>
> http://www.youtube.com/watch?v=d8MtmKrXlKQ
>
> http://www.rc4nomore.com/vanhoef-usenix2015.pdf
>
>
>> BTW, you wrote earlier:
>> "In the past I have already demonstrated that one of your PRNG's,
>> namely PERMPOLYPRNG which passed the Maurer test, is massively flawed."
>> The latest version of that software is 3.1, so at least its first
>> version was fairly unsatisfactory even to myself. However, all the
>> revisions were based on thoughts of myself, not of any other person. It
>> can certainly not be excluded that even the latest version may indeed
>> be "massively flawed". But where is your "demonstration"?? I just
>> checked and found that the thread in the group where PERMPOLYPRNG is
>> published is exceptionally short and all posts in it were from me and
>> not from any other person.
>>
> Over here are some visual results displaying the massive bias as well as
> the source code and test tools used
> http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/
>
> Especially these two images are mostly interesting in regards of
> displaying the bias
>
> http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_1MB.bin_rnd.jpg
>
>
> http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_4MB.bin_rnd.jpg
>
>
>
>
> The original posting is over here
> http://s13.zetaboards.com/Crypto/single/?p=8045140&t=7392575

Ok, I forgot that history, as it laid quite a time back. From the date
of that post of yours, you were testing version 1.0 not the later
versions.

I dont have the opinion that any crypto scheme be justified only by its
passing a single statistical test. When some other plausible reasoning
are available in the positive direction, then such a test gives in my
view substantial support for its goodness. Formal proof of security
would be ideal, but in practice that's a difficult to attain goal.
I can't remember/know now how much work I had spent to check version
1.0 with Maurer's test and whether I might have done mistakes there and
so any bad behavior of Version 1.0 should not be interpreted to be
non-sensitiveness of Maurer's test. In fact, in the current case of
TESTCOMBINE-SP, certain arguments seemed to indicate that its resulting
sequences would be fairly biased though Maurer's test came out always to
be ok and I started to doubt the sensitivity of Maurer's test. But, if
my later computations are correct, this can be explained by the fact
that the underlying factors of the said arguments turned out not to
be strong enough in their influence in practice and hence Maurer's test
can't be blamed in that context.

M. K. Shen
>
>
>
>> M. K. Shen
>>>
>>>
>>>> Views could indeed be entirely different. Schneier demanded also that
>>>> one who is interested in crypto should first be proficient in analyzing
>>>> the diverse classical schemes. If everyone follows that advice, I am
>>>> quite sure that a non-trivial percentage of persons currently in the
>>>> crypto groups would have been absent because they haven't yet been able
>>>> to finish the work that they are required to do.
>>>>
>>> This would hold mostly true for you then.
>>>
>>>
>>>> M. K. Shen
>>>>
>>>
>>>
>>
>
>

Concerning the reference you gave of the break of RC-4, I read there
"Our attack is not limited to decrypting cookies. Any data or
information that is repeatedly encrypted can be recovered".

Do you know how much is this the special fault of RC-4? I mean, would
other PRNGs also be liable to that attack just as effectively?

M. K. Shen

On 2017-07-13, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
> Concerning the reference you gave of the break of RC-4, I read there
> "Our attack is not limited to decrypting cookies. Any data or
> information that is repeatedly encrypted can be recovered".
>
> Do you know how much is this the special fault of RC-4? I mean, would
> other PRNGs also be liable to that attack just as effectively?

It is in that case a fault of RC4. It had long been known that RC4 had
biases, especially in the first bytes that came out of RC4. These
attacks show that those biases are
useable in an attack. RC4 is broken.
Note that your scheme here seems to be using 4 text files as a "key" Ie,
to encrypt 1000 plaintexts you need to find 4000 different text fiels to
use, and you have to communicate to the recipient what those files are
without also telling the attacker what those files are. Plus as I said,
your ouput will have loads of biases in it simply because of hthe strong
long range correlations in any text file.

> M. K. Shen
>

On 13.07.17 16:04, Mok-Kong Shen wrote:
> Am 13.07.2017 um 14:08 schrieb Karl.Frank:
>> On 13.07.17 13:29, Mok-Kong Shen wrote:
>>> Am 13.07.2017 um 11:39 schrieb Karl.Frank:
>>>> On 13.07.17 08:36, Mok-Kong Shen wrote:
>>>>> Am 13.07.2017 um 02:15 schrieb Karl.Frank:
>>>>>> On 12.07.17 22:44, Mok-Kong Shen wrote:
>>>>>>> Am 12.07.2017 um 18:44 schrieb Karl.Frank:
>>>>>>>> On 12.07.17 17:37, Mok-Kong Shen wrote:
>>>>>>>>> Am 11.07.2017 um 22:01 schrieb William Unruh:
>>>>>>>>>> You are repeating yourself. Do you think that if you say it three
>>>>>>>>>> times
>>>>>>>>>> (as with the Bellman) it will suddenly become worthwhile?
>>>>>>>>>> As I have said, this is a horrible scheme. text has many long
>>>>>>>>>> range
>>>>>>>>>> correlations (from charater pairs to paragraphs, etc) , which
>>>>>>>>>> would
>>>>>>>>>> mess up the random stream. (make it non-random).
>>>>>>>>>> Bad idea.
>>>>>>>>>> And you might want to look at what Shannon and others actually
>>>>>>>>>> did.
>>>>>>>>>>
>>>>>>>>>> Note that even if the letters really were completely random, your
>>>>>>>>>> method
>>>>>>>>>> of combining them would make the output non-random.
>>>>>>>>>
>>>>>>>>> See the reference I gave of the paper about entropy and the test
>>>>>>>>> statistic of Maurer's test.
>>>>>>>>>
>>>>>>>> Did it ever occur to you that verifying randomness only by the
>>>>>>>> Maurer
>>>>>>>> test is not sufficient?
>>>>>>>>
>>>>>>>> What about ENT, test for bias or, much more important, Pierre
>>>>>>>> L'Ecuyer's
>>>>>>>> TestU01 suite?
>>>>>>>>
>>>>>>>> In the past I have already demonstrated that one of your PRNG's,
>>>>>>>> namely
>>>>>>>> PERMPOLYPRNG which passed the Maurer test, is massively flawed.
>>>>>>>
>>>>>>> The unfortunate situation with PRN generation in general is that
>>>>>>> there
>>>>>>> are lots of different tests. I don't have expertise in such and use
>>>>>>> just
>>>>>>> one test for the sake of convenience. Further, my targeted users,
>>>>>>> the
>>>>>>> common people who need security protection of their personal
>>>>>>> communications, have only very limited volumes, so cryptanalytical
>>>>>>> risks
>>>>>>> associated with assumptions of large volumes of encrypted materials
>>>>>>> are
>>>>>>> not intimidating for them IMHO.
>>>>>>>
>>>>>>> M. K. Shen
>>>>>>
>>>>>> It is not an unfortunate situation. In contrast these test suites
>>>>>> are of
>>>>>> great help in revealing weak or even totally useless PRNG's for the
>>>>>> common people - let alone in terms of cryptography.
>>>>>>
>>>>>> So now you're promoting another cryptographic scheme for the
>>>>>> obfuscation
>>>>>> of your little sisters diary? - as Bruce Schneier called it once.
>>>>>
>>>>> If the volume of materials available for analyze is small, then
>>>>> exploiting tiny biases would be more difficult, isn't it?
>>>>>
>>>> Depending on how heavy the bias of the PRNG in question is. You might
>>>> notice that the most recent break of RC4 was basically managed with
>>>> very
>>>> tiny ciphertexts.
>>>
>>> I don't yet know that recent break. How tiny? Could you give a
>>> reference?
>>>
>> Just tiny short 16-character cookies.
>>
>> http://www.rc4nomore.com/
>>
>> http://www.youtube.com/watch?v=d8MtmKrXlKQ
>>
>> http://www.rc4nomore.com/vanhoef-usenix2015.pdf
>>
>>
>>> BTW, you wrote earlier:
>>> "In the past I have already demonstrated that one of your PRNG's,
>>> namely PERMPOLYPRNG which passed the Maurer test, is massively flawed."
>>> The latest version of that software is 3.1, so at least its first
>>> version was fairly unsatisfactory even to myself. However, all the
>>> revisions were based on thoughts of myself, not of any other person. It
>>> can certainly not be excluded that even the latest version may indeed
>>> be "massively flawed". But where is your "demonstration"?? I just
>>> checked and found that the thread in the group where PERMPOLYPRNG is
>>> published is exceptionally short and all posts in it were from me and
>>> not from any other person.
>>>
>> Over here are some visual results displaying the massive bias as well as
>> the source code and test tools used
>> http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/
>>
>> Especially these two images are mostly interesting in regards of
>> displaying the bias
>>
>> http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_1MB.bin_rnd.jpg
>>
>>
>> http://www.freecx.co.uk/crypto/cryptanalysis/PERMPOLYPRNG/permpolyprng_4MB.bin_rnd.jpg
>>
>>
>>
>>
>> The original posting is over here
>> http://s13.zetaboards.com/Crypto/single/?p=8045140&t=7392575
>
> Ok, I forgot that history, as it laid quite a time back. From the date
> of that post of yours, you were testing version 1.0 not the later
> versions.
>
> I dont have the opinion that any crypto scheme be justified only by its
> passing a single statistical test. When some other plausible reasoning
> are available in the positive direction, then such a test gives in my
> view substantial support for its goodness. Formal proof of security
> would be ideal, but in practice that's a difficult to attain goal.
> I can't remember/know now how much work I had spent to check version
> 1.0 with Maurer's test and whether I might have done mistakes there and
> so any bad behavior of Version 1.0 should not be interpreted to be
> non-sensitiveness of Maurer's test. In fact, in the current case of
> TESTCOMBINE-SP, certain arguments seemed to indicate that its resulting
> sequences would be fairly biased though Maurer's test came out always to
> be ok and I started to doubt the sensitivity of Maurer's test. But, if
> my later computations are correct, this can be explained by the fact
> that the underlying factors of the said arguments turned out not to
> be strong enough in their influence in practice and hence Maurer's test
> can't be blamed in that context.
>
Any PRNG used for cryptographic purpose *HAS* to *pass* *all* of these
very harsh and intense tests for randomness quality, especially TestU01
"crush" and "big crush". This is the first ever measurement that a PRNG
designer has to take very seriously. If a proposed CSPRNG does not pass
these test it has to be dropped or re-designed, because a *failure* of
these test *indicate* a *non-random* *output* no matter what one
believes are the plausible reasons why it would still be "sufficiently
random" for cryptographic purposes.

If you would read the mentioned thread from the beginning you will
realised that my intention on starting it was my critique that the
Maurer test is seemingly not reliable, as even the keystream output of
one of the most miserably designed cipher algorithms, namely the
Crystalline cipher, passes the Maurer test. This indicates in my view
the importance not to rely solely on this result but always run the
whole bunch of available test tools.

Just one example of the Crystalline output by the Maurer test
http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/maurer-test-result_10MB.txt

....and that's what the simple test for bias reveals
http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/bias-result_(8)_10MB.txt

Both test results are based on the same keystream.

Am 13.07.2017 um 17:40 schrieb William Unruh:
> On 2017-07-13, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
>> Concerning the reference you gave of the break of RC-4, I read there
>> "Our attack is not limited to decrypting cookies. Any data or
>> information that is repeatedly encrypted can be recovered".
>>
>> Do you know how much is this the special fault of RC-4? I mean, would
>> other PRNGs also be liable to that attack just as effectively?
>
> It is in that case a fault of RC4. It had long been known that RC4 had
> biases, especially in the first bytes that came out of RC4. These
> attacks show that those biases are
> useable in an attack. RC4 is broken.
> Note that your scheme here seems to be using 4 text files as a "key" Ie,
> to encrypt 1000 plaintexts you need to find 4000 different text fiels to
> use, and you have to communicate to the recipient what those files are
> without also telling the attacker what those files are. Plus as I said,
> your ouput will have loads of biases in it simply because of hthe strong
> long range correlations in any text file.

In fact I had also done a simple counting to see whether the least
significant bit of my particular encoding of input letters leads to
strong bias, for it seems that that bit could be sensitive. It turned
out that the ratio of counts of 0 to total counts varies for the
individual files are in the range of [0.52, 0.57}. When 2 files are
combined with xor this range is reduced. With 4 files the range is
reduced to almost exactly 0.5.

M. K. Shen

Am 13.07.2017 um 18:02 schrieb Karl.Frank:

> Any PRNG used for cryptographic purpose *HAS* to *pass* *all* of these
> very harsh and intense tests for randomness quality, especially TestU01
> "crush" and "big crush". This is the first ever measurement that a PRNG
> designer has to take very seriously. If a proposed CSPRNG does not pass
> these test it has to be dropped or re-designed, because a *failure* of
> these test *indicate* a *non-random* *output* no matter what one
> believes are the plausible reasons why it would still be "sufficiently
> random" for cryptographic purposes.
>
> If you would read the mentioned thread from the beginning you will
> realised that my intention on starting it was my critique that the
> Maurer test is seemingly not reliable, as even the keystream output of
> one of the most miserably designed cipher algorithms, namely the
> Crystalline cipher, passes the Maurer test. This indicates in my view
> the importance not to rely solely on this result but always run the
> whole bunch of available test tools.
>
> Just one example of the Crystalline output by the Maurer test
> http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/maurer-test-result_10MB.txt
>
>
> ...and that's what the simple test for bias reveals
> http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/bias-result_(8)_10MB.txt
>
>
> Both test results are based on the same keystream.
>
> Not sure if the latest version of your PERMPOLYPRNG or TESTCOMBINE-SP
> would pass these test.

I know naturally that, if one employs more different kinds of tests,
that's anyway better than less and that this is in fact true for all
field of science. Long time ago I thought of the NIST test suite.
However, my OS is Windows and I read on Internet that there were some
difficulties to have that test suite run on Windows and dropped the
idea. Which good alternatives in your experience run on Windows
straightforwardly?

M. K. Shen

On 13.07.17 20:10, Mok-Kong Shen wrote:
> Am 13.07.2017 um 18:02 schrieb Karl.Frank:
>
>> Any PRNG used for cryptographic purpose *HAS* to *pass* *all* of these
>> very harsh and intense tests for randomness quality, especially TestU01
>> "crush" and "big crush". This is the first ever measurement that a PRNG
>> designer has to take very seriously. If a proposed CSPRNG does not pass
>> these test it has to be dropped or re-designed, because a *failure* of
>> these test *indicate* a *non-random* *output* no matter what one
>> believes are the plausible reasons why it would still be "sufficiently
>> random" for cryptographic purposes.
>>
>> If you would read the mentioned thread from the beginning you will
>> realised that my intention on starting it was my critique that the
>> Maurer test is seemingly not reliable, as even the keystream output of
>> one of the most miserably designed cipher algorithms, namely the
>> Crystalline cipher, passes the Maurer test. This indicates in my view
>> the importance not to rely solely on this result but always run the
>> whole bunch of available test tools.
>>
>> Just one example of the Crystalline output by the Maurer test
>> http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/maurer-test-result_10MB.txt
>>
>>
>> ...and that's what the simple test for bias reveals
>> http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/bias-result_(8)_10MB.txt
>>
>>
>> Both test results are based on the same keystream.
>>
>> Not sure if the latest version of your PERMPOLYPRNG or TESTCOMBINE-SP
>> would pass these test.
>
> I know naturally that, if one employs more different kinds of tests,
> that's anyway better than less and that this is in fact true for all
> field of science. Long time ago I thought of the NIST test suite.
> However, my OS is Windows and I read on Internet that there were some
> difficulties to have that test suite run on Windows and dropped the
> idea. Which good alternatives in your experience run on Windows
> straightforwardly?
>
> M. K. Shen
>
I am not aware of any randomness test tool running straightforwardly on
Windows. However on the TestU01 website you'll find a description of
what the requirements are and how to install the test battery

http://simul.iro.umontreal.ca/testu01/install.html

In my opinion the installation of cygwin is mostly recommended as
it enables the Windows user the execution of all those different test
tools for *NIX/BSD

http://www.cygwin.com/

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

On 13.07.17 19:33, Mok-Kong Shen wrote:
> Am 13.07.2017 um 17:40 schrieb William Unruh:
>> On 2017-07-13, Mok-Kong Shen <mok-kong.shen@t-online.de> wrote:
>>> Concerning the reference you gave of the break of RC-4, I read there
>>> "Our attack is not limited to decrypting cookies. Any data or
>>> information that is repeatedly encrypted can be recovered".
>>>
>>> Do you know how much is this the special fault of RC-4? I mean, would
>>> other PRNGs also be liable to that attack just as effectively?
>>
>> It is in that case a fault of RC4. It had long been known that RC4 had
>> biases, especially in the first bytes that came out of RC4. These
>> attacks show that those biases are
>> useable in an attack. RC4 is broken.
>> Note that your scheme here seems to be using 4 text files as a "key" Ie,
>> to encrypt 1000 plaintexts you need to find 4000 different text fiels to
>> use, and you have to communicate to the recipient what those files are
>> without also telling the attacker what those files are. Plus as I said,
>> your ouput will have loads of biases in it simply because of hthe strong
>> long range correlations in any text file.
>
> In fact I had also done a simple counting to see whether the least
> significant bit of my particular encoding of input letters leads to
> strong bias, for it seems that that bit could be sensitive. It turned
> out that the ratio of counts of 0 to total counts varies for the
> individual files are in the range of [0.52, 0.57}. When 2 files are
> combined with xor this range is reduced. With 4 files the range is
> reduced to almost exactly 0.5.
>
> M. K. Shen
>
Just counting the appearance of zeros and ones is not sufficient.
Important is how they are spread over the whole output. As an example
you might be interested in this particular explanation found on WikiPedia

Quote:
-------------------------------------------------------------------
These practical tests make it possible to compare the randomness of
strings. On probabilistic grounds, all strings of a given length have
the same randomness. However different strings have a different
Kolmogorov complexity. For example, consider the following two strings.

String 1:
0101010101010101010101010101010101010101010101010101010101010101

String 2:
1100100001100001110111101110110011111010010000100101011110010110

String 1 admits a short linguistic description, namely "32 repetitions
of '01'", which consists of 64 characters, and it can be efficiently
constructed out of some basis sequences. String 2 has no obvious simple
description other than writing down the string itself, which has 64
characters, and it has no comparably efficient basis function
representation. Using linear Hadamard spectral tests (see Hadamard
transform), the first of these sequences will be found to be of much
less randomness than the second one, which agrees with intuition.

https://en.wikipedia.org/wiki/Randomness_tests#Specific_tests_for_randomness
-------------------------------------------------------------------

As you can very easily determin String 1 is far from being random.

Of course there is the test for bit-wise statistics by Winston Rayburn
which a PRNG should obviously pass. But still it does only reveal if
there is some problematic deviance of the bit in a given file.

For instance if we look again on the bit-wise statistic result of the
before mentioned Crystalline cipher output we find a perfectly good
distribution of zeros and ones

http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/bitstat-result_10MB.txt

As you can see yourself, based on the Maurer test as well as on the bit
statistic it would lead us to the false conclusion that the output of
Crystalline is of perfect random quality.

However quick visual test soon reveals the disastrous output quality

http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/zero_4MB.bin.crystalline_rnd.jpg

http://www.freecx.co.uk/crypto/cryptanalysis/Crystalline/zero_10MB.bin.crystalline_rnd.jpg

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

Am 13.07.2017 um 21:08 schrieb Karl.Frank:

> Just counting the appearance of zeros and ones is not sufficient.
> Important is how they are spread over the whole output. As an example
> you might be interested in this particular explanation found on WikiPedia
[snip]

The correlation is to my knowledge commonly investigated with the
autocorrelation test. Maurer's test should in a sense be a superset
covering that test. To check that I'll code the autocorrelation test
and apply it to the result of my example and hope to be able to report
on that issue not later than tomorrow evening anyway.

M. K. Shen

On 13.07.17 21:55, Mok-Kong Shen wrote:
> Am 13.07.2017 um 21:08 schrieb Karl.Frank:
>
>> Just counting the appearance of zeros and ones is not sufficient.
>> Important is how they are spread over the whole output. As an example
>> you might be interested in this particular explanation found on WikiPedia
> [snip]
>
> The correlation is to my knowledge commonly investigated with the
> autocorrelation test. Maurer's test should in a sense be a superset
> covering that test. To check that I'll code the autocorrelation test
> and apply it to the result of my example and hope to be able to report
> on that issue not later than tomorrow evening anyway.
>
> M. K. Shen

Perhaps you can upload a 1MB, a 4MB and a 10BM file of the output to
your website and post the link so I can give it a quick check.

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

On 13.07.17 17:40, William Unruh wrote:
> On 2017-07-13, Mok-Kong Shen<mok-kong.shen@t-online.de> wrote:
>> Concerning the reference you gave of the break of RC-4, I read there
>> "Our attack is not limited to decrypting cookies. Any data or
>> information that is repeatedly encrypted can be recovered".
>>
>> Do you know how much is this the special fault of RC-4? I mean, would
>> other PRNGs also be liable to that attack just as effectively?
>
> It is in that case a fault of RC4. It had long been known that RC4 had
> biases, especially in the first bytes that came out of RC4. These
> attacks show that those biases are
> useable in an attack. RC4 is broken.
> Note that your scheme here seems to be using 4 text files as a "key" Ie,
> to encrypt 1000 plaintexts you need to find 4000 different text fiels to
> use, and you have to communicate to the recipient what those files are
> without also telling the attacker what those files are. Plus as I said,
> your ouput will have loads of biases in it simply because of hthe strong
> long range correlations in any text file.
>
>
This sounds like a generation of some sort of one-time-pad to me,
inheriting the key exchange problem as well as the necessity to keep
track of which files already used etc..., rendering it extremely useless
to the "common people" which it was originally designed for.

But regarding the bias I consider that it /might/ be possible to
generate a good pseudo-random encryption key if we would XOR four JPEG
files of nearly the same size and of which the header and footer is dropped.

Or even better, if we seed a regular PRNG and now cycle over these files
byte-wise, picking four byte based on four consecutive 8bit values drawn
from the PRNG on each step. Once we reached the end of the files we
start again at the top. Of course this implies a PRNG that has a very
long period and can be seeded with at least a 64bit seed. This way we
might share the four basic key files as innocent JPEG images.

>> M. K. Shen
>>

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

On 13.07.17 22:41, Karl.Frank wrote:
> On 13.07.17 17:40, William Unruh wrote:
>> On 2017-07-13, Mok-Kong Shen<mok-kong.shen@t-online.de> wrote:
>>> Concerning the reference you gave of the break of RC-4, I read there
>>> "Our attack is not limited to decrypting cookies. Any data or
>>> information that is repeatedly encrypted can be recovered".
>>>
>>> Do you know how much is this the special fault of RC-4? I mean, would
>>> other PRNGs also be liable to that attack just as effectively?
>>
>> It is in that case a fault of RC4. It had long been known that RC4 had
>> biases, especially in the first bytes that came out of RC4. These
>> attacks show that those biases are
>> useable in an attack. RC4 is broken.
>> Note that your scheme here seems to be using 4 text files as a "key" Ie,
>> to encrypt 1000 plaintexts you need to find 4000 different text fiels to
>> use, and you have to communicate to the recipient what those files are
>> without also telling the attacker what those files are. Plus as I said,
>> your ouput will have loads of biases in it simply because of hthe strong
>> long range correlations in any text file.
>>
>>
> This sounds like a generation of some sort of one-time-pad to me,
> inheriting the key exchange problem as well as the necessity to keep
> track of which files already used etc..., rendering it extremely useless
> to the "common people" which it was originally designed for.
>
> But regarding the bias I consider that it /might/ be possible to
> generate a good pseudo-random encryption key if we would XOR four JPEG
> files of nearly the same size and of which the header and footer is
> dropped.
>
> Or even better, if we seed a regular PRNG and now cycle over these files
> byte-wise, picking four byte based on four consecutive 8bit values drawn
> from the PRNG on each step. Once we reached the end of the files we
> start again at the top. Of course this implies a PRNG that has a very
> long period and can be seeded with at least a 64bit seed. This way we
> might share the four basic key files as innocent JPEG images.
>
>
Sounds intriguing, doesn't it?

But does anybody realised why the above mentioned construct is a very
bad idea?

Just imaging the four key files were images having a very similar
motive, .i.e lost of blue sky. If we XOR them all those similar parts
would clearly eliminating the byte values making them all zero. And this
would lead to the ciphertext being exactly the plaintext with no
encryption at all.

And now imagine what would happen when you XOR text files. In my opinion
there is a great chance that you run into the same problem having no
encryption at all.

Additionally the range of possible byte value when XORing text files
might be extremely limited.

>
>>> M. K. Shen
>>>
>
>

--
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=

I am extremely sorry to say that I was unfortunately misled by some
erroneous
computations in the design stage such that I like to retract this software
(instead of attempting certain more complicated redesign) and sincerely ask
for pardon from readers of this thread for having wasted their precious
time.

M. K. Shen