1. What is best way to set_rand?
- Posted by SnakeCharmer Jan 24, 2013
- 1272 views
I read documentation:
When set_rand() is called with a sequence of exactly two integers/atoms the internal seeds are set to the parameter values.
When set_rand() is called with an empty sequence, the internal seeds are set to random values and are unpredictable. This is how to reset the generator.
When set_rand() is called with any other sequence, the internal seeds are set based on the length of the sequence and the hashed value of the sequence.
I want to use two values as seeds for set_rand. How many bit these values have in width? What is best way to use hashes (of external data) as seeds?
atom Hash1 = hash(data1, ALGORITHM) atom Hash2 = hash(data2, ALGORITHM) set_rand({Hash1, Hash2})
It is correct? What ALGORITHM is better in this case? HSIEH30 or something 32-bit? 64-bit? More? When I put a line to set_rand, in what it turns? HSIEH32?
2. Re: What is best way to set_rand?
- Posted by mattlewis (admin) Jan 24, 2013
- 1292 views
I want to use two values as seeds for set_rand. How many bit these values have in width? What is best way to use hashes (of external data) as seeds?
atom Hash1 = hash(data1, ALGORITHM) atom Hash2 = hash(data2, ALGORITHM) set_rand({Hash1, Hash2})
It is correct? What ALGORITHM is better in this case? HSIEH30 or something 32-bit? 64-bit? More? When I put a line to set_rand, in what it turns? HSIEH32?
What are you trying to do? Normally, the only reason to set the random seed is to get consistent results in a simulation, or something like that.
I don't think it really matters how you go about picking a seed. The seed doesn't affect the "randomness" of the sequence of random numbers. It just gives you the ability to get a consistent sequence subject to the pseudo random algorithm that's used.
The seeds are all 32-bits, regardless of platform.
Matt
3. Re: What is best way to set_rand?
- Posted by DerekParnell (admin) Jan 24, 2013
- 1305 views
I want to use two values as seeds for set_rand. How many bit these values have in width? What is best way to use hashes (of external data) as seeds?
There is no best way. It all depends on what you are trying to achieve.
The purpose of set_rand() is ensure that the next set of random numbers generated by the library are able to be repeated. That is to say, you call set_rand() before a series of calls to generate random numbers, so that the set you get is always the same set.
If you actually want the next set of generated random numbers to be unpredictable, call set_rand() with an empty sequence. Every other way of calling set_rand() will make the next set of numbers predictable in the sense that the set will be able to be repeated. The parameters given to set_rand() merely dictate which set of random numbers will be generated next.
It doesn't matter that much what you use to define the next set of random numbers, so using any algorithm in hash() to create some seeds would be just as good as any other algorithm.
For example ...
include std/rand.e sequence S,T set_rand("test") S = {} for i = 1 to 10 do S &= rand(100) end for set_rand("test") T = {} for i = 1 to 10 do T &= rand(100) end for ? equal(S,T) --> 1 (Both sets are identical) set_rand("") S = {} for i = 1 to 10 do S &= rand(100) end for set_rand("") T = {} for i = 1 to 10 do T &= rand(100) end for ? equal(S,T) --> 0 (Both sets are different)
4. Re: What is best way to set_rand?
- Posted by SnakeCharmer Jan 24, 2013
- 1239 views
What are you trying to do? Normally, the only reason to set the random seed is to get consistent results in a simulation, or something like that.
I want to create the primitive utility for password generation by key-file (or key-picture) and key-phrase.
Hash1 = getCRC32(File) Hash2 = getCRC32(String) set_seed({Hash1, Hash2}) sequence Password = {} for i = 1 to PasswordLength do Password &= Alphabet[rand(AlphabetLength)] end for
I don't think it really matters how you go about picking a seed. The seed doesn't affect the "randomness" of the sequence of random numbers. It just gives you the ability to get a consistent sequence subject to the pseudo random algorithm that's used.
I knew all this still in those days when I had ZX Spectrum. Oh my God, what I old! By the way, I am a birthday man today.
The seeds are all 32-bits, regardless of platform.
Thus, the best way is sending two atoms with values from 0 to 2^32-1? I am surprised by impossibility to set seeds as sequence with four 16-bit integers.
5. Re: What is best way to set_rand?
- Posted by SnakeCharmer Jan 24, 2013
- 1232 views
I want to use two values as seeds for set_rand. How many bit these values have in width? What is best way to use hashes (of external data) as seeds?
There is no best way. It all depends on what you are trying to achieve.
Speed!
I assume that it is the only case when function doesn't make additional operations with input data.
6. Re: What is best way to set_rand?
- Posted by mattlewis (admin) Jan 25, 2013
- 1252 views
The seeds are all 32-bits, regardless of platform.
Thus, the best way is sending two atoms with values from 0 to 2^32-1? I am surprised by impossibility to set seeds as sequence with four 16-bit integers.
That's the explicit way to set the seeds. Internally, we use two 32-bit integers. If you really want to use 4 16-bit integers, then you'll need to put them together into 2 32-bit integers.
There is no best way. It all depends on what you are trying to achieve.
Speed!
I assume that it is the only case when function doesn't make additional operations with input data.
That's true. But I don't see speed being a factor in your algorithm. Calculating the CRCs (or whatever you eventually decide on) should dominate setting the random seeds. Here is the set_rand() function in the back end:
static object set_rand(object x) /* set random number generator */ { intptr_t r; s1_ptr x_ptr; int slen; object_ptr obp; if (!ASEQ(x)) { // Simple case - just a single value supplied. r = get_int(x); seed1 = r+1; seed2 = ~(r) + 999; } else { // We got a sequence given to us. x_ptr = SEQ_PTR(x); slen = x_ptr->length; if (slen == 0) { // Empty sequence means randomize the generator. setran(); } else { obp = x_ptr->base; // A sequence of two atoms explictly supplies seed1 and seed2 values. if ((slen == 2) && !ASEQ(obp[1]) && !ASEQ(obp[2])) { seed1 = get_int(obp[1]); seed2 = get_int(obp[2]); } else { // Complex case - an arbitary sequence supplied. seed1 = get_int(calc_hash(x, slen)); seed2 = get_int(calc_hash(slen, make_atom32( seed1 ) )); } } } rand_was_set = TRUE; return ATOM_1; }
It's pretty simple, really. The fastest might be to simply pass the image or whatever as a sequence and let euphoria calculate the hash. That's probably faster than a CRC or other hash function written in pure euphoria.
Matt
7. Re: What is best way to set_rand?
- Posted by CoJaBo2 Jan 25, 2013
- 1235 views
I want to create the primitive utility for password generation by key-file (or key-picture) and key-phrase.
Hash1 = getCRC32(File) Hash2 = getCRC32(String) set_seed({Hash1, Hash2}) sequence Password = {} for i = 1 to PasswordLength do Password &= Alphabet[rand(AlphabetLength)] end for
Passwords generated this way should be crackable in a few microseconds using modern techniques (the keyspace is small enough to generate rainbow-tables of all possible generated passwords).
The correct way would to use Bcrypt (or sha2/3 if you are less paranoid) for secure hashing. Eu does not appear to implement any of these. Unfortunately, this means Eu is probably a poor choice, currently, for writing any kind of cryptographic software.
Note that keyfiles often aren't as secure as they may seem-
- I've managed to recover one, using a rather weak machine, in just under a week, simply by trying all files on the system (starting with likely candidates such as Documents). The error here was keeping the keyfile on the same drive as the protected volume.
- Many file formats don't provide a lot of entropy; the majority of them may stay the same between different files in the same format.
- If the file is from the internet, shared, or otherwise not your own private data, chances are it is hashed in a database somewhere already.
You'd have generate the keyfile from a secure random source and keep it on a flash drive, in addition to keeping a passphrase elsewhere (e.g., memorized), and of course use them with software that is itself strong.
8. Re: What is best way to set_rand?
- Posted by jimcbrown (admin) Jan 25, 2013
- 1192 views
Unfortunately, this means Eu is probably a poor choice, currently, for writing any kind of cryptographic software.
Well, you could just read bytes out of /dev/random.
9. Re: What is best way to set_rand?
- Posted by CoJaBo2 Jan 25, 2013
- 1195 views
Unfortunately, this means Eu is probably a poor choice, currently, for writing any kind of cryptographic software.
Well, you could just read bytes out of /dev/random.
/dev/random isn't necessarily guaranteed to be cryptographically secure on all platforms (openssl, etc, generally implement their own), also its not available at all on Windows. It doesn't matter in this case, as the question is really about hashing, and Eu lacks any secure hashing algos..
10. Re: What is best way to set_rand?
- Posted by jimcbrown (admin) Jan 25, 2013
- 1214 views
Unfortunately, this means Eu is probably a poor choice, currently, for writing any kind of cryptographic software.
Well, you could just read bytes out of /dev/random.
/dev/random isn't necessarily guaranteed to be cryptographically secure on all platforms (openssl, etc, generally implement their own), also its not available at all on Windows. It doesn't matter in this case, as the question is really about hashing, and Eu lacks any secure hashing algos..
Well, you could write a wrapper and borrow one from openssl or gnutls....
11. Re: What is best way to set_rand?
- Posted by DerekParnell (admin) Jan 25, 2013
- 1270 views
I want to create the primitive utility for password generation by key-file (or key-picture) and key-phrase.
Hash1 = getCRC32(File) Hash2 = getCRC32(String) set_seed({Hash1, Hash2}) sequence Password = {} for i = 1 to PasswordLength do Password &= Alphabet[rand(AlphabetLength)] end for
Passwords generated this way should be crackable in a few microseconds using modern techniques (the keyspace is small enough to generate rainbow-tables of all possible generated passwords).
Wouldn't that depend upon the length of the password and the length of the Alphabet?
A limit of 20 character passwords from an Alphabet of 96 characters produces a very large number of potential passwords, around 4.5*10^39. So if our cracking software does a billion passwords per second, that would still take about 10^25 years on average to crack one password.
The correct way would to use Bcrypt (or sha2/3 if you are less paranoid) for secure hashing. Eu does not appear to implement any of these. Unfortunately, this means Eu is probably a poor choice, currently, for writing any kind of cryptographic software.
I don't see the connection between generating passwords and hashing? It is possible to generate passwords without using any hashing function at all.
The currently implemented hashing algorithms in Eu are not designed for cryptographic purposes, however a few of them are quite secure of most other purposes.
The design of the hash() built-in function allows for other, including modern, algorithms to be implemented in future.
12. Re: What is best way to set_rand?
- Posted by CoJaBo2 Jan 26, 2013
- 1119 views
Wouldn't that depend upon the length of the password and the length of the Alphabet?
A limit of 20 character passwords from an Alphabet of 96 characters produces a very large number of potential passwords, around 4.5*10^39. So if our cracking software does a billion passwords per second, that would still take about 10^25 years on average to crack one password.
If we assume the two 32-bit hashes were cryptographically secure, they were then concatenated, and fed to a cryptographically secure stream cipher, this places an absolute upper bound of 2^64 passwords that could possibly be generated.
Assuming 65 billion keys per second (based on what was used to crack DES some years ago), this would take no longer than a year to crack (less if using more or faster machines).
However, the hashes used are not concatenated, they are ones with very well known mathematical properties, and the "stream cipher" in this case is Eu's random number generator which is very much not suited for that purpose. This could dramatically reduce the time it takes to brute-force. And, if either the keyfile or passphrase are known, it would be downright trivial to brute-force, even without caring about other weaknesses.
I don't see the connection between generating passwords and hashing? It is possible to generate passwords without using any hashing function at all.
The question in this thread was about how to hash data to generate a password.
The currently implemented hashing algorithms in Eu are not designed for cryptographic purposes, however a few of them are quite secure of most other purposes.
The design of the hash() built-in function allows for other, including modern, algorithms to be implemented in future.
This should probably be done "soon"..
Though it may be a good idea to name the function for cryptographic hashes something else.
13. Re: What is best way to set_rand?
- Posted by DerekParnell (admin) Jan 26, 2013
- 1115 views
Wouldn't that depend upon the length of the password and the length of the Alphabet?
A limit of 20 character passwords from an Alphabet of 96 characters produces a very large number of potential passwords, around 4.5*10^39. So if our cracking software does a billion passwords per second, that would still take about 10^25 years on average to crack one password.
If we assume the two 32-bit hashes were cryptographically secure, they were then concatenated, and fed to a cryptographically secure stream cipher, this places an absolute upper bound of 2^64 passwords that could possibly be generated.
I see. Your assumption is that the generated password is first converted to a 64-bit key via a hashing algorithm, then that key is then the one to crack. And this is a valid assumption as many systems store and/or use passwords this way. But a 64-bit key is not the only way to do it.
A 64-bit key is equivalent to an 8-character long password built from an alphabet of 256 characters, or a 64-character one built from an alphabet of 2 characters - which is not what I was saying.
I was suggesting passwords from 1 to 20 characters long, built from an alphabet of 96 characters. That would be similar to a 132-bit key and not a 64-bit one.
Assuming 65 billion keys per second (based on what was used to crack DES some years ago), this would take no longer than a year to crack (less if using more or faster machines).
Ok, given that that rate is accurate - I don't know how to confirm that, but let's assume its correct. It would actually take 4.5 years on average to crack one of your passwords.
Specifically, using the scenario you mention above...
Min length 64 Max length 64 Alphabet 2 Permutations 1.84467e+019 Rate = 6.5e+010/second 283796062.67 seconds (worst case) 3284.68 days (worst case) 9.00 years (worst case) 4.50 years (average case) 3.21397e-010 Universe Lifetimes (average case)
The figures for the scenario that I was talking about are ...
Min length 1 Max length 20 Alphabet 96 Permutations 4.46655e+039 Rate = 6.5e+010/second 68716167858175135713194934272.00 seconds (worst case) 795326016877027076866048.00 days (worst case) 2178975388704183681024.00 years (worst case) 1089487694352091840512.00 years (average case) 7.78205e+010 Universe Lifetimes (average case)
Or explicitly using a 132-bit hash ...
Min length 132 Max length 132 Alphabet 2 Permutations 5.44452e+039 Rate = 6.5e+010/second 83761813395923318488164204544.00 seconds (worst case) 969465432823186598133760.00 days (worst case) 2656069678967634264064.00 years (worst case) 1328034839483817132032.00 years (average case) 9.48596e+010 Universe Lifetimes (average case)
But seeing that 132 bits is not a convenient number for computers, let try 128-bits...
Min length 128 Max length 128 Alphabet 2 Permutations 3.40282e+038 Rate = 6.5e+010/second 5235113337245207405510262784.00 seconds (worst case) 60591589551449162383360.00 days (worst case) 166004354935477141504.00 years (worst case) 83002177467738570752.00 years (average case) 5.92873e+009 Universe Lifetimes (average case)
Ok, let's get a bit silly and use 256 bit hash.
Min length 256 Max length 256 Alphabet 2 Permutations 1.15792e+077 Rate = 6.5e+010/second 1781416757497172330966974207074918049576683195587658169300239253504.00 seconds (worst case) 20618249508069125084842994879798559325453885474692661415772160.00 days (worst case) 56488354816627734612404904132517902578297360597153505869824.00 years (worst case) 28244177408313867306202452066258951289148680298576752934912.00 years (average case) 2.01744e+048 Universe Lifetimes (average case)
However, the hashes used are not concatenated, they are ones with very well known mathematical properties, and the "stream cipher" in this case is Eu's random number generator which is very much not suited for that purpose. This could dramatically reduce the time it takes to brute-force. And, if either the keyfile or passphrase are known, it would be downright trivial to brute-force, even without caring about other weaknesses.
I don't see the connection between generating passwords and hashing? It is possible to generate passwords without using any hashing function at all.
The question in this thread was about how to hash data to generate a password.
Actually, I don't think that was the question. My reading of it was "What is the best way to initialize the random number generator?". The passwords were then to be creating by using the set of numbers returned by the RNG - no hashing involved.
The quality of a password manufactured by using a random number generator is more a function of the quality of the RNG and not of any hashing algorithm used after the password has been selected. If, for example, the RNG started repeating its cycle after a few thousand numbers, or it didn't evenly return all its possible values, the quantity of unique passwords that could be manufactured would be greatly limited. I'm pretty sure that Eu's RNG is quite a deal better than that.
The key factor in creating passwords in this manner is to ensure that the next character to be used in the password is as unpredictable as possible. So it might be a strategy to re-seed the RNG after every X characters used, based on an external (real-time entropy) value, to avoid cyclic repetition.
And you are is correct in implying that no matter what password/pass-phrase/pass-file you actually use, the limiting factor is the value of the key created from that pass.
By the way, I agree with your comments about using a pass-file. If used, it must be physically secured such that it cannot be accessed by the cracker. Otherwise it is equivalent to writing your password on a slip of paper and leaving that on your desk.
14. Re: What is best way to set_rand?
- Posted by SnakeCharmer Jan 27, 2013
- 1093 views
Password generation by random numbers was bad idea. It was just a fast-n-dirty solution. I need any modern cryptographic function now. I programmed some checksums and hashes already (for Euphoria-skill). I try to implement on Euphoria one of the modern crypto-hash function: SHA-3 (formerly known as Keccak).