OpenEuphoria

2. Re: Check if files equal

• Posted by Kat <gertie at PELL.NET> Jul 06, 2002
• 409 views

On 7 Jul 2002, at 1:51, 10963508 at europeonline.com wrote:

> 
> What is the fastest way of checking if two very large files (~500 MB) are
> equal?
> I was thinking about this:
> -name
> -size
> -date last modified
> -pick about 10 random positions and check if bytes at those positions in
> both files match.

I would not trust those tests at all.

> Is there any better and faster way that I'm not aware of?

Open file
while not eof do
Read them in, one buffer size at a time, 
compare, 
if not equal { tell me it's not equal, abort}
end while

Kat

4. Re: Check if files equal

• Posted by Juergen Luethje <jluethje at gmx.de> Jul 07, 2002
• 418 views

Hi, Kat wrote:

> On 7 Jul 2002, at 1:51, 10963508 at europeonline.com wrote:
                          ^^^^^^^^
It would be nice, to see a name here. (Just my opinion.)

>> What is the fastest way of checking if two very large files (~500 MB) are
>> equal?

I assume you mean equal content, not equal name, equal date, ..

>> I was thinking about this:
>> -name

Name doesn't matter concerning the content.

>> -size
>> -date last modified

Date doesn't matter concerning the content.

>> -pick about 10 random positions and check if bytes at those positions in
>> both files match.

> I would not trust those tests at all.

First, I would compare the size of the files, this is very fast.
Whether this comparison can be trusted or not, depends on it's result!
If both files don't have the same size, it's 100% sure that they are
not equal. If they have the same size, further testing is needed.
The same logic goes for CRC tests and the comparison of random bytes.

>> Is there any better and faster way that I'm not aware of?

I think it would be the best, first to make some _fast_ tests, that
will find unequal files in some probability. (I don't know how fast CRC
testing is.)
Then, if these tests didn't prove that the files are unequal, more
precise tests must follow. Of course, the most precise test is this:

> Open file
> while not eof do
> Read them in, one buffer size at a time, 
> compare, 
> if not equal { tell me it's not equal, abort}
> end while

> Kat

Best regards,
   Juergen

6. Re: Check if files equal

• Posted by Juergen Luethje <jluethje at gmx.de> Jul 07, 2002
• 427 views

Kat wrote:

> On 7 Jul 2002, at 11:21, Juergen Luethje wrote:

>> Hi, Kat wrote:
>> 
>> > On 7 Jul 2002, at 1:51, 10963508 at europeonline.com wrote:
>>                           ^^^^^^^^
>> It would be nice, to see a name here. (Just my opinion.)

> This isn't my problem, you know. 

I know that, of course. Sorry if it looked as if I was meaning you!
I wrote it there because I assume, that the one who starts a thread,
reads all the posts in it.


When you want to discuss about a text, please don't snip the decisive
part away! Here it is again, what I wrote in my previous post:
---------------------------------------------------------------------
If both files don't have the same size, it's 100% sure that they are
not equal. If they have the same size, further testing is needed.
---------------------------------------------------------------------

>> The same logic goes for CRC tests and the comparison of random bytes.

> To get a CRC, you need to perform math on all the bytes.

I know.

> How will you get that math done, if you don't read all the bytes?

I didn't write that. In the above text between the two lines, just
replace "size" with "CRC", and then you'll see what I mean.

> Kat

Regards,
   Juergen

8. Re: Check if files equal

• Posted by 10963508 at europeonline.com Jul 07, 2002
• 420 views

Files are not databases, they are .zip .avi and .mp3 files mainly (stuff
coming down from satellite) - so they are compressed in some way.
Speed is more important than accuracy.

Tone Skoda

----- Original Message -----
From: "Derek Parnell" <Derek.Parnell at SYD.RABOBANK.COM>
To: "EUforum" <EUforum at topica.com>
Subject: RE: Check if files equal


>
> Hi all,
> in the original question put to the list by Tone, he mentioned 500MB+
files.
> I'm guessing that these are databases, given the size. If so, checking the
> bytes from the front of the files is probably a good idea because a lot of
> database systems keep pointers and stamps near the front of the database
> file. Thus, even if the file sizes haven't been changed, a short scan will
> probably find a changed stamp or pointer. If you get about 25% through the
> files and haven't found a mismatch, you might take a risk that they are
the
> same - or you might like to check the last 25% too. Just a thought.
>
> ---------
> Derek.
>
> ==================================================================
>
>
> ==================================================================
>
>
>
>

10. Re: Check if files equal

• Posted by Igor Kachan <kinz at peterlink.ru> Jul 08, 2002
• 434 views

Hi Tone,

----------
> Îò: 10963508 at europeonline.com
> Êîìó: EUforum <EUforum at topica.com>
> Òåìà: Check if files equal
> Äàòà: 7 èþëÿ 2002 ã. 3:51
> 
> What is the fastest way of checking if two very large files (~500 MB) are
> equal?
> I was thinking about this:
> -name
> -size
> -date last modified
> -pick about 10 random positions and check if bytes at those positions in
> both files match.
> 
> Is there any better and faster way that I'm not aware of?
> 
> Tone ©koda
> 

There is a good program by RDS:

http://www.RapidEuphoria.com/dupfile.zip

Regards,
Igor Kachan
kinz at peterlink.ru

12. Re: Check if files equal

• Posted by petelomax at blueyonder.co.uk Jul 12, 2002
• 463 views

{{{ On Mon, 8 Jul 2002 11:37:33 +0800, Derek Parnell <Derek.Parnell at SYD.RABOBANK.COM> wrote:

<snip>

Just my $0.02: I'd use md5.

There's a dos/windows version here: http://www.fourmilab.ch/md5/ and linux, DLL, & more here: http://userpages.umbc.edu/mabzug1/cs/md5/md5.html