1. [OT] files/dir on windoze
I ran into a problem last night, and can't figure out why this is. In one dir, i
have 14,301 txt files. So i go to create a new one, and Eu reports a bad file. I
try it manually, and Windoze reports:
Unable to create the file "New Text Document.txt'
The directory of file cannot be created.
Scandisk reported a filename too long in the directory, but it didn't fix
anything, and i cannot find any error. Does anyone have any experience with
this?
Kat
2. Re: [OT] files/dir on windoze
Kat wrote:
> I ran into a problem last night, and can't figure out why this is. In one dir,
> i
> have 14,301 txt files. So i go to create a new one, and Eu reports a bad file.
> I
> try it manually, and Windoze reports:
>
> Unable to create the file "New Text Document.txt'
> The directory of file cannot be created.
>
> Scandisk reported a filename too long in the directory, but it didn't fix
> anything, and i cannot find any error. Does anyone have any experience with
> this?
I had experienced things like that in the past, too. Fortunately,
recently this did not happen to me any more, so I recall this problem
rather vaguely. IIRC Windows allows you to create files that have too
long names, and after creation, it can't handle them any more. 
The first question IMHO is: How long is (or would have been) the *full*
name (including path) of "New Text Document.txt"? Windows only allaws a
maximum of 260 characters or so.
Maybe you should move all "good" files in the regarding directory into a
new directory (with a rather short name), and then (try to) delete that
old corrupted directory.
Regards,
Juergen
3. Re: [OT] files/dir on windoze
On 3 Jul 2004, at 19:20, Juergen Luethje wrote:
>
>
> Kat wrote:
>
> > I ran into a problem last night, and can't figure out why this is. In one
> > dir,
> > i have 14,301 txt files. So i go to create a new one, and Eu reports a bad
> > file. I try it manually, and Windoze reports:
> >
> > Unable to create the file "New Text Document.txt'
> > The directory of file cannot be created.
> >
> > Scandisk reported a filename too long in the directory, but it didn't fix
> > anything, and i cannot find any error. Does anyone have any experience with
> > this?
>
> I had experienced things like that in the past, too. Fortunately,
> recently this did not happen to me any more, so I recall this problem
> rather vaguely. IIRC Windows allows you to create files that have too
> long names, and after creation, it can't handle them any more.
>
> The first question IMHO is: How long is (or would have been) the *full*
> name (including path) of "New Text Document.txt"? Windows only allaws a
> maximum of 260 characters or so.
There are path+filenames which are longer than "123.txt" in that directory. I
can drag/drop/copy a file to the directory, like "new.txt" , but then cannot
rename it to "123.txt",, but i can rename it to "NEW.txt" !
> Maybe you should move all "good" files in the regarding directory into a
> new directory (with a rather short name), and then (try to) delete that
> old corrupted directory.
I'll try that in a few days, when the program ends this run. Chances are, it
will
crash when the new dir also has 14,301 files in it? The directory is NOT
flagged as system, archive, hidden, or read-only.
Kat
4. Re: [OT] files/dir on windoze
Kat wrote:
> On 3 Jul 2004, at 19:20, Juergen Luethje wrote:
>
>> Kat wrote:
>>
>>> I ran into a problem last night, and can't figure out why this is. In one
>>> dir,
>>> i have 14,301 txt files. So i go to create a new one, and Eu reports a bad
>>> file. I try it manually, and Windoze reports:
>>>
>>> Unable to create the file "New Text Document.txt'
>>> The directory of file cannot be created.
>>>
>>> Scandisk reported a filename too long in the directory, but it didn't fix
>>> anything, and i cannot find any error. Does anyone have any experience with
>>> this?
>>
>> I had experienced things like that in the past, too. Fortunately,
>> recently this did not happen to me any more, so I recall this problem
>> rather vaguely. IIRC Windows allows you to create files that have too
>> long names, and after creation, it can't handle them any more.
>>
>> The first question IMHO is: How long is (or would have been) the *full*
>> name (including path) of "New Text Document.txt"? Windows only allaws a
>> maximum of 260 characters or so.
>
> There are path+filenames which are longer than "123.txt" in that directory. I
> can drag/drop/copy a file to the directory, like "new.txt" , but then cannot
> rename it to "123.txt",, but i can rename it to "NEW.txt" !
Very strange. Do you think the problem has got something to do with the
digits in the name?
>> Maybe you should move all "good" files in the regarding directory into a
>> new directory (with a rather short name), and then (try to) delete that
>> old corrupted directory.
>
> I'll try that in a few days, when the program ends this run. Chances are, it
> will
> crash when the new dir also has 14,301 files in it? The directory is NOT
> flagged as system, archive, hidden, or read-only.
As far as I know, on DOS and Windows the number of entries (directories
and files) is only limited in the *root directory* of a disk. As far as
there is enough free disk space, any other directory should be able to
accept an unlimited number of entries, no?
Regards,
Juergen
5. Re: [OT] files/dir on windoze
On 3 Jul 2004, at 19:20, Juergen Luethje wrote:
>
>
> Kat wrote:
>
> > I ran into a problem last night, and can't figure out why this is. In one
> > dir,
> > i have 14,301 txt files. So i go to create a new one, and Eu reports a bad
> > file. I try it manually, and Windoze reports:
> >
> > Unable to create the file "New Text Document.txt'
> > The directory of file cannot be created.
> >
> > Scandisk reported a filename too long in the directory, but it didn't fix
> > anything, and i cannot find any error. Does anyone have any experience with
> > this?
>
> I had experienced things like that in the past, too. Fortunately,
> recently this did not happen to me any more, so I recall this problem
> rather vaguely. IIRC Windows allows you to create files that have too
> long names, and after creation, it can't handle them any more.
>
> The first question IMHO is: How long is (or would have been) the *full*
> name (including path) of "New Text Document.txt"? Windows only allaws a
> maximum of 260 characters or so.
>
> Maybe you should move all "good" files in the regarding directory into a
> new directory (with a rather short name), and then (try to) delete that
> old corrupted directory.
It's happened again, this time with a count of 14,172 files in another
directory. Is there a magic number i missed somewhere?
Kat
6. Re: [OT] files/dir on windoze
I had this happen a few times, once in a VERY long dir (like
"C:\qwertyuiopasdfghjklzxcvbnm\qwertyuiopasdfghjklzxcvbnm\qwertyuiopasdfghjklzxcvbnm")
and once in a rather short dir that had some currupt file names
(no name, symbols, ...) try a disk repair utility (not scandisk,
I've never had any luck with it).
Kat wrote:
>
> On 3 Jul 2004, at 19:20, Juergen Luethje wrote:
>
> >
> > Kat wrote:
> >
> > > I ran into a problem last night, and can't figure out why this is. In one
> > > dir,
> > > i have 14,301 txt files. So i go to create a new one, and Eu reports a bad
> > > file. I try it manually, and Windoze reports:
> > >
> > > Unable to create the file "New Text Document.txt'
> > > The directory of file cannot be created.
> > >
> > > Scandisk reported a filename too long in the directory, but it didn't fix
> > > anything, and i cannot find any error. Does anyone have any experience
> > > with
> > > this?
> >
> > I had experienced things like that in the past, too. Fortunately,
> > recently this did not happen to me any more, so I recall this problem
> > rather vaguely. IIRC Windows allows you to create files that have too
> > long names, and after creation, it can't handle them any more.
> >
> > The first question IMHO is: How long is (or would have been) the *full*
> > name (including path) of "New Text Document.txt"? Windows only allaws a
> > maximum of 260 characters or so.
> >
> > Maybe you should move all "good" files in the regarding directory into a
> > new directory (with a rather short name), and then (try to) delete that
> > old corrupted directory.
>
> It's happened again, this time with a count of 14,172 files in another
> directory. Is there a magic number i missed somewhere?
>
> Kat
>
>
7. Re: [OT] files/dir on windoze
Kat wrote:
> On 3 Jul 2004, at 19:20, Juergen Luethje wrote:
>
>> Kat wrote:
>>
>>> I ran into a problem last night, and can't figure out why this is. In one
>>> dir,
>>> i have 14,301 txt files. So i go to create a new one, and Eu reports a bad
>>> file. I try it manually, and Windoze reports:
>>>
>>> Unable to create the file "New Text Document.txt'
>>> The directory of file cannot be created.
>>>
>>> Scandisk reported a filename too long in the directory, but it didn't fix
>>> anything, and i cannot find any error. Does anyone have any experience with
>>> this?
>>
>> I had experienced things like that in the past, too. Fortunately,
>> recently this did not happen to me any more, so I recall this problem
>> rather vaguely. IIRC Windows allows you to create files that have too
>> long names, and after creation, it can't handle them any more.
>>
>> The first question IMHO is: How long is (or would have been) the *full*
>> name (including path) of "New Text Document.txt"? Windows only allaws a
>> maximum of 260 characters or so.
>>
>> Maybe you should move all "good" files in the regarding directory into a
>> new directory (with a rather short name), and then (try to) delete that
>> old corrupted directory.
>
> It's happened again, this time with a count of 14,172 files in another
> directory. Is there a magic number i missed somewhere?
You make me curious, and now I want to see what happens on my system.
On my harddrive, there are 4096 bytes per cluster. When I want to create
say 17000 small files, I need 17000*4096 bytes (about 66 MB) free disk
space. Well, I've got that free space, so it's now Euphoria's turn,
to create the files:
sequence number
atom t
integer fn
t = time()
for i = 1 to 17000 do
number = sprintf("%05d", {i})
fn = open(number & ".txt", "w")
puts(fn, "This is file #" & number)
close(fn)
end for
t = time()-t
? t/60
puts(1, " minutes")
if getc(0) then end if
It took about 18 minutes on my old PC. The last file is '17000.txt',
containing the line "This is file #17000". I looks as if everything is
correct on my system (Windows 98, FAT32).
I think you are using Windows 95, and FAT16, right? But using the FAT16
system shouldn't cause the problem, should it?
You wrote that Scabdisk didn't find an error. Maybe you should try a
tool such as Norton Utilities?
Sorry Kat, I think I can't be of much help ATM.
Regards,
Juergen
8. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
> It took about 18 minutes on my old PC. The last file is '17000.txt',
> containing the line "This is file #17000". I looks as if everything is
> correct on my system (Windows 98, FAT32).
> I think you are using Windows 95, and FAT16, right? But using the FAT16
> system shouldn't cause the problem, should it?
Perhaps you could try the test again, using long file names.
I'll bet it's either that (LFN) or FAT 16.
I haven't found any info on google about this, so it's not a very
common problem.
Irv
9. Re: [OT] files/dir on windoze
Irv Mullins wrote:
> Juergen Luethje wrote:
>
>> It took about 18 minutes on my old PC. The last file is '17000.txt',
>> containing the line "This is file #17000". I looks as if everything is
>> correct on my system (Windows 98, FAT32).
>> I think you are using Windows 95, and FAT16, right? But using the FAT16
>> system shouldn't cause the problem, should it?
>
> Perhaps you could try the test again, using long file names.
> I'll bet it's either that (LFN) or FAT 16.
> I haven't found any info on google about this, so it's not a very
> common problem.
Oops, I almost forgot about LFN. New program (as previously, using the
exw.exe 2.4 interpreter BTW):
sequence number
atom t
integer fn
t = time()
for i = 1 to 17000 do
number = sprintf("%05d", {i})
fn = open(number & " is the number of this file. Euphoria is cool."
& " The weather is fine today.txt", "w")
-- Each file name (including the extension '.txt') has 81
-- characters. The directory, in which I ran the program was
-- 'c:\temp\'. So the full qualified name of each file consisted
-- of 89 characters.
puts(fn, "This is file #" & number)
close(fn)
end for
t = time()-t
? t/60
puts(1, "minutes")
if getc(0) then end if
Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
"Can't create error message file: ex.err"
Now it's your turn. 
Regards,
Juergen
10. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
> Oops, I almost forgot about LFN. New program (as previously, using the
> exw.exe 2.4 interpreter BTW):
> Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
> "Can't create error message file: ex.err"
>
> Now it's your turn.
Pass. Now it's Rob's turn :)
Oh, you might try redirecting the ex.err to another directory and
see if Eu could write it there.
Irv
11. Re: [OT] files/dir on windoze
On 8 Jul 2004, at 8:20, Juergen Luethje wrote:
>
>
> Irv Mullins wrote:
>
> > Juergen Luethje wrote:
> >
> >> It took about 18 minutes on my old PC. The last file is '17000.txt',
> >> containing the line "This is file #17000". I looks as if everything is
> >> correct on my system (Windows 98, FAT32).
> >> I think you are using Windows 95, and FAT16, right? But using the FAT16
> >> system shouldn't cause the problem, should it?
> >
> > Perhaps you could try the test again, using long file names.
> > I'll bet it's either that (LFN) or FAT 16.
> > I haven't found any info on google about this, so it's not a very
> > common problem.
Irv, i don't do "normal" very well.
If it's worth doing for me, it's probably never been done, or done right,
before.
This project is 14 years old, and i still have no reliable all-purpose internet
file
fetcher!
> Oops, I almost forgot about LFN. New program (as previously, using the
> exw.exe 2.4 interpreter BTW):
>
> }}}
<eucode>
> sequence number
> atom t
> integer fn
>
> t = time()
> for i = 1 to 17000 do
> number = sprintf("%05d", {i})
> fn = open(number & " is the number of this file. Euphoria is cool."
> & " The weather is fine today.txt", "w")
> -- Each file name (including the extension '.txt') has 81
> -- characters. The directory, in which I ran the program was
> -- 'c:\temp\'. So the full qualified name of each file consisted
> -- of 89 characters.
> puts(fn, "This is file #" & number)
> close(fn)
> end for
> t = time()-t
> ? t/60
> puts(1, "minutes")
> if getc(0) then end if
> </eucode>
{{{
>
> Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
> "Can't create error message file: ex.err"
>
> Now it's your turn.
Thanks for duplicating the bug, Juergen. By any chance did it stop you from
using the computer for anything else before you got out of that crashed
filename program?
I have a suspicion this is related to the total bytecount of the filenames and
the allowed space in the FAT per directory. It occured with me using LFN on
win95B, on a 15gb FAT32 partition. Keep in mind the bytecount of the
filename may be higher than the filename as displayed to you.
Now, if only i can get around the lockup cause by a program sleep(120)ing,
or err'ing out and waiting on a keypress, when using urlmon.dll. I still have
that problem, it stops all mirc activity, and stops me from launching *any*
shortcut'ed programs. Got any more ideas on that one?
Kat
12. Re: [OT] files/dir on windoze
On 8 Jul 2004, at 4:58, irv mullins wrote:
>
>
> posted by: irv mullins <irvm at ellijay.com>
>
> Juergen Luethje wrote:
>
> > Oops, I almost forgot about LFN. New program (as previously, using the
> > exw.exe 2.4 interpreter BTW):
>
> > Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
> > "Can't create error message file: ex.err"
> >
> > Now it's your turn.
>
> Pass. Now it's Rob's turn :)
>
> Oh, you might try redirecting the ex.err to another directory and
> see if Eu could write it there.
Too bad that doesn't fix the original problem: the program could not create a
file for the programmer, which caused Eu to fault and try to write it's own
ex.err, which wasn't going to happen either. Windos Eu returns only a -1 to
let the programmer know there is an error, but not what the error is,, unless
RobC can extract the extended error msgs to give us.
Kat
13. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
>
> Irv Mullins wrote:
>
> > Juergen Luethje wrote:
> Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
> "Can't create error message file: ex.err"
>
> Now it's your turn.
2.524166667 minutes to create 17000 files on Linux. No errors.
Irv
14. Re: [OT] files/dir on windoze
On 8 Jul 2004, at 7:47, irv mullins wrote:
>
>
> posted by: irv mullins <irvm at ellijay.com>
>
> Juergen Luethje wrote:
> >
> > Irv Mullins wrote:
> >
> > > Juergen Luethje wrote:
>
> > Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
> > "Can't create error message file: ex.err"
> >
> > Now it's your turn.
>
> 2.524166667 minutes to create 17000 files on Linux. No errors.
Come on Irv, that's 0.002474673 millisec per file, what harddrive responds
that fast? Tell us!
Kat
15. Re: [OT] files/dir on windoze
Kat wrote:
>
> On 8 Jul 2004, at 7:47, irv mullins wrote:
>
> >
> > 2.524166667 minutes to create 17000 files on Linux. No errors.
>
> Come on Irv, that's 0.002474673 millisec per file, what harddrive responds
> that fast? Tell us!
>
That's nothing. I did it in 0.4913_ minutes in Win2K (2.4GHz).
I can only assume that there's a whole lot of caching going on. I
added a flush() right before close, and it only took 0.4145 minutes.
(First try with flush() took 0.541 minutes, but I hadn't deleted the
files from the previous run, so I think that wasn't really a fair test.)
Matt Lewis
16. Re: [OT] files/dir on windoze
Kat wrote:
>
> On 8 Jul 2004, at 7:47, irv mullins wrote:
> > 2.524166667 minutes to create 17000 files on Linux. No errors.
>
> Come on Irv, that's 0.002474673 millisec per file, what harddrive responds
> that fast? Tell us!
That's minutes, not seconds, Kat. I thought it was pretty slow.
However, since you asked:
User: irv
Drive hda is: Maxtor 5T030H3
Capacity: 60030432
Cache: 2048
Driver: ide-disk version 1.18
Geometry: physical 16383/16/63
Drive hdb is: Maxtor 90340D2
Capacity: 6640704
Cache: 512
Driver: ide-disk version 1.18
Geometry: physical 6588/16/63
17. Re: [OT] files/dir on windoze
Kat wrote:
<big snip>
> Thanks for duplicating the bug, Juergen. By any chance did it stop you from
> using the computer for anything else before you got out of that crashed
> filename program?
I noticed that other programs worked slower than normally. I think this
is nothing special, because the Eu program needs some CPU time, and
writing to disk isn't a fast process. But the program didn't stop me
from using the computer for anything else.
> I have a suspicion this is related to the total bytecount of the filenames and
> the allowed space in the FAT per directory.
In the meantime, I have the same suspicion.
> It occured with me using LFN on
> win95B, on a 15gb FAT32 partition. Keep in mind the bytecount of the
> filename may be higher than the filename as displayed to you.
Absolutely. I made some tests in the root directory of a 1.4 MB floppy
disk (e.g. 9+3 means a filename such as '123456789.txt'):
length of filenames max. number of files
------------------- --------------------
1+3 112
..
9+3 112
10+3 74
..
22+3 74
23+3 56
Maybe someone can discover the appropriate formula, how much bytes are
actually occupied by a filename, depending on the number of its
characters. It might look somehow like this:
occupiedBytes = ceil(numberOfCharacters/a)*a + b
Maybe a similar formula would apply to subdirectories on a hard disk?
<snip>
Regards,
Juergen
18. Re: [OT] files/dir on windoze
Kat wrote:
> On 8 Jul 2004, at 4:58, irv mullins wrote:
>
>> posted by: irv mullins <irvm at ellijay.com>
>>
>> Juergen Luethje wrote:
>>
>>> Oops, I almost forgot about LFN. New program (as previously, using the
>>> exw.exe 2.4 interpreter BTW):
>>
>>> Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
>>> "Can't create error message file: ex.err"
>>>
>>> Now it's your turn.
>>
>> Pass. Now it's Rob's turn :)
>>
>> Oh, you might try redirecting the ex.err to another directory and
>> see if Eu could write it there.
>
> Too bad that doesn't fix the original problem: the program could not create a
> file for the programmer, which caused Eu to fault and try to write it's own
> ex.err, which wasn't going to happen either. Windos Eu returns only a -1 to
> let the programmer know there is an error, but not what the error is,, unless
> RobC can extract the extended error msgs to give us.
Yes, another test showed, that the 'ex.err' file says:
"bad file number (-1)".
Perhaps someone wants to write a test program, that uses Windows API
functions for writing the files. There is an API function called
'GetLastError' or so, that might return a meaningful value.
Regards,
Juergen
19. Re: [OT] files/dir on windoze
On 8 Jul 2004, at 22:11, Juergen Luethje wrote:
>
>
> Kat wrote:
>
> <big snip>
>
> > Thanks for duplicating the bug, Juergen. By any chance did it stop you from
> > using the computer for anything else before you got out of that crashed
> > filename program?
>
> I noticed that other programs worked slower than normally. I think this
> is nothing special, because the Eu program needs some CPU time, and
> writing to disk isn't a fast process. But the program didn't stop me
> from using the computer for anything else.
>
> > I have a suspicion this is related to the total bytecount of the filenames
> > and
> > the allowed space in the FAT per directory.
>
> In the meantime, I have the same suspicion.
>
> > It occured with me using LFN on
> > win95B, on a 15gb FAT32 partition. Keep in mind the bytecount of the
> > filename may be higher than the filename as displayed to you.
>
> Absolutely. I made some tests in the root directory of a 1.4 MB floppy
> disk (e.g. 9+3 means a filename such as '123456789.txt'):
>
> length of filenames max. number of files
> ------------------- --------------------
> 1+3 112
> ..
> 9+3 112
> 10+3 74
> ..
> 22+3 74
> 23+3 56
>
> Maybe someone can discover the appropriate formula, how much bytes are
> actually occupied by a filename, depending on the number of its
> characters. It might look somehow like this:
> occupiedBytes = ceil(numberOfCharacters/a)*a + b
>
> Maybe a similar formula would apply to subdirectories on a hard disk?
Last i saw (for win9x), it was
name = space
1.0 = 8.3
8.3 = 8+3
9.3 = 7+flagbyte+3+8.3
15.3 = 7+flagbyte+3+8.3
16.3 = 7+flagbyte+3+7+flagbyte+3+8.3
The other catch is one filename can consume a whole cluster, even if only
10.3 long. Floppies are different than harddrives in that regard.
Basically, if x in x.y is over 8, remove the last byte (making 7 remaining),
replace the last byte space with a flag byte (a normally illegal char), put the
last byte in the next 8.3 space. It's a crude hack, yes. It also makes it
possible to use non-LFN apps to overwrite LFN files if the filename isn't
changed. Backwards compatability and all that. I don't know about the
FAT32 drives. FAT32 in win9x is probably all different than w2k, ME, and xp.
Kat
20. Re: [OT] files/dir on windoze
>From: Juergen Luethje <j.lue at gmx.de>
>Reply-To: EUforum at topica.com
>To: EUforum at topica.com
>Subject: Re: [OT] files/dir on windoze
>Date: Thu, 08 Jul 2004 08:20:41 +0200
>
>Irv Mullins wrote:
> > Juergen Luethje wrote:
> >> It took about 18 minutes on my old PC. The last file is '17000.txt',
> >> containing the line "This is file #17000". I looks as if everything is
> >> correct on my system (Windows 98, FAT32).
> >> I think you are using Windows 95, and FAT16, right? But using the FAT16
> >> system shouldn't cause the problem, should it?
> > Perhaps you could try the test again, using long file names.
> > I'll bet it's either that (LFN) or FAT 16.
> > I haven't found any info on google about this, so it's not a very
> > common problem.
>
>Oops, I almost forgot about LFN. New program (as previously, using the
>exw.exe 2.4 interpreter BTW):
>Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
>"Can't create error message file: ex.err"
>
>Now it's your turn.
>
I had about the same, but... 8191 text files, plus the program, the
error file was still written, though. "file number invalid (-1)" was the
error. I ran it in "D:\d". It took forever to try to delete the files, so i
just deleted the directory, and recreated the program. There were no
problems with the internet, or any other program, really.
Here is the program in Windows API. All it does is replace the builtin
functions. There's no error checking on write, which could be done, i
suppose. The error still occurs at the same time, though. I get an error
code of 82, which is ERROR_CANNOT_MAKE: "The directory or file cannot be
created."
include dll.e
include machine.e
constant
kernel32 = open_dll("Kernel32.dll"),
GENERIC_WRITE = #40000000,
CREATE_ALWAYS = 2,
FILE_ATTRIBUTE_NORMAL = #00000080,
INVALID_HANDLE_VALUE = #FFFFFFFF,
xGetLastError = define_c_func(kernel32, "GetLastError", {}, C_ULONG),
xCreateFile = define_c_func(kernel32, "CreateFileA", {C_POINTER, C_LONG,
C_LONG, C_POINTER, C_LONG, C_LONG, C_POINTER}, C_POINTER),
xWriteFile = define_c_func(kernel32, "WriteFile", {C_POINTER, C_POINTER,
C_ULONG, C_POINTER, C_POINTER}, C_INT),
xCloseHandle = define_c_func(kernel32, "CloseHandle", {C_POINTER}, C_INT)
function open(sequence name, sequence form)
atom ret, lpName
lpName = allocate_string(name)
ret = c_func(xCreateFile, {lpName, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL, NULL})
free(lpName)
if ret = INVALID_HANDLE_VALUE then
printf(1, "Error: %d\n", {c_func(xGetLastError, {})})
ret = getc(0)
abort(0)
end if
return ret
end function
procedure close(atom hFile)
atom c
c = c_func(xCloseHandle, {hFile})
end procedure
procedure puts(atom hFile, sequence text)
atom lpText, r, lpWritten
lpText = allocate_string(text)
lpWritten = allocate(4)
r = c_func(xWriteFile, {hFile, lpText, length(text), lpWritten, NULL})
free(lpWritten)
free(lpText)
end procedure
sequence number
atom t
atom fn
t = time()
for i = 1 to 17000 do
?i
number = sprintf("%05d", {i})
fn = open(number & " is the number of this file. Euphoria is cool."
& " The weather is fine today.txt", "w")
-- Each file name (including the extension '.txt') has 81
-- characters. The directory, in which I ran the program was
-- 'd:\d\'. So the full qualified name of each file consisted
-- of 86 characters.
puts(fn, "This is file #" & number)
close(fn)
end for
t = time()-t
printf(1, "%d minutes", {t / 60})
if getc(0) then end if
>Regards,
> Juergen
>
~[ WingZone ]~
http://wingzone.tripod.com/
21. Re: [OT] files/dir on windoze
Elliott Sales de Andrade wrote:
>> From: Juergen Luethje
>> Date: Thu, 08 Jul 2004 08:20:41 +0200
>>
>> Irv Mullins wrote:
>>> Juergen Luethje wrote:
>>>> It took about 18 minutes on my old PC. The last file is '17000.txt',
>>>> containing the line "This is file #17000". I looks as if everything is
>>>> correct on my system (Windows 98, FAT32).
>>>> I think you are using Windows 95, and FAT16, right? But using the FAT16
>>>> system shouldn't cause the problem, should it?
>>> Perhaps you could try the test again, using long file names.
>>> I'll bet it's either that (LFN) or FAT 16.
>>> I haven't found any info on google about this, so it's not a very
>>> common problem.
>>
>> Oops, I almost forgot about LFN. New program (as previously, using the
>> exw.exe 2.4 interpreter BTW):
>> Anything worked fine, up to file #08191 (inclusive). Then Euphoria said:
>> "Can't create error message file: ex.err"
>>
>> Now it's your turn.
>>
>
> I had about the same, but... 8191 text files, plus the program, the
> error file was still written, though. "file number invalid (-1)" was the
> error. I ran it in "D:\d". It took forever to try to delete the files, so i
> just deleted the directory, and recreated the program. There were no
> problems with the internet, or any other program, really.
> Here is the program in Windows API. All it does is replace the builtin
> functions. There's no error checking on write, which could be done, i
> suppose. The error still occurs at the same time, though. I get an error
> code of 82, which is ERROR_CANNOT_MAKE: "The directory or file cannot be
> created."
Unfortunately, this error message is not much more specific than
Euphoria's error massage. Anyway, there is increasing evidence that this
is a general Windows issue, isn't it? But nobody on this list has heard
about it before ...?
[snipped the code]
Regards,
Juergen
22. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
>
> Unfortunately, this error message is not much more specific than
> Euphoria's error massage. Anyway, there is increasing evidence that this
> is a general Windows issue, isn't it? But nobody on this list has heard
> about it before ...?
No, and I haven't been able to find anything via Google, either.
Granted, not many people are likely to have 17,000 files in any
one directory (with LFN's). We know the problem does not occur
with Eu on Linux, so it's presumably not a Eu problem.
Does this happen with all versions of Windows, or just some?
Irv
23. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
> > I had about the same, but... 8191 text files, plus the program, ...
That makes 8192.
There may be some significance to that number.
Irv
24. Re: [OT] files/dir on windoze
irv mullins wrote:
>
> Juergen Luethje wrote:
>
> > > I had about the same, but... 8191 text files, plus the program, ...
>
> That makes 8192.
>
> There may be some significance to that number.
>
> Irv
>
The Linux file system is not constructed or FORMATED the same way
that dos/windows is constructed so it will not have a limit.
When you initially format a hard drive for dos/windows depending on
cluster size, LFN's, etc. The ROOT directory has a limited number
of entries which is FIXED during the INITIAL FORMATING. Any hidden file,
system file, temp file, directory or normal file COUNTS against this LIMIT.
So if you are creating these in the ROOT DIRECTORY you will reach
a LIMIT and no longer be able to WRITE another file or directory.
Are you creating the files in a directory OTHER THAN THE ROOT DIRECTORY ?
Bernie
25. Re: [OT] files/dir on windoze
Bernard Ryan wrote:
>
>
> irv mullins wrote:
> >
> > Juergen Luethje wrote:
> >
> > > > I had about the same, but... 8191 text files, plus the program, ...
> >
> > That makes 8192.
> >
> > There may be some significance to that number.
> >
> > Irv
> >
>
> The Linux file system is not constructed or FORMATED the same way
> that dos/windows is constructed so it will not have a limit.
>
> When you initially format a hard drive for dos/windows depending on
> cluster size, LFN's, etc. The ROOT directory has a limited number
> of entries which is FIXED during the INITIAL FORMATING. Any hidden file,
> system file, temp file, directory or normal file COUNTS against this LIMIT.
> So if you are creating these in the ROOT DIRECTORY you will reach
> a LIMIT and no longer be able to WRITE another file or directory.
Correct, the root directory can only hold 512 entries.
> Are you creating the files in a directory OTHER THAN THE ROOT DIRECTORY ?
Yes.
Irv
26. Re: [OT] files/dir on windoze
Irv:
I found this on the internet at
http://www.systemsmedic.com/SoftwareEdu2.htm
This is probably what is happening :
" Under the FAT system, files are always allocated an integral number
of clusters. On a 1.2GB hard disk with clusters of 32K, a text file
containing only the words "hello, world" may have a directory listing
of just 12 bytes, but the file will actually take up 32K of disk space.
The unused part of the cluster is called slack. Small files may have
slack equal to almost an entire cluster; on average, a file will have
half a cluster of slack. "
Bernie
27. Re: [OT] files/dir on windoze
Irv Mullins wrote:
> Juergen Luethje wrote:
>>
>> Unfortunately, this error message is not much more specific than
>> Euphoria's error massage. Anyway, there is increasing evidence that this
>> is a general Windows issue, isn't it? But nobody on this list has heard
>> about it before ...?
>
> No, and I haven't been able to find anything via Google, either.
> Granted, not many people are likely to have 17,000 files in any
> one directory (with LFN's). We know the problem does not occur
> with Eu on Linux, so it's presumably not a Eu problem.
I also don't think that it's an Eu problem. I made an additional test to
prove that. My new test program wrote 10,000 files not to 1 directory,
but to 2 directories:
constant
stuff = " is the number of this file. Euphoria is cool."
& " The weather is fine today.txt"
sequence path, number, name
atom t
integer fn
t = time()
path = "c:\\c1\\"
for i = 0 to 4999 do
number = sprintf("%04d", {i})
name = number & stuff
fn = open(path & name, "w")
puts(fn, "This is file #" & number)
close(fn)
end for
path = "c:\\c2\\"
for i = 5000 to 9999 do
number = sprintf("%04d", {i})
name = number & stuff
fn = open(path & name, "w")
puts(fn, "This is file #" & number)
close(fn)
end for
t = time()-t
printf(1, "2x5000 files written to 2 dirs, each filename"
& " consisting of %d characters."
& "\n%.2f minutes"
& "\n\nPress Enter ...", {length(name), t/60})
if getc(0) then end if
As expected, there was no problem running this program, since there were
less than the "magic limit" of 8192 (me) or 8193 (Elliot) files per
directory.
Note: Contrary to the previous test, now each filename consists of only
80 characters, because I used
number = sprintf("%04d", {i}).
Previously, each filename consisted of 81 characters, because
number = sprintf("%05d", {i})
was used.
However, my assumption was that also in this case the "magic limit"
might be the same as before.
Then I tried to copy all files from dir 'c:\c2\' to 'c:\c1\', using
Total Commander on Windows. Total Commander aborted with an error, that
said (translation to English by me):
"Error: Unable to write 'c:\c1\8191 is the number ...'"
Then, examinating directory 'c:\c1\', I found all files intact, from
#0000 to #8190.
Summary:
Using Euphoria to create files with names that consist of 81 characters
each, 8192 files per directory are possible on my system.
Using Total Commander to copy files with names that consist of 80
characters each, 8191 files per directory are possible.
Doesn't sound logically, but is true.
> Does this happen with all versions of Windows, or just some?
Kat wrote she was using Win 95B, I was using Win 98 (partition with
FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
on what Windows version did you run your test?
Regards,
Juergen
28. Re: [OT] files/dir on windoze
Irv Mullins wrote:
> Juergen Luethje wrote:
(The following was from Elliot:)
>>> I had about the same, but... 8191 text files, plus the program, ...
I also had 8191 text files, plus the program in the regarding directory.
> That makes 8192.
Elliot additionally had the 'ex.err' file in the directory (= 8193).
On my system, Euphoria also tried to write an 'ex.err' file in the
directory, but didn't succeed. Funny, isn't it?
> There may be some significance to that number.
Just an idea, since for modern computers the number 2 has a special
meaning: 8192 is exactly 2^13 ...
Regards,
Juergen
29. Re: [OT] files/dir on windoze
Bernard Ryan wrote:
> Irv:
> I found this on the internet at
> http://www.systemsmedic.com/SoftwareEdu2.htm
>
> This is probably what is happening :
>
> " Under the FAT system, files are always allocated an integral number
> of clusters. On a 1.2GB hard disk with clusters of 32K, a text file
> containing only the words "hello, world" may have a directory listing
> of just 12 bytes, but the file will actually take up 32K of disk space.
> The unused part of the cluster is called slack. Small files may have
> slack equal to almost an entire cluster; on average, a file will have
> half a cluster of slack. "
I know about this phenomenon. At least on my system, I don't think that
the problem is caused by this say "slack problem".
As I wrote previously, on my harddrive there are 4096 bytes per cluster.
So in order to create 17000 small files (smaller than one cluster),
I need 17000*4096 bytes (about 66 MB) free disk space. I have got much
more than that free space!
Also I just wrote 10,000 files with long filenames (80 characters) to my
hard disk. No problem, when I only write 5,000 files per directory. But
in *one* directory, only 8191 files were possible.
I can't imagine that this problem is caused by cluster size and slack.
Regards,
Juergen
30. Re: [OT] files/dir on windoze
Bernard Ryan wrote:
>
> Irv:
> I found this on the internet at
> <a
> href="http://www.systemsmedic.com/SoftwareEdu2.htm">http://www.systemsmedic.com/SoftwareEdu2.htm</a>
>
> This is probably what is happening :
>
> " Under the FAT system, files are always allocated an integral number
> of clusters. On a 1.2GB hard disk with clusters of 32K, a text file
> containing only the words "hello, world" may have a directory listing
> of just 12 bytes, but the file will actually take up 32K of disk space.
> The unused part of the cluster is called slack. Small files may have
> slack equal to almost an entire cluster; on average, a file will have
> half a cluster of slack. "
True, but I don't think that is what is happening in this case, for
two reasons:
1. Different people (with no doubt different amounts of free disk space)
are coming up with the same limit of 8192 files.
2. When trying the test with 8.3 filenames, many more files can be
written, even though the files themselves take up the same amount of space.
I think it's likely caused by a programming error in the LFN system.
Irv
31. Re: [OT] files/dir on windoze
irv mullins wrote:
>
> Juergen Luethje wrote:
> >
> > Unfortunately, this error message is not much more specific than
> > Euphoria's error massage. Anyway, there is increasing evidence that this
> > is a general Windows issue, isn't it? But nobody on this list has heard
> > about it before ...?
>
> No, and I haven't been able to find anything via Google, either.
> Granted, not many people are likely to have 17,000 files in any
> one directory (with LFN's). We know the problem does not occur
> with Eu on Linux, so it's presumably not a Eu problem.
> Does this happen with all versions of Windows, or just some?
I was able to write the files (17,000 of them) under Win2K, so it would
appear to be a FAT32 issue, as NTFS didn't have any problem.
Matt Lewis
32. Re: [OT] files/dir on windoze
On 11 Jul 2004, at 20:41, Juergen Luethje wrote:
>
>
> Irv Mullins wrote:
>
> > Juergen Luethje wrote:
>
> (The following was from Elliot:)
>
> >>> I had about the same, but... 8191 text files, plus the program, ...
>
> I also had 8191 text files, plus the program in the regarding directory.
>
> > That makes 8192.
>
> Elliot additionally had the 'ex.err' file in the directory (= 8193).
> On my system, Euphoria also tried to write an 'ex.err' file in the
> directory, but didn't succeed. Funny, isn't it?
>
> > There may be some significance to that number.
>
> Just an idea, since for modern computers the number 2 has a special
> meaning: 8192 is exactly 2^13 ...
This all makes sense, except for one thing: i was topping out at a little over
14,000 files per directory. This happened 2 times, but not on *exactly* the
same number of files, crashing the two separate programs, at which point i
modified all the programs to sort to multiple directories, resulting in 1000's
of
directories, with 100's of files in each. All these files are on the small side,
only 1000 bytes.
Win95B, LFN, 2nd 15gig partition on a 30gig drive, 10.5gig free, FAT32,
using exw40.exe (registered) (a few winapi calls, dos32 gui and file calls
otherwise).
Kat
33. Re: [OT] files/dir on windoze
Kat wrote:
> This all makes sense, except for one thing: i was topping out at a little over
>
> 14,000 files per directory. This happened 2 times, but not on *exactly* the
> same number of files, crashing the two separate programs, at which point i
> modified all the programs to sort to multiple directories, resulting in 1000's
> of
> directories, with 100's of files in each. All these files are on the small
> side,
> only 1000 bytes.
Even a 1 byte file takes up quite a bit of space, but I don't think you
were running out of disk space, just out of long file name tables
(or whatever scheme is used to store LFN's).
> Win95B, LFN, 2nd 15gig partition on a 30gig drive, 10.5gig free, FAT32,
> using exw40.exe (registered) (a few winapi calls, dos32 gui and file calls
> otherwise).
The options seem to be:
1. Use XP or NT
2. split up the files into different directories, that's probably a hassle.
3. find some way to use only 8.3 filenames
4. Use Linux.
Irv
34. Re: [OT] files/dir on windoze
Me wrote:
> Irv Mullins wrote:
>
>> Juergen Luethje wrote:
>>>
>>> Unfortunately, this error message is not much more specific than
>>> Euphoria's error massage. Anyway, there is increasing evidence that this
>>> is a general Windows issue, isn't it? But nobody on this list has heard
>>> about it before ...?
>>
>> No, and I haven't been able to find anything via Google, either.
>> Granted, not many people are likely to have 17,000 files in any
>> one directory (with LFN's). We know the problem does not occur
>> with Eu on Linux, so it's presumably not a Eu problem.
And it's presumably not a Linux problem, and as Matt pointed out, it's
presumably not an NTFS problem, but a FAT issue.
We have considered the problem from many sides, and so I googled for
"fat lfn limit number entries" (without the double-quotes).
Among other pages, I found
<http://www.danisoft.com/services/FileSystems.jsp>
and I believe there is the formula that I was looking for:
function ceil (object x) -- ceil() is the opposite of floor()
return -floor(-x)
end function
function number_of_occupied_dir_entries (integer lengthOfFileName)
return 1 + ceil(lengthOfFileName/13)
end function
<snip>
> Using Euphoria to create files with names that consist of 81 characters
> each, 8192 files per directory are possible on my system.
Unfortunately, this statement contains a mistake:
The Eu program created 8191 files with names that consisted of 81
characters each. The 8192th file was the program itself. The name of the
program did consist of much less than 81 characters (I don't remember --
say 8 characters, when the name was "test.exw").
So the total number of occupied directory entries was:
? 8191 * number_of_occupied_dir_entries(81)
+ 1 * number_of_occupied_dir_entries( 8) -- prints 65530
One more file with a 81-character name would have resulted in 65538
total entries, which obviously isn't possible.
> Using Total Commander to copy files with names that consist of 80
> characters each, 8191 files per directory are possible.
In this case, the total number of occupied directory entries was:
? 8191 * number_of_occupied_dir_entries(80) -- prints 65528
One more file with a 80-character name would have resulted in 65536
total entries, which obviously isn't possible.
> Doesn't sound logically, but is true.
Was not completely true ... 
>> Does this happen with all versions of Windows, or just some?
>
> Kat wrote she was using Win 95B, I was using Win 98 (partition with
> FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
> on what Windows version did you run your test?
My current hypothesis is:
On several (if not all) Windows FAT systems, there is a maximum of
65535 (biggest unsigned 16-bit integer) entries per directory.
The number of entries occupied by a filename depends on its length,
and is calculated by the function number_of_occupied_dir_entries()
(see above).
Regards,
Juergen
35. Re: [OT] files/dir on windoze
Irv Mullins wrote:
> Kat wrote:
>
>> This all makes sense, except for one thing: i was topping out at a little
>> over
>> 14,000 files per directory. This happened 2 times, but not on *exactly* the
>> same number of files, crashing the two separate programs, at which point i
>> modified all the programs to sort to multiple directories, resulting in
>> 1000's of
>> directories, with 100's of files in each. All these files are on the small
>> side,
>> only 1000 bytes.
>
> Even a 1 byte file takes up quite a bit of space, but I don't think you
> were running out of disk space, just out of long file name tables
> (or whatever scheme is used to store LFN's).
I think my other post, where the function number_of_occupied_dir_entries()
is introduced, explains anything.
>> Win95B, LFN, 2nd 15gig partition on a 30gig drive, 10.5gig free, FAT32,
>> using exw40.exe (registered) (a few winapi calls, dos32 gui and file calls
>> otherwise).
>
> The options seem to be:
> 1. Use XP or NT
> 2. split up the files into different directories, that's probably a hassle.
I assume, also the number of possible subdirectories is limited
(depending on the length of their names).
> 3. find some way to use only 8.3 filenames
3a. Find some way to use as short filenames as possible (It's not just
'short' vs. 'long'.)
> 4. Use Linux.
5. If possible, use a few big files instead of many small files.
Regards,
Juergen
36. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
> 5. If possible, use a few big files instead of many small files.
6. If the files are actually small, it seems a database would be
the way to go.
Irv
37. Re: [OT] files/dir on windoze
>From: Juergen Luethje <j.lue at gmx.de>
>Reply-To: EUforum at topica.com
>To: EUforum at topica.com
>Subject: Re: [OT] files/dir on windoze
>Date: Sun, 11 Jul 2004 20:41:31 +0200
>
> > Does this happen with all versions of Windows, or just some?
>
>Kat wrote she was using Win 95B, I was using Win 98 (partition with
>FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
>on what Windows version did you run your test?
>
Windows XP Pro, but still FAT32, not NTFS. This obviously seems to be the
problem.
>Regards,
> Juergen
>
~[ WingZone ]~
http://wingzone.tripod.com/
38. Re: [OT] files/dir on windoze
Elliott Sales de Andrade wrote:
>> From: Juergen Luethje
>> Date: Sun, 11 Jul 2004 20:41:31 +0200
>>
>>> Does this happen with all versions of Windows, or just some?
>>
>> Kat wrote she was using Win 95B, I was using Win 98 (partition with
>> FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
>> on what Windows version did you run your test?
>>
>
> Windows XP Pro, but still FAT32, not NTFS. This obviously seems to be the
> problem.
Two of my programs split Unix mailbox files into separate e-mail files.
All e-mail files that originate from the same mailbox file are written
to one directory.
One user wrote, that he has got mailbox files with a size up to about
100 MB, containing about 28,000 messages. My programs build the file
names after the subject of the respective mail. He didn't report a
problem (maybe he uses NTFS), but I'm just calculating:
- If any file name has 8+3 (+ the dot = 12) characters, they will occupy
28000 * (1 + ceil(12/13)) = 28000*2 = 56000 entries in the regarding
directory. This is OK.
- If any file name has say 14 characters, all file names would occupy
28000*3 = 84000 entries. This is not possible on a FAT system, as we
know now.
Do you, or someone else, know a method, how an Euphoria program can
detect, what file system a given drive uses?
Regards,
Juergen
39. Re: [OT] files/dir on windoze
>From: Juergen Luethje <j.lue at gmx.de>
>Reply-To: EUforum at topica.com
>To: EUforum at topica.com
>Subject: Re: [OT] files/dir on windoze
>Date: Mon, 12 Jul 2004 09:33:35 +0200
>
>Do you, or someone else, know a method, how an Euphoria program can
>detect, what file system a given drive uses?
>
Here's a function that will do it. You just specify a drive letter
("C", "C:", "C:\\", or even "C:\\Documents and Settings\\") or a server
drive location ("\\\\MyServer\\MyServedDrive\\"). It will return {DriveType,
FileDirectoryComponent} DriveType is a string specifying FAT, NTFS, etc.
FileDirectoryComponent is the maximum length of a "directory component" This
is the maximum length of a bit between the slashes, ie a single file or
directory name. I figure that might be interesting, though it probably only
affects FAT systems without LFN.
include dll.e
include machine.e
constant
kernel32 = open_dll("kernel32.dll"),
lstrlen = define_c_func(kernel32, "lstrlenA", {C_POINTER}, C_ULONG),
xGetVolumeInformation = define_c_func(kernel32, "GetVolumeInformationA",
{C_POINTER, C_POINTER, C_ULONG, C_POINTER, C_POINTER, C_POINTER, C_POINTER,
C_ULONG}, C_INT)
global function getVolInfo(sequence drive)
atom lpDrive, lpComponent, lpType
integer ret
sequence info
if length(drive) = 1 then
drive &= ":\\"
else
if equal(drive[1..2], "\\\\") then --server locations...
if drive[length(drive)] != '\\' then
drive &= '\\'
end if
else
if length(drive) != 3 then
drive = drive[1] & ":\\"
end if
end if
end if
lpDrive = allocate_string(drive)
lpComponent = allocate(4)
lpType = allocate(20) --? maybe longer...
ret = c_func(xGetVolumeInformation, {lpDrive, NULL, 0, NULL, lpComponent,
NULL, lpType, 20})
info = {peek({lpType, c_func(lstrlen, {lpType})}), peek4u(lpComponent)}
free(lpDrive)
free(lpComponent)
free(lpType)
return info
end function
--*** TEST ***
printf(1, "%s:%d\n%s:%d\n%s:%d\n%s:%d\n",
getVolInfo("C")&getVolInfo("D:")&getVolInfo("E:\\")&getVolInfo("F:\\"))
>Regards,
> Juergen
>
~[ WingZone ]~
http://wingzone.tripod.com/
40. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
>
> Elliott Sales de Andrade wrote:
>
> >> From: Juergen Luethje
> >> Date: Sun, 11 Jul 2004 20:41:31 +0200
> >>
> >>> Does this happen with all versions of Windows, or just some?
> >>
> >> Kat wrote she was using Win 95B, I was using Win 98 (partition with
> >> FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
> >> on what Windows version did you run your test?
> >>
> >
> > Windows XP Pro, but still FAT32, not NTFS. This obviously seems to be the
> > problem.
>
> Two of my programs split Unix mailbox files into separate e-mail files.
> All e-mail files that originate from the same mailbox file are written
> to one directory.
>
> One user wrote, that he has got mailbox files with a size up to about
> 100 MB, containing about 28,000 messages. My programs build the file
> names after the subject of the respective mail. He didn't report a
> problem (maybe he uses NTFS), but I'm just calculating:
>
> - If any file name has 8+3 (+ the dot = 12) characters, they will occupy
> 28000 * (1 + ceil(12/13)) = 28000*2 = 56000 entries in the regarding
> directory. This is OK.
> - If any file name has say 14 characters, all file names would occupy
> 28000*3 = 84000 entries. This is not possible on a FAT system, as we
> know now.
>
I found this on the web which explains some of the file limitations
NTFS Size Limits
Maximum file size
Theory: 16 exabytes minus 1 KB (264 bytes minus 1 KB)
Implementation: 16 terabytes minus 64 KB (244 bytes minus 64 KB)
Maximum volume size
Theory: 264 clusters minus 1 cluster
Implementation: 256 terabytes minus 64 KB ( 232 clusters minus 1 cluster)
Files per volume 4,294,967,295 (232 minus 1 file)
Maximum Sizes on FAT32 Volumes
A FAT32 volume must have a minimum of 65,527 clusters.
Windows XP Professional can format FAT32 volumes up to 32 GB, but it
can mount larger FAT32 volumes created by other operating systems.
FAT32 Size Limits
Maximum file size 4 GB minus 1 byte (232 bytes minus 1 byte)
Maximum volume size 32 GB (implementation)
Files per volume 4,177,920
Maximum number of files and subfolders within a single folder 65,534
(The use of long file names can significantly reduce the number of
available files and subfolders within a folder.)
Maximum Sizes on FAT16 Volumes
FAT16 supports a maximum of 65,524 clusters per volume.
FAT16 Size Limits
Maximum file size 4 GB minus 1 byte (232 bytes minus 1 byte)
Maximum volume size 4 GB
Files per volume Approximately 65,536 (216 files)
Maximum number of files and folders within the root folder 512 (
Long file names can reduce the number of available files and folders
in the root folder.)
regards Pete.
41. Re: [OT] files/dir on windoze
I wrote:
>
> I found this on the web which explains some of the file limitations
>
> NTFS Size Limits
> Maximum file size
> Theory: 16 exabytes minus 1 KB (264 bytes minus 1 KB)
> Implementation: 16 terabytes minus 64 KB (244 bytes minus 64 KB)
>
> Maximum volume size
> Theory: 264 clusters minus 1 cluster
> Implementation: 256 terabytes minus 64 KB ( 232 clusters minus 1 cluster)
>
> Files per volume 4,294,967,295 (232 minus 1 file)
>
> Maximum Sizes on FAT32 Volumes
> A FAT32 volume must have a minimum of 65,527 clusters.
> Windows XP Professional can format FAT32 volumes up to 32 GB, but it
> can mount larger FAT32 volumes created by other operating systems.
>
> FAT32 Size Limits
> Maximum file size 4 GB minus 1 byte (232 bytes minus 1 byte)
> Maximum volume size 32 GB (implementation)
> Files per volume 4,177,920
> Maximum number of files and subfolders within a single folder 65,534
> (The use of long file names can significantly reduce the number of
> available files and subfolders within a folder.)
>
> Maximum Sizes on FAT16 Volumes
> FAT16 supports a maximum of 65,524 clusters per volume.
>
> FAT16 Size Limits
> Maximum file size 4 GB minus 1 byte (232 bytes minus 1 byte)
> Maximum volume size 4 GB
> Files per volume Approximately 65,536 (216 files)
> Maximum number of files and folders within the root folder 512 (
> Long file names can reduce the number of available files and folders
> in the root folder.)
42. Re: [OT] files/dir on windoze
Actually, I wouldn't try and use longer filenames if I could...
Why not use as many characters as you can -
01234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ - and store them that way....
5.3 format (assuming all the extensions have the same 3 letters) gives
you 60,466,176 combinations - 60 million ought to be plenty...
If the name of the file was to have meaning, then you could embed that
in the file itself as meta-data - after all, the files would have to
be parsed anyway, to see what the header info was.
What was the driving force in using separate files?
On Mon, 12 Jul 2004 09:33:35 +0200, Juergen Luethje <j.lue at gmx.de> wrote:
>
> Elliott Sales de Andrade wrote:
>
> >> From: Juergen Luethje
> >> Date: Sun, 11 Jul 2004 20:41:31 +0200
> >>
> >>> Does this happen with all versions of Windows, or just some?
> >>
> >> Kat wrote she was using Win 95B, I was using Win 98 (partition with
> >> FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
> >> on what Windows version did you run your test?
> >>
> >
> > Windows XP Pro, but still FAT32, not NTFS. This obviously seems to be the
> > problem.
>
> Two of my programs split Unix mailbox files into separate e-mail files.
> All e-mail files that originate from the same mailbox file are written
> to one directory.
>
> One user wrote, that he has got mailbox files with a size up to about
> 100 MB, containing about 28,000 messages. My programs build the file
> names after the subject of the respective mail. He didn't report a
> problem (maybe he uses NTFS), but I'm just calculating:
>
> - If any file name has 8+3 (+ the dot = 12) characters, they will occupy
> 28000 * (1 + ceil(12/13)) = 28000*2 = 56000 entries in the regarding
> directory. This is OK.
> - If any file name has say 14 characters, all file names would occupy
> 28000*3 = 84000 entries. This is not possible on a FAT system, as we
> know now.
>
> Do you, or someone else, know a method, how an Euphoria program can
> detect, what file system a given drive uses?
>
> Regards,
> Juergen
>
>
>
>
>
--
MrTrick
43. Re: [OT] files/dir on windoze
On 13 Jul 2004, at 9:50, Patrick Barnes wrote:
>
>
> Actually, I wouldn't try and use longer filenames if I could...
> Why not use as many characters as you can -
> 01234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ - and store them that way....
>
> 5.3 format (assuming all the extensions have the same 3 letters) gives
> you 60,466,176 combinations - 60 million ought to be plenty...
> If the name of the file was to have meaning, then you could embed that
> in the file itself as meta-data - after all, the files would have to
> be parsed anyway, to see what the header info was.
>
> What was the driving force in using separate files?
Data mining, and discovering after 2 days, the directory was full, and there
was no room to store more data, unless i came up with an artificial way to
categorise the data, and the data from other programs, and a different more
complicated method to merge and search the datum. It's like,, i haveto know
how much data there is, in what form, before i can even store the
10megabytes on a 10gig drive??
Kat
44. Re: [OT] files/dir on windoze
Kat wrote:
> > What was the driving force in using separate files?
>
> Data mining, and discovering after 2 days, the directory was full, and there
> was no room to store more data, unless i came up with an artificial way to
> categorise the data, and the data from other programs, and a different more
> complicated method to merge and search the datum. It's like,, i haveto know
> how much data there is, in what form, before i can even store the
> 10megabytes on a 10gig drive??
>
> Kat
Well, if you don't want to come up with a scheme then it's time to move on
from Win95. You might as well be playing with DOS and Windows 3.x. And for
those who like touting their personal flavor of *nix... just take it offline
with Kat 'cause I'm not the only one sick of hearing it (you know, the whole
"this *nix is best, and here are 20 more unrelated posts on how to use it...").
-- Brian
(and if you want to have an OS war, then take it offline with me...)
45. Re: [OT] files/dir on windoze
Brian Broker wrote:
> Well, if you don't want to come up with a scheme then it's time to move on
> from Win95. You might as well be playing with DOS and Windows 3.x. And for
> those who like touting their personal flavor of *nix... just take it offline
> with Kat 'cause I'm not the only one sick of hearing it (you know, the whole
> "this *nix is best, and here are 20 more unrelated posts on how to use
> it...").
>
> -- Brian
> (and if you want to have an OS war, then take it offline with me...)
The fact is, Kat asked because she was trying to get some work done
using Euphoria. Others here proved that the problem was with Windows.
I mentioned 4 alternatives which would work for Kat, one of which
was Linux.
If that upsets you so much, you probably shouldn't be reading this
list.
Irv
46. Re: [OT] files/dir on windoze
Elliott Sales de Andrade wrote:
>> From: Juergen Luethje
>> Date: Mon, 12 Jul 2004 09:33:35 +0200
>>
>> Do you, or someone else, know a method, how an Euphoria program can
>> detect, what file system a given drive uses?
>>
>
> Here's a function that will do it. You just specify a drive letter
> ("C", "C:", "C:\\", or even "C:\\Documents and Settings\\") or a server
> drive location ("\\\\MyServer\\MyServedDrive\\"). It will return {DriveType,
> FileDirectoryComponent} DriveType is a string specifying FAT, NTFS, etc.
> FileDirectoryComponent is the maximum length of a "directory component" This
> is the maximum length of a bit between the slashes, ie a single file or
> directory name. I figure that might be interesting, though it probably only
> affects FAT systems without LFN.
[snipped the code]
I knew that you would know a solution.
Thank you, Elliot!
Regards,
Juergen
47. Re: [OT] files/dir on windoze
Patrick Barnes wrote:
[rearranged quoting order]
> On Mon, 12 Jul 2004 09:33:35 +0200, Juergen Luethje <j.lue at gmx.de> wrote:
>>
>> Elliott Sales de Andrade wrote:
>>
>>>> From: Juergen Luethje
>>>> Date: Sun, 11 Jul 2004 20:41:31 +0200
>>>>
>>>>> Does this happen with all versions of Windows, or just some?
>>>>
>>>> Kat wrote she was using Win 95B, I was using Win 98 (partition with
>>>> FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
>>>> on what Windows version did you run your test?
>>>>
>>>
>>> Windows XP Pro, but still FAT32, not NTFS. This obviously seems to be the
>>> problem.
>>
>> Two of my programs split Unix mailbox files into separate e-mail files.
>> All e-mail files that originate from the same mailbox file are written
>> to one directory.
>>
>> One user wrote, that he has got mailbox files with a size up to about
>> 100 MB, containing about 28,000 messages. My programs build the file
>> names after the subject of the respective mail. He didn't report a
>> problem (maybe he uses NTFS), but I'm just calculating:
>>
>> - If any file name has 8+3 (+ the dot = 12) characters, they will occupy
>> 28000 * (1 + ceil(12/13)) = 28000*2 = 56000 entries in the regarding
>> directory. This is OK.
>> - If any file name has say 14 characters, all file names would occupy
>> 28000*3 = 84000 entries. This is not possible on a FAT system, as we
>> know now.
>>
>> Do you, or someone else, know a method, how an Euphoria program can
>> detect, what file system a given drive uses?
>>
>> Regards,
>> Juergen
>
> Actually, I wouldn't try and use longer filenames if I could...
> Why not use as many characters as you can -
> 01234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ - and store them that way....
>
> 5.3 format (assuming all the extensions have the same 3 letters) gives
> you 60,466,176 combinations - 60 million ought to be plenty...
> If the name of the file was to have meaning, then you could embed that
> in the file itself as meta-data
The subject of the mail is already contained *in* any mail anyway. But
users and I myself wanted the *filename* to be equal to the subject. I
have limited the filename to 64 characters, and nobody has complained so
far. But as I wrote previoously, even 14 characters sometimes can be to
much.
> - after all, the files would have to
> be parsed anyway, to see what the header info was.
Yes, of course.
> What was the driving force in using separate files?
The whole purpose of the programs is to split one or more Unix mailbox
files into separate e-mail files.
E.g. when someone uses an mail client that uses Unix mailbox files, and
wants to migrate to another mail client, that isn't able to import Unix
mailbox files, it's easy to split the mailbox files, and then import the
separate EML files.
Someone else e.g. uses the program to split the mailbox file that
contains the incoming mails in his company, and then he stores the mails
separately, according to the co-worker who is the recipient.
> MrTrick
Regards,
Juergen
48. Re: [OT] files/dir on windoze
On Mon, 12 Jul 2004 22:17:14 -0500, Kat <gertie at visionsix.com> wrote:
>
>
> On 13 Jul 2004, at 9:50, Patrick Barnes wrote:
>
> >
> > Actually, I wouldn't try and use longer filenames if I could...
> > Why not use as many characters as you can -
> > 01234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ - and store them that way....
> >
> > 5.3 format (assuming all the extensions have the same 3 letters) gives
> > you 60,466,176 combinations - 60 million ought to be plenty...
> > If the name of the file was to have meaning, then you could embed that
> > in the file itself as meta-data - after all, the files would have to
> > be parsed anyway, to see what the header info was.
> >
> > What was the driving force in using separate files?
>
> Data mining, and discovering after 2 days, the directory was full, and there
> was no room to store more data, unless i came up with an artificial way to
> categorise the data, and the data from other programs, and a different more
> complicated method to merge and search the datum. It's like,, i haveto know
> how much data there is, in what form, before i can even store the
> 10megabytes on a 10gig drive??
>
> Kat
Well, there's more than one way to skin a cat... oops...
What's wrong with EDS with/without EuSQL? or what about bucketing all
of the files into subfolders? put all the A* files in subfolder A, all
the B* files in subfolder B, etc. It makes it easier to find whether a
file exists that way. Or even some hashing function, and as many
folders as the OS can handle?
If '95 can handle the number, then any win platform after that should do fine.
--
MrTrick
49. Re: [OT] files/dir on windoze
Pete wrote:
> Juergen Luethje wrote:
>>
>> Elliott Sales de Andrade wrote:
>>
>>>> From: Juergen Luethje
>>>> Date: Sun, 11 Jul 2004 20:41:31 +0200
>>>>
>>>>> Does this happen with all versions of Windows, or just some?
>>>>
>>>> Kat wrote she was using Win 95B, I was using Win 98 (partition with
>>>> FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
>>>> on what Windows version did you run your test?
>>>>
>>>
>>> Windows XP Pro, but still FAT32, not NTFS. This obviously seems to be the
>>> problem.
<snip>
> I found this on the web which explains some of the file limitations
>
> NTFS Size Limits
> Maximum file size
> Theory: 16 exabytes minus 1 KB (264 bytes minus 1 KB)
> Implementation: 16 terabytes minus 64 KB (244 bytes minus 64 KB)
>
> Maximum volume size
> Theory: 264 clusters minus 1 cluster
> Implementation: 256 terabytes minus 64 KB ( 232 clusters minus 1 cluster)
>
> Files per volume 4,294,967,295 (232 minus 1 file)
>
> Maximum Sizes on FAT32 Volumes
> A FAT32 volume must have a minimum of 65,527 clusters.
> Windows XP Professional can format FAT32 volumes up to 32 GB, but it
> can mount larger FAT32 volumes created by other operating systems.
>
> FAT32 Size Limits
> Maximum file size 4 GB minus 1 byte (232 bytes minus 1 byte)
> Maximum volume size 32 GB (implementation)
> Files per volume 4,177,920
> Maximum number of files and subfolders within a single folder 65,534
> (The use of long file names can significantly reduce the number of
> available files and subfolders within a folder.)
>
> Maximum Sizes on FAT16 Volumes
> FAT16 supports a maximum of 65,524 clusters per volume.
>
> FAT16 Size Limits
> Maximum file size 4 GB minus 1 byte (232 bytes minus 1 byte)
> Maximum volume size 4 GB
> Files per volume Approximately 65,536 (216 files)
> Maximum number of files and folders within the root folder 512 (
> Long file names can reduce the number of available files and folders
> in the root folder.)
>
> regards Pete.
Interesting, thank you. Just out of curiousity: Can you please tell me
the URL from where you got this information?
Regards,
Juergen
50. Re: [OT] files/dir on windoze
On 14 Jul 2004, at 8:39, Patrick Barnes wrote:
>
>
> On Mon, 12 Jul 2004 22:17:14 -0500, Kat <gertie at visionsix.com> wrote:
<snip>
> >
> > Data mining, and discovering after 2 days, the directory was full, and there
> > was no room to store more data, unless i came up with an artificial way to
> > categorise the data, and the data from other programs, and a different more
> > complicated method to merge and search the datum. It's like,, i haveto know
> > how much data there is, in what form, before i can even store the
> > 10megabytes
> > on a 10gig drive??
> >
> > Kat
>
> Well, there's more than one way to skin a cat... oops...
>
> What's wrong with EDS with/without EuSQL? or what about bucketing all
> of the files into subfolders? put all the A* files in subfolder A, all
> the B* files in subfolder B, etc. It makes it easier to find whether a
> file exists that way. Or even some hashing function, and as many
> folders as the OS can handle?
> If '95 can handle the number, then any win platform after that should do fine.
The number of folders per folder would be the same as the number of files per
folder. But as for the solution, merging all the files into one file would be a
real disaster for me. But i have made /a/ and /b/ and etc folders, and
subdivided the files on that basis. I don't expect that will be a working
solution for very long.
I am seriously considering digging out an olde TP smartdrive/ramdrive i did 8
years or so ago, and revamping it (or recoding it into Eu form), to recognise
overflows from the win/dos limits, making drive letters obsolete, and making
folders like /a1/ , /a2/ , etc, all show up as being in /a/ in some cases. I
dunno. I hear nix doesn't have the same limits, but i prefer to see an OS
working in real life before i dedicate hours to learning how to fix it when it
breaks.
What i particularly need is an OS that doesn't tie up the drive with a
scandisk after abnormal shutdown, and which i can run simple tasks on, like
a lan fileserver. The only person i know in real life who has ever used nix said
it could not be done with a windows box on the lan, so that shot down using
nix.
Kat
51. Re: [OT] files/dir on windoze
Juergen Luethje wrote:
>
> Pete wrote:
>
> > Juergen Luethje wrote:
> >>
> >> Elliott Sales de Andrade wrote:
> >>
> >>>> From: Juergen Luethje
> >>>> Date: Sun, 11 Jul 2004 20:41:31 +0200
> >>>>
> >>>>> Does this happen with all versions of Windows, or just some?
> >>>>
> >>>> Kat wrote she was using Win 95B, I was using Win 98 (partition with
> >>>> FAT 32, 7.58 GB total, 5.14 GB free, 4096 bytes per cluster). Elliot,
> >>>> on what Windows version did you run your test?
> >>>>
> >>>
> >>> Windows XP Pro, but still FAT32, not NTFS. This obviously seems to be
> >>> the
> >>> problem.
>
> <snip>
>
> > I found this on the web which explains some of the file limitations
> >
> > NTFS Size Limits
> > Maximum file size
> > Theory: 16 exabytes minus 1 KB (264 bytes minus 1 KB)
> > Implementation: 16 terabytes minus 64 KB (244 bytes minus 64 KB)
<snip>
> > regards Pete.
>
> Interesting, thank you. Just out of curiousity: Can you please tell me
> the URL from where you got this information?
>
> Regards,
> Juergen
>
It was from a googled microsoft page (I didn't notice when posting but
some of the fonts got confused- some of the numbers should have been
powers) Beware long URL...
http://www.microsoft.com/resources/documentation/Windows/XP/all/reskit/en-us/Default.asp?url=/resources/documentation/Windows/XP/all/reskit/en-us/prkc_fil_tdrn.asp
Regards Pete.
52. Re: [OT] files/dir on windoze
Kat wrote:
...
> I am seriously considering digging out an olde TP smartdrive/ramdrive i did 8
> years or so ago, and revamping it (or recoding it into Eu form), to recognise
> overflows from the win/dos limits, making drive letters obsolete, and making
> folders like /a1/ , /a2/ , etc, all show up as being in /a/ in some cases. I
> dunno. I hear nix doesn't have the same limits, but i prefer to see an OS
> working in real life before i dedicate hours to learning how to fix it when it
>
> breaks.
>
> What i particularly need is an OS that doesn't tie up the drive with a
> scandisk after abnormal shutdown, and which i can run simple tasks on, like
> a lan fileserver. The only person i know in real life who has ever used nix
> said
> it could not be done with a windows box on the lan, so that shot down using
> nix.
Sigh...
I learned a long time ago that people rarely let facts get in the way
of their prejudices.
But best of luck, anyway.
Irv
53. Re: [OT] files/dir on windoze
On 15 Jul 2004, at 6:48, irv mullins wrote:
>
>
> posted by: irv mullins <irvm at ellijay.com>
>
> Kat wrote:
> ...
> > I am seriously considering digging out an olde TP smartdrive/ramdrive i did
> > 8
> > years or so ago, and revamping it (or recoding it into Eu form), to
> > recognise
> > overflows from the win/dos limits, making drive letters obsolete, and making
> > folders like /a1/ , /a2/ , etc, all show up as being in /a/ in some cases. I
> > dunno. I hear nix doesn't have the same limits, but i prefer to see an OS
> > working in real life before i dedicate hours to learning how to fix it when
> > it
> > breaks.
> >
> > What i particularly need is an OS that doesn't tie up the drive with a
> > scandisk after abnormal shutdown, and which i can run simple tasks on, like
> > a
> > lan fileserver. The only person i know in real life who has ever used nix
> > said
> > it could not be done with a windows box on the lan, so that shot down using
> > nix.
>
> Sigh...
>
> I learned a long time ago that people rarely let facts get in the way
> of their prejudices.
Meaning? Like the facts i have been unable to set up a nix box here? That a
$100K/yr HP engineer couldn't either? That when i asked to buy a box
aready set up, that all i had to do was add my DB drives, i wasn't able to buy
such a server? Which facts got in my way? Or were you on windoze's side
this time?
> But best of luck, anyway.
Thanks.
Kat
54. Re: [OT] files/dir on windoze
Pete wrote:
[big snip]
> It was from a googled microsoft page (I didn't notice when posting but
> some of the fonts got confused- some of the numbers should have been
> powers) Beware long URL...
>
> http://www.microsoft.com/resources/documentation/Windows/XP/all/reskit/en-us/Default.asp?url=/resources/documentation/Windows/XP/all/reskit/en-us/prkc_fil_tdrn.asp
>
> Regards Pete.
Thank you!
Regards,
Juergen
