1. archives
- Posted by Kat Jul 03, 2008
- 828 views
I decompressed all the files in the archives. The stats are:
decompressed files are 925megabytes total,
24,192 files,
in 2,689 contributions.
Kat
2. Re: archives
- Posted by ZNorQ Jul 03, 2008
- 826 views
I decompressed all the files in the archives. The stats are:
decompressed files are 925megabytes total,
24,192 files,
in 2,689 contributions.
Kat
Holy s*beep*..! Will you be uploading that?
Kenneth / ZNorQ
3. Re: archives
- Posted by Kat Jul 03, 2008
- 856 views
I decompressed all the files in the archives. The stats are:
decompressed files are 925megabytes total,
24,192 files,
in 2,689 contributions.
Kat
Holy s*beep*..! Will you be uploading that?
Kenneth / ZNorQ
I don't have the bandwidth now. And the zip files combined in a 7z'd file is only 282 megabytes, even un-7z'd it's 335megabytes of RDS site content plus the zipped user contribs. It's when everything is uncompressed that there's a size explosion to nearly a gigabyte, and even that isn't a huge database.
But i am indexing *some* of the files. There's some files that contain tons of non-program text (there's *.ew files that contain txt fiction, 8 contained large wordlists, there's several dictionaries, etc), some contain bitmaps (shoot-em-up games, mostly), etc, so i had to cut out some petabytes of non-Euphoria-related content, and likely missed something.
It's not even done indexing, and already being able to search for certain words has been much handier than merely searching the application description like the RDS archive/contribution pages do.
Kat
4. Re: archives
- Posted by CChris Jul 03, 2008
- 832 views
I decompressed all the files in the archives. The stats are:
decompressed files are 925megabytes total,
24,192 files,
in 2,689 contributions.
Kat
Holy s*beep*..! Will you be uploading that?
Kenneth / ZNorQ
I don't have the bandwidth now. And the zip files combined in a 7z'd file is only 282 megabytes, even un-7z'd it's 335megabytes of RDS site content plus the zipped user contribs. It's when everything is uncompressed that there's a size explosion to nearly a gigabyte, and even that isn't a huge database.
But i am indexing *some* of the files. There's some files that contain tons of non-program text (there's *.ew files that contain txt fiction, 8 contained large wordlists, there's several dictionaries, etc), some contain bitmaps (shoot-em-up games, mostly), etc, so i had to cut out some petabytes of non-Euphoria-related content, and likely missed something.
It's not even done indexing, and already being able to search for certain words has been much handier than merely searching the application description like the RDS archive/contribution pages do.
Kat
I have a gmail account with like 2Go of unused file storage. That account could serve for temporary storage for files you'd email at your convenience, given the bandwidth you have. I could then parcel those and use www.sendthisfile.com to send them back to RDS if need be.
Email me at zc2004oe hat gmail doubt com if this is of any use.
CChris
5. Re: archives
- Posted by Jeremy Cowgar Jul 03, 2008
- 767 views
Is RDS missing files from the Archive? I think they are restored, just the script is not online yet due to a pending code audit by Rob?
Jeremy Cowgar
6. Re: archives
- Posted by Kat Jul 03, 2008
- 798 views
- Last edited Jul 04, 2008
Is RDS missing files from the Archive? I think they are restored, just the script is not online yet due to a pending code audit by Rob?
Jeremy Cowgar
When i noticed they were restored, i stopped with the upload, days ago. But it's been discussed (years) before, and i have done this before, to de-zip the contributions so the contributions could be searched for Euphoria keywords or OS api keywords, or whatever, instead of just the brief application description. So that's what i am doing. I could never upload a gigabyte now, and somehow i doubt any code i write will be running on anyone else's computer, so i wasn't even considering donating it, unless asked for it, and being hurt when it isn't used. But with hosting packages routinely offering gigabyte storage, and the possibility to send up only the index, with further cleaning of garbage from it, i still don't see why we can't search the RDS archives (or openeuphoria.org archives) in this way. You can see how Al Getz uses allocate(), or see who has used poke4(), or look up the proper use of some wierd Microsoft or *nix api call.
Kat
7. Re: archives
- Posted by euphoric (admin) Jul 03, 2008
- 807 views
- Last edited Jul 04, 2008
But with hosting packages routinely offering gigabyte storage, and the possibility to send up only the index, with further cleaning of garbage from it, i still don't see why we can't search the RDS archives (or openeuphoria.org archives) in this way. You can see how Al Getz uses allocate(), or see who has used poke4(), or look up the proper use of some wierd Microsoft or *nix api call.
That would be a very educational and useful tool. I hope somebody can set that up.
In fact, I could probably host it at www.UsingEuphoria.com, so let me know if that's needed.
8. Re: archives
- Posted by Kat Jul 04, 2008
- 834 views
But with hosting packages routinely offering gigabyte storage, and the possibility to send up only the index, with further cleaning of garbage from it, i still don't see why we can't search the RDS archives (or openeuphoria.org archives) in this way. You can see how Al Getz uses allocate(), or see who has used poke4(), or look up the proper use of some wierd Microsoft or *nix api call.
That would be a very educational and useful tool. I hope somebody can set that up.
In fact, I could probably host it at www.UsingEuphoria.com, so let me know if that's needed.
But that site is down!
Kat
9. Re: archives
- Posted by euphoric (admin) Jul 04, 2008
- 822 views
But with hosting packages routinely offering gigabyte storage, and the possibility to send up only the index, with further cleaning of garbage from it, i still don't see why we can't search the RDS archives (or openeuphoria.org archives) in this way. You can see how Al Getz uses allocate(), or see who has used poke4(), or look up the proper use of some wierd Microsoft or *nix api call.
That would be a very educational and useful tool. I hope somebody can set that up.
In fact, I could probably host it at www.UsingEuphoria.com, so let me know if that's needed.
But that site is down!
Kat
Oopsies. Well, no worries. It's just down until I'm confident there are no security holes. I admit I don't know how long that will be.
10. Re: archives
- Posted by Kat Jul 04, 2008
- 857 views
From: Self <Kat@designerthinking.com> To: euphoric@cklester.com Subject: Re: archives Copies to: jeremy Cowgar <jeremy@cowgar.com> Date sent: Thu, 3 Jul 2008 22:11:08 -0500
But with hosting packages routinely offering gigabyte storage, and the possibility to send up only the index, with further cleaning of garbage from it, i still don't see why we can't search the RDS archives (or openeuphoria.org archives) in this way. You can see how Al Getz uses allocate(), or see who has used poke4(), or look up the proper use of some wierd Microsoft or *nix api call.
That would be a very educational and useful tool. I hope somebody can set that up.
In fact, I could probably host it at www.UsingEuphoria.com, so let me know if that's needed.
But that site is down!
Kat
Oopsies. Well, no worries. It's just down until I'm confident there are no security holes. I admit I don't know how long that will be.
Really, the problem isn't the hosting, it's getting the data to it. If the archive is 100% zipped, i dunno how to unzip it, and all the nested zips, on the host. If i try to ftp the data up unzipped, it will use over a weeks worth of my bandwidth here.
I'd luv to have it available for searching tho. There's a dir of 183,264 unique indexed words, with pointers to the files containing them. And for each file indexed, there's a list of unique words for that file, and how many times that word occured in the file. I know poke4 was mentioned in 1250 files, and poke255 is mentioned in mio.e. Getbitmapinfo is mentioned in 11 files, and GETSYSTEMMETRICS is in 61 files, in case you needed to see how to correctly use those commands. Gravitation is mentioned in one file, and gravitational is in 2 files. Since the searching, minus wildcards, is already done, it's trivially fast to retrieve filenames, the application package the file is in, and even return the line in the already unzipped file for inpection. It should consume very little cpu resources on a host. I have not added any search code to it.
Maybe i can hack the local library computers again and ftp up the data on their gigabit line.....
On the other paw, i wasn't successful in trying to be helpful last nite in #euphoria by using the data to locate a *suitable* SQL database wrapper. I found one that was windows only that turned out to not be useful on a *nix host, and after that none of what i suggested was even looked at. I'll just pack it up in the back of an older drive and forget it.
Kat
