1. EBCDIC Formatted File (Help Needed)

I've got a file from a government agency that is in EBCDIC format (shoot
me now). It is almost 140MB in size. I'm trying to use Hayden McKay's
EBCDIC converter, and I'm sure it will work, but I can't even get the
raw file loaded up! I tried using "rb" and gets() but that was taking
a million years to load the file and I couldn't interrupt it nicely. Then
I tried using "rb" and getc(), but that will obviously take a million years
also, but at least I could put a tracking counter on screen and use SPACE
to stop it. Am I just going to have to be patient loading this big boy into
a variable so I can txt = ebcdic_to_ascii( txt ) it? AAARRRRRRRGGGHH!!!

Here's my program, in case anybody wants to revise/advise/contemplize.

---start
-- EBCDIC file converter

include ebcdic.e
include get.e
include graphics.e
include wildcard.e

object fn, line, junk
sequence txt, fname

txt = ""
fname = "orf850"

puts(1,"Opening " & fname & "...")
fn = open(fname,"rb")
clear_screen()

line = getc(fn)
while line != -1 do
	txt &= line
	position(1,1)
	?length(txt)
	line = getc(fn)
	junk = get_key()
	if junk = ' ' then
		line = -1
	end if
end while
close(fn)

puts(1,"Convert? (Y/N) ")
junk = wait_key()
if upper(junk) = 'Y' then
	txt = ebcdic_to_ascii( txt )
	fn = open("orf850a","wb")
	puts(fn,txt)
	close(fn)
end if

puts(1,"\nFinished.")
junk = wait_key()
---end

-=ck
"Programming in a state of EUPHORIA."
http://www.cklester.com/euphoria/

new topic     » topic index » view message » categorize

2. Re: EBCDIC Formatted File (Help Needed)

cklester wrote:
> I've got a file from a government agency that is in EBCDIC format (shoot
> me now). It is almost 140MB in size. I'm trying to use Hayden McKay's
> EBCDIC converter, and I'm sure it will work, but I can't even get the
> raw file loaded up! I tried using "rb" and gets() but that was taking
> a million years to load the file and I couldn't interrupt it nicely. Then
> I tried using "rb" and getc(), but that will obviously take a million years
> also, but at least I could put a tracking counter on screen and use SPACE
> to stop it. Am I just going to have to be patient loading this big boy into
> a variable so I can txt = ebcdic_to_ascii( txt ) it? AAARRRRRRRGGGHH!!!
> 
> Here's my program, in case anybody wants to revise/advise/contemplize.
> 
> ---start
> -- EBCDIC file converter
> 
> include ebcdic.e
> include get.e
> include graphics.e
> include wildcard.e
> 
> object fn, line, junk
> sequence txt, fname
> 
> txt = ""
> fname = "orf850"
> 
> puts(1,"Opening " & fname & "...")
> fn = open(fname,"rb")
> clear_screen()
> 
> line = getc(fn)
> while line != -1 do
> 	txt &= line
> 	position(1,1)
> 	?length(txt)
> 	line = getc(fn)
> 	junk = get_key()
> 	if junk = ' ' then
> 		line = -1
> 	end if
> end while
> close(fn)
> 
> puts(1,"Convert? (Y/N) ")
> junk = wait_key()
> if upper(junk) = 'Y' then
> 	txt = ebcdic_to_ascii( txt )
> 	fn = open("orf850a","wb")
> 	puts(fn,txt)
> 	close(fn)
> end if
> 
> puts(1,"\nFinished.")
> junk = wait_key()
> ---end

I think it's better if you convert it in chunks:
read a chunk of bytes, convert it, write it to file,
rinse and repeat.

Something like this (untested code):
include ebcdic.e
include get.e
include wildcard.e

integer fn_in, fn_out
sequence fname_in, fname_out, chunk

fname_in = "orf850"
fname_out = "orf850a"

fn_in = open(fname_in, "rb")
fn_out = open(fname_out, "wb")

chunk = get_bytes(fn_out, 1000)
while length(chunk) do
	puts(fn_out, ebcdic_to_ascii(chunk))
	chunk = get_bytes(fn_in, 1000)
end while
close(fn_in)
close(fn_out)


--
tommy online: http://users.telenet.be/tommycarlier
tommy.blog: http://tommycarlier.blogspot.com
Euphoria Message Board: http://uboard.proboards32.com
Empire for Euphoria: http://empire.iwireweb.com

new topic     » goto parent     » topic index » view message » categorize

3. Re: EBCDIC Formatted File (Help Needed)

cklester wrote:
> 
> I've got a file from a government agency that is in EBCDIC format (shoot
> me now). 

HEY!!!  What's wrong with EBCDIC?  ;)

IF they were thinking ahead, most IBM (and I'm assuming it's IBM) machines
with a current OS release support translation from EBCDIC to ASCII.  Maybe
you could ask them for this?  Wait...big government agency...might take 
a few months!

I work on an EBCDIC machine so if I can offer any assistance I will.  I 
haven't used the EBCDIC to ASCII routines that you referenced but 
hopefully it will translate correctly meaning EBCDIC has CCSID/codepages
just like ASCII.

Jonas Temple
http://www.yhti.net/~jktemple

new topic     » goto parent     » topic index » view message » categorize

4. Re: EBCDIC Formatted File (Help Needed)

Jonas Temple wrote:
> cklester wrote:
> > I've got a file from a government agency that is in EBCDIC format (shoot
> > me now). 
> HEY!!!  What's wrong with EBCDIC?  ;)

http://www.dynamoo.com/technical/ascii-ebcdic.htm

I actually had no experience with it up until today. :)

-=ck
"Programming in a state of EUPHORIA."
http://www.cklester.com/euphoria/

new topic     » goto parent     » topic index » view message » categorize

5. Re: EBCDIC Formatted File (Help Needed)

Tommy Carlier wrote:
> 
> cklester wrote:
> > I've got a file from a government agency that is in EBCDIC format (shoot
> > me now). It is almost 140MB in size. I'm trying to use Hayden McKay's
> > EBCDIC converter, and I'm sure it will work, but I can't even get the
> > raw file loaded up!

> I think it's better if you convert it in chunks:
> read a chunk of bytes, convert it, write it to file,
> rinse and repeat.

Yeah, you're right. :)

Final proggie:

include ebcdic.e
include get.e
include wildcard.e
include graphics.e

integer fn_in, fn_out
sequence fname_in, fname_out, chunk

fname_in = "orf850"
fname_out = "orf850a"

clear_screen()

fn_in = open(fname_in, "rb")
fn_out = open(fname_out, "wb")

chunk = get_bytes(fn_in, 1000)
while length(chunk) do
	position(1,1)
	printf(1,"%d",{numchunks})
	puts(fn_out, ebcdic_to_ascii(chunk))
	chunk = get_bytes(fn_in, 1000)
end while
close(fn_in)
close(fn_out)


Worked great!

Hayden, regarding your ebcdic_to_ascii() function, I had to change it
to the following to account for zeroes... Is this proper/correct?

-- Encodes an EBCDIC formatted string to ASCII.
global function ebcdic_to_ascii(sequence string)
    sequence ascii
    ascii = {}
    for i = 1 to length(string) do
    	if string[i] = 0 then -- because EBCDIC[0] is bad
    		ascii &= 0
    	else
		    if string[i] < 64 then
				ascii &= find(EBCDIC[string[i]], ASCII)
			else
				ascii &= EBCDIC[string[i]]
		    end if
		end if
    end for
    return ascii
end function


-=ck
"Programming in a state of EUPHORIA."
http://www.cklester.com/euphoria/

new topic     » goto parent     » topic index » view message » categorize

6. Re: EBCDIC Formatted File (Help Needed)

cklester wrote:
> 
> 
> <a
> href="http://www.dynamoo.com/technical/ascii-ebcdic.htm">http://www.dynamoo.com/technical/ascii-ebcdic.htm</a>
> 
> I actually had no experience with it up until today. :)
> 
Well, let's check the facts:

1. EBCDIC was around long before ASCII.  Doesn't necessarily make it 
better.  That's like saying electric cars are better than gasoline just
because they're newer.  EBCDIC and gasoline cars are what was needed at 
the time they were created because nothing else was out there.  They both
continue to run fine today.
2. The article cited some comparisons of programming in EBCDIC versus
ASCII.  The fact is that NO ONE programs like that on an IBM machine.  We
don't care what the control characters are because that is all handled
by the OS!
3. The article specifically mentioned the AS/400.  Yep, that's what I 
work on!  It cast the 400 in a "antique" light.  Hmmm, let's check a 400
against a PC:
- Built-in database: Yes PC: No
- Built-in IP services (HTTP, FTP, telnet, etc.): Yes PC: No
- 99.9% uptime: Yes PC: No
- Can get viruses: No PC: Yes
- Can support multiple OS on single processor: Yes PC: No 
    specifically: OS/400, AIX, Linux, Windows, SSP
- Partitionable: Yes PC: No
    not talking about the "I have a Linux partition on my PC" stuff...I'm
    talking about running MULTIPLE OS at the same time.
- Can partition on a SINGLE processor machine: Yes PC: No
- Can manage/store PC/Unix style files: Yes PC: No
- Same OS CDs can be installed on the smallest or largest machine: Yes 
PC: <bust out laughing>
- Can write programs in: C/C++/Java/Cobol/RPG/RPGII/RPGIII/RPGIV/REXX

Need I go on?  The fact is it's near impossible to run a large business on
PCs only.  How do I know?  Ask MS...in the past they tried to put their
money where there mouth was and run their software distribution on PCs
attached to a network.  They gave a large software company a contract to
rewrite their systems for their own OS.  The end result?  They threw it all
out and ordered new AS/400s!!!!!

I have an AS/400 in my basement.  It's an older model but since I got the
OS installed and configured I haven't touched it.  Yet I continue to spend
hours on my PC trying to clean off viruses, spyware, adware and re-install
the OS when all-of-a-sudden it starts shutting itself off at random intervals!
I've been working on the 400 for 14 years and I have NEVER re-installed the
OS because something was just acting "flaky".  Try to tell the president of
a company that you have to re-install the OS and he won't be able to take
orders for a couple of days.  Yeah, right!

Sorry for the rant but I've had it people dissin' the best machine out
there!

Jonas Temple
http://www.yhti.net/~jktemple

new topic     » goto parent     » topic index » view message » categorize

7. Re: EBCDIC Formatted File (Help Needed)

Jonas Temple wrote:
> cklester wrote:
> > <a
> > href="http://www.dynamoo.com/technical/ascii-ebcdic.htm">http://www.dynamoo.com/technical/ascii-ebcdic.htm</a>
> > I actually had no experience with it up until today. :)
> Sorry for the rant but I've had it people dissin' the best machine out
> there!

I was dissing the EBCDIC file format... not the AS/400. :P

(Does sound like a very cool machine, though!) :)

-=ck
"Programming in a state of EUPHORIA."
http://www.cklester.com/euphoria/

new topic     » goto parent     » topic index » view message » categorize

8. Re: EBCDIC Formatted File (Help Needed)

cklester wrote:
> I was dissing the EBCDIC file format... not the AS/400. :P
> 
> (Does sound like a very cool machine, though!) :)
> 
CK,

I know you weren't dissin', sorry if you took it that way.

I was just using the opportunity to "vent".  And yes, it is a very cool
machine.  You can learn more at:

http://www-1.ibm.com/servers/eserver/iseries/

Yeah, it's called an i5 now.  Before that it was iSeries, before that
it was AS/400, before that it was System/38, etc., etc.  The new i5 
comes in 1-way to 32-way processors.

Oh, BTW, did I mention that the AS/400 was 64 bit?  Has been since about
2002!  AAMOF, the conversion from 32 to 64 bit for the 400 world was no
more than restoring the 32 bit backup to the 64 bit machine.  The OS 
recompiled all the code during the restore.  99.9999% of all software ran
without modification (even stuff that was written 20 years before!).  
Put that in your pipe and smoke it, MS!

For the curious I would even be willing to allow access to my 400.

Jonas Temple
http://www.yhti.net/~jktemple

new topic     » goto parent     » topic index » view message » categorize

9. Re: EBCDIC Formatted File (Help Needed)

Jonas Temple wrote:

> better.  That's like saying electric cars are better than gasoline just
> because they're newer.  EBCDIC and gasoline cars are what was needed at 

Jonas:
    Electric Cars have been around since 1830's
    Go here for more details.

http://inventors.about.com/library/weekly/aacarselectrica.htm


Bernie

My files in archive:

http://www.rapideuphoria.com/w32engin.zip
http://www.rapideuphoria.com/mixedlib.zip
http://www.rapideuphoria.com/eu_engin.zip
http://www.rapideuphoria.com/win32eru.zip

new topic     » goto parent     » topic index » view message » categorize

10. Re: EBCDIC Formatted File (Help Needed)

Jonas Temple wrote:
> Oh, BTW, did I mention that the AS/400 was 64 bit?  Has been since about
> 2002!  AAMOF, the conversion from 32 to 64 bit for the 400 world was no
> more than restoring the 32 bit backup to the 64 bit machine.  The OS 
> recompiled all the code during the restore.  99.9999% of all software ran
> without modification (even stuff that was written 20 years before!).  
> Put that in your pipe and smoke it, MS!
> 
Oops...I meant it's been 64 bit since 1992!

Jonas Temple
http://www.yhti.net/~jktemple

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu