1. [WEE] code page mystery (solved)

Testing WEE on Mint Linux.

(1) write a "hello world" program: the file is utf8
(2) insert a utf8 unicode character: the file becomes iso-8859-1

_tom

new topic     » topic index » view message » categorize

2. Re: [WEE] code page mystery

_tom said...

Testing WEE on Mint Linux.

(1) write a "hello world" program: the file is utf8
(2) insert a utf8 unicode character: the file becomes iso-8859-1

_tom

Which utf8 character are you inserting?

And how are you checking the file's text encoding type? If, e.g., using /bin/file then it just may be a case where the heuristics used give a wrong answer.

(Keep in mind every file can be interpreted as valid iso-8859, unlike utf-8 there are no byte sequences which are illegal.)

new topic     » goto parent     » topic index » view message » categorize

3. Re: [WEE] code page mystery

Here is the simplest demo of my observation:

  • Resources
    • Mint Linux
    • WEE 0.49
    • Euphoria Interpreter v4.1.0 development
      64-bit Linux, Using System Memory
      Revision Date: 2015-02-02 14:18:53, Id: 5861:57179171dbed

grab some unicode from LibreOffice. Insert three characters "ÄßC"

puts(1, "ÄßC" ) 

After inserting, the unicode characters look correct.

Run and I see:

��CPress any key... 

Check, terminal is utf8

Set terminal encoding to ISO-8859-1

Now I see:

ÄßC 

I discovered this because I was typing on the same file with both Geany and WEE. Geany reports the encoding on a status line.

_tom

new topic     » goto parent     » topic index » view message » categorize

4. Re: [WEE] code page mystery

Testing on a new Mint version I conclude that you can not paste utf8 text from LibreOffice into WEE.

The first test used low code-point values ('ß' shows up) high code-point values (like box drawing) show up as '?'.

WEE never did utf8.

_tom

new topic     » goto parent     » topic index » view message » categorize

5. Re: [WEE] code page mystery

One of the biggest problems, I find, is that in the file open dialog of both (eg) gedit on linux and notepad on windows there is a character encoding drop-down.
I have managed to craft a windows-only file open, instead of IupFileDlg, that has the same, but that I fear would be far harder on linux (with IUP, and sadly google("IupFileDlg character encoding dropdown") yields nothing useful)

Otherwise I might have suggested Edix, but it is still very (very) ropey and progress is agonizingly slow.

Pete

new topic     » goto parent     » topic index » view message » categorize

6. Re: [WEE] code page mystery

_tom said...

Testing on a new Mint version I conclude that you can not paste utf8 text from LibreOffice into WEE.

The first test used low code-point values ('ß' shows up) high code-point values (like box drawing) show up as '?'.

WEE never did utf8.

_tom

LibreOffice should be able to open plain text utf8 files and save them directly.

Have you tried opening up the file in LibreOffice, using LibreOffice to insert the characters, saving the file and reopening in Geany/WEE ?

It's possible that the issue is not with WEE but with the copy-and-paste process itself.

new topic     » goto parent     » topic index » view message » categorize

7. Re: [WEE] code page mystery

jimcbrown said...

It's possible that the issue is not with WEE but with the copy-and-paste process itself.

I'm convinced my WEE editor is just not working in utf8. If I save a file in Geany with unicode characters and open it in WEE I see the weird boxes where the unicode characters should be. A LibreOffice (saving as a txt file) with unicode characters opens in Geany Ok but is weird in WEE. So it is not just the clipboard.

Learning how to write a "hello world" program has proven to be difficult.

_tom

new topic     » goto parent     » topic index » view message » categorize

8. Re: [WEE] code page mystery

OK, I downloaded Scite (which uses the same scintilla editor as Wee), and it also shows the weird boxes. Then, I changed Scite's global settings by commenting out code.page=0 and uncommenting code.page=65001. Now the puts(1, "ÄßC" ) is readable.

So...Try adding the first line below to wee.exw:

    ssm(SCI_SETCODEPAGE,SC_CP_UTF8) -- added by Irv 
    ssm(SCI_SETMARGINWIDTHN, 0, 48*line_numbers) -- line numbers margin visible 
    ssm(SCI_SETMARGINWIDTHN, 1, 0) -- non-folding symbols margin hidden 
 

Now, the line is readable in Geany, Wee, Scite, GEdit, LibreOffice, Xed, and of course, displays correctly when run in a Gnome terminal set to UTF-8.

new topic     » goto parent     » topic index » view message » categorize

9. Re: [WEE] code page mystery (solved)

irv said...

...

Now, the line is readable in Geany, Wee, Scite, GEdit, LibreOffice, Xed, and of course, displays correctly when run in a Gnome terminal set to UTF-8.

Thank you Irv,

The WEE editor is now utf8.

_tom

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu