1. [WEE] code page mystery (solved)
- Posted by _tom (admin) Oct 31, 2017
- 1733 views
- Last edited Nov 06, 2017
Testing WEE on Mint Linux.
(1) write a "hello world" program: the file is utf8
(2) insert a utf8 unicode character: the file becomes iso-8859-1
_tom
2. Re: [WEE] code page mystery
- Posted by jimcbrown (admin) Nov 01, 2017
- 1684 views
Testing WEE on Mint Linux.
(1) write a "hello world" program: the file is utf8
(2) insert a utf8 unicode character: the file becomes iso-8859-1
_tom
Which utf8 character are you inserting?
And how are you checking the file's text encoding type? If, e.g., using /bin/file then it just may be a case where the heuristics used give a wrong answer.
(Keep in mind every file can be interpreted as valid iso-8859, unlike utf-8 there are no byte sequences which are illegal.)
3. Re: [WEE] code page mystery
- Posted by _tom (admin) Nov 01, 2017
- 1670 views
Here is the simplest demo of my observation:
- Resources
- Mint Linux
- WEE 0.49
- Euphoria Interpreter v4.1.0 development
64-bit Linux, Using System Memory
Revision Date: 2015-02-02 14:18:53, Id: 5861:57179171dbed
grab some unicode from LibreOffice. Insert three characters "ÄßC"
puts(1, "ÄßC" )
After inserting, the unicode characters look correct.
Run and I see:
��CPress any key...
Check, terminal is utf8
Set terminal encoding to ISO-8859-1
Now I see:
ÄßC
I discovered this because I was typing on the same file with both Geany and WEE. Geany reports the encoding on a status line.
_tom
4. Re: [WEE] code page mystery
- Posted by _tom (admin) Nov 02, 2017
- 1626 views
Testing on a new Mint version I conclude that you can not paste utf8 text from LibreOffice into WEE.
The first test used low code-point values ('ß' shows up) high code-point values (like box drawing) show up as '?'.
WEE never did utf8.
_tom
5. Re: [WEE] code page mystery
- Posted by petelomax Nov 02, 2017
- 1609 views
One of the biggest problems, I find, is that in the file open dialog of both (eg) gedit on linux and notepad on windows there is a character encoding drop-down.
I have managed to craft a windows-only file open, instead of IupFileDlg, that has the same, but that I fear would be far harder on linux (with IUP, and sadly google("IupFileDlg character encoding dropdown") yields nothing useful)
Otherwise I might have suggested Edix, but it is still very (very) ropey and progress is agonizingly slow.
Pete
6. Re: [WEE] code page mystery
- Posted by jimcbrown (admin) Nov 02, 2017
- 1586 views
Testing on a new Mint version I conclude that you can not paste utf8 text from LibreOffice into WEE.
The first test used low code-point values ('ß' shows up) high code-point values (like box drawing) show up as '?'.
WEE never did utf8.
_tom
LibreOffice should be able to open plain text utf8 files and save them directly.
Have you tried opening up the file in LibreOffice, using LibreOffice to insert the characters, saving the file and reopening in Geany/WEE ?
It's possible that the issue is not with WEE but with the copy-and-paste process itself.
7. Re: [WEE] code page mystery
- Posted by _tom (admin) Nov 03, 2017
- 1578 views
It's possible that the issue is not with WEE but with the copy-and-paste process itself.
I'm convinced my WEE editor is just not working in utf8. If I save a file in Geany with unicode characters and open it in WEE I see the weird boxes where the unicode characters should be. A LibreOffice (saving as a txt file) with unicode characters opens in Geany Ok but is weird in WEE. So it is not just the clipboard.
Learning how to write a "hello world" program has proven to be difficult.
_tom
8. Re: [WEE] code page mystery
- Posted by irv Nov 03, 2017
- 1544 views
OK, I downloaded Scite (which uses the same scintilla editor as Wee), and it also shows the weird boxes. Then, I changed Scite's global settings by commenting out code.page=0 and uncommenting code.page=65001. Now the puts(1, "ÄßC" ) is readable.
So...Try adding the first line below to wee.exw:
ssm(SCI_SETCODEPAGE,SC_CP_UTF8) -- added by Irv ssm(SCI_SETMARGINWIDTHN, 0, 48*line_numbers) -- line numbers margin visible ssm(SCI_SETMARGINWIDTHN, 1, 0) -- non-folding symbols margin hidden
Now, the line is readable in Geany, Wee, Scite, GEdit, LibreOffice, Xed, and of course, displays correctly when run in a Gnome terminal set to UTF-8.
9. Re: [WEE] code page mystery (solved)
- Posted by _tom (admin) Nov 06, 2017
- 1409 views
...
Now, the line is readable in Geany, Wee, Scite, GEdit, LibreOffice, Xed, and of course, displays correctly when run in a Gnome terminal set to UTF-8.
Thank you Irv,
The WEE editor is now utf8.
_tom