QGIS » QGIS Application

So, this turned out a practically unsolvable problem in GRASS. Quoting Glynn Clements:

===
There are two issues for which there is no viable solution:

1. OEM encoding.
2. Shift-JIS.

Regarding #1: GRASS neither knows nor cares whether a string is in
ANSI or OEM encoding. Much of it doesn't care about encodings at all,
and just treats strings as sequences of bytes. Anything which needs to
care about the encoding (e.g. the GUI) will just use "the locale's
encoding", which on Windows means "the ANSI codepage". If you use the
OEM codepage for anything, you lose.

Suggestions as to how to determine whether a string uses the ANSI or
OEM page are welcome, if unlikely.

Regarding #2: On Windows, any byte within the range 0-127 is assumed
to represent the corresponding ASCII character. For encodings which
assign other characters to any byte within that range (either
individually or as part of a multi-byte sequence), that is likely to
cause problems.

The most obvious example is that any occurrence of the byte 0x5C
within a filename is assumed to be a directory separator.
Unfortunately, Shift-JIS uses 0x5C as the second byte of a multi-byte
sequence, meaning that Japanese filenames may be parsed incorrectly.

Neither EUC-JP nor UTF-8 have this problem (as these only re-purpose
codes above 128), but unfortunately Windows doesn't provide locales
which uses either of these encodings.

And I can't think of any solution which doesn't involve re-writing all
code which handles pathnames.

Similar issues may exist with the other punctuation characters which
are "mingled" with the alphabetic characters, i.e. "[\\]^_{|}~" (e.g. |
is commonly used as a field separator, so tabular data which includes
Japanese text may be parsed incorrectly).

While such cases are probably less common than the pathname issue, a
fix is even less viable (i.e. fixing all string-handling code).

-- Glynn Clements <[email protected]> ===

So the solution seems just to switch to EN, just for Windows.
Seems an easy fix.

May be a duplicate of #3164. Please close it if this is the case.

Hi Minoru,
the patch looks good to me.

I'm adding a check so if we are not able to get the encoding from the XML declaration (using utf8 and the regular expression) then we'll let Qt detects the encoding of the XML (current behaviour).

This will make it working even whether the encoding name is not found, e.g. the encoding attribute is missing (though we are quite sure GRASS won't remove it) or the XML content is a UTF-16 or UTF-32 encoded string (the regexp doesn't match the text).

Since I cannot test it with Japanese lang, please, could you try the branch grass_jp_enc from my repo and report if it works?

Giuseppe Sucameli wrote:

Since I cannot test it with Japanese lang, please, could you try the branch grass_jp_enc from my repo and report if it works?

I've just tested your branch and got good result. Thanks!