Bug report #9425

grass_script.bat always fails if USERNAME contains MBCS

Added by Tarot Osuji over 5 years ago. Updated 8 months ago.

Status:Closed
Priority:Normal
Assignee:Victor Olaya
Category:Processing/GRASS
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:Yes Resolution:end of life
Crashes QGIS or corrupts data:No Copied to github as #:18019

Description

On Windows, if USERNAME contains Multi Byte Character Set, processing with GRASS (grass_script.bat) always fails.
The reason is following bug:
#8915

Attached patch solves the problem from "Always failing" to "Available, if all value contains no MBCS". Other backends have same problem.

I think the fundamental solution is convert string value to temporal safe ascii value before create batch.
Unfortunately many tools don't consider MBCS. However if such option exists, many processing tools are able to use more widely.

patch.zip (1.21 KB) Tarot Osuji, 2014-01-26 01:27 AM

error.png (148 KB) Tarot Osuji, 2016-03-11 06:28 PM

qgis_log_panel.png (226 KB) Médéric RIBREUX, 2016-03-12 01:08 PM

error_log160313.png (15.5 KB) Tarot Osuji, 2016-03-12 08:17 PM

Grass7Utils.py Magnifier (17 KB) Médéric RIBREUX, 2016-03-13 03:22 AM

error_log160313-2.png (14.1 KB) Tarot Osuji, 2016-03-13 04:14 AM

Grass7Utils.py Magnifier (17.2 KB) Médéric RIBREUX, 2016-03-13 10:26 AM

error160315.png (114 KB) Tarot Osuji, 2016-04-09 02:01 AM


Related issues

Related to QGIS Application - Bug report #12657: Processing/GRASS don't work under Windows if username or ... Closed 2015-04-27
Related to QGIS Application - Bug report #16250: Processing ogr tools problems in QGIS 2.18.10 when specia... Closed 2017-02-28

History

#1 Updated by Tarot Osuji over 5 years ago

I'm sorry, the report explains 2 bugs.
Former is "Windows batch file which ends LF can't process MBCS". Attached patch converts the line-end of batch file: LF to CRLF.
Latter is "Many tools which invoked from batch can't operate with MBCS". It is a different matter from the former.

#2 Updated by Giovanni Manghi about 5 years ago

  • Project changed from 78 to QGIS Application
  • Category deleted (59)
  • Affected QGIS version set to 2.4.0
  • Crashes QGIS or corrupts data set to No

#3 Updated by Giovanni Manghi about 5 years ago

  • Category set to Processing/GRASS

#4 Updated by Giovanni Manghi over 4 years ago

  • Status changed from Open to Feedback

duplicate of #12657 ?

#5 Updated by Giovanni Manghi almost 4 years ago

  • Status changed from Feedback to Open

#6 Updated by Médéric RIBREUX over 3 years ago

  • Status changed from Open to In Progress
  • Affected QGIS version changed from 2.4.0 to master

Hello,

I am working on it (only made with GRASS7)...

I have found a way to deal with grass7_script.bat. The main problem was a MS-Windows Shell (cmd.exe) problem. MS-Windows shell can only execute scripts (.bat files) with the same encoding than the shell encoding. This encoding follows the configured language on the computer, so I had to find a way to get this encoding to build grass7_script.bat with the correct encoding.

Finally, I succeeded in launching a GRASS7 algorithm under an account with a username which has french accents.

I have made the fix tests on MS-Windows XP for the moment (the only VM with MS I have). I will try it on MS-Windows 7 next week and make a Pull-Request for this soon (so you can test it).

Stay tuned !

#7 Updated by Médéric RIBREUX over 3 years ago

Hello,

the PR is published...

Can you make some tests ?

#8 Updated by Tarot Osuji over 3 years ago

Although I tested with MBCS USERNAME, I get error.
Where do the error logs are stored ?

#9 Updated by Médéric RIBREUX over 3 years ago

Hello and thank for your report,

logs are in the Log Messages Panel (and I need them to go further) !
You can display it by clicking on the rightest bottom button (red marked in screenshot).

I really need the logs to know where the error is. Could you paste them here (or send an attached file) ?

The pull request is working on a french version of MS Windows but I suspected that there could be other problems on different language versions of MS Windows.

I have also worked on another workaround (which should work but relies on short filenames (which is bad !)) so we have another solution if the first is not working for you.

#10 Updated by Tarot Osuji over 3 years ago

Hi,

attached error log screen.
Thank you.

#11 Updated by Médéric RIBREUX over 3 years ago

Hello,

I can see that you are using GRASS6. The fix I have made is only for GRASS7 (but I can backport it for GRASS6 once we will be sure it is really fixed).

Here is a quick test procedure:
  • You have to install QGIS 2.14 (or a nightly build).
  • Copy the attached version of Grass7Utils.py into C:\\OSGeo4W\\apps\\qgis\\python\\plugins\\processing\\algs\\grass7\\ directory.
  • Then, start "QGIS with GRASS 7.0.3".
  • Go to Processing Menu -> Options -> Provider -> GRASS GIS 7 Commands.
  • Activate and set the GRASS7 folder (C:\\OSGeo4W\\apps\\grass\\grass-7.0.3). Set also the MSYS folder (C:\\OSGeo4W\\apps\\msys).
  • Restart QGIS to be sure that GRASS7 is activated into Processing.
  • Try to open a GRASS7 algorithm. If the algorithm dialog box opens, it means that your problem is fixed (but you can go further and launch the algorithm).
  • If the algorithm don't open, send me the log from the log panel (on the Python and Processing tabs).

Thanks for your test...

#12 Updated by Tarot Osuji over 3 years ago

Hi,

thanks for teaching method to activate GRASS 7.
I didn't notice activation is required.

I tried but unfortunately error occured.

I installed QGIS 2.14 with OSGeo4W. And yesterday, I download the trunk from github.
And I replaced contents of processing folder to downloaded trunk.
And I replaced the Grass7Utils.py you uploaded.

Thank you.

#13 Updated by Médéric RIBREUX over 3 years ago

Ok, I suspect this gonna be painful because I don't have the same problems than you and I can't reproduce them.

You will be my eyes for remote testing...

Can you try with the attached Grass7Utils.py and mail me the results (to not pollute the bug tracker) ?

Thanks !

#14 Updated by Tarot Osuji over 3 years ago

About a month ago, I sent the following messege:
----------

Hi,

I exchanged Grass7Utils.py and run, DEBUG log only show
tempPython=['C: .... cmdEncoding.py
and occured error with no hint.

So I added QgsMessageLog.logMessage() to find where is the problem.
Then I find runalg() fails.
next sentence,
if not os.path.exists(result['output']):
becomes True, and translated error message dialog fired.

I attached the screen. 'cp932' is Japanese subset of mbcs.

Thanks

#15 Updated by Giovanni Manghi over 2 years ago

  • Regression? set to No
  • Easy fix? set to No

#16 Updated by Giovanni Manghi 8 months ago

  • Status changed from In Progress to Closed
  • Resolution set to end of life

Also available in: Atom PDF