Bug report #20244

Processing/GRASS do not work if Windows regional settings are not set to EN

Added by Marcin Domański over 5 years ago. Updated over 5 years ago.

Status:Closed
Priority:High
Assignee:-
Category:Processing/GRASS
Affected QGIS version:3.4.0 Regression?:Yes
Operating System:Windows 10 Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed/implemented
Crashes QGIS or corrupts data:No Copied to github as #:28065

Description

Hi,

Each time I want run any Grass algorithm fromprocessing toolbox, I have this kind of error:

Traceback (most recent call last):
File "C:/OSGEO4~1/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Algorithm.py", line 411, in processAlgorithm
Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
File "C:/OSGEO4~1/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Utils.py", line 369, in executeGrass
for line in iter(proc.stdout.readline, ''):
File "C:\OSGEO4~1\apps\Python37\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb3 in position 50: invalid start byte

0xb3 is polish ł letter, but I don't know which data are read on this level...

error_processing_grass.jpg - error processing grass for r.latlong with default profile (212 KB) Olivier ATHIMON, 2018-11-05 02:45 PM

error_processing_grass_witn_another_profile.jpg - error processing grass for r.basins.fill with new profile (269 KB) Olivier ATHIMON, 2018-11-05 02:46 PM


Related issues

Duplicated by QGIS Application - Bug report #20608: r.viewshed fails in Windows due to unicode Closed 2018-11-23

History

#1 Updated by Giovanni Manghi over 5 years ago

  • Status changed from Open to Feedback
  • Easy fix? changed from Yes to No

do you have special chars in the datasources paths or names?

#2 Updated by Pedro Venâncio over 5 years ago

Similar error here, on QGIS 3.4:

Traceback (most recent call last):
File "C:/OSGEO4~1/apps/qgis-rel-dev/./python/plugins\processing\algs\grass7\Grass7Algorithm.py", line 411, in processAlgorithm
Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
File "C:/OSGEO4~1/apps/qgis-rel-dev/./python/plugins\processing\algs\grass7\Grass7Utils.py", line 372, in executeGrass
for line in iter(proc.stdout.readline, ''):
File "C:\OSGEO4~1\apps\Python37\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc7 in position 4: invalid continuation byte

Giovanni, data and paths don't have any complex characters, and work well on QGIS 2.18.25.

Cleaning C:\Users\PedroVenancio\AppData\Roaming\QGIS\QGIS3\profiles\default does not solved the problem.

#3 Updated by Giovanni Manghi over 5 years ago

  • Priority changed from Normal to High
  • Affected QGIS version changed from 3.2.3 to 3.4.0
  • Subject changed from UTF-8 decode error each time when I try to run any Grass algorithm to UTF-8 decode error each time when I try to run any GRASS algorithm
  • Status changed from Feedback to Open
  • Regression? changed from No to Yes

#4 Updated by Giovanni Manghi over 5 years ago

  • Status changed from Open to Feedback

I assume you are using a localized OS, can you test with an OS in English?

#5 Updated by Pedro Venâncio over 5 years ago

Changing windows regional settings to English (United States) solves the problem!

So, the problem is related to some regional setting(s).

#6 Updated by Giovanni Manghi over 5 years ago

  • Subject changed from UTF-8 decode error each time when I try to run any GRASS algorithm to Processing/GRASS do not work if Windows regional settings are not set to EN
  • Status changed from Feedback to Open

#7 Updated by Cong Ning over 5 years ago

I' m a chinese. I meet the same problem on QGIS3.4. However I make sure there is no any special chars in path.(no chinese and I also try to use temporary layer)
I change the windows region to US, including system language. But the UI language still shows chinese. I think i don't install EN language package in system.
I also change QGIS locale to US.
Of couse, the prolem exists.

#8 Updated by Pedro Venâncio over 5 years ago

Some inputs from Nyall about this one:

I had a super-brief look into this one. For anyone's who's investigating a fix, it's something like this:

https://adminswerk.de/python3-unicodedecodeerror-popen/

(But it's not directly that -- if I do the exact same change then I get a lockup).

Hope that helps!
Nyall

http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-Early-3-4-point-release-tp5383869p5384478.html

#9 Updated by Olivier ATHIMON over 5 years ago

Same problem with a french or english windows region for default profile and new profile With QGIS 3.4.1 (standalone or OSGeo4W64 installation) and Windows 7 (and 10), +when i am executing any function GRASS from processing, i have no results and this error message:

"Traceback (most recent call last):
File "C:/PROGRA~1/QGIS3~1.4/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Algorithm.py", line 415, in processAlgorithm
Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
File "C:/PROGRA~1/QGIS3~1.4/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Utils.py", line 372, in executeGrass
for line in iter(proc.stdout.readline, ''):
File "C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 1: invalid continuation byte

Échec d'exécution au bout de 0.34 secondes

Chargement des couches de résultat
Les couches suivantes n'ont pas été générées correctement.<ul><li>D:/test.tif</li></ul>Vous pouvez vérifier le Panel de messages du journal dans la fenêtre principale de QGIS pour trouver plus d'informations à propos de l'exécution de l'algorithme."

It's running without problem with QGIS 2.18.25 (OSGeo4W installation) and Windows 7 (and 10)...

#10 Updated by Cong Ning over 5 years ago

Well,the problem still exists in QGIS3.4.1. But I found there are no problems in QGIS3.2.3 without locale change.(Default region is China)

#11 Updated by mr twister over 5 years ago

Same here with GRASS (Processing works fine though) with 3.4.1 on Windows 7 and de_ch as regional settings:

Starting GRASS GIS...
Traceback (most recent call last):
File "C:/OSGEO4~1/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Algorithm.py", line 415, in processAlgorithm
Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
File "C:/OSGEO4~1/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Utils.py", line 372, in executeGrass
for line in iter(proc.stdout.readline, ''):
File "C:\OSGEO4~1\apps\Python37\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 81: invalid start byte

After switching OS regional setting to en_us and also QGIS locale to en_us, everything works fine. Probably related to #20344?

#12 Updated by Giovanni Manghi over 5 years ago

  • Category changed from GRASS to Processing/GRASS

#13 Updated by Jürgen Fischer over 5 years ago

#14 Updated by mr twister over 5 years ago

For me, this issue is solved with 3.4.2. Can anybody confirm this?

#15 Updated by Cong Ning over 5 years ago

Of coure, the issue is solved too in Chinese system with QGIS3.4.2.

#16 Updated by Nyall Dawson over 5 years ago

  • Resolution set to fixed/implemented
  • Status changed from Open to Closed

#17 Updated by Pedro Venâncio over 5 years ago

Fix confirmed also here!

Thanks!

#18 Updated by Pedro Venâncio over 5 years ago

Can someone with regional settings different from English test this issue again?

I'm getting a similar error, even after this 0c36b90eafc80e4b6bfe7091b9c07ecf80afc744

Traceback (most recent call last):
File "C:/OSGEO4~1/apps/qgis-rel-dev/./python/plugins\processing\algs\grass7\Grass7Algorithm.py", line 413, in processAlgorithm
Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
File "C:/OSGEO4~1/apps/qgis-rel-dev/./python/plugins\processing\algs\grass7\Grass7Utils.py", line 375, in executeGrass
for line in iter(proc.stdout.readline, ''):
File "C:\OSGEO4~1\apps\Python37\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'CP_UTF8' codec can't decode byte 0xc7 in position 4: No mapping for the Unicode character exists in the target code page.

The difference is:

UnicodeDecodeError: 'CP_UTF8' codec can't decode byte 0xc7 in position 4: No mapping for the Unicode character exists in the target code page.

#19 Updated by Pedro Venâncio over 5 years ago

Ok, the problem is this one, commented in this b39e5a0937e5e317b42088e8a8ccae327ae8e033

#20 Updated by Frédéric Lévesque over 5 years ago

Hi, is this the same problem?

I run 3.4.2 on a french canadian windows 10 machine, qgis is made portable with a batch script.

Processing algorithm…
Algorithm 'v.clean' starting…
Input parameters:
{ '-b' : False, '-c' : False, 'GRASS_MIN_AREA_PARAMETER' : 0.0001, 'GRASS_OUTPUT_TYPE_PARAMETER' : 0, 'GRASS_REGION_PARAMETER' : None, 'GRASS_SNAP_TOLERANCE_PARAMETER' : -1, 'GRASS_VECTOR_DSCO' : '', 'GRASS_VECTOR_LCO' : '', 'error' : 'C:/Users/my_username/AppData/Local/Temp/processing_287288bf2b804ee6b24c5552db1bb035/f7dda6a0a1ce48a59c7093e54c901e38/error.gpkg', 'input' : 'C:\\Users\\my_username\\Desktop\\temp\\rmareatest.shp|layername=rmareatest', 'output' : 'C:/Users/my_username/AppData/Local/Temp/processing_287288bf2b804ee6b24c5552db1bb035/5e410c8fe7594c23bf3f92bb2285e722/output.gpkg', 'threshold' : '', 'tool' : [10], 'type' : [4] }

v.in.ogr min_area=0.0001 snap=-1.0 input="C:\Users\my_username\Desktop\temp\rmareatest.shp" layer="rmareatest" output="vector_5c46404173fb23" --overwrite -o
g.region n=5062643.2 s=5048824.38 e=306671.14 w=294370.81
v.clean input=vector_5c46404173fb23 type="area" tool="rmarea" output=outputb4bb433870364941aae38cbb027e68be error=errorb4bb433870364941aae38cbb027e68be --overwrite
v.out.ogr -c type="auto" input="outputb4bb433870364941aae38cbb027e68be" output="C:\Users\my_username\AppData\Local\Temp\processing_287288bf2b804ee6b24c5552db1bb035\5e410c8fe7594c23bf3f92bb2285e722\output.gpkg" format="GPKG" --overwrite
v.out.ogr -c type="auto" input="errorb4bb433870364941aae38cbb027e68be" output="C:\Users\my_username\AppData\Local\Temp\processing_287288bf2b804ee6b24c5552db1bb035\f7dda6a0a1ce48a59c7093e54c901e38\error.gpkg" format="GPKG" --overwrite
Le chemin d'accŠs sp‚cifi‚ est introuvable.
Le chemin d'accŠs sp‚cifi‚ est introuvable.
File "C:\_Cportable\QGIS_3.4.2_x64\qgis\apps\Python37\lib\site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Appuyez sur une touche pour continuer...
Traceback (most recent call last):
File "C:\_Cportable\QGIS_3.4.2_x64\qgis/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Algorithm.py", line 413, in processAlgorithm
Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
File "C:\_Cportable\QGIS_3.4.2_x64\qgis/apps/qgis/./python/plugins\processing\algs\grass7\Grass7Utils.py", line 411, in executeGrass
startupinfo=si if isWindows() else None
File "C:\_Cportable\QGIS_3.4.2_x64\qgis\apps\Python37\lib\subprocess.py", line 756, in __init__
restore_signals, start_new_session)
File "C:\_Cportable\QGIS_3.4.2_x64\qgis\apps\Python37\lib\subprocess.py", line 1155, in _execute_child
startupinfo)
OSError: [WinError 87] Paramètre incorrect

Execution failed after 0.63 seconds

Loading resulting layers
The following layers were not correctly generated.<ul><li>C:/Users/my_username/AppData/Local/Temp/processing_287288bf2b804ee6b24c5552db1bb035/5e410c8fe7594c23bf3f92bb2285e722/output.gpkg</li><li>C:/Users/my_username/AppData/Local/Temp/processing_287288bf2b804ee6b24c5552db1bb035/f7dda6a0a1ce48a59c7093e54c901e38/error.gpkg</li></ul>You can check the 'Log Messages Panel' in QGIS main window to find more information about the execution of the algorithm.

#21 Updated by Jürgen Fischer over 5 years ago

  • Description updated (diff)

Also available in: Atom PDF