Bug report #16833
Unicode character in query in db_manager makes appear a never ending hourglass
|Affected QGIS version:||2.18.10||Regression?:||Yes|
|Operating System:||Easy fix?:||No|
|Pull Request or Patch supplied:||Yes||Resolution:|
|Crashes QGIS or corrupts data:||No||Copied to github as #:||24732|
Experiencing big regression which put QGIS in an unstable state (endless rolling cursor) under Windows 10.
The minimal step to reproduce is to put this query in DB Manager connected to a Postgresql instance
SELECT 'é'::text --being an unicode accentuated character
This however work fine under 2.18.3 that is still installed on my colleague workstation.
Might also be related to this bug report #16756
Test that PostGIS query can be passed as both unicode and string literal
Do not assume input SQL is ASCII
#1 Updated by Giovanni Manghi over 5 years ago
It works as expected up until 2.18.9
#2 Updated by Martin HOFFMANN over 5 years ago
Guess it was time I learn how to downgrade using OSGeo4W...
I confirm that downgrading to 2.18.9 solve my issue.
I strongly discourage Windows10 users that need unicode support to upgrade until this is solved.
#3 Updated by Jürgen Fischer over 5 years ago
- Subject changed from Unicode caracter crash db_manager to Unicode character crashes db_manager
#4 Updated by Giovanni Manghi over 5 years ago
- Subject changed from Unicode character crashes db_manager to Unicode caracter in query in db_manager makes appear a never ending hourglass
- Crashes QGIS or corrupts data changed from Yes to No
It is also true that neither DB Manager of QGIS crash or are blocked/froze. Just the very annoying hourglass.
#5 Updated by Martin HOFFMANN over 5 years ago
Sorry for the confusion, in fact the hourglass don't prevent manipulating the application and permit for instance to save the current document.
What seems to happen is that the query is stuck and never get_back to the preview window or a layer.
Apparently the error is only triggered where the input SQL query contain Unicode character, Unicode character retunned by the database are handled just fine.
Here is a test table to create in your DB using external tool to check that only input is concerned.
SELECT * INTO test_unicode FROM (VALUES ('ascii','e'),('unicode','é')) foo(test,value)
So if you try to access to that table through database manager query
SELECT * FROM test_unicode --This will work SELECT * FROM test_unicode WHERE value='e' --This as well SELECT * FROM test_unicode WHERE value='é' --This will never return and start and endless hourglass
I tested against Debian Jessie and I get the same error, here is the python log for the event
2017-07-11T13:50:05 1 Traceback (most recent call last): File "/usr/share/qgis/python/plugins/db_manager/dlg_sql_window.py", line 169, in loadAsLayerToggled self.fillColumnCombos() File "/usr/share/qgis/python/plugins/db_manager/dlg_sql_window.py", line 312, in fillColumnCombos c = connector._execute(None, sql) File "/usr/share/qgis/python/plugins/db_manager/db_plugins/connector.py", line 81, in _execute cursor.execute(str(sql)) UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 23: ordinal not in range(128)
This seems to confirm that this might actually be the same bug than #16756
Sorry for duplicating I'm more a user than a developer and maybe I don't communicate "the good way".
#6 Updated by Borys Jurgiel over 5 years ago
- Operating System deleted (
Windows 10 (with OSGeo4W))
- Assignee set to Sandro Santilli
Assigning to Sandro, as 14ab5eb0bda is the reason of this regression (line 81 of connector.py)
#7 Updated by Jürgen Fischer over 5 years ago
- Subject changed from Unicode caracter in query in db_manager makes appear a never ending hourglass to Unicode character in query in db_manager makes appear a never ending hourglass
#8 Updated by Sandro Santilli over 5 years ago
I think I made those changes because I could not build otherwise, but trying again now I can build and confirm the revert fixes the issue. PR is here: https://github.com/qgis/QGIS/pull/4882 -- it would be good to have a testcase for this but I don't know when I'll have time for that.
#9 Updated by Sandro Santilli over 5 years ago
- Status changed from Open to In Progress
#10 Updated by Sandro Santilli over 5 years ago
- Pull Request or Patch supplied changed from No to Yes
#11 Updated by Sandro Santilli over 5 years ago
A second, simpler, pull request is here:
This second one removes some assumptions about the input being ASCII (or encoded utf8).
Can the original reporter run some tests with that ?
A test is included and I run a manual test, but maybe you have more paths to test..
#12 Updated by Sandro Santilli over 5 years ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
Applied in changeset qgis|3120068bf108e8929189cd560435f778290a648d.
#13 Updated by Sandro Santilli over 5 years ago
Fix is now merged