Skip to content

Commit 14ab5eb

Browse files
committedJun 6, 2017
More unicode/utf8 fixes
1 parent 4f4d63a commit 14ab5eb

File tree

1 file changed

+7
-7
lines changed

1 file changed

+7
-7
lines changed
 

‎python/plugins/db_manager/db_plugins/connector.py

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ def _execute(self, cursor, sql):
7878
if cursor is None:
7979
cursor = self._get_cursor()
8080
try:
81-
cursor.execute(unicode(sql))
81+
cursor.execute(str(sql))
8282

8383
except self.connection_error_types() as e:
8484
raise ConnectionError(e)
@@ -98,7 +98,7 @@ def _execute_and_commit(self, sql):
9898
def _get_cursor(self, name=None):
9999
try:
100100
if name is not None:
101-
name = unicode(name).encode('ascii', 'replace').replace('?', "_")
101+
name = str(name).encode('ascii', 'replace').replace('?', "_")
102102
self._last_cursor_named_id = 0 if not hasattr(self,
103103
'_last_cursor_named_id') else self._last_cursor_named_id + 1
104104
return self.connection.cursor("%s_%d" % (name, self._last_cursor_named_id))
@@ -181,16 +181,16 @@ def _get_cursor_columns(self, c):
181181

182182
@classmethod
183183
def quoteId(self, identifier):
184-
if hasattr(identifier, '__iter__'):
184+
if hasattr(identifier, '__iter__') and not isinstance(identifier, str):
185185
ids = list()
186186
for i in identifier:
187187
if i is None or i == "":
188188
continue
189189
ids.append(self.quoteId(i))
190190
return u'.'.join(ids)
191191

192-
identifier = unicode(
193-
identifier) if identifier is not None else unicode() # make sure it's python unicode string
192+
identifier = str(
193+
identifier) if identifier is not None else str() # make sure it's python unicode string
194194
return u'"%s"' % identifier.replace('"', '""')
195195

196196
@classmethod
@@ -204,12 +204,12 @@ def quoteString(self, txt):
204204
txts.append(self.quoteString(i))
205205
return u'.'.join(txts)
206206

207-
txt = unicode(txt) if txt is not None else unicode() # make sure it's python unicode string
207+
txt = str(txt) if txt is not None else str() # make sure it's python unicode string
208208
return u"'%s'" % txt.replace("'", "''")
209209

210210
@classmethod
211211
def getSchemaTableName(self, table):
212-
if not hasattr(table, '__iter__'):
212+
if not hasattr(table, '__iter__') and not isinstance(table, str):
213213
return (None, table)
214214
if isinstance(table, str):
215215
table = table.split('.')

10 commit comments

Comments
 (10)

jusabatier commented on Jun 26, 2017

@jusabatier

Since this commit, I get this error when trying to import data via 'cadastre' module in a PostGIS database :

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 104: ordinal not in range(128) 
Traceback (most recent call last):
  File "C:/Users/julien.sabatier/.qgis2/python/plugins\cadastre\cadastre_dialogs.py", line 1090, in processImport
    qi.importMajic()
  File "C:/Users/julien.sabatier/.qgis2/python/plugins\cadastre\cadastre_import.py", line 419, in importMajic
    self.executeSqlScript(s, False, item.has_key('constraints'))
  File "C:/Users/julien.sabatier/.qgis2/python/plugins\cadastre\cadastre_import.py", line 1064, in executeSqlScript
    self.executeSqlQuery(sql, ignoreError)
  File "C:/Users/julien.sabatier/.qgis2/python/plugins\cadastre\cadastre_import.py", line 1115, in executeSqlQuery
    c = self.connector._execute_and_commit(sql)
  File "C:/PROGRA~1/QGIS2~1.18/apps/qgis/./python/plugins\db_manager\db_plugins\connector.py", line 95, in _execute_and_commit
    self._execute(None, sql)
  File "C:/PROGRA~1/QGIS2~1.18/apps/qgis/./python/plugins\db_manager\db_plugins\connector.py", line 81, in _execute
    cursor.execute(str(sql))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 104: ordinal not in range(128)


Version de Python : 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] 
Version de QGIS : 2.18.10 Las Palmas, 59e0f78 

It look like the use of the str function in the connector create this error.

Have you some solution to fix this ? Is it a configuration problem with my environment ?
I tried both on a Deian and Windows platform and each time with the 2.18 version of QGis I have this error.

Gustry commented on Jun 26, 2017

@Gustry
Contributor

Hum, yes, users with some accents in the attribute table may have some problems with this commit I think

jusabatier commented on Jun 26, 2017

@jusabatier

To my knowledge, this addon doesn't generate tables with accent in their attributes.

Just add comments on columns with accents : https://github.com/3liz/QgisCadastrePlugin/blob/master/scripts/plugin/commun_create_metier.sql#L1142

Maybe this is the problem.

But I think it have to be solved since it's a regression.

mdouchin commented on Jul 5, 2017

@mdouchin
Contributor

In fact, the "cadastre" french plugin uses dbmanager tools to run queries. Il would have to adapt the code to be able to manage QGIS before and after this commit.
3liz/QgisCadastrePlugin#106

mdouchin commented on Jul 5, 2017

@mdouchin
Contributor

It seems I can find a workaround by replacing in my code L 461 of cadastre_dialogs.py
c = connector._execute(None,unicode(sql))
by
c = connector._execute(None,unicode(sql).encode('utf-8'))

@jusabatier Please continue the issue conversation here for the cadastre plugin
3liz/QgisCadastrePlugin#106

borysiasty commented on Jul 13, 2017

@borysiasty
Member

@strk could you take a look at https://issues.qgis.org/issues/16833 please?
What was wrong with the previous unicode()? Looks like a mistaken backport from master/Py3? :)

strk commented on Jul 17, 2017

@strk
ContributorAuthor

strk commented on Jul 18, 2017

@strk
ContributorAuthor

Revert PR is in #4882

strk commented on Jul 19, 2017

@strk
ContributorAuthor

Reverting the commit triggers an infinite loop, see https://travis-ci.org/qgis/QGIS/jobs/255027295

strk commented on Jul 19, 2017

@strk
ContributorAuthor

NOTE: infinite loop only occurs with python3, python2 is fine with the revert. We need a solution working for both.

Please sign in to comment.