Bug report #9704

Gdal Modules Errors on OsX Mavericks

Added by Rhenriques Henriques about 6 years ago. Updated about 6 years ago.

Status:Closed
Priority:Severe/Regression
Assignee:William Kyngesburye
Category:GDAL Tools
Affected QGIS version:2.2.0 Regression?:No
Operating System:OsX Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed/implemented
Crashes QGIS or corrupts data:No Copied to github as #:18267

Description

Several GDAL Modules give me error dialogues while selecting some options.
MacOSX Mavericks.

The following examples occur while trying to select the output file:

Gdal modules errors

Translate

Traceback (most recent call last):
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/doTranslate.py", line 161, in fillOutputFileEdit
    outputFile = Utils.FileDialog.getSaveFileName(self, self.tr( "Select the raster file to save the results to" ), Utils.FileFilter.allRastersFilter(), lastUsedFilter )
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/GdalTools_utils.py", line 402, in getSaveFileName
    return self.getDialog(parent, caption, QFileDialog.AcceptSave, QFileDialog.AnyFile, filter, selectedFilter, useEncoding)
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/GdalTools_utils.py", line 351, in getDialog
    dialog.selectNameFilter(selectedFilter[0])
TypeError: QFileDialog.selectNameFilter(QString): argument 1 has unexpected type 'QPyNullVariant'

Python version:
2.7.5 (default, Aug 25 2013, 00:04:04) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]

Clipper


Traceback (most recent call last):
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/doClipper.py", line 115, in fillOutputFileEdit
    outputFile = Utils.FileDialog.getSaveFileName(self, self.tr( "Select the raster file to save the results to" ), Utils.FileFilter.allRastersFilter(), lastUsedFilter)
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/GdalTools_utils.py", line 402, in getSaveFileName
    return self.getDialog(parent, caption, QFileDialog.AcceptSave, QFileDialog.AnyFile, filter, selectedFilter, useEncoding)
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/GdalTools_utils.py", line 351, in getDialog
    dialog.selectNameFilter(selectedFilter[0])
TypeError: QFileDialog.selectNameFilter(QString): argument 1 has unexpected type 'QPyNullVariant'

Python version:
2.7.5 (default, Aug 25 2013, 00:04:04) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]

Cheers

GdalTools_utils.py Magnifier (27.9 KB) Rhenriques Henriques, 2014-03-06 03:04 PM

Associated revisions

Revision 073dbf17
Added by Larry Shaffer about 6 years ago

Fix #9704, GdalTools file dialogs return QPyNullVariant due to QSettings undefined return type

Revision 9c139c28
Added by Larry Shaffer about 6 years ago

Merge pull request #1230 from dakcarto/issue_9704_gdal-qsettings

Fix #9704, GdalTools file dialogs return QPyNullVariant due to QSettings undefined return type

History

#1 Updated by Giovanni Manghi about 6 years ago

  • Operating System set to OsX
  • OS version set to mavericks

did it happened in previous qgis versions?

#2 Updated by Giovanni Manghi about 6 years ago

  • Status changed from Open to Feedback

#3 Updated by Francesco Vidotto about 6 years ago

Same problem here with Mavericks.
It never occurred with previous versions of QGIS

Warp (reproject)

Traceback (most recent call last):
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/doWarp.py", line 129, in fillOutputFileEdit
    outputFile = Utils.FileDialog.getSaveFileName(self, self.tr( "Select the raster file to save the results to" ), Utils.FileFilter.allRastersFilter(), lastUsedFilter )
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/GdalTools_utils.py", line 402, in getSaveFileName
    return self.getDialog(parent, caption, QFileDialog.AcceptSave, QFileDialog.AnyFile, filter, selectedFilter, useEncoding)
  File "/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/GdalTools/tools/GdalTools_utils.py", line 351, in getDialog
    dialog.selectNameFilter(selectedFilter[0])
TypeError: QFileDialog.selectNameFilter(QString): argument 1 has unexpected type 'QPyNullVariant'

Python version:
2.7.5 (default, Aug 25 2013, 00:04:04) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]

QGIS version:
2.2.0-Valmiera Valmiera, 

Python path: ['/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins/processing', '/Applications/QGIS.app/Contents/MacOS/../Resources/python', u'/Users/vidotto/.qgis2/python', u'/Users/vidotto/.qgis2/python/plugins', '/Applications/QGIS.app/Contents/MacOS/../Resources/python/plugins', '/Library/Frameworks/SQLite3.framework/Versions/B/Python/2.7', '/Library/Frameworks/GEOS.framework/Versions/3/Python/2.7', '/Library/Python/2.7/site-packages/numpy-override', '/Library/Python/2.7/site-packages/matplotlib-override', '/Library/Frameworks/GDAL.framework/Versions/1.10/Python/2.7/site-packages', '/Library/Python/2.7/site-packages/PySAL-1.7.0-py2.7.egg', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/Users/vidotto/Library/Python/2.7/lib/python/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', '/Library/Python/2.7/site-packages', '/Library/Python/2.7/site-packages/PIL', '/Applications/QGIS.app/Contents/Resources/python/plugins/fTools/tools']

Cheers

#4 Updated by Rhenriques Henriques about 6 years ago

As Francesco Vidotto said, it was working in previous versions. I do lots of reprojections (warp module) and format transformations. This way I would have noticed if this was not working in previous versions.

Cheers

#5 Updated by Francesco Vidotto about 6 years ago

I have trashed QGIS 2.2 and reinstalled 2.0.1-Dufour (I had to reinstall matplotlib). Now the warp module works like a charm.

#6 Updated by Rhenriques Henriques about 6 years ago

Francesco, I have been able to make 2.2 GDAL modules work by pickup the gdal folder from version 2.0.1 located in "/Applications/QGIS.app/Contents/Resources/python/plugins" (use show package contents over QGIS) and replace by the same folder in 2.2. Of course that modifications have been made since version 2.0.1 and this is not the most elegant way to make modules work but, by now, all that I need is working. We can also conclude that the poblem is certainly in the sintax of the newer GDAL .py modules.

Cheers

#7 Updated by Francesco Vidotto about 6 years ago

Rhenriques, I tried your method and it works! Many thanks

Cheers

#8 Updated by Giovanni Manghi about 6 years ago

  • Priority changed from Normal to Severe/Regression
  • Subject changed from Gdal Modules Errors to Gdal Modules Errors on OsX Mavericks
  • Assignee set to William Kyngesburye
  • Target version changed from Version 2.2 to Version 2.4

#9 Updated by William Kyngesburye about 6 years ago

Could a problem with preferences from previous version. Try trashing org.qgis.QGIS2.plist in your home library/preferences (library is hidden, use Finder Go to folder). If that doesn't do it, also trash ~/.qgis2 folder (also hidden), or at least all contents of it.

#10 Updated by Francesco Vidotto about 6 years ago

William, thanks for your suggestion. Unfortunately this did not solve the problem in my case.

#11 Updated by Rhenriques Henriques about 6 years ago

Hi William

As Francesco said, your suggestion did not solve the problem either. I guess that the problem is in the .py modules code. Anyone checked, for instance,"doWarp.py" line 129? as the error message warns.
Cheers

#12 Updated by Larry Shaffer about 6 years ago

Looks like the return type from QSettings needs to be explicitly defined, to avoid QPyNullVariant from being returned, and that there are incorrect comparison operators used for comparing None.

I can not reproduce the error here on 10.7.5, but try opening up the GdalTools_utils.py file and change the following:

line ~412: return settings.value( "/GdalTools/" + typeName + "FileFilter", u"" )

to

return settings.value( "/GdalTools/" + typeName + "FileFilter", u"", type=str )  # add ', type=str' 

and replace occurences of:

selectedFilter != None

with (correct None comparison operator)

selectedFilter is not None

It looks like defining the QSettings return type and None comparison need updated across all of GdalTools. I can only surmise that Mac OS X builds using the latest PyQt4 versions are the first to exhibit the needed changes.

#13 Updated by Larry Shaffer about 6 years ago

Hi Giovanni,

A GdalTools update could be pushed out via the official plugin repository channel. So, fixes would not have to wait for QGIS 2.4 release.

The only issue (that may be fixed now) is when QGIS 2.4 comes out, the core GdalTools plugin version number needs to be higher than any delta-update versions that may be released in between QGIS releases. This would keep the delta-updates of GdalTools plugin, installed in ~/.qgis2/python/plugins, from being used instead of the new one from core.

I know Victor has done this with Processing plugin a couple of times (at least I'm pretty sure).

#14 Updated by Andrew Mercer about 6 years ago

William Kyngesburye wrote:

Could a problem with preferences from previous version. Try trashing org.qgis.QGIS2.plist in your home library/preferences (library is hidden, use Finder Go to folder). If that doesn't do it, also trash ~/.qgis2 folder (also hidden), or at least all contents of it.

This worked for me on OSX 10.7.5 QGIS 2.2.0 doing "Extraction/Clipper" which failed before trashing the plist

#15 Updated by Rhenriques Henriques about 6 years ago

Hi William

I just trashed everything that I had before, installed your new compilation 2.2.0-7 but the same problem persists. I don't know if this compilation was meant to solve also this issue. Got back my backup with the GDAL copied folder trick, which is working. Thanks for the effort anyway. I'll be paying attention to next compilations.

Cheers

#16 Updated by Larry Shaffer about 6 years ago

Rhenriques,

The edit noted above (add ', type=str' for QSettings value read) has reportedly fixed the issue. Please verify, if you have the time.

If this is the proper fix (which I think it is), it may take some time to fix all of GdalTools and get the tested fix into another QGIS 2.2 Mac installer.

#17 Updated by Rhenriques Henriques about 6 years ago

Hi Larry

In "La Mouche!". The fix solved the issue with the "Save Dialog Box call" in the GDAL modules. I'm uploading the file as a quick fix for MacOSX users.
Replace the file in "/Applications/QGIS.app/Contents/Resources/python/plugins/GdalTools/tools " with this one.
William, is it possible to include the modified file into your compilation?

Cheers

#18 Updated by William Kyngesburye about 6 years ago

I can drop this into an update. Make sure to backport to release 2.2 branch in case there is a source update release.

#19 Updated by Larry Shaffer about 6 years ago

I can work on this tonight, or tomorrow morning. I noticed several other QSettings reads that need attention in GdalTools. I should make an effort to fix all of them.

#20 Updated by rcline - about 6 years ago

The proposed fix of "selectedFilter is not None" does not fix the problem.

When the program fails, selectedFilter contains this list: [NULL]
Line 350
if selectedFilter != None:
should be replaced with
if selectedFilter0 != None and selectedFilter0 != NULL:

Will it ever contain None?

#21 Updated by Larry Shaffer about 6 years ago

rcline - wrote:

The proposed fix of "selectedFilter is not None" does not fix the problem.

That is not the proposed fix. I was merely changing the comparison to the appropriate operator for the None singleton.

The fix is to ensure the returned type from individual QSettings reads are correct, relative to what is expected, as noted above. Otherwise, QPyNullVariant (NULL in your test) might be returned.

Will it ever contain None?

Actually, it is just a question of whether it is an empty string (after above returned type fix), so simply if selectedFilter should be all that's needed, maybe with an additional verification that the filter is actually in the available list.

#22 Updated by Larry Shaffer about 6 years ago

  • Resolution set to fixed/implemented

@rcline I was incorrect. selectedFilter will always be None if the associated keyword arg was undefined (None is its default value). When defined, it should be a string, regardless of whether that string is empty.

Should be fixed with this pull request. Any testers here please verify the fix by downloading the files from the PR. Works for me here on 10.9.2. If it works for others, I'll commit to master and 2.2 branches.

Seems there are other core Python plugins that need this fix, e.g. DB Manager.

#23 Updated by Rhenriques Henriques about 6 years ago

Hi Larry

Tested after the code changing from https://github.com/qgis/QGIS/pull/1230/files and everything seems to be working just fine.
Thank you so much.

Cheers

#24 Updated by Francesco Vidotto about 6 years ago

Hi Larry, I have made the modifications indicated in https://github.com/qgis/QGIS/pull/1230/files and I confirm that everything seems working correctly on 10.9.2.
Many many thanks

Francesco

#25 Updated by Larry Shaffer about 6 years ago

  • Status changed from Feedback to Closed

#26 Updated by Larry Shaffer about 6 years ago

Hi,

This has been committed to master branch (2.3), but not to the release-2_2, because there is some question as to whether devs are allowed to commit to that branch right now.

Also available in: Atom PDF