Bug report #13061

QGIS crashes on close if data was read through GDAL in Python Console

Added by Radoslaw Guzinski over 4 years ago. Updated over 2 years ago.

Status:Closed
Priority:High
Assignee:-
Category:PyQGIS Console
Affected QGIS version:2.18.11 Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed/implemented
Crashes QGIS or corrupts data:Yes Copied to github as #:21131

Description

To reproduce start QGIS, run the following in Python Console and then close QGIS:

from osgeo import gdal
inFilename = r"C:\\test.tif" 
myImg = gdal.Open(inFilename, gdal.GA_ReadOnly)
myBand = myImg.GetRasterBand(1)
myData = myBand.ReadAsArray()
myData = None
myBand = None
myImg = None

The crash happens around 90% of the time. The same happens when the code is run in a Processing script. If the data is not read (i.e. "myData = myBand.ReadAsArray()" is removed) the crash does not occur.

Tested on 64 OSGeo4W installation of QGIS 2.8.2, 2.10 and nightly (f3fe29d) on a number of Windows 7 machines.

History

#1 Updated by Giovanni Manghi over 4 years ago

  • Category set to PyQGIS Console
  • Priority changed from Normal to High
  • Crashes QGIS or corrupts data changed from No to Yes

#2 Updated by Casper Børgesen about 4 years ago

I can verify that it still exists in MASTER 824fd7b

I found the issue seperately when instead of calling ReadAsArray(), I called WriteArray(...) in my python plugin. Without calling WriteArray(...) no minidump is created on exit.

It happens in Windows 7, x64 running both OSGeo4W x64 MASTER and stand alone installer v. 2.12 x64 - both versions affected.

#3 Updated by Leonard Kong almost 4 years ago

I can verify that it still exists with QGIS 2.12. My plugin crashes in exactly the same location.

#4 Updated by Leonard Kong almost 4 years ago

This is a blocking issue for me, . . . I have been thinking that this is a gdal problem and I think most people looking at this think the same, . . . perhaps it's not. Perhaps this is a QGIS problem in the function that is called on exit. Maybe there is an exception in that function that needs to be caught. Is that a python function or is it a QGIS c++ call at that location?

#5 Updated by Giovanni Manghi almost 4 years ago

  • Status changed from Open to Feedback

Leonard Kong wrote:

This is a blocking issue for me, . . . I have been thinking that this is a gdal problem and I think most people looking at this think the same, . . . perhaps it's not. Perhaps this is a QGIS problem in the function that is called on exit. Maybe there is an exception in that function that needs to be caught. Is that a python function or is it a QGIS c++ call at that location?

please raise this issue in the developers mailing list.

#6 Updated by Sebastian Dietrich almost 4 years ago

I can reproduce the crash using the mentioned Python code.

Under Windows, GDAL is quite picky with its memory, because it uses its own heap. Also ownership is an issue, e.g. delete myBand; would be an error in C++. But we assume this is all properly taken care of in the python bindings.

As a workaround you can use ReadRaster() and WriteRaster(). The following code does not crash QGIS on exit under Windows:

from osgeo import gdal
inFilename = r"C:\\test.tif" 
myImg = gdal.Open(inFilename, gdal.GA_ReadOnly)
myBand = myImg.GetRasterBand(1)

scanline = myBand.ReadRaster( 0, 0, myBand.XSize, 1, myBand.XSize, 1, gdal.GDT_Byte )
import array
array_of_bytes = array.array('B', scanline)

myBand = None
myImg = None

#7 Updated by Sandro Santilli almost 4 years ago

Could you please try with the code in https://github.com/qgis/QGIS/pull/2755 ?
I cannot reproduce due to some problem with the python binding of gdal on my local system

#8 Updated by Sandro Santilli almost 4 years ago

I tried myself, this issue cannot be reproduced (for me) by using the code in https://github.com/qgis/QGIS/pull/2755

#9 Updated by Sandro Santilli almost 4 years ago

I actually cannot reproduce this with master branch either (as of b6c714ac20ea595ebf3136ae8facae0520acbe5f)
Is the .tif file to use irrelevant ? I'm using one from the testsuite:

from osgeo import gdal                                                          
inFilename = r"./output/python/plugins/processing/tests/data/raster.tif"        
myImg = gdal.Open(inFilename, gdal.GA_ReadOnly)                                 
myBand = myImg.GetRasterBand(1)                                                 
myData = myBand.ReadAsArray()                                                   
myData = None                                                                   
myBand = None                                                                   
myImg = None

#10 Updated by Radoslaw Guzinski almost 4 years ago

I just tried with 2.12.3 and the latest nightly build (up to commit 0a64c16) with raster.tif and the issue is still present in both cases. It's a clean install on Windows 7 using OSGeo4W 64 bit installer.

I don't have the build environment set up so I haven't tested the pull request but I will try to give it a go.

#11 Updated by Casper Børgesen almost 4 years ago

Radoslaw Guzinski wrote:

I just tried with 2.12.3 and the latest nightly build (up to commit 0a64c16) with raster.tif and the issue is still present in both cases. It's a clean install on Windows 7 using OSGeo4W 64 bit installer.

I have the same result on the latest nightly build (0a64c16).

#12 Updated by Giovanni Manghi over 3 years ago

  • Status changed from Feedback to Open

#13 Updated by Giovanni Manghi over 3 years ago

this is still an issue on the latest master (tested on windows).

#14 Updated by Matthias Kuhn over 3 years ago

Can't reproduce on latest master / Fedora.

#15 Updated by Nyall Dawson over 3 years ago

Matthias - I've only been able to reproduce this under Windows.

#16 Updated by Giovanni Manghi about 3 years ago

see also: #14648

it affects also a few Processing tools.

#17 Updated by Giovanni Manghi over 2 years ago

  • Status changed from Open to Feedback
  • Target version set to Version 2.18
  • Affected QGIS version changed from master to 2.18.4

This still affects 2.18.4, but what about qgis3 master?

#18 Updated by Giovanni Manghi over 2 years ago

  • Status changed from Feedback to Open
  • Description updated (diff)
  • Affected QGIS version changed from 2.18.4 to 2.18.7

#19 Updated by Giovanni Manghi over 2 years ago

  • Regression? set to No
  • Easy fix? set to No

#20 Updated by Radoslaw Guzinski over 2 years ago

Just tested with GDAL 2.2.0 on Windows: the issue is still present in 2.18.8 but I couldn't reproduce it with latest 2.99 nightly (d55fa22).

#21 Updated by Andreas Ekstrand over 2 years ago

I'm also having this problem with gdal_merge.py from QGIS 2.18.11 64-bit on Win10. It's a real show-stopper for me since it's blocking my scripting.

#22 Updated by Giovanni Manghi over 2 years ago

  • Affected QGIS version changed from 2.18.7 to 2.18.11

Andreas Ekstrand wrote:

I'm also having this problem with gdal_merge.py from QGIS 2.18.11 64-bit on Win10. It's a real show-stopper for me since it's blocking my scripting.

Please raise your concerns about this long standing issue in the developers mailing list, at some point someone will need to have a look at it or support the work to have a look at it and fix it.

#23 Updated by Jürgen Fischer over 2 years ago

Probably a duplicate of OSGeo4W #466

#24 Updated by Giovanni Manghi over 2 years ago

  • Resolution set to fixed/implemented
  • Status changed from Open to Closed

Jürgen Fischer wrote:

Probably a duplicate of OSGeo4W #466

after the osgeo4w/gdal update no more crashes. Also there were a few plugins (like Lecos) that I believe were making qgis crash on exit that now do not make it crash anymore.

#25 Updated by Radoslaw Guzinski over 2 years ago

I confirm that the issue was fixed by the gdal update. Thanks Jurgen!

Also available in: Atom PDF