Bug report #20895

Raster Merge Failure

Added by Corey Rueckheim over 1 year ago. Updated over 1 year ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Build/Install
Affected QGIS version:3.4.3 Regression?:No
Operating System:Windows 10 Easy fix?:No
Pull Request or Patch supplied:No Resolution:up/downstream
Crashes QGIS or corrupts data:No Copied to github as #:28714

Description

After loading two geotiff DEMs into separate layers, then attempting a Raster --> Miscellaneous --> Merge... of the two layers, it takes only a few seconds to fail, resulting in a new layer of the correct size, but with no elevation data. Near the beginning of the log it says "RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa" and near the end it says "No module named _gdal_array". This worked fine in the previous version of QGIS I had and is a critical part of my work.

I thought maybe I needed to delete my older version of QGIS, so ultimately I deleted all QGIS files I could find throughout my system and then reinstalled, but that didn't help.

Here is the entire log:

Processing algorithm…
Algorithm 'Merge' starting…
Input parameters:
{ 'DATA_TYPE' : 5, 'INPUT' : ['D:/SteamLibrary/steamapps/common/assettocorsa/content/tracks/tuttleridge/resources/LIDAR/Monroe/monroe','D:/SteamLibrary/steamapps/common/assettocorsa/content/tracks/tuttleridge/resources/LIDAR/Vernon/vernon'], 'NODATA_INPUT' : None, 'NODATA_OUTPUT' : None, 'OPTIONS' : '', 'OUTPUT' : 'C:/Users/corey/AppData/Local/Temp/processing_6399c68c009946eeb88963854255a4cd/1feb7c0f4e8542e8a6583730da4eebcb/OUTPUT.tif', 'PCT' : False, 'SEPARATE' : False }

GDAL command:
cmd.exe /C gdal_merge.bat -ot Float32 -of GTiff -o C:/Users/corey/AppData/Local/Temp/processing_6399c68c009946eeb88963854255a4cd/1feb7c0f4e8542e8a6583730da4eebcb/OUTPUT.tif --optfile C:/Users/corey/AppData/Local/Temp/processing_6399c68c009946eeb88963854255a4cd\mergeInputFiles.txt
GDAL command output:
0RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa

Traceback (most recent call last):

File "C:\QGIS\bin\gdal_merge.py", line 611, in <module>

sys.exit(main())

File "C:\QGIS\bin\gdal_merge.py", line 596, in main

fi.copy_into(t_fh, band, band, nodata)

File "C:\QGIS\bin\gdal_merge.py", line 343, in copy_into

nodata_arg)

File "C:\QGIS\bin\gdal_merge.py", line 124, in raster_copy

m_band)

File "C:\QGIS\bin\gdal_merge.py", line 180, in raster_copy_with_mask

t_xsize, t_ysize)

File "C:\QGIS\apps\Python27\lib\site-packages\osgeo\gdal.py", line 2635, in ReadAsArray

from osgeo import gdalnumeric

File "C:\QGIS\apps\Python27\lib\site-packages\osgeo\gdalnumeric.py", line 2, in <module>

from osgeo.gdal_array import *

File "C:\QGIS\apps\Python27\lib\site-packages\osgeo\gdal_array.py", line 17, in <module>

_gdal_array = swig_import_helper()

File "C:\QGIS\apps\Python27\lib\site-packages\osgeo\gdal_array.py", line 16, in swig_import_helper

return importlib.import_module('_gdal_array')

File "C:\QGIS\apps\Python27\lib\importlib\__init__.py", line 37, in import_module

__import__(name)

ImportError: No module named _gdal_array

Execution completed in 2.28 seconds
Results:
{'OUTPUT': <QgsProcessingOutputLayerDefinition {'sink':C:/Users/corey/AppData/Local/Temp/processing_6399c68c009946eeb88963854255a4cd/1feb7c0f4e8542e8a6583730da4eebcb/OUTPUT.tif, 'createOptions': {'fileEncoding': 'System'}}>}

Loading resulting layers
Algorithm 'Merge' finished

Related issues

Duplicated by QGIS Application - Bug report #20947: Merging DEM Closed 2019-01-09
Duplicated by QGIS Application - Bug report #20977: Merge Raster fails with ImportError: No module named _gda... Closed 2019-01-11
Duplicated by QGIS Application - Bug report #21075: Update to 3.4.3 via OSGeo4W gets wrong numpy Closed 2019-01-23

History

#1 Updated by Gabriel De Luca over 1 year ago

Can be solved by manually uninstalling the numpy module and reinstalling it:

  • Run as Administrator the OSGeo4W.bat file, located at the QGIS 3.4 folder (C:\Program Files\QGIS 3.4 by default), that calls enviroment variables and opens a shell with administrator privileges (needed to make the changes effective for all users)
  • run: python -m pip uninstall numpy, returns Successfully uninstalled numpy-1.12.1+mkl
  • run: python -m pip install numpy, returns Successfully installed numpy-1.15.4
  • close the shell and run the Merge function in QGIS again, it will work fine now.

#2 Updated by Alexander Bruy over 1 year ago

  • Category changed from GDAL Tools to Build/Install
  • Status changed from Open to Feedback
  • Assignee deleted (Corey Rueckheim)

This is not a QGIS issue, your installation of GDAL and related packages (namely numpy) is/was broken. Probably you have installed numpy separately and overwrite package that was included in the QGIS distribution.

#3 Updated by Giovanni Manghi over 1 year ago

Alexander Bruy wrote:

This is not a QGIS issue, your installation of GDAL and related packages (namely numpy) is/was broken. Probably you have installed numpy separately and overwrite package that was included in the QGIS distribution.

agree

#4 Updated by Gabriel De Luca over 1 year ago

Alexander Bruy wrote:

Probably you have installed numpy separately and overwrite package that was included in the QGIS distribution.

That's not my case.

The Merge tool fails after clean installations of QGIS Distributions http://qgis.org/downloads/QGIS-OSGeo4W-3.4.3-1-Setup-x86.exe and http://qgis.org/downloads/QGIS-OSGeo4W-3.4.3-1-Setup-x86_64.exe, on Windows 7 and Windows 10 machines.

The proposed solution is a post-installation manual solution to a failure found in the distribution.

#5 Updated by Nyall Dawson over 1 year ago

This is an osgeo4w packaging issue - it needs filing at https://trac.osgeo.org/osgeo4w/newticket

#6 Updated by Gabriel De Luca over 1 year ago

Nyall Dawson wrote:

This is an osgeo4w packaging issue - it needs filing at https://trac.osgeo.org/osgeo4w/newticket

Done, at https://trac.osgeo.org/osgeo4w/ticket/588.

#7 Updated by Jürgen Fischer over 1 year ago

  • Status changed from Feedback to Closed
  • Resolution set to up/downstream

#8 Updated by Jürgen Fischer over 1 year ago

#9 Updated by Jürgen Fischer over 1 year ago

  • Duplicated by Bug report #20977: Merge Raster fails with ImportError: No module named _gdal_array added

#10 Updated by Jürgen Fischer over 1 year ago

  • Duplicated by Bug report #21075: Update to 3.4.3 via OSGeo4W gets wrong numpy added

#11 Updated by Loren Amelang over 1 year ago

For anyone who updated QGIS via the OSGeo4W installer and is trying to fix this, as I was after #21075, it appears just using system-wide pip to update numpy does not help. OSGeo4W has its own numpy in its installation folder. (Strangely, that version was 1.12, not the 0xa shown in my error message. System-wide pip saw numpy 1.10... Whatever can that mean?) Run the OSGeo4W.bat file in the OSGeo4W folder as Administrator, and pip it from that console. You should see numpy 1.16 (as of 23jan19) in the OSGeo4W folder now. (It is possible my system-wide install was also necessary, no way to tell now, but it wasn't sufficient.)

#12 Updated by Jürgen Fischer over 1 year ago

Loren Amelang wrote:

For anyone who updated QGIS via the OSGeo4W installer and is trying to fix this, as I was after #21075, it appears just using system-wide pip to update numpy does not help. OSGeo4W has its own numpy in its installation folder. (Strangely, that version was 1.12, not the 0xa shown in my error message. System-wide pip saw numpy 1.10... Whatever can that mean?) Run the OSGeo4W.bat file in the OSGeo4W folder as Administrator, and pip it from that console. You should see numpy 1.16 (as of 23jan19) in the OSGeo4W folder now. (It is possible my system-wide install was also necessary, no way to tell now, but it wasn't sufficient.)

OSGeo4W was already updated.

#13 Updated by Jürgen Fischer over 1 year ago

  • Description updated (diff)

Also available in: Atom PDF