Bug report #16422

Processing (on Windows): external SAGA does not work anymore

Added by Giovanni Manghi over 2 years ago. Updated over 2 years ago.

Status:Closed
Priority:High
Assignee:Victor Olaya
Category:Processing/SAGA
Affected QGIS version:2.18.5 Regression?:Yes
Operating System:Windows Easy fix?:No
Pull Request or Patch supplied:Yes Resolution:fixed/implemented
Crashes QGIS or corrupts data:No Copied to github as #:24331

Description

If SAGA is not installed with osgeo4w then Processing has always allowed to use and external SAGA installation by configuring the proper path in Processing configurations.

While testing the support for SAGA LTR I therefore downloaded

https://downloads.sourceforge.net/project/saga-gis/SAGA%20-%202.3/SAGA%202.3.1/saga_2.3.1_x64.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fsaga-gis%2Ffiles%2FSAGA%2520-%25202.3%2FSAGA%25202.3.1%2F&ts=1491557320&use_mirror=netcologne

uncompressed and pointed Processing to it.

Processing correctly recognizes SAGA and SAGA version, tools open and are run, but ultimately they fail. This does not happen if used SAGA ltr is the one shipped with osgeo4w. This anyway used to work as expected in the past.

Algorithm Analytical hillshading starting...
io_gdal 0 -TRANSFORM 1 -RESAMPLING 0 -GRIDS "C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\51bd78a192e243b9ba5d841fc68d84f8\\madeiramdt100.sgrd" -FILES "C:\\Users\\qgis\\Desktop\\data\
asters\\madeira_mdt_100.tif" 
ta_lighting "Analytical Hillshading" -ELEVATION "C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\51bd78a192e243b9ba5d841fc68d84f8\\madeiramdt100.sgrd" -METHOD 0 -AZIMUTH 315 -DECLINATION 45 -EXAGGERATION 4 -SHADE "C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\38d16b8c0e654334921f0a6a2ca622a1\\SHADE.sdat" 

C:\\OSGEO4~1\\bin>set SAGA=C:\\Users\\qgis\\Desktop\\saga231

C:\\OSGEO4~1\\bin>set SAGA_MLB=C:\\Users\\qgis\\Desktop\\saga231\\modules

C:\\OSGEO4~1\\bin>PATH=C:\\OSGEO4~1\\apps\\qgis\\bin;C:\\OSGEO4~1\\apps\\Python27\\Scripts;C:\\OSGEO4~1\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\WBem;C:\\OSGEO4~1\\apps\\msys\\bin;C:\\Users\\qgis\\Desktop\\saga231;C:\\Users\\qgis\\Desktop\\saga231\\modules

C:\\OSGEO4~1\\bin>saga_cmd io_gdal 0 -TRANSFORM 1 -RESAMPLING 0 -GRIDS "C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\51bd78a192e243b9ba5d841fc68d84f8\\madeiramdt100.sgrd" -FILES "C:\\Users\\qgis\\Desktop\\data\
asters\\madeira_mdt_100.tif" 
____________________________

##### ## ##### ##
### ### ## ###
### # ## ## #### # ##
### ##### ## # #####
##### # ## ##### # ##
____________________________

SAGA Version: 2.3.1 (64 bit)

76 loaded tool libraries (659 tools):
- climate
- climate_tools
- contrib_perego
- db_odbc
- db_pgsql
- docs_html
- docs_pdf
- garden_3d_viewer
- garden_fractals
- garden_games
- garden_learn_to_program
- garden_webservices
- grid_analysis
- grid_calculus
- grid_calculus_bsl
- grid_filter
- grid_gridding
- grid_spline
- grid_tools
- grid_visualisation
- imagery
- imagery_classification
- imagery_isocluster
- imagery_maxent
- imagery_opencv
- imagery_photogrammetry
- imagery_segmentation
- imagery_svm
- imagery_tools
- io_esri_e00
- io_gps
- io_grid
- io_grid_image
- io_shapes
- io_shapes_dxf
- io_shapes_las
- io_table
- io_virtual
- pj_georeference
- pj_geotrans
- pj_proj4
- pointcloud_tools
- pointcloud_viewer
- shapes_grid
- shapes_lines
- shapes_points
- shapes_polygons
- shapes_tools
- shapes_transect
- sim_cellular_automata
- sim_ecosystems_hugget
- sim_erosion
- sim_fire_spreading
- sim_hydrology
- sim_ihacres
- sim_qm_of_esp
- sim_rivflow
- statistics_grid
- statistics_kriging
- statistics_points
- statistics_regression
- ta_channels
- ta_compound
- ta_hydrology
- ta_lighting
- ta_morphometry
- ta_preprocessor
- ta_profiles
- ta_slope_stability
- table_calculus
- table_tools
- terrain_analysis
- tin_tools
- tin_viewer
- toolchains
- tta_tools

type -h or --help for further information

Error: select a library

C:\\OSGEO4~1\\bin>saga_cmd ta_lighting "Analytical Hillshading" -ELEVATION "C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\51bd78a192e243b9ba5d841fc68d84f8\\madeiramdt100.sgrd" -METHOD 0 -AZIMUTH 315 -DECLINATION 45 -EXAGGERATION 4 -SHADE "C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\38d16b8c0e654334921f0a6a2ca622a1\\SHADE.sdat" 
____________________________

##### ## ##### ##
### ### ## ###
### # ## ## #### # ##
### ##### ## # #####
##### # ## ##### # ##
____________________________

SAGA Version: 2.3.1 (64 bit)

____________________________
library path: C:\\Users\\qgis\\Desktop\\saga231\\modules\\
library name: ta_lighting
library : Lighting, Visibility
tool : Analytical Hillshading
author : O.Conrad, V.Wichmann (c) 2003-2013
processors : 2 [2]
____________________________

Load grid: C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\51bd78a192e243b9ba5d841fc68d84f8\\madeiramdt100.sgrd...
failed

Usage: saga_cmd ta_lighting 0 [-ELEVATION ] [-SHADE ] [-METHOD ] [-AZIMUTH ] [-DECLINATION ] [-EXAGGERATION ] [-SHADOW ] [-NDIRS ] [-RADIUS ]
-ELEVATION: Elevation
Grid (input)
-SHADE: Analytical Hillshading
Grid (output)
-METHOD: Shading Method
Choice
Available Choices:
[0] Standard
[1] Standard (max. 90Degree)
[2] Combined Shading
[3] Ray Tracing
[4] Ambient Occlusion
Default: 0
-AZIMUTH: Azimuth [Degree]
Floating point
Default: 315.000000
-DECLINATION: Height [Degree]
Floating point
Default: 45.000000
-EXAGGERATION: Exaggeration
Floating point
Default: 4.000000
-SHADOW: Shadow
Choice
Available Choices:
[0] slim
[1] fat
Default: 1
-NDIRS: Number of Directions
Integer
Minimum: 2
Default: 8
-RADIUS: Search Radius
Floating point
Minimum: 0.001000
Default: 10.000000
Error: input file [C:\\Users\\qgis\\AppData\\Local\\Temp\\processing926e3fb9581345ea92a61a65c9b211c4\\51bd78a192e243b9ba5d841fc68d84f8\\madeiramdt100.sgrd]
Error: Elevation

C:\\OSGEO4~1\\bin>exit
Converting outputs
Loading resulting layers

The following layers were not correctly generated.
Analytical Hillshading
You can check the log messages to find more information about the execution of the algorithm

Related issues

Related to QGIS Application - Bug report #16423: Broken io_gdal raster import in SAGA LTR package shipped ... Closed 2017-04-07

Associated revisions

Revision b9929d4c
Added by Alexander Bruy over 2 years ago

[processing][needs-docs] always use SAGA shipped with QGIS (refs #16422)

Revision bbcc8725
Added by Alexander Bruy over 2 years ago

Merge pull request #4587 from alexbruy/processing-saga

[processing][needs-docs] always use SAGA shipped with QGIS (refs #16422)

History

#1 Updated by Pedro Venâncio over 2 years ago

For some reason, SAGA 2.3.1 io_gdal module is not loaded using SAGA from
https://downloads.sourceforge.net/project/saga-gis/SAGA%20-%202.3/SAGA%202.3.1/saga_2.3.1_x64.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fsaga-gis%2Ffiles%2FSAGA%2520-%25202.3%2FSAGA%25202.3.1%2F&ts=1491557320&use_mirror=netcologne
and Processing SAGA algs does not work.

The same using OSGeo4W Shell:

C:\\OSGeo4W64\\bin>set SAGA=C:/saga_2.3.1_x64

C:\\OSGeo4W64\\bin>set SAGA_MLB=C:/saga_2.3.1_x64\\modules

C:\\OSGeo4W64\\bin>PATH=C:\\OSGEO4~1\\apps\\Python36\\lib\\site-packages\
umpy\\core;C:\\OSGEO4~1\\apps\\qgis-dev\\bin;C:\\OSGEO4~1\\apps\\grass\\grass-7.2.0\\lib;C:\\OSGEO4~1\\apps\\grass\\grass-7.2.0\\bin;C:\\OSGEO4~1\\apps\\Python36;C:\\OSGEO4~1\\apps\\Python36\\Scripts;C:\\OSGEO4~1\\apps\\qt5\\bin;C:\\OSGEO4~1\\apps\\Python27\\Scripts;C:\\OSGEO4~1\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\WBem;C:\\OSGEO4~1\\apps\\msys\\bin;C:/saga_2.3.1_x64;C:/saga_2.3.1_x64\\modules

C:\\OSGeo4W64\\bin>saga_cmd io_gdal
____________________________

##### ## ##### ##
### ### ## ###
### # ## ## #### # ##
### ##### ## # #####
##### # ## ##### # ##
____________________________

SAGA Version: 2.3.1 (64 bit)

77 loaded tool libraries (670 tools):
- climate
- climate_tools
- contrib_perego
- db_odbc
- db_pgsql
- docs_html
- docs_pdf
- garden_3d_viewer
- garden_fractals
- garden_games
- garden_learn_to_program
- garden_webservices
- grid_analysis
- grid_calculus
- grid_calculus_bsl
- grid_filter
- grid_gridding
- grid_spline
- grid_tools
- grid_visualisation
- imagery
- imagery_classification
- imagery_isocluster
- imagery_maxent
- imagery_opencv
- imagery_photogrammetry
- imagery_segmentation
- imagery_svm
- imagery_tools
- imagery_vigra
- io_esri_e00
- io_gps
- io_grid
- io_grid_image
- io_shapes
- io_shapes_dxf
- io_shapes_las
- io_table
- io_virtual
- pj_georeference
- pj_geotrans
- pj_proj4
- pointcloud_tools
- pointcloud_viewer
- shapes_grid
- shapes_lines
- shapes_points
- shapes_polygons
- shapes_tools
- shapes_transect
- sim_cellular_automata
- sim_ecosystems_hugget
- sim_erosion
- sim_fire_spreading
- sim_hydrology
- sim_ihacres
- sim_qm_of_esp
- sim_rivflow
- statistics_grid
- statistics_kriging
- statistics_points
- statistics_regression
- ta_channels
- ta_compound
- ta_hydrology
- ta_lighting
- ta_morphometry
- ta_preprocessor
- ta_profiles
- ta_slope_stability
- table_calculus
- table_tools
- terrain_analysis
- tin_tools
- tin_viewer
- toolchains
- tta_tools

type -h or --help for further information

Running saga_cmd directly from the folder, it already works:

C:\\saga_2.3.1_x64>saga_cmd
____________________________

##### ## ##### ##
### ### ## ###
### # ## ## #### # ##
### ##### ## # #####
##### # ## ##### # ##
____________________________

SAGA Version: 2.3.1 (64 bit)

78 loaded tool libraries (680 tools):
- climate
- climate_tools
- contrib_perego
- db_odbc
- db_pgsql
- docs_html
- docs_pdf
- garden_3d_viewer
- garden_fractals
- garden_games
- garden_learn_to_program
- garden_webservices
- grid_analysis
- grid_calculus
- grid_calculus_bsl
- grid_filter
- grid_gridding
- grid_spline
- grid_tools
- grid_visualisation
- imagery
- imagery_classification
- imagery_isocluster
- imagery_maxent
- imagery_opencv
- imagery_photogrammetry
- imagery_segmentation
- imagery_svm
- imagery_tools
- imagery_vigra
- io_esri_e00
- io_gdal
- io_gps
- io_grid
- io_grid_image
- io_shapes
- io_shapes_dxf
- io_shapes_las
- io_table
- io_virtual
- pj_georeference
- pj_geotrans
- pj_proj4
- pointcloud_tools
- pointcloud_viewer
- shapes_grid
- shapes_lines
- shapes_points
- shapes_polygons
- shapes_tools
- shapes_transect
- sim_cellular_automata
- sim_ecosystems_hugget
- sim_erosion
- sim_fire_spreading
- sim_hydrology
- sim_ihacres
- sim_qm_of_esp
- sim_rivflow
- statistics_grid
- statistics_kriging
- statistics_points
- statistics_regression
- ta_channels
- ta_compound
- ta_hydrology
- ta_lighting
- ta_morphometry
- ta_preprocessor
- ta_profiles
- ta_slope_stability
- table_calculus
- table_tools
- terrain_analysis
- tin_tools
- tin_viewer
- toolchains
- tta_tools

Error: select a library
type -h or --help for further information

C:\\saga_2.3.1_x64>saga_cmd io_gdal
____________________________

##### ## ##### ##
### ### ## ###
### # ## ## #### # ##
### ##### ## # #####
##### # ## ##### # ##
____________________________

SAGA Version: 2.3.1 (64 bit)

Library:    GDAL/OGR
Category:   Import/Export
File:   C:\\saga_2.3.1_x64\\modules\\io_gdal.dll

Tools:
0    Import Raster
1    Export Raster
2    Export GeoTIFF
3    Import Shapes
4    Export Shapes
5    Export Shapes to KML
6    Import NetCDF
7    Create Raster Catalog from Files
8    Create Raster Catalog from Directory
9    Import TMS Image

Error: select a tool
type -h or --help for further information

#2 Updated by Jürgen Fischer over 2 years ago

Reason for not using the binaries from the saga project is that they have a lot of own dependencies, that are already included in OSGeo4W. The version in OSGeo4W is patched to allow usage of OSGeo4W dependencies (ie. ported to GDAL C-API - patches were submitted and merged upstream; https://sourceforge.net/p/saga-gis/code/merge-requests/3/). Unfortunately the new support for SAGA LTR in processing of 2.18 wasn't anticipated. This was meant to be used with QGIS3 (ie. plenty of time to spot and fix any issues introduced by this).

#3 Updated by Giovanni Manghi over 2 years ago

I also just noticed that saga-ltr installed with osgeo4w misses io_gdal completely

c:\\OSGeo4W64\\apps\\saga-ltr>saga_cmd
____________________

  1. ## ##### ## ### ### ## ### ### # ## ## #### # ## ### ##### ## # #####
  2. # ## ##### # ##
    ____________________

SAGA Version: 2.3.2 (64 bit)

66 loaded tool libraries (585 tools):
- climate_tools
- contrib_perego
- db_odbc
- docs_html
- garden_3d_viewer
- garden_fractals
- garden_games
- garden_learn_to_program
- garden_webservices
- grid_analysis
- grid_calculus
- grid_calculus_bsl
- grid_filter
- grid_gridding
- grid_spline
- grid_tools
- grid_visualisation
- imagery_classification
- imagery_isocluster
- imagery_maxent
- imagery_photogrammetry
- imagery_segmentation
- imagery_svm
- imagery_tools
- io_esri_e00
- io_gps
- io_grid
- io_grid_image
- io_shapes
- io_shapes_dxf
- io_table
- io_virtual
- pj_georeference
- pj_geotrans
- pointcloud_tools
- pointcloud_viewer
- shapes_grid
- shapes_lines
- shapes_points
- shapes_polygons
- shapes_tools
- shapes_transect
- sim_cellular_automata
- sim_ecosystems_hugget
- sim_erosion
- sim_fire_spreading
- sim_hydrology
- sim_ihacres
- sim_qm_of_esp
- sim_rivflow
- statistics_grid
- statistics_kriging
- statistics_points
- statistics_regression
- ta_channels
- ta_compound
- ta_hydrology
- ta_lighting
- ta_morphometry
- ta_preprocessor
- ta_profiles
- ta_slope_stability
- table_calculus
- table_tools
- tin_tools
- tin_viewer

Error: select a library
type -h or --help for further information

#4 Updated by Giovanni Manghi over 2 years ago

Giovanni Manghi wrote:

I also just noticed that saga-ltr installed with osgeo4w misses io_gdal completely

SAGA LTR shipped with osgeo4w seems to work only in osgeo4w shell.

#5 Updated by Jürgen Fischer over 2 years ago

Giovanni Manghi wrote:

Giovanni Manghi wrote:

I also just noticed that saga-ltr installed with osgeo4w misses io_gdal completely

SAGA LTR shipped with seems to work only in osgeo4w shell.

As it's supposed to - why is that a problem?

#6 Updated by Giovanni Manghi over 2 years ago

As it's supposed to - why is that a problem?

that was just a comment to my previous one. Like a note/reminder. But anyway your comment made be think about it, and if I must be honest I'm not totally convinced that this is expected. It is at least a change compared to previous SAGA osgeo4w installation, if I'm not wrong.

Returning to the subject of this ticket: if now SAGA that ships with osgeo4w is compiled (and not just a copy of the upstream binaries) then I guess that in processing allowing use external SAGA installations should be disabled/removed(?).

#7 Updated by Giovanni Manghi over 2 years ago

  • Regression? set to Yes

#8 Updated by Giovanni Manghi over 2 years ago

  • Priority changed from Severe/Regression to High

#9 Updated by Giovanni Manghi over 2 years ago

  • Easy fix? set to No

#10 Updated by Alexander Bruy over 2 years ago

  • Pull Request or Patch supplied changed from No to Yes
  • Description updated (diff)

Here is PR to remove support for external SAGA https://github.com/qgis/QGIS/pull/4587.

Not sure if same should be done for 2.18, as 2.18 still supports different SAGA versions.

#11 Updated by Giovanni Manghi over 2 years ago

Not sure if same should be done for 2.18, as 2.18 still supports different SAGA versions.

I would say do it only for qgis 3, for now at least. QGIS LTR is broken on Windows (packaging issue) and I don't see anyone tacking #16423

#12 Updated by Jürgen Fischer over 2 years ago

  • Description updated (diff)

#13 Updated by Jürgen Fischer over 2 years ago

  • Related to Bug report #16423: Broken io_gdal raster import in SAGA LTR package shipped with OSGeo4W added

#14 Updated by Giovanni Manghi over 2 years ago

  • Resolution set to fixed/implemented
  • Status changed from Open to Closed

Alexander Bruy wrote:

Here is PR to remove support for external SAGA https://github.com/qgis/QGIS/pull/4587.

Not sure if same should be done for 2.18, as 2.18 still supports different SAGA versions.

now that #16423 is fixed this can be applied also to 2.18.*

Also available in: Atom PDF