Bug report #13279
Processing & SAGA
Status: | Closed | ||
---|---|---|---|
Priority: | High | ||
Assignee: | Victor Olaya | ||
Category: | Processing/SAGA | ||
Affected QGIS version: | master | Regression?: | No |
Operating System: | Easy fix?: | No | |
Pull Request or Patch supplied: | No | Resolution: | |
Crashes QGIS or corrupts data: | No | Copied to github as #: | 21339 |
Description
SAGA 2.2.0 is currently on Debian (testing and sid). As a result, a number of commend do not work, e.g.:
2015-08-28T14:22:11 0 SAGA execution commands io_gdal -TRANSFORM -INTERPOL 0 -GRIDS "/tmp/processing/c97eff6cecfa4d57ab70373762af8c64/dtm.sgrd" -FILES "/home/paolo/Documents/Didattica/Corsi_Formazione_Faunalia/QGIS/QGIS_data/dtm.tif" ta_morphometry "Slope, Aspect, Curvature" -ELEVATION "/tmp/processing/c97eff6cecfa4d57ab70373762af8c64/dtm.sgrd" -METHOD 6 -UNIT_SLOPE 1 -UNIT_ASPECT 1 -SLOPE "/tmp/processing/603f7f0a7b254a94830e2b22f2f3ee17/SLOPE.sdat" -ASPECT "/tmp/processing/90fd8d1c501748d1bb8b5a097ca8063b/ASPECT.sdat" -C_GENE "/tmp/processing/b9631b34c537485f8cfda97691c8510c/CGENE.sdat" -C_PLAN "/tmp/processing/254e029f993f4a6aa22607ab48d613da/CPLAN.sdat" -C_PROF "/tmp/processing/74d209817d3f421991325581eaa4f530/CPROF.sdat" -C_TANG "/tmp/processing/b9843ee4dbeb4df4a79338363be582b0/CTANG.sdat" -C_LONG "/tmp/processing/38ff7d18c92743d3af99c53ec29a2b61/CLONG.sdat" -C_CROS "/tmp/processing/b1efc31b658742a19640e67f7b47d84a/CCROS.sdat" -C_MINI "/tmp/processing/3b0079a261934cd9a30d1e4f764a67ec/CMINI.sdat" -C_MAXI "/tmp/processing/347e1e2bd02744419fa4a8f7e8c16d02/CMAXI.sdat" -C_TOTA "/tmp/processing/683095d37fb34739b1957c09781e5fdc/CTOTA.sdat" -C_ROTO "/tmp/processing/dd85022b267b43cdac433ae9f11be8fe/CROTO.sdat" 2015-08-28T14:22:12 0 Risultato della console di esecuzione di SAGA Error: select a tool ____________________________ ##### ## ##### ## ### ### ## ### ### # ## ## #### # ## ### ##### ## # ##### ##### # ## ##### # ## ____________________________ SAGA Version: 2.2.0 tools: 0 - GDAL: Import Raster 1 - GDAL: Export Raster 2 - GDAL: Export Raster to GeoTIFF 3 - OGR: Import Vector Data 4 - OGR: Export Vector Data 5 - OGR: Export Shapes to KML 6 - GDAL: Import NetCDF type -h or --help for further information Error: Grid file could not be opened. Error: input file [/tmp/processing/c97eff6cecfa4d57ab70373762af8c64/dtm.sgrd] Error: Elevation
Related issues
Associated revisions
[processing]fixed SAGA when using a version newer than the latest supported one
fixes #13279
History
#1 Updated by Giovanni Manghi about 9 years ago
- Status changed from Open to Feedback
On Windows the above 2.2.0 module works fine. My test under Linux is inconclusive because SAGA 2.2.0 io_gdal seg faults on Ubuntu.
Do you have any other evidence that already existing modules have parameters names changed?
#2 Updated by Paolo Cavallini about 9 years ago
Tried a few more tools, I always get:
Error: select a tool
Chengelog is here:
#3 Updated by Paolo Cavallini about 9 years ago
Here one can find the diff in interfaces:
SAGA 2.2.0
https://sourceforge.net/p/saga-gis/wiki/Changelog%202.2.0/attachment/saga_cmd_interface_v2_2_0.txt
SAGA 2.1.4
https://sourceforge.net/p/saga-gis/wiki/Changelog%202.1.4/attachment/saga_cmd_interface_v2_1_4.txt
So one can simply do a diff between both files to see the changes. This also works with files created for older SAGA versions, e.g. in case you have skipped SAGA 2.1.4.
Thanks Volker for pointing it out.
#4 Updated by Giovanni Manghi about 9 years ago
So one can simply do a diff between both files to see the changes.
there are a few changes that will need to be addressed, but the above module (ta_morphometry "Slope, Aspect, Curvature") is identical in 2.1.4 and 2.2.0. As you get the same error for many other modules and as it generally seems to work (tested just on Windows for now, but haven't found yet a 2.2 module that does not work with the 2.1.4 descriptions) it seems anyway that you may have a local issue.
#5 Updated by Paolo Cavallini about 9 years ago
Agreed, thanks for pointing this out.
My system is all from standard packages, a local issue is unlikely, or anyway unexpected.
Does anyone have the same?
#6 Updated by Paolo Cavallini about 9 years ago
The options:
`io_gdal -TRANSFORM `
should be replaced by:
`io_gdal 0 -TRANSFORM 1`
so the import does not fail.
Thanks Giovanni for spotting this.
#7 Updated by Paolo Cavallini about 9 years ago
- Tag set to easy
#8 Updated by Paolo Cavallini about 9 years ago
Basic problem fixed with 72665e0. More comprehensive solution needed.
#9 Updated by Victor Olaya about 9 years ago
If this patch fixes the problem and 2.2.0 works fine, i think this can be closed, considering that we have 2.2.0 now as a supported version in the code
#10 Updated by Paolo Cavallini about 9 years ago
Works well here, I leave it open for further feedback.
#11 Updated by Jeff G about 9 years ago
I applied this patch on my 2.8.1 QGIS installation on Linux Mint, using SAGA 2.2.0. It solved the problem for me. (Thank you Paolo!)
#12 Updated by Paolo Cavallini about 9 years ago
Thank Victor for this.
Unfortunately now (saga 2.2.1) I can see only one (!) saga alg. From the log:
...
2015-09-30T21:19:25 2 Could not open SAGA algorithm:
/usr/share/qgis/python/plugins/processing/algs/saga/description/2.1.4/FragmentationClassesfromDensityandConnectivity.txt
'2.2.1'
2015-09-30T21:19:25 2 Could not open SAGA algorithm:
/usr/share/qgis/python/plugins/processing/algs/saga/description/2.1.4/FlatDetection.txt
'2.2.1'
#13 Updated by Victor Olaya about 9 years ago
- Status changed from Feedback to Closed
Fixed in changeset a9dab06a6ada0595802e2dcacbe5835f3d957a37.
#14 Updated by Paolo Cavallini almost 9 years ago
- Status changed from Closed to Reopened
- Affected QGIS version changed from 2.10.1 to master
2.2.2 is out. Have to check whether there is some API changes.
Cannot find clear info here:
http://sourceforge.net/p/saga-gis/wiki/SAGA%20Release%20Information/
#15 Updated by Paolo Cavallini almost 9 years ago
- File saga_diff_2_2_0_vs_2_2_2.txt added
Diff in command syntax between 2.2.0 and 2.2.2 attached here.
#16 Updated by Paolo Cavallini almost 9 years ago
Steps to reproduce also for other versions:
wget http://sourceforge.net/p/saga-gis/wiki/Changelog%202.2.2/attachment/saga_cmd_interface_v2_2_2.txt
wget http://sourceforge.net/p/saga-gis/wiki/Changelog%202.2.0/attachment/saga_cmd_interface_v2_2_0.txt
diff saga_cmd_interface_v2_2_2.txt saga_cmd_interface_v2_2_0.txt > saga_diff_2_2_0_vs_2_2_2.txt
#17 Updated by Paolo Cavallini almost 9 years ago
- Subject changed from Please upgrade Processing to SAGA 2.2.0 to Please upgrade Processing to SAGA 2.2.x
#18 Updated by Victor Olaya almost 9 years ago
- File algdiffs.txt added
Attached you can find a processed version of the diff file, showing only those algorithms with differences
I added a new folder with 2.2.2 descriptions in the QGIS repo, so if anyone wants to start applying those changes, now you can. I suggest leaving this issue open until all changes have been applied
Here's is the scritp that i used, just for the record. I added some manual editing in the end to separate new algs from modified ones
with open("diff.txt") as f: lines = f.readlines() lines = [line for line in lines if "Usage" in line] for old, new in zip(lines,lines[1:])[::2]: old = old.replace("[","").replace("]","")[1:] new = new.replace("[","").replace("]","")[1:] if old != new: print "---\ %s%s" %(old,new)
Thanks to the SAGA crew for making this so hard to mantain!
#19 Updated by Paolo Cavallini over 8 years ago
- Status changed from Reopened to Feedback
Seems fixed now, right?
#20 Updated by Alexander Bruy over 8 years ago
No, thanks to the SAGA devs. In SAGA 2.2.3 there are some commands with changed parameters, see #14300
#21 Updated by Giovanni Manghi over 8 years ago
I still find modules description for 2.2.0 (and by consequence on 2.2.2 and 2.2.3) that are wrong (just made a pull request for "resampling") because they were not changed when the files were copied from 2.1.*
Is there a way we can check what is the actual status in master?
#22 Updated by Paolo Cavallini over 8 years ago
Thanks a lot Giovanni for this work!
In the meantime, saga 2.2.4 is out, so probably there will be more to fix.
#23 Updated by Giovanni Manghi over 8 years ago
Paolo Cavallini wrote:
In the meantime, saga 2.2.4 is out, so probably there will be more to fix.
this is getting ridiculous, we should really stick to one SAGA version and eventually update only for LTR releases.
Meanwhile I would really appreciate if someone is able to easily check what
- modules we still miss to add in 2.2.0 and 2.2.2 (and now 2.2.4). Yesterday I realized we were missing 5 tools from the shapes_polygons group. I already added the new ones in 2.2.3.
- modules in 2.2.* that we still need to update (since we copied the description files from 2.1.*, and is not unusual still to find broken tools, see the above comment about "resampling").
#24 Updated by Paolo Cavallini over 8 years ago
Agreed, it's really a PITA. Unfortunately, the upgrade cycle for Linux distros at least is outside our control.
#25 Updated by Giovanni Manghi over 8 years ago
Paolo Cavallini wrote:
Agreed, it's really a PITA. Unfortunately, the upgrade cycle for Linux distros at least is outside our control.
Honestly as a Linux user I don't care about this specific problem. For example Linux users have virtually disappeared from trainings, and I would prefer much more to be able to rely 100% on a specific version of SAGA on Windows (that is what the vast majority of users use). Linux users can always be instructed on how to get/compile a specific version of SAGA.
#26 Updated by Paolo Cavallini over 8 years ago
This would be discriminatory, and obviously against our policy.
One possible solution would be to embed a specific saga version, to be merged from upstream when appropriate.
The whole issue should be discussed in the ML or as a QEP however.
#27 Updated by Giovanni Manghi over 8 years ago
Paolo Cavallini wrote:
This would be discriminatory, and obviously against our policy.
it would be a practical decision, not a discriminatory one.
Because otherwise among ~240 (and growing) tools we will have always a good chunk broken, and shipping at each release broken tools is admittedly very bad for the users and qgis image.
If the project want to task me to keep the descriptions files up to date at each saga release I'm available.
One possible solution would be to embed a specific saga version, to be merged from upstream when appropriate.
this would work for me.
#28 Updated by Alexander Bruy over 8 years ago
If SAGA devs don't care about their users its their problem, not ours. Maybe users will move to another tools, e.g. GRASS and OTB which are much more stable and SAGA will die in neareast future :-).
I think that we should not add support for new SAGA versions without real need. Our priority should be average users, not early adopters. Maybe I'm wrong, but new SAGA version appears in repositories, espesially stable repositories, with delay, not in the same day as it was released. So there is no harm in providing support only for version from stable repositories. We already do this with lot of dependencies, e.g. minimum supported Qt version is 4.8.0 not latest 4.8.6 and so on. Why not use same approach with SAGA? Declare support for version, available for example in Debian Stable (or another repo, we use as minimal supported version), and make sure it works. All other versions may be supported but without any warranty.
Also Victor and me discussed SAGA support recently. Maybe we can implement something like "real-time" description generation for SAGA algs using its Python API. But this will work only for some algs, because many single SAGA algs should be splitted into two or more Processing algs and also there are some differences in parameters handling (e.g. extent passed to SAGA differently). So such automated generator is not a "silver bullet", a lot of manual work will be needed. So not much difference with current situation.
#29 Updated by Giovanni Manghi over 8 years ago
I think that we should not add support for new SAGA versions without real need. Our priority should be average users, not early adopters. Maybe I'm wrong, but new SAGA version appears in repositories, especially stable repositories, with delay, not in the same day as it was released. So there is no harm in providing support only for version from stable repositories.
agree
#30 Updated by Giovanni Manghi over 8 years ago
Can we close this ticket?
#31 Updated by Paolo Cavallini over 8 years ago
IMHO making SAGA usable for all is still an open issue, although the solution may not be a simple one.
Please keep it open until we have a proper solution.
#32 Updated by Giovanni Manghi over 8 years ago
Paolo Cavallini wrote:
IMHO making SAGA usable for all is still an open issue, although the solution may not be a simple one.
Please keep it open until we have a proper solution.
what this has to do with the description/subject of this ticket?
#33 Updated by Giovanni Manghi over 8 years ago
- Priority changed from Severe/Regression to High
- Subject changed from Please upgrade Processing to SAGA 2.2.x to Processing & SAGA
- Status changed from Feedback to Open
- Tag deleted (
easy)
This ticket is not a matter anymore of updating SAGA to 2.2.2 (done) or whatever else SAGA version. Now is about what to do with SAGA in QGIS: recently a lot of new SAGA releases have been published (6 releases since october) and they not only contain new algorithms, that is fine, but also changes in parameter names of existing tools (like recently a fundamental parameter in io_gdal) that is not so cool (broken scripts and models the first side effect).
#34 Updated by Alexander Bruy over 8 years ago
According to https://sourceforge.net/p/saga-gis/mailman/message/35146184/ SAGA 2.3 will support reading data via GDAL.
#35 Updated by Alexander Bruy almost 8 years ago
- Status changed from Open to Closed
Closed as not relevant.