Bug report #18082

using "v.external" as Processing option (to handle input vectors) is not working

Added by Alister Hood over 2 years ago. Updated over 1 year ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Processing/GRASS
Affected QGIS version:3.4.3 Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:worksforme
Crashes QGIS or corrupts data:No Copied to github as #:25978

Description

...
C:\OSGeo4W64\bin>v.to.rast input=vector_5a821b37cb30a3 type="point,line,area" use="attr" attribute_column=ELEV value=1 memory=300 output=output85773b68dae446d8a8b60ca718ce6adb --overwrite 
WARNING: Missing key column name 
ERROR: Column <ELEV> not found 

C:\OSGeo4W64\bin>g.region raster=output85773b68dae446d8a8b60ca718ce6adb 
ERROR: Raster map <output85773b68dae446d8a8b60ca718ce6adb> not found
...

Please see the attached output and the batch job files.
Comparing the grass_batch_job.* files, the obvious difference is that on linux it is using v.in.ogr but on Windows it is using v.external (my linux build hasn't been updated for several days, but I know the problem on Windows pre-dates it).
This has been broken in master for a long time, but I thought there used to be a slightly different error message - something about no features being selected.
Note that the v.to.rast algorithm in master replaces two separate algorithms in QGIS 2.x (which worked on Windows).

grass_batch_job.cmd (844 Bytes) Alister Hood, 2018-02-13 12:20 AM

grass_batch_job.sh Magnifier (836 Bytes) Alister Hood, 2018-02-13 12:20 AM

log windows.txt Magnifier (4.37 KB) Alister Hood, 2018-02-13 12:20 AM

log linux.txt Magnifier (4.86 KB) Alister Hood, 2018-02-13 12:20 AM

History

#1 Updated by Alister Hood over 2 years ago

Ah, I see that in the preferences there is an option for processing to use v.external, and if I turn that off then it works.

There is a very old mention of a "bug in this module" (v.external) at https://sourceforge.net/p/qgis/mailman/qgis-user/?viewmonth=200602, which might be related. The "[1]" link provided does not work, and I haven't had a good look to see if I can find the reference anywhere.

I also haven't investigated more widely to see if v.external always fails on Windows, or if it is also used in algorithms that run successfully.

#2 Updated by Alister Hood over 2 years ago

  • Subject changed from v.to.rast does not work on Windows in QGIS3 (works on linux) to v.to.rast does not work on Windows in QGIS3 using v.external
  • Operating System deleted (Windows)

The v.external option was obviously disabled in my linux testing - if I enable it then it fails there too, so this isn't a Windows issue.

Is v.external an option because it is expected to cause problems sometimes?
If so then this ticket could probably be closed, although it would probably cause less headaches for everybody if the option was removed (or hidden, with the problems well documented)!

#3 Updated by Alister Hood over 2 years ago

There is another issue with v.to.rast in master on Windows:
When using it in a processing model, if you specify the "GRASS GIS 7 region cellsize" (under "advanced parameters"), the cellsize is ignored on Windows.
It is not ignored on Linux, or if using the algorithm directly (not in a model).

But I have just realised that the latest master build in osgeo4w is 5 days old, so there's a chance it has been fixed and that's why it tested ok on linux. When there is a newer build to test on Windows I'll check and file another ticket for it if the issue is still present.

#4 Updated by Giovanni Manghi over 2 years ago

  • Priority changed from Normal to High

#5 Updated by Alister Hood about 2 years ago

There is another issue with v.to.rast in master on Windows:
When using it in a processing model, if you specify the "GRASS GIS 7 region cellsize" (under "advanced parameters"), the cellsize is ignored on Windows.

For the record, this appears to have been fixed.

#6 Updated by Nyall Dawson about 2 years ago

  • Status changed from Open to Closed
  • Resolution set to fixed/implemented

#7 Updated by Alister Hood about 2 years ago

  • Status changed from Closed to Reopened
  • Resolution deleted (fixed/implemented)

Sorry Nyall, but unless you've worked on a fix for this in current master I think you misinterpreted my last comment.

This ticket is still valid, at least to QGIS 3.2. What appears to be fixed is the additional issue that I described in comment 3, and said I would post as a new ticket if I found it was still present.

#8 Updated by Nyall Dawson about 2 years ago

Using v.external defaults to False now though - so you're only experiencing this because you're using a non-default, (and I'd say experimental) setting

#9 Updated by Alister Hood about 2 years ago

Ah, good.
I wasn't sure what was default before, anyway. The trouble is that someone looking through the settings can enable something like this and then later when they have forgotten about it they encounter an error that seems weird and hard to troubleshoot.

As long as there isn't a real fix, I think it would almost be worth making processing print a warning when running GRASS algorithms with this option enabled. Would that be feasible?

#10 Updated by Nyall Dawson about 2 years ago

  • Resolution set to up/downstream

Ultimately it's an upstream issue, and needs filing and resolving in grass itself.

But maybe we should rename that option in QGIS to include "(experimental)"?

#11 Updated by Alister Hood about 2 years ago

Definitely, unless you use stronger wording.
FWIW, at the moment this is the only upstream ticket I can see that might be related: https://trac.osgeo.org/grass/ticket/981

#12 Updated by Paolo Cavallini almost 2 years ago

Could you please check again on current release?
Thanks.

#13 Updated by Paolo Cavallini almost 2 years ago

  • Assignee deleted (Victor Olaya)

#14 Updated by Giovanni Manghi almost 2 years ago

  • Status changed from Reopened to Feedback

Paolo Cavallini wrote:

Could you please check again on current release?
Thanks.

Please change status to "feedback" when needed.

#15 Updated by Alister Hood almost 2 years ago

  • Status changed from Feedback to Open

Paolo Cavallini wrote:

Could you please check again on current release?
Thanks.

V.external option still causes algorithm to fail on 3.4.3
The ui still does not indicate that this feature is broken or "experimental".

#16 Updated by Giovanni Manghi over 1 year ago

  • Priority changed from High to Normal
  • Affected QGIS version changed from master to 3.4.3
  • Regression? changed from Yes to No
  • Resolution deleted (up/downstream)
  • Subject changed from v.to.rast does not work on Windows in QGIS3 using v.external to using "v.external" as Processing option (to handle input vectors) is not working
  • Status changed from Open to Feedback

Is also not working on linux. Also I'm unsure why this is an upstream issue, is v.external not working on native GRASS?

#17 Updated by Alexander Bruy over 1 year ago

Please note that by default using v.external is turned off and instead v.in.org is used. Also seems this not Processing issue, but bug in the GRASS module. Can we close this?

#18 Updated by Alister Hood over 1 year ago

No, please don't. It is very problematic to have even an optional feature that is so broken and has no warning and no way for the user to know what the problem is if they happen to have enabled the feature.

I guess a warning in the preferences would be a relatively simple edit, so I will look into it in the next few days. BTW I think "experimental" is definitely not strong enough wording, and I'm still inclined to think it would be better to disable the feature completely until it does work.

#19 Updated by Nyall Dawson over 1 year ago

+1 to totally removing this option and disabling the offending code. Better safe then sorry...

#20 Updated by Alister Hood over 1 year ago

  • Status changed from Feedback to Closed
  • Resolution set to worksforme

Testing today using both latest Osgeo4W and Arch Linux it seems to work!
I'm not sure what might have changed to fix it: QGIS, Grass, Gdal or something else. Hopefully it isn't specific to certain PCs or certain input files; of course I'll reopen if I see it again.
The only way I could get v.torast to fail was using a zipped shapefile as the input layer, but that is a different issue as it fails regardless of whether or not v.external is enabled.

Also available in: Atom PDF