Bug report #13174

Regression: Modeler issue with temp outputs under Windows

Added by Bernd Vogelgesang over 5 years ago. Updated about 5 years ago.

Status:Closed
Priority:Severe/Regression
Assignee:-
Category:Processing/Modeller
Affected QGIS version:2.10.1 Regression?:No
Operating System:Windows Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:21237

Description

Using the GDAL "Dissolve Polygons" in Processing and in a model in Ubuntu runs fine.

Using it under Windows under Processing works too.

Using it in a model under Windows creates some strange path problems:

GDAL command:
cmd.exe /C ogr2ogr.exe C:\\Users\\BERNDV~1\\AppData\\Local\\Temp\\processing\\1a5ff9aa24844ba08fdb3350c62b46e1\\OUTPUTLAYERALGGDALOGRDISSOLVEPOLYGONS2.shp :\\Users\\BERNDV~1\\AppData\\Local\\Temp\\processing\\b6850e80399344a2992ef795a0ae3e98\\OUTPUTLAYERALGQGISFIELDCALCULATOR4.sh OUTPUTLAYERALGQGISFIELDCALCULATOR4 -dialect sqlite -sql "SELECT ST_Union(geometry),*, SUM(AW) AS sum_diss, MIN(AW) AS min_diss, MAX(AW) AS max_diss, AVG(AW) AS avg_diss FROM 'OUTPUTLAYERALGQGISFIELDCALCULATOR4' GROUP BY FLAECHE2" -explodecollections 
GDAL command output:
FAILURE: 
Unable to open datasource `:\\Users\\BERNDV~1\\AppData\\Local\\Temp\\processing\\b6850e80399344a2992ef795a0ae3e98\\OUTPUTLAYERALGQGISFIELDCALCULATOR4.sh' with the following drivers. 
-> OCI 
-> SOSI 
-> ESRI Shapefile 
...

the same algo in Processing:

GDAL command:
cmd.exe /C ogr2ogr.exe C:\\Users\\BERNDV~1\\AppData\\Local\\Temp\\processing\\4b0e510c709e4e0a97402e9d7d4f4bd1\\OUTPUTLAYER.shp C:\\Users\\BERNDV~1\\AppData\\Local\\Temp\\processing\\b6850e80399344a2992ef795a0ae3e98\\OUTPUTLAYERALGQGISFIELDCALCULATOR4.shp OUTPUTLAYERALGQGISFIELDCALCULATOR4 -dialect sqlite -sql "SELECT ST_Union(geometry),*, SUM(AW) AS sum_diss, MIN(AW) AS min_diss, MAX(AW) AS max_diss, AVG(AW) AS avg_diss FROM 'OUTPUTLAYERALGQGISFIELDCALCULATOR4' GROUP BY FLAECHE2" -explodecollections 
GDAL command output:
layernames ignored in combination with -sql

From the output path, the first and the last letters are missing (C:\\ ... .sh p )

Maybe this also occurs with other algos.

Win7, OSGEO4W, fresh simple install 2.10 with processing 2.10.1

p.s. horrible wysiwig editor

Dissolve_Polygons_Bug_Example.zip - Example data (20.5 KB) Bernd Vogelgesang, 2015-08-04 04:33 PM

Associated revisions

Revision 72af56d8
Added by Victor Olaya about 5 years ago

[processing] quote return string when returning uri in ogrConnectionString

Fixes #13174

History

#1 Updated by Jürgen Fischer over 5 years ago

  • Assignee deleted (Giovanni Manghi)

#2 Updated by Giovanni Manghi over 5 years ago

  • Assignee set to Giovanni Manghi

#3 Updated by Giovanni Manghi over 5 years ago

  • Status changed from Open to Feedback

Just tested on both Linux and Windows (osgeo4w, qgis 2.10, Processing 2.10) and works just fine.

Can you please attach sample data? thanks.

#4 Updated by Bernd Vogelgesang over 5 years ago

Hi Giovanni,
I attach a folder with the two necessary shape files and a model which works under Linux, and the other which doesn't under Windows. They might not perfectly be the same cause I had to remodel due to GRASS-differences, but critical part under Windows is the trimmed path, as far as I can see. Just retested, it's really like I posted over here.

#5 Updated by Giovanni Manghi over 5 years ago

  • Status changed from Feedback to Open
  • Operating System set to Windows
  • Assignee deleted (Giovanni Manghi)
  • Subject changed from Modeler: GDAL Dissolve with path problems in Windows to Modeler issue with temp outputs under Windows

Bernd Vogelgesang wrote:

Hi Giovanni,
I attach a folder with the two necessary shape files and a model which works under Linux, and the other which doesn't under Windows. They might not perfectly be the same cause I had to remodel due to GRASS-differences, but critical part under Windows is the trimmed path, as far as I can see. Just retested, it's really like I posted over here.

Hi Bernd,
it works for me under Windows, but only if I choose to save outputs not as temp files. If I leave the outputs spaces blank the resulting path to the Windows temp folder seems very long. So to make a counter test I choose to save manually outputs in the same folder where temp ones are created, and it fails.

Seems to me an issue with the Modeller or Processing/Core, not with the tool.

#6 Updated by Anita Graser over 5 years ago

  • Priority changed from Normal to Severe/Regression

This seems to be a regression between Processing 2.9 and 2.10 where 2.10 fails to run models because it cannot find the intermediate results it produced in a previous step. The model runs perfectly in 2.9.

Bellow you can see an example to reproduce: just chain QGIS buffer and OGR dissolve and it fails because it's looking for a file :

\\Users\\agraser\\AppData\\Local\\Temp\\processing\\7f77fa47bcec4799818d1f5ea41cc8a9\\OUTPUT.sh
(without C: and .sh instead of .shp)

Algorithm dissolve starting...
Prepare algorithm: QGISFIXEDDISTANCEBUFFER_1
Running Fixed distance buffer [1/2]
Parameters: INPUT =C:/Users/agraser/Downloads/qgis_sample_data/shapefiles/popp.shp, DISTANCE =10000, SEGMENTS =5, DISSOLVE =True
Converting outputs
OK. Execution took 67.825 ms (1 outputs).
Prepare algorithm: GDALOGRDISSOLVEPOLYGONS_1
Running Dissolve polygons [2/2]
Parameters: INPUT_LAYER =C:\\Users\\agraser\\AppData\\Local\\Temp\\processing\\7f77fa47bcec4799818d1f5ea41cc8a9\\OUTPUT.shp, GEOMETRY =geometry, FIELD =TYPE, MULTI =False, FIELDS =False, COUNT =False, AREA =False, STATS =False, STATSATT =TYPE, OPTIONS =
GDAL command:
cmd.exe /C ogr2ogr.exe C:\\Users\\agraser\\AppData\\Local\\Temp\\processing\\a44629a1c1f34bc4bf9d26ed974f2bb6\\OUTPUTLAYERALGGDALOGRDISSOLVEPOLYGONS1.shp :\\Users\\agraser\\AppData\\Local\\Temp\\processing\\7f77fa47bcec4799818d1f5ea41cc8a9\\OUTPUT.sh OUTPUT -dialect sqlite -sql "SELECT ST_Union(geometry),TYPE FROM 'OUTPUT' GROUP BY TYPE" -explodecollections
GDAL command output:
FAILURE:
Unable to open datasource `:\\Users\\agraser\\AppData\\Local\\Temp\\processing\\7f77fa47bcec4799818d1f5ea41cc8a9\\OUTPUT.sh' with the following drivers.
-> ESRI Shapefile
-> MapInfo File
-> UK .NTF
-> SDTS
-> TIGER
-> S57
-> DGN
-> VRT
-> REC
-> Memory
-> BNA
-> CSV
-> NAS
-> GML
-> GPX
-> KML
-> GeoJSON
-> Interlis 1
-> Interlis 2
-> GMT
-> GPKG
-> SQLite
-> ODBC
-> WAsP
-> PGeo
-> MSSQLSpatial
-> OGDI
-> PostgreSQL
-> MySQL
-> PCIDSK
-> OpenFileGDB
-> XPlane
-> AVCBin
-> AVCE00
-> DXF
-> Geoconcept
-> GeoRSS
-> GPSTrackMaker
-> VFK
-> PGDump
-> OSM
-> GPSBabel
-> SUA
-> OpenAir
-> PDS
-> WFS
-> HTF
-> AeronavFAA
-> Geomedia
-> EDIGEO
-> GFT
-> GME
-> SVG
-> CouchDB
-> Idrisi
-> ARCGEN
-> SEGUKOOA
-> SEGY
-> XLS
-> ODS
-> XLSX
-> ElasticSearch
-> PDF
-> Walk
-> CartoDB
-> SXF
Converting outputs
OK. Execution took 0.125 ms (1 outputs).
Model processed ok. Executed 2 algorithms total
Converting outputs
Loading resulting layers

#7 Updated by Anita Graser over 5 years ago

  • Subject changed from Modeler issue with temp outputs under Windows to Regression: Modeler issue with temp outputs under Windows

#8 Updated by Victor Olaya about 5 years ago

  • Status changed from Open to Closed

Also available in: Atom PDF