Bug report #8878

OSM plugin fails to download all data within given extent

Added by Mathieu Pellerin - nIRV over 6 years ago. Updated over 2 years ago.

Status:Closed
Priority:Normal
Assignee:Martin Dobias
Category:Data Provider
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:wontfix
Crashes QGIS or corrupts data:No Copied to github as #:17558

Description

Steps to reproduce:
1. Load the shapefile attached to this issue (ICJ-DMZ.zip)
2. Download OSM data for the map extent covering the above shapefile
3. Import topology, then export topology to polylines

The resulting spatialite line layer will be missing lots of OSM open ways (see qgis-import_vs_actual-osm.png). You can overlay an OpenLayers' OSM layer to easily see what's missing.

It's an issue affecting both QGIS 2.0 and QGIS master (2.1).

ICJ-DMZ.zip (995 Bytes) Mathieu Pellerin - nIRV, 2013-10-16 07:21 PM

qgis-import_vs_actual-osm.png (122 KB) Mathieu Pellerin - nIRV, 2013-10-16 07:21 PM

GDAL-import.png (84.3 KB) Andre Joost, 2014-05-27 09:53 AM


Related issues

Related to QGIS Application - Bug report #10790: OpenStreetMap plugin only loads 32 bits numbers ID, then ... Closed 2014-07-03
Duplicated by QGIS Application - Bug report #12727: Openstreetmap Import to spatialite incomplete Closed 2015-05-09

History

#1 Updated by Mathieu Pellerin - nIRV over 6 years ago

Forgot to expand on the qgis-import_vs_actual-osm.png; the imported lines are displayed using red squares along path, overlaid on top of OSM tiles. In total, three ways are imported, and a bunch of visible tracks / paths / etc. ways are clearly missing.

#2 Updated by Martin Dobias over 6 years ago

QGIS currently uses this kind of query for download:

http://overpass-api.de/api/interpreter?data=(node(14.3833,104.638,14.4167,104.706);<;out;

What it does is that it returns all nodes in the bounding box + ways and relations referring to those nodes. Now, the ways containing nodes that are outside of the bounding box cannot be reconstructed properly and they are silently ignored.

What would be the best way to deal with this situation? We can query also nodes outside of the bounding box if used by some ways like this:

http://overpass-api.de/api/interpreter?data=(node(14.3833,104.638,14.4167,104.706);way(bn);node(w););out;

Upon reconstruction, this may give a bit surprising result, because often big ways (that span through big area, e.g. border line) may completely dominate the actual area of interest. Probably we should have an option to clip the resulting layer to originally asked extent.

Also, I guess we need to check whether the .osm file being loaded is complete (e.g. no missing nodes for ways) and if not, refuse opening such file.

Thoughts?

#3 Updated by Jukka Rahkonen over 6 years ago

I don't think that users should be forced to download always full features and relations because they can be quite big. And if user has received some OSM data from external source it would be unfriendly to prevent opening the dataset totally just because some features are not complete.

OpenJUMP OSM reader is writing into attributes if features has been resolved totally. That might be worth thinking.
http://sourceforge.net/apps/mediawiki/jump-pilot/index.php?title=Working_with_OpenStreetMap_data

#4 Updated by Mathieu Pellerin - nIRV over 6 years ago

Martin, there are ways that are confined within the bounding box (see the yellow tertiary highway, some small stair ways, and tracks) yet are not downloaded / exported to lines. Wondering whether there are two issues here. One with how to deal with partial ways, and the other with ways well within bounding box not being retrieved or exported to polylines.

#5 Updated by Alexandre Neto about 6 years ago

I have found this same issue and the workaround is to download the .osm files using the export button on www.openstreetmap.org

#6 Updated by Giovanni Allegri about 6 years ago

The tertiary road is correctly downloaded, I can see it in the ways table (id=204549674, http://www.openstreetmap.org/way/204549674).
The problem is that it itsn't put into the polylines spatialite layer.

#7 Updated by Giovanni Allegri about 6 years ago

Martin, your suggestion causes recursions which, for areas bigger then this example, will make the query fail (Overpass will kill it).

http://overpass-api.de/api/interpreter?data=(node(14.3833,104.638,14.4167,104.706);way(bn);node(w););out;

#8 Updated by Andre Joost about 6 years ago

If I use the QGIS OSM plugin to download the data (step 1 and 2), but then use `Add vector Layer` to import the .osm, I get all ways inside the bbox as expected (see attached GDAL-import.png).

This uses the GDAL osm driver. I don't know what they are doing better (it should be a spatialite database as well).

I suggest to merge the two importers.

#9 Updated by Anita Graser almost 5 years ago

  • Assignee set to Martin Dobias
  • Affected QGIS version changed from 2.0.1 to master

@Martin: I've assigned this to you now since I guessed from Github that you added many parts of the OSM tool. Maybe you could also reply to the discussion in http://osgeo-org.1560.x6.nabble.com/Suggestion-Remove-Vector-Openstreetmap-tool-from-QGIS-2-12-td5226614.html and add your opinion.

I hope that's ok, thanks!

#10 Updated by Giovanni Manghi about 3 years ago

  • Easy fix? set to No
  • Regression? set to No

#11 Updated by Andre Joost almost 3 years ago

Maybe related to #10790

#12 Updated by Jürgen Fischer almost 3 years ago

  • Related to Bug report #10790: OpenStreetMap plugin only loads 32 bits numbers ID, then start with negative numbers added

#13 Updated by Nyall Dawson over 2 years ago

  • Resolution set to wontfix
  • Description updated (diff)
  • Status changed from Open to Closed

OSM downloader has been removed for 3.0

Also available in: Atom PDF