Bug report #9693

OpenStreetMap XML import takes a long time

Added by Giovanni Allegri over 5 years ago. Updated over 5 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Vectors
Affected QGIS version:2.2.0 Regression?:No
Operating System:Windows Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:18258

Description

The OSM import from a downloaded XML to Spatialite takes much more time on Windows then on Linux.
With the following file Windows it takes about 3 minutes to parse it and import it. It seems that the problem is in the XML parsing phase, because the progress bar runs fast at the end of the process, as fast as on Linux (Looking into the code the progress emits after the OSM parse, during the insert phase).

Test file: https://dl.dropboxusercontent.com/u/13861666/osmtest.osm

Associated revisions

Revision 4d5d26a9
Added by Martin Dobias over 5 years ago

Merge pull request #1399 from giohappy/fix_osm_initspatialmetadata

Fix #9693 (Make OSM DB metadata initialization in transaction)

History

#1 Updated by Giovanni Allegri over 5 years ago

I've noticed that during the freezed time a Sqlite journal file is continously created and destroyed. I don't know what is all this journaling activity for, before inserting data...

#2 Updated by Giovanni Manghi over 5 years ago

  • Category changed from Browser to Vectors

Giovanni Allegri wrote:

I've noticed that during the freezed time a Sqlite journal file is continously created and destroyed. I don't know what is all this journaling activity for, before inserting data...

This bug affected 2.0.1 always that was needed to create a new SL db: off-line editing, save a vector as SL, new SL db from "layer" menu, etc.

Strange that affects 2.2 only on Windows as the same tool on Linux is ok.

By the way, once added the line layer from the imported osm file it seems that many features are missing.

#3 Updated by Martin Dobias over 5 years ago

  • Assignee deleted (Martin Dobias)

One just needs to call "SELECT InitSpatialMetadata(1)" for SpatiaLite >= 4.1 instead of just "SELECT InitSpatialMetadata()" in order to init metadata in a transaction. (see #9157 for similar problem) On linux it may work just because of different SpatiaLite version.

Regarding missing features, it may be #8878

#5 Updated by Giovanni Allegri over 5 years ago

Probably the best would be both were using the spatialite provider which already does the correct choice (https://github.com/qgis/QGIS/blob/master/src/providers/spatialite/qgsspatialiteprovider.cpp#L4983)

#7 Updated by Martin Dobias over 5 years ago

  • Status changed from Open to Closed

Also available in: Atom PDF