Bug report #11965
Performance issues editing in 2.6.x/master compared to 2.4
|Affected QGIS version:||master||Regression?:||No|
|Operating System:||Easy fix?:||No|
|Pull Request or Patch supplied:||No||Resolution:|
|Crashes QGIS or corrupts data:||No||Copied to github as #:||20172|
Our users have been complaining of performance bugs when editing a spatialite layer in 2.6 and 2.6.1 when compared to 2.4 which they were using previously.
Whenever the user makes an edit to the spatialite vector layer (eg split, merge, change an attribute) QGIS 2.6.x hangs and increases the length of time it takes to perform their work. These problems did not occur with the same database in 2.4.
The spatialite DB was created in FME and is indexed. It is 1.5GB in size and contains 629,201 features. A new database has been created using FME which also repaired geometry in the process but the problems still persist in 2.6.x
I have uploaded a video of the issue to illustrate what is going on:
Fix #11965 - improve performance of selectedFeatures()
The slowness of merge / split features tools was caused by the change in the logic
in selectedFeatures(): instead of fetching individual features one by one by ID,
the whole layer is traversed. Such approach makes sense when many features are selected,
but with few features there is considerable delay when dealing with big layers.
The implementation is not ideal, but for some common cases the performance is much better.
Merging of features now does not request selected features when not necessary.
When rendering, avoid using layer's extent() method that may force expensive
calculation of layer's extent.
#2 Updated by Matthew Yandell-Thomas over 5 years ago
Hi Nathan, just about to test that. Disabling them would do the trick for trying this out or do they need to be completely uninstalled?
After disabling all 'after market' plugins and restarting QGIS the problem still remains
Plugins used are: Go2Streetview, Dockable Mirror Map, Auto saver and a modified version of the Quick Multi Attribute Edit tool
#5 Updated by Giovanni Manghi over 5 years ago
- Category set to Vectors
- Status changed from Open to Feedback
Matthew Yandell-Thomas wrote:
Just tried those suggestions and after doing both the slow response to edits still occurred.
I have tested with a SL database created within QGIS and I cannot see such issue, but now I cannot test a large dataset.
Do you see the same issue if creating the DB within QGIS?
Can you attach a minimal sample of your DB where the difference is still visible?
#6 Updated by Matthew Yandell-Thomas over 5 years ago
Yea, the problem persists if I create the DB in both QGIS 2.4 and 2.6
Due to commercial reasons we aren't able to upload any data at this time as it's not supposed to be publicly available yet. It's due for release later in the year and after that point we would be allowed to upload it freely. We may be able to send it directly to someone to take a look at if that's a possible solution?
#8 Updated by Jukka Rahkonen over 5 years ago
It is simple and fast to create big Spatialite databases from OpenStreetMap data with GDAL. Try to repeat your issue for example with the German dataset http://download.geofabrik.de/europe/germany-latest.osm.pbf.
ogr2ogr -f SQLite -dsco spatialite=yes germany.sqlite germany-latest.osm.pbf -gt 20000 -progress --config OGR_SQLITE_SYNCHRONOUS OFF --config OSM_COMPRESS_NODES YES
#9 Updated by Matthew Yandell-Thomas over 5 years ago
We've just finished creating that DB using ogr and created a spatialite file that was nearly 14gig.
Loaded the polygon layer into 2.6 and it still hangs, but for even longer than before. And in 2.4 it performs all edits instantly.
Can I ask what OS you are running? We are getting these problems on all our PCs in the office which are Windows 7. We are about to try it on a linux box shortly
#10 Updated by Giovanni Manghi over 5 years ago
- Affected QGIS version changed from 2.6.1 to master
- Subject changed from Performance issues editing SpatiaLite DBs in 2.6.x compared to 2.4 to Performance issues editing in 2.6.x/master compared to 2.4
- Status changed from Feedback to Open
- Priority changed from Normal to Severe/Regression
- Target version set to Version 2.8
- Operating System deleted (
- OS version deleted (
I confirm that there is a very clear decrease of speed in vector editing operation, and it affects also PostGIS (and maybe shapefiles, not tested yet).
To see clearly a user must use a very large vector.
It affects also master.