Bug report #12578
Large pause before snapping to large layer
|Affected QGIS version:||master||Regression?:||No|
|Operating System:||Easy fix?:||No|
|Pull Request or Patch supplied:||No||Resolution:|
|Crashes QGIS or corrupts data:||Yes|
steps to replicate:
1 add a layer A with a lot of vertexes (the more vertexes the more this issue will be clear)
2 add another layer B
3 enable snapping to layer A
4 start editing layer B, snap a vertex of B to A -> a certain amount of RAM starts to be used in addition to the one that qgis was using before 4)
5 snapping more B vertexes to A does not change the amount of memory used
6 add a layer C with a lot of vertexes and enable snapping to C
7 snap a vertex from B to C -> a certain amount of RAM starts to be used in addition to the one that qgis was using before 7)
8 repeat until memory finishes
Dear all, we noticed that qGis use a lot of memory when start to snap to a big layer during an edit session.
This issue is verified when we edit a layer (with few features) and the snap to another very big layer is activated: at first edit activity (add a new vertex of a new feature or first move a vertex of an existing feature), qGis starts to load a lot of memory and meanwhile qGis is blocked until all the memory is loaded.
We discovered this issue with a very big postgis layer (more than 1 million polygons), the waiting time is about 1 minute and the RAM occupied is 800Mb.
qGis free the RAM only when the project is closed but it is occupied during all the editing session.
The occupied RAM is dependent from the number of layer in TOC
For bigger layers, or also if we have more than layer in the TOC, sometimes qGIs crashes (due to the memory limits, I guess).
How to reproduce this issue:
- load a small polygon layer (edit)
- load a very big polygon layer (snapTo)
- activate the snap on the shapTo layer (every kind of snap)
- start editing the edit layer
- move one vertex
You will see qGis blocked and a lot of memory loading (in task manager).
You can download the test dataset from:
#2 Updated by Giovanni Manghi over 3 years ago
I just tested your project on qgis master on Linux, and I cannot replicate this issue. When starting editing the used memory by qgis raises to around 400mb, but then it remains stable.
I see two possibilities: the issue has been fixed on master or it is just a Windows thing. I cannot do more tests right now.
#3 Updated by francesco marucci over 3 years ago
we noticed this issue only on 2.8.1, untill 2.6 all was correct.
I tested only in win7, with the osgeo4w 32&64 installer and also with the standalone installers (32&64)). i tried also with the master build 33.
i will try to reproduce the dataset i used to do the tests (bigger than i sent) and i will send you.
in any case in my win7 pc, with thae dataset i sent, before starting digitising the memory occupied by qgis was 140mb, after more than 300mb. if you have more snapped layers in toc, the memory used is the summatory of each layer.
#4 Updated by francesco marucci over 3 years ago
we grab a video showing the problem:
in the toc we have tree "snapped" layers with a total of 20 millions vertices...
you can see that the memory used when the first editing activity (new polygon) starts is about 1 Gb.
#5 Updated by Giovanni Manghi about 3 years ago
- Affected QGIS version changed from 2.8.1 to master
- Subject changed from Problem with the snapping to layer function: a lot of memory loaded during the first action to Snapping to layer function: memory use grows indefinitely
- Status changed from Feedback to Open
#7 Updated by Giovanni Manghi about 3 years ago
- Resolution deleted (
- Target version set to Future Release - High Priority
- Status changed from Closed to Reopened
It would be appreciated if you can verify with tomorrows nightly build and reopen if the problem persists.
unfortunately still there.
#11 Updated by Saber Razmjooei about 3 years ago
The problem is in 2.10 for sure. See Martin's email (2nd in the thread) regarding the new geometry engine and snapping cache:
#12 Updated by Saber Razmjooei almost 3 years ago
- Status changed from Reopened to Feedback
Tried it with a 0.5 GB shapefile:
and it works fine with the latest master. I have deleted previous spatial index and created a new one.
Could you upload or link to some sample data?
#13 Updated by Nyall Dawson almost 3 years ago
I can't reproduce with master either. There were a few (tiny) leaks flagged which I'll fix shortly, but nothing significant.
Just to confirm - when you say it's still present, are you referring to the memory leak or the initial hang when the snapping index is built?
#15 Updated by Saber Razmjooei almost 3 years ago
- Status changed from Feedback to Open
I think it will be good to give user feedback during the initial indexing. You can see in this screencast, nothing is happening between 0:57 and 1:25 and then the polygon capture becomes active:
#17 Updated by Saber Razmjooei almost 3 years ago
- Using a PostGIS layer (os mastermap) with 1034924 features, the initial lag was too long and QGIS became completely unusable my PC (32 GB RAM, 1 TB SSD, PG layer is from localhost)
- Saving the same layer as Spatialite, things improved significantly. There was an initial lag, but after that it was smooth digitising.
One can replicate it using OSM as PG layer and Spatialite layer.