Bug report #13963
QGIS greadily allocates memory and crashes when editing moderately large shapefiles with the node tool
Status: | Closed | ||
---|---|---|---|
Priority: | High | ||
Assignee: | - | ||
Category: | Digitising | ||
Affected QGIS version: | 2.8.4 | Regression?: | No |
Operating System: | Linux / Ubuntu | Easy fix?: | No |
Pull Request or Patch supplied: | No | Resolution: | fixed/implemented |
Crashes QGIS or corrupts data: | Yes | Copied to github as #: | 21977 |
Description
When editing moderately large shapefiles (228428 points, 1.1MB << 1% total RAM). QGIS starts to allocate huge amounts of memory and either crashes the system when no ulimit is set or is killed by the kernel. This happens reproducibly by adding the dataset to a new project, going into edit mode, selecting the node tool, selecting more than 1/3 of the points at once and attempting to delete them. This is a downsampled dataset which can be generated and edited with other software on the same machine without hickups. QGIS version is 1:2.8.1-0+20trusty6.
Associated revisions
change QgsAbstractGeometryV2::coordinateSequence() to return a
implicitly shared copy of an internal cache instead of recreating the
coordinate sequence again and again.
Improves performance of the nodetool on large features a lot (refs #13963)
Also introduce Qgs(Coordinate|Ring|Point)SequenceV2 typedefs.
Much faster selection of large number of nodes using node tool
Refs #13963
History
#1 Updated by Nyall Dawson almost 9 years ago
- Status changed from Open to Feedback
Please test with 2.8.4 or 2.12.1
#2 Updated by Karl Kastner almost 9 years ago
Release 2.12.1 segfaults immediately at start up. This has to do with python plugins, because it starts up if the package python-qgis is removed. Release 2.8.4 is not in the repositories, at least neither in http://qgis.org/ubuntugis nor in http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu/.
#3 Updated by Karl Kastner almost 9 years ago
- File dmesg-2.12.1.txt added
Release 2.12.1 segfaults immediately at start up. This has to do with python plugins, because it starts up if the package python-qgis is removed. Release 2.8.4 is not in the repositories, at least neither in http://qgis.org/ubuntugis nor in http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu/.
#4 Updated by Jürgen Fischer almost 9 years ago
Karl Kastner wrote:
Release 2.8.4 is not in the repositories, at least neither in http://qgis.org/ubuntugis nor in http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu/.
Because 2.8.4 is the LTR, which is in debian-ltr or ubuntugis-ltr (see download page)
#5 Updated by Karl Kastner almost 9 years ago
- File dmesg-2.8.4.txt added
Thanks for the advice to add the LTR repository. The bug also affects 2.8.4. It does not occur on every deletion though, the steps had to be repeated a couple of times until Qgis crashed. Qgis becomes also relatively slow when activating the node tool, as it automatically starts a background validation. Note that the dataset has some crossing lines due to the downsampling, but this should not make Qgis crash.
dpkg -l qgis Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-========================================-=========================-=========================-===================================================================================== ii qgis 1:2.8.4+20trusty-ubuntugi amd64 Geographic Information System (GIS)
#6 Updated by Jürgen Fischer almost 9 years ago
The kernel output is mostly useless. Please attach the shape file.
#7 Updated by Karl Kastner almost 9 years ago
- File kapuas-polygon-merged-100.tgz added
#8 Updated by Jürgen Fischer almost 9 years ago
The archive is missing the dbf.
#9 Updated by Karl Kastner almost 9 years ago
There is no dbf, as there are no attributes. This is a Matlab export. Matlab does not create a dbf if there are no attributes.
#10 Updated by Giovanni Manghi almost 9 years ago
- Priority changed from Low to High
- Category changed from Actions to Digitising
- Status changed from Feedback to Open
Karl Kastner wrote:
There is no dbf, as there are no attributes. This is a Matlab export. Matlab does not create a dbf if there are no attributes.
weird, but anyway I confirm the issue, at least on 2.8.4 on ubuntu 14.04.
#11 Updated by Giovanni Manghi almost 9 years ago
- Affected QGIS version changed from 2.8.1 to 2.8.4
#12 Updated by Hynek Suchy almost 9 years ago
- File Shapefile_test.zip added
Hello!
I would like to confirm this bug.
I am on Windows Vista 32bits Ultimate, QGIS 2.12.1 Lyon, standalone installer.
I am working on a small shapefile with some 300 points.
While editing the shapefile, mainly deleting points or moving them, qgis starts to
slow down and in the end freezes.
I tried to open the shp file outside of current project and I easily recreated the bug.
1. select one point
2. delete it
repeat 11 times and qgis freezes (not responding), but after some time
refreshes and I can delete another point, after 14 repetitions qgis doesn't recover
and I have to kill it from Windows Task Manager.
Enclosing shapefile.
#13 Updated by Giovanni Manghi almost 9 years ago
Jurgen, Nyall, shouldn't this be tagged as blocker?
#14 Updated by Gerhard Spieles almost 9 years ago
Hi Hynek,
maybe a problem of installed plugins.
See #13869
#15 Updated by Giovanni Manghi almost 9 years ago
Gerhard Spieles wrote:
Hi Hynek,
maybe a problem of installed plugins.
See #13869
usually in my testing boxes I have no 3rd party plugins, but I can test again.
#16 Updated by Hynek Suchy almost 9 years ago
Sorry for making a false alarm!
It was CadTools, I deinstalled them (and Digitizing Tools to be sure) and now it works smoothly!
Thank you!
#17 Updated by Karl Kastner almost 9 years ago
On my machine the bug can be reproduced when all plugins are disabled or by temporally moving the ~/.qgis2 folder before starting qgis.
The bug is also be reproduced with data sets having a valid geometry. (The polygon in the original sample shape file was self intersecting.)
The behaviour depends on the geometry for the same data points:
Polygon: QGIS becomes a memory bomb and crashes.
Line: QGIS takes very long (minutes) to complete the tasks, but does not fill the memory.
Point: QGIS deletes all selected points flawlessly within less than a second.
#18 Updated by Giovanni Manghi almost 9 years ago
Karl Kastner wrote:
On my machine the bug can be reproduced when all plugins are disabled or by temporally moving the ~/.qgis2 folder before starting qgis.
yes, this is my observation too.
#19 Updated by Jürgen Fischer over 8 years ago
Improvements in PR #2817
#20 Updated by Giovanni Manghi over 8 years ago
- Status changed from Open to Feedback
#21 Updated by Jürgen Fischer over 8 years ago
50f01a2 also reduces the memory consumption of the geometry classes, which should also help with this issue.
#22 Updated by Giovanni Manghi over 8 years ago
- Status changed from Feedback to Closed
- Resolution set to fixed/implemented
seems fixed to me at least on 2.14.3, please reopen if necessary
#23 Updated by baditaflorin - over 8 years ago
- Status changed from Closed to Reopened
Tested in 2.14.3 on the Canada provinces and the results are horrible.
If i delete less then 10-20 nodes at a time it takes around 1-2 seconds.
If i select around 100-1000 nodes, it can take up to 10 minutes.
You can download the file from here https://www.dropbox.com/s/j5i1m1tt2cam1qy/canada_states.zip?dl=0
Here it`s a youtube video showing the selection https://youtu.be/tV7VOK4YAZE
#24 Updated by Giovanni Manghi over 7 years ago
- Status changed from Reopened to Closed
baditaflorin - wrote:
Tested in 2.14.3 on the Canada provinces and the results are horrible.
If i delete less then 10-20 nodes at a time it takes around 1-2 seconds.
If i select around 100-1000 nodes, it can take up to 10 minutes.
You can download the file from here https://www.dropbox.com/s/j5i1m1tt2cam1qy/canada_states.zip?dl=0
Here it`s a youtube video showing the selection https://youtu.be/tV7VOK4YAZE
the original report/issue is solved, there is no more leak if you the node tool.
So in this cases please file a different ticket.
Anyway I tested you dataset on both 2.18.4 and master (qgis3) on both Windows and Linux and is not that slow removing nodes when you selected a lot of them. Is not lighting fast, but it takes a few seconds when you have selected a reasonable large amount. Please try with the latest releases and in case file a new ticket.