Bug report #18420
Excessive, erratic memory usage
|Affected QGIS version:||3.0.0||Regression?:||No|
|Operating System:||Easy fix?:||No|
|Pull Request or Patch supplied:||Yes||Resolution:||fixed/implemented|
|Crashes QGIS or corrupts data:||Yes||Copied to github as #:||26309|
QGIS uses excessive amounts of memory, seemingly leaking some until it takes down my whole system. This seems to depend on adding and removing layers. In my case I noticed it with GeoJSON and Shapefile formats, I have no idea if it affects others).
QGIS takes about 320M of RAM with a clean (no project loaded, but some plugins) start for me.
1670M after loading the geojson after a clean start 2500M after adding the shp to that 3300M after removing the shp layer 4160M after adding the shp again 5000M after removing the shp layer again (5000M after removing the geojson layer as well but that seems to get at least partially reclaimed as adding both the shp and the geojson again only made it grow to 5075M...)
365M after loading the shp after a clean start 1720M after adding the geojson to that 1700M after removing the geojson layer 1760M after adding the geojson again
So this seemed fine (although 1.7G for a 25M GeoJSON is crazy).
365M after loading the shp after a clean start 1720M after adding the geojson to that 2530M after removing the shp layer 3350M after adding the shp again
I did not bother going further, it would have taken more memory again.
So, loading and unloading the shp file while the geojson is loaded makes QGIS gradually eat all RAM and crash the system.
QGIS version 3.0.0-Girona
QGIS code branch Release 3.0
[ogr] Fix ref/unref mismatch when loading OGR layers
Causes an extra connection reference which is never removed,
blocking ogr dataset closing.
Fixes #18420, probably others
#2 Updated by Johannes Kroeger over 2 years ago
I could not test on the same machine yet. On my other one (same 64 bit Archlinux but different GDAL) I could not replicate the high memory usage even in an older qgis (r44842.6ed078c889) so I am a bit confused. Maybe the GDAL version plays a role too? I will try to test it tomorrow.
#4 Updated by Johannes Kroeger over 2 years ago
Yeah, that one had the latest GDAL from SVN I think.
On the machine where I noticed the problem:
QGIS 3.1.0-Master (193c554b4c) with the patch applied.
GDAL 2.2.3, released 2017/11/20
1660M added GeoJSON
2033M added shp
3000M removed shp
3800M added shp
4200M removed shp
It also does not free after unloading all layers or creating a new project.
#6 Updated by Johannes Kroeger over 2 years ago
- Status changed from Closed to Reopened
- Assignee set to Nyall Dawson
No luck :(
I built from Git r46587.10044fb1dd with the same GDAL 2.2.3, released 2017/11/20 and tried the files again:
5000 new project
5100 open random other project