Bug report #18740
is a WFS layer is large enough then identifying a feature in QGIS with the "auto open form" option active leads to memory leak/crashes
| Status: | Closed | ||
|---|---|---|---|
| Priority: | High | ||
| Assignee: |  Even Rouault | ||
| Category: | Web Services clients/WFS | ||
| Affected QGIS version: | 2.18.19 | Regression?: | No |
| Operating System: | Easy fix?: | No | |
| Pull Request or Patch supplied: | No | Resolution: | fixed/implemented |
| Crashes QGIS or corrupts data: | Yes | Copied to github as #: | 26627 |
Description
This happens regardless the server used (I tested QGIS Server and Mapserver).
If you load one of those WFS layers the first identify works ok.
On the second identify it seems that loading the entire layer is triggered (by default the WFS client has the option "only request the features overlapping the view extent" enabled). The dialog that asks to "hide" or "abort" the operation shows, but the buttons are not responsive.
At this point QGIS starts eating memory (in my case 16gb od ram and almost 16gb of swap). A crash/freeze is usual, but if there are enough resources and a long time is waited then ultimately QGIS regains life and the identify dialog shows.
Note that this does not happen at all if the "auto open form" is not enabled, even with huge WFS layers.
You can test this (layer "ine_inspire_portugal")
http://www.naturalgis.pt/cgi-bin/opendata/qgis_mapserv.fcgi?service=WFS&request=GetCapabilities
or layer "portugal_addresses" in
http://www.naturalgis.pt/cgi-bin/opendata/mapserv?
this also affects QGIS 3.
Associated revisions
Dummy commit to indicate that 0addae5b783e541953522eabe178426e820bf852 was about refs #18740
[WFS provider] Avoid request by feature id to cause a full layer download (fixes #18740)
[WFS provider] Fix excessive memory consumption on big layers (refs #18740)
There was a pseudo memory leak in qgsgml.cpp
And the WFS downloader could also have to process big replies, causing a
lot of features to be instanciated at once.
Was seen on the 'portugal_addresses' layer of http://www.naturalgis.pt/cgi-bin/opendata/mapserv?
Backport of
0addae5b783e541953522eabe178426e820bf852
c3c5c9760a0b300548f9a669ff64f513988add7c
6cf1c502d1e057aa63ba7b48e2a532324a98194d
[WFS provider] Avoid request by feature id to cause a full layer download (fixes #18740)
Backport of 6cf1c502d1e057aa63ba7b48e2a532324a98194d
Merge pull request #7052 from rouault/fix_18740_qgis2_18
[WFS provider] 2.18 / Avoid request by feature id to cause a full layer download (fixes #18740)History
#1
Updated by Jeremy Palmer over 6 years ago
- Assignee set to Even Rouault
I also get this problem for large layers, but only for QGIS 3.0. When I load a layer with restrictToRequestBBOX='1' it loads into the map ok, but on the first feature info it causes the provider to download the whole layer. This didn't cause a cause for me, but did lock up the UI for about 20mins in downloading 5GB of data for a very large layer.
I was testing on QGIS 3.0.1 MacOSX.
Even are you able to help?
#2
Updated by Even Rouault over 6 years ago
I didn't do significant changes in the WFS provider in months, so if this is a regression, perhaps a "git bisect" exercise could help find the offending commit (probably in another component)
#3
Updated by Giovanni Manghi over 6 years ago
Even Rouault wrote:
I didn't do significant changes in the WFS provider in months, so if this is a regression, perhaps a "git bisect" exercise could help find the offending commit (probably in another component)
I'm not sure is a regression. I think is just that no one has possibly ever tested with a very large (millions of features) using the "auto open form" option enabled.
#4
Updated by Even Rouault over 6 years ago
- Status changed from Open to Closed
- % Done changed from 0 to 100
Applied in changeset qgis|6cf1c502d1e057aa63ba7b48e2a532324a98194d.
#5
Updated by Even Rouault over 6 years ago
- Resolution set to fixed/implemented
- Affected QGIS version changed from 3.1(master) to 2.18.19
Fixed in master 3.1dev and backported to 2.18 branch