Bug report #11141
Parallel multi-core rendering crashes QGIS
Status: | Closed | ||
---|---|---|---|
Priority: | Severe/Regression | ||
Assignee: | - | ||
Category: | Map Canvas | ||
Affected QGIS version: | 2.6.0 | Regression?: | No |
Operating System: | Windows | Easy fix?: | No |
Pull Request or Patch supplied: | No | Resolution: | fixed/implemented |
Crashes QGIS or corrupts data: | Yes | Copied to github as #: | 19469 |
Description
We rolled out QGIS 2.4 recently and face regular crashes when setting a group containing many layers visible. All vector layers are PostGIS layers (dedicated DB server in the LAN) and some of the layers have data-defined symbology, one is a raster layer (raster on network drive). All layers are in the same CRS as the project.
When unticking Render layers in parallel using many CPU cores the crash does not happen. Even when Render layers in parallel using many CPU cores has been activated again after setting the group visible no crash when changing the group's visiblity again => The crash is only happening immediately after program start.
- QGIS 2.4, Win7, 4 cores, 4GB RAM => crash
- QGIS 2.5 (master, OSGEO-install), Win7 VM, 4 cores, 2GB RAM => crash
- QGIS 2.5 (master, self compiled), OpenSuse 12.2 (64 bit), 4 cores 4 GB RAM => no crash
I attached the minidump file created on Windows hoping it helps.
History
#1 Updated by Bernhard Ströbl about 10 years ago
seems the minidump file was to large to upload. I can provide it if someone needs it to analyze teh problem.
#2 Updated by Martin Dobias about 10 years ago
In order to track down the likely source of the problem, I would need some more information regarding what kind of functionality has to be turned on in order to get crashes. For the start:
- will it crash if you stop using data-defined symbology on those layers? (ideally convert to single symbol renderer with just "simple" symbol)
- do you use labeling / diagrams? if yes, will it crash if you disable it?
- do you use joins? if yes, will it crash if you disable joins?
- will it crash if you use a different provider for the data than PostGIS?
Feel free to send minidump file to me, but I'm not sure if I will be able to make use of it (I work rarely on windows)
#3 Updated by Bernhard Ströbl about 10 years ago
- File qgis_style.png added
Hi Martin,
Martin Dobias wrote:
In order to track down the likely source of the problem, I would need some more information regarding what kind of functionality has to be turned on in order to get crashes. For the start:
- will it crash if you stop using data-defined symbology on those layers? (ideally convert to single symbol renderer with just "simple" symbol)
I was misstaken (mixed it up with a different group): There is no data-defined ymbology but rule-based renderer. I changed to simple styles to no avail => crash persists
- do you use labeling / diagrams? if yes, will it crash if you disable it?
No diagrams; switching off all labels has no effect
- do you use joins? if yes, will it crash if you disable joins?
no joins in QGIS
- will it crash if you use a different provider for the data than PostGIS?
this would be quite heavy, transferring some 100000 datasets to e.g. SpatialLite. Will do that as last ration, though.
Currently I have nailed down the problem to a single layer within the group: If I set it to "Single symbol" all works ok (see attached screenshot). The setting looks weird because this project originally was a version 2.0.1 project (min/max scale seems exchanged). I replaced the modulo (%) with a simple / and the crash is gone!
#4 Updated by Bernhard Ströbl about 10 years ago
ok I fixed the layer in question by duplicating it and using the modulo in the filter instead of the rule-based renderer. Would be great if you could still look into it.
#5 Updated by Bernhard Ströbl about 10 years ago
I just realized that the position of the input lines for min/max scale was exchanged since 2.2, that's while it looked weird to me, not reading the label but been focussed on the position of an input element :-)
#6 Updated by Martin Dobias about 10 years ago
So, do I understand correctly that in your case the crash happens only when modulo operator is used in rule-based renderer for a layer within the group?
Are there more layers with rule-based renderer?
To me it seems a bit unlikely that modulo operator would be causing the crash... (Also when checking the code.) Could you please try to do some more tests to confirm it is really the case?
#7 Updated by Bernhard Ströbl about 10 years ago
I tested a lot this morning, results:
- the modulo operator is not the cause of the crashes
- the rule-based renderer is not the cause of the crashes
- labels or scale based visibility are not the cause of the crashes
- nor can a single layer be identified to cause the crashes
- the minidump message is shown almost immediately after setting the group visible
- the minidump message is displayed while the MapCanvas continues drawing until the map is finished
- the minidump message is not shown at every crash; there are hard crasehes, too
- the crash is reproducible:
- it only happens when setting the group to visible immediately after QGIS has finished loading the project
- it only occures when Render layers in parallel using many CPU cores is enabled
#8 Updated by Jürgen Fischer about 10 years ago
Bernhard Ströbl wrote:
- the crash is reproducible:
- it only happens when setting the group to visible immediately after QGIS has finished loading the project
- it only occures when Render layers in parallel using many CPU cores is enabled
Can you really reproduce it reliably? I was unable to reproduce this kind of crashes so far. The minidumps are unconclusive (at least to me). And reopening the project a couple of times usually makes it work again. It seems to be some subtle race condition that isn't triggered too often.
#9 Updated by Bernhard Ströbl about 10 years ago
Jürgen Fischer wrote:
Can you really reproduce it reliably? I was unable to reproduce this kind of crashes so far. The minidumps are unconclusive (at least to me). And reopening the project a couple of times usually makes it work again. It seems to be some subtle race condition that isn't triggered too often.
I did not look at the minidump, however it is reproducible: I start QGIS which loads the project by default. When loading has finished and I set the group to visible QGIS always crashes (at least this morning in about 20 cases). If I e.g. open the group and set one layer to visible first and afterwards set the whole group to visible there is no crash.
#10 Updated by Martin Dobias about 10 years ago
Thanks for further testing.
Will you still get a crash if you reduce the group to 5-10 layers? If yes, would replacing these few layers with other backend (shapefile?) avoid crashes?
Are there any visible postgis layers outside of that group?
I have a suspicion that the crash could be triggered by PQconnectDb() or something related to that. I found a very old (1998) mailing list discussion about its thread safety, but libpq seems to have evolved a lot since then...
#11 Updated by Bernhard Ströbl about 10 years ago
Martin Dobias wrote:
Thanks for further testing.
Thanks for your patience
Will you still get a crash if you reduce the group to 5-10 layers?
yes
If yes, would replacing these few layers with other backend (shapefile?) avoid crashes?
no crash with shape files
Are there any visible postgis layers outside of that group?
No, BUT (tada!) if I set another PostGIS layer visible in the project, store the project, restart QGIS and set the group visible there is no crash!
I have a suspicion that the crash could be triggered by PQconnectDb() or something related to that. I found a very old (1998) mailing list discussion about its thread safety, but libpq seems to have evolved a lot since then...
#12 Updated by Andreas Neumann about 10 years ago
Hi,
I have very similar problems like Bernhard. The difference is that in my case the crash already happens at the end of project load, before the user get's any chance to interact with QGIS. If I turn off parallel rendering the crashes disappear.
My projects are similar to Bernhards:
- lots of layers
- all layer are loaded from PostgreSQL
- no joins so far (we use PostgreSQL views) - except the value relation widgets
- lots of nested groups
- embeded layers and groups from other projects
I had a look at it with Jürgen already but we weren't able to track down the exact cause. The crashes did not happen always, but often.
Hope that we can solve this problem before the next release.
#13 Updated by Bernhard Ströbl about 10 years ago
Hi Andreas,
my workaround is to have one PostGIS layer (with one feature) visible in the project so the connection gets established upon loading (before that only a WMS layer was visible). Maybe your crashes are because you have too many set visible? (This is not an acceptable behaviour, though).
I currently have no embedded layers in my projects.
#14 Updated by Andreas Neumann about 10 years ago
Yes - if I set visibility off for all except one layer, QGIS does not crash. If I then change visibility state to visible for a whole group (with subgroups and many layers), QGIS crashes.
It behaves quite similar to what Bernhard experiences.
#15 Updated by Martin Dobias about 10 years ago
Please try again with lastest master (242754) - some possible crashes were fixed when expressions were used with parallel rendering.
#16 Updated by Andreas Neumann about 10 years ago
It works fine for me. No more crashes for my projects.
#17 Updated by Giovanni Manghi about 10 years ago
- Resolution set to fixed/implemented
- Status changed from Open to Closed
Andreas Neumann wrote:
It works fine for me. No more crashes for my projects.
I assume that this should be fixed. Please test and reopen if necessary, cheers!
#18 Updated by Bernhard Ströbl almost 10 years ago
- Affected QGIS version changed from master to 2.6.0
- Status changed from Closed to Reopened
- Resolution deleted (
fixed/implemented)
Sorry, I have to reopen this again. I checked back with Andreas and the problem persists on QGIS 2.6 on Win7 64 bit
#19 Updated by Nathan Woodrow almost 10 years ago
Try setting the max thread count in the options to max - 1. e.g if you can have max threads of 8 then use 7 or lower. I have round using max threads will crash Roam but if I set it to max-1 then it never crashes.
#20 Updated by Niccolo' Marchi over 9 years ago
Bernhard Ströbl wrote:
Sorry, I have to reopen this again. I checked back with Andreas and the problem persists on QGIS 2.6 on Win7 64 bit
Hi guys,
I have similar problems with 2.6.1(osgeo) on Win8.1 64bit: lots of layers (raster+vector; no particular labelling or style), almost no 3rd party plugins (TomBio prod tools, openlayers).
Zooming in&out quickly, QGIS randomly crashes. The problem disappears if MTR is switched off.
Settings of MTR were the default ones from installation. The processor is an i5 dual-core.
Hope this can help in some way.
Thank you all.
#21 Updated by Andreas Neumann over 9 years ago
#22 Updated by Giovanni Manghi over 9 years ago
- Status changed from Reopened to Feedback
#23 Updated by Bernhard Ströbl over 9 years ago
Confimred, problem seems gone in f4c3f3d (OSGEO-Install)
#24 Updated by Giovanni Manghi over 9 years ago
- Status changed from Feedback to Closed
- Resolution set to fixed/implemented
reopen if necessary.