Bug report #11141

Parallel multi-core rendering crashes QGIS

Added by Bernhard Ströbl over 9 years ago. Updated almost 9 years ago.

Status:Closed
Priority:Severe/Regression
Assignee:-
Category:Map Canvas
Affected QGIS version:2.6.0 Regression?:No
Operating System:Windows Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed/implemented
Crashes QGIS or corrupts data:Yes Copied to github as #:19469

Description

We rolled out QGIS 2.4 recently and face regular crashes when setting a group containing many layers visible. All vector layers are PostGIS layers (dedicated DB server in the LAN) and some of the layers have data-defined symbology, one is a raster layer (raster on network drive). All layers are in the same CRS as the project.
When unticking Render layers in parallel using many CPU cores the crash does not happen. Even when Render layers in parallel using many CPU cores has been activated again after setting the group visible no crash when changing the group's visiblity again => The crash is only happening immediately after program start.

Tests performed:
  • QGIS 2.4, Win7, 4 cores, 4GB RAM => crash
  • QGIS 2.5 (master, OSGEO-install), Win7 VM, 4 cores, 2GB RAM => crash
  • QGIS 2.5 (master, self compiled), OpenSuse 12.2 (64 bit), 4 cores 4 GB RAM => no crash

I attached the minidump file created on Windows hoping it helps.

qgis_style.png (4.01 KB) Bernhard Ströbl, 2014-09-05 03:08 AM

History

#1 Updated by Bernhard Ströbl over 9 years ago

seems the minidump file was to large to upload. I can provide it if someone needs it to analyze teh problem.

#2 Updated by Martin Dobias over 9 years ago

In order to track down the likely source of the problem, I would need some more information regarding what kind of functionality has to be turned on in order to get crashes. For the start:
- will it crash if you stop using data-defined symbology on those layers? (ideally convert to single symbol renderer with just "simple" symbol)
- do you use labeling / diagrams? if yes, will it crash if you disable it?
- do you use joins? if yes, will it crash if you disable joins?
- will it crash if you use a different provider for the data than PostGIS?

Feel free to send minidump file to me, but I'm not sure if I will be able to make use of it (I work rarely on windows)

#3 Updated by Bernhard Ströbl over 9 years ago

Hi Martin,
Martin Dobias wrote:

In order to track down the likely source of the problem, I would need some more information regarding what kind of functionality has to be turned on in order to get crashes. For the start:
- will it crash if you stop using data-defined symbology on those layers? (ideally convert to single symbol renderer with just "simple" symbol)

I was misstaken (mixed it up with a different group): There is no data-defined ymbology but rule-based renderer. I changed to simple styles to no avail => crash persists

- do you use labeling / diagrams? if yes, will it crash if you disable it?

No diagrams; switching off all labels has no effect

- do you use joins? if yes, will it crash if you disable joins?

no joins in QGIS

- will it crash if you use a different provider for the data than PostGIS?

this would be quite heavy, transferring some 100000 datasets to e.g. SpatialLite. Will do that as last ration, though.

Currently I have nailed down the problem to a single layer within the group: If I set it to "Single symbol" all works ok (see attached screenshot). The setting looks weird because this project originally was a version 2.0.1 project (min/max scale seems exchanged). I replaced the modulo (%) with a simple / and the crash is gone!

#4 Updated by Bernhard Ströbl over 9 years ago

ok I fixed the layer in question by duplicating it and using the modulo in the filter instead of the rule-based renderer. Would be great if you could still look into it.

#5 Updated by Bernhard Ströbl over 9 years ago

I just realized that the position of the input lines for min/max scale was exchanged since 2.2, that's while it looked weird to me, not reading the label but been focussed on the position of an input element :-)

#6 Updated by Martin Dobias over 9 years ago

So, do I understand correctly that in your case the crash happens only when modulo operator is used in rule-based renderer for a layer within the group?

Are there more layers with rule-based renderer?

To me it seems a bit unlikely that modulo operator would be causing the crash... (Also when checking the code.) Could you please try to do some more tests to confirm it is really the case?

#7 Updated by Bernhard Ströbl over 9 years ago

Hi Martin,
I tested a lot this morning, results:
  1. the modulo operator is not the cause of the crashes
  2. the rule-based renderer is not the cause of the crashes
  3. labels or scale based visibility are not the cause of the crashes
  4. nor can a single layer be identified to cause the crashes
  5. the minidump message is shown almost immediately after setting the group visible
  6. the minidump message is displayed while the MapCanvas continues drawing until the map is finished
  7. the minidump message is not shown at every crash; there are hard crasehes, too
  8. the crash is reproducible:
    • it only happens when setting the group to visible immediately after QGIS has finished loading the project
    • it only occures when Render layers in parallel using many CPU cores is enabled

#8 Updated by Jürgen Fischer over 9 years ago

Bernhard Ströbl wrote:

  1. the crash is reproducible:
    • it only happens when setting the group to visible immediately after QGIS has finished loading the project
    • it only occures when Render layers in parallel using many CPU cores is enabled

Can you really reproduce it reliably? I was unable to reproduce this kind of crashes so far. The minidumps are unconclusive (at least to me). And reopening the project a couple of times usually makes it work again. It seems to be some subtle race condition that isn't triggered too often.

#9 Updated by Bernhard Ströbl over 9 years ago

Jürgen Fischer wrote:

Can you really reproduce it reliably? I was unable to reproduce this kind of crashes so far. The minidumps are unconclusive (at least to me). And reopening the project a couple of times usually makes it work again. It seems to be some subtle race condition that isn't triggered too often.

I did not look at the minidump, however it is reproducible: I start QGIS which loads the project by default. When loading has finished and I set the group to visible QGIS always crashes (at least this morning in about 20 cases). If I e.g. open the group and set one layer to visible first and afterwards set the whole group to visible there is no crash.

#10 Updated by Martin Dobias over 9 years ago

Thanks for further testing.

Will you still get a crash if you reduce the group to 5-10 layers? If yes, would replacing these few layers with other backend (shapefile?) avoid crashes?

Are there any visible postgis layers outside of that group?

I have a suspicion that the crash could be triggered by PQconnectDb() or something related to that. I found a very old (1998) mailing list discussion about its thread safety, but libpq seems to have evolved a lot since then...

#11 Updated by Bernhard Ströbl over 9 years ago

Martin Dobias wrote:

Thanks for further testing.

Thanks for your patience

Will you still get a crash if you reduce the group to 5-10 layers?

yes

If yes, would replacing these few layers with other backend (shapefile?) avoid crashes?

no crash with shape files

Are there any visible postgis layers outside of that group?

No, BUT (tada!) if I set another PostGIS layer visible in the project, store the project, restart QGIS and set the group visible there is no crash!

I have a suspicion that the crash could be triggered by PQconnectDb() or something related to that. I found a very old (1998) mailing list discussion about its thread safety, but libpq seems to have evolved a lot since then...

#12 Updated by Andreas Neumann over 9 years ago

Hi,

I have very similar problems like Bernhard. The difference is that in my case the crash already happens at the end of project load, before the user get's any chance to interact with QGIS. If I turn off parallel rendering the crashes disappear.

My projects are similar to Bernhards:

  • lots of layers
  • all layer are loaded from PostgreSQL
  • no joins so far (we use PostgreSQL views) - except the value relation widgets
  • lots of nested groups
  • embeded layers and groups from other projects

I had a look at it with Jürgen already but we weren't able to track down the exact cause. The crashes did not happen always, but often.

Hope that we can solve this problem before the next release.

#13 Updated by Bernhard Ströbl over 9 years ago

Hi Andreas,
my workaround is to have one PostGIS layer (with one feature) visible in the project so the connection gets established upon loading (before that only a WMS layer was visible). Maybe your crashes are because you have too many set visible? (This is not an acceptable behaviour, though).
I currently have no embedded layers in my projects.

#14 Updated by Andreas Neumann over 9 years ago

Yes - if I set visibility off for all except one layer, QGIS does not crash. If I then change visibility state to visible for a whole group (with subgroups and many layers), QGIS crashes.

It behaves quite similar to what Bernhard experiences.

#15 Updated by Martin Dobias over 9 years ago

Please try again with lastest master (242754) - some possible crashes were fixed when expressions were used with parallel rendering.

#16 Updated by Andreas Neumann over 9 years ago

It works fine for me. No more crashes for my projects.

#17 Updated by Giovanni Manghi over 9 years ago

  • Resolution set to fixed/implemented
  • Status changed from Open to Closed

Andreas Neumann wrote:

It works fine for me. No more crashes for my projects.

I assume that this should be fixed. Please test and reopen if necessary, cheers!

#18 Updated by Bernhard Ströbl over 9 years ago

  • Affected QGIS version changed from master to 2.6.0
  • Status changed from Closed to Reopened
  • Resolution deleted (fixed/implemented)

Sorry, I have to reopen this again. I checked back with Andreas and the problem persists on QGIS 2.6 on Win7 64 bit

#19 Updated by Nathan Woodrow over 9 years ago

Try setting the max thread count in the options to max - 1. e.g if you can have max threads of 8 then use 7 or lower. I have round using max threads will crash Roam but if I set it to max-1 then it never crashes.

#20 Updated by Niccolo' Marchi about 9 years ago

Bernhard Ströbl wrote:

Sorry, I have to reopen this again. I checked back with Andreas and the problem persists on QGIS 2.6 on Win7 64 bit

Hi guys,
I have similar problems with 2.6.1(osgeo) on Win8.1 64bit: lots of layers (raster+vector; no particular labelling or style), almost no 3rd party plugins (TomBio prod tools, openlayers).
Zooming in&out quickly, QGIS randomly crashes. The problem disappears if MTR is switched off.
Settings of MTR were the default ones from installation. The processor is an i5 dual-core.
Hope this can help in some way.
Thank you all.

#21 Updated by Andreas Neumann almost 9 years ago

Can you please all check if the problem still persists? At my system the crashes are gone since the recent modifications of Jürgen regarding MTR and Postgis: 1e96813 and c550af2

#22 Updated by Giovanni Manghi almost 9 years ago

  • Status changed from Reopened to Feedback

#23 Updated by Bernhard Ströbl almost 9 years ago

Confimred, problem seems gone in f4c3f3d (OSGEO-Install)

#24 Updated by Giovanni Manghi almost 9 years ago

  • Status changed from Feedback to Closed
  • Resolution set to fixed/implemented

reopen if necessary.

Also available in: Atom PDF