Bug report #9626

Crash when adding OpenLayers layer

Added by Saber Razmjooei about 6 years ago. Updated almost 6 years ago.

Status:Closed
Priority:Normal
Assignee:Martin Dobias
Category:Map Canvas
Affected QGIS version:master Regression?:No
Operating System:Ubuntu Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:Yes Copied to github as #:18199

Description

Error I get prior to crashing QGIS:

undefined[0]: TypeError: 'null' is not an object

This happens immediately after MTR merge. Maybe due to the fact that MTR does not support plugin layer.

console-output.txt Magnifier (6.57 KB) Tobias Preuss, 2014-02-27 08:51 AM


Subtasks

Bug report #9860: openlayers plugin crashes qgis masterRejected

Associated revisions

Revision 4b820362
Added by Jürgen Fischer about 6 years ago

fix crash in network access manager (fixes #9626)

History

#1 Updated by Saber Razmjooei about 6 years ago

QGIS code revision: bc87d51

Affected version should be QGIS 2.3.0-Master

#2 Updated by Etienne Tourigny about 6 years ago

confirmed. Here is debug output.

CANVAS refresh!
CACHE VALID: 0
src/core/qgsmaprendererjob.cpp: 546: (prepareJobs) Rendering at layer item OpenLayers_plugin_layer20140223184848905
src/core/qgsmaprendererjob.cpp: 563: (prepareJobs) layer Google Streets:  minscale:0  maxscale:1e+08  scaledepvis:0  extent:-20037508.3399999998509884,-20037508.3399999998509884 : 20037508.3399999998509884,20037508.3399999998509884  blendmode:0
src/core/qgsmaprendererjob.cpp: 579: (prepareJobs)   extent 1: -35297280.4943747147917747,-21039383.7569999992847443 : 35297280.4943747147917747,21039383.7569999992847443
src/core/qgsmaprendererjob.cpp: 580: (prepareJobs)   extent 2: Empty
src/core/qgsmaprendererjob.cpp: 891: (renderLayerStatic) job 81b3720 start
OpenlayersLayer draw
page file: file:////home/tourigny/.qgis2/python/plugins/openlayers_plugin/html/google_streets.html
QGIS died on signal 11CANVAS update timer!
CANVAS update timer!
CANVAS update timer!
CANVAS update timer!
CANVAS update timer!
Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No thread selected
No stack.
gdb returned 0
Aborted

#3 Updated by Etienne Tourigny about 6 years ago

  • Subject changed from QGIS 2.3.0 Master freezes when adding OpenLayers layer to QGIS 2.3.0 Master crashes when adding OpenLayers layer

#4 Updated by Etienne Tourigny about 6 years ago

  • Crashes QGIS or corrupts data changed from No to Yes

#5 Updated by Jürgen Fischer about 6 years ago

  • Status changed from Open to Closed

#6 Updated by Jürgen Fischer about 6 years ago

Jürgen Fischer wrote:

Fixed in changeset 4b820362c2eb3529bd19e37e40210233b03c92b6.

Still crashes - but now in QWebPage. I guess it's invoked by the plugin from a rendering thread, which QWebPage doesn't support (see QThreadPool , QRunnable and objects that need to be created in main thread).

#7 Updated by Etienne Tourigny about 6 years ago

Quick fix would be to render all plugin layers in the main thread to avoid any problems like this one.

Ideally, plugin layers should have a variable/function canRenderMultiThread() to let simple plugin layers be rendered in a separate thread (but by default, rendered in the main thread).

For this particular plugin, perhaps it can be tweaked following this : http://qt-project.org/doc/qt-4.8/qwebpage.html#using-qwebpage-in-a-widget-less-environment

#8 Updated by Martin Dobias about 6 years ago

Actually it is not a general problem of plugin layers. For example, the Crayfish plugin (which also uses QgsPluginLayer) works fine without any updates.

I strongly prefer not to have optional rendering of plugin layers in main thread as this would make everything much more complex - imagine that if QGIS wants to render 5 layers and the plugin layer is in the middle of the list, in sequential rendering you would need to: render two layers in a worker thread, when it is done render plugin layer in main thread and then continue in the worker thread.

My preferred solution would be to adapt plugins that have this kind of issue. In addition to implementing QgsMapLayer::draw() they can use new API for map layer rendering:
- implement QgsMapLayer::createMapRenderer() that will return a new QgsMapLayerRenderer object. This method is called in main thread at the beginning of the rendering. Ideally the method should not do any heavy work as this would block the main thread
- implement render() method in a subclass of QgsMapLayerRenderer - this function is called in the worker thread and should do the rendering

#9 Updated by Nathan Woodrow about 6 years ago

Couldn't we solve this issue by exposing the QgsDataProvider stuff to Python and implement this stuff as data providers rather then plugin layers. The open layers plugin should really be done as a raster data provider, and something like crayfish done as a vector provider.

It's not going to solve it directly but I guess it moves the logic to the correct location.

#10 Updated by Martin Dobias about 6 years ago

Nathan: currently there is a technical problem that providers are handled by core library, but Python plugins are handled at the level of QGIS application. It also means that in custom applications the custom data providers would not be available. Of course it is nothing that couldn't be sorted out, but at the same time it is quite nice to have providers more under control (I think of them as slightly more low-level pieces of code).

In general I agree that openlayers could be done as a raster data provider (preferably inside qgis source tree) at some point.

#11 Updated by Etienne Tourigny about 6 years ago

The core problem specific to the openlayers plugin, as Jürgen wrote, is that it relies on QWebPage and QWebFrame, which not thread-safe because of GUI elements, supposedly.

That is regardless of whether or not it is implemented as a plugin layer or data provider.

Is there an alternative to QWebPage that is thread-safe?

Also, there might be other plugins with a similar problem (i.e. not thread-safe). It might prove necessary to allow those to render in the main thread, but I guess we can wait and see if any pop up.

#12 Updated by Nathan Woodrow about 6 years ago

  • Tag set to MTR

Added the MTR tag to keep track of any threaded rendering issues.

#13 Updated by Martin Dobias about 6 years ago

Etienne: I believe there is no need to look for alternatives of QWebPage - it is just about changing how the QWebPage is being used. I see no problem in doing all communication with QWebPage in main thread using a helper object that would then pass the rendered image through queued signal to the worker thread (which would start its event loop to receive the signal).

#14 Updated by Etienne Tourigny about 6 years ago

Martin - I have just tested my plugin tileindex, which works with qgis 2.2, and it crashes also, because it uses QPixMap to generate a preview pixmap using QgsRasterLayer::previewAsPixmap.

TileIndex plugin : reading raster /data/research/work/gls/LE72280622009192_RGB.tif
Warning: QPixmap: It is not safe to use pixmaps outside the GUI thread
Stacktrace (piped through c++filt):
/home/softdev/bin/qgis[0x55e5b7]
/home/softdev/bin/qgis(myMessageOutput(QtMsgType, char const*)+0xe2)[0x55e996]
/usr/lib/x86_64-linux-gnu/libQtCore.so.4(qt_message_output(QtMsgType, char const*)+0x30)[0x7ffff2ca14d0]
/usr/lib/x86_64-linux-gnu/libQtCore.so.4(+0x71938)[0x7ffff2ca1938]
/usr/lib/x86_64-linux-gnu/libQtCore.so.4(qWarning(char const*, ...)+0x94)[0x7ffff2ca2084]
/usr/lib/x86_64-linux-gnu/libQtGui.so.4(+0x29f28c)[0x7ffff222228c]
/usr/lib/x86_64-linux-gnu/libQtGui.so.4(QPixmap::QPixmap(QSize const&)+0x34)[0x7ffff2223264]
/home/softdev/lib/libqgis_core.so.2.3.0(QgsRasterLayer::previewAsPixmap(QSize, QColor)+0x43)[0x7ffff42f5c51]
/home/softdev/share/qgis/python/qgis/core.so(+0x3dbe10)[0x7fffcd363e10]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5e75)[0x7fffceb138d5]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5c03)[0x7fffceb13663]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x81a)[0x7fffceba72fa]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0xff4a6)[0x7fffceba74a6]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x4e)[0x7fffcebd9d8e]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0xa106d)[0x7fffceb4906d]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x4e)[0x7fffcebd9d8e]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47)[0x7fffcebdab37]
/usr/lib/python2.7/dist-packages/sip.so(+0x98e2)[0x7fffce8938e2]
/home/softdev/share/qgis/python/qgis/core.so(sipVH_core_45(PyGILState_STATE, void (*)(_sipSimpleWrapper*, PyGILState_STATE), _sipSimpleWrapper*, _object*, QgsFeature&)+0x70)[0x7fffcd2052ef]
/home/softdev/share/qgis/python/qgis/core.so(sipQgsFeatureRendererV2::symbolForFeature(QgsFeature&)+0x87)[0x7fffcd2f0cf1]

Program received signal SIGFPE, Arithmetic exception.
[Switching to Thread 0x7fffbb14f700 (LWP 11969)]
0x00007ffff42f5d7c in QgsRasterLayer::previewAsPixmap (this=0x7fffb0011000, size=..., bgColor=...) at /data/src/qgis/qgis-etiennesky/src/core/raster/qgsrasterlayer.cpp:1110
1110      if ( myExtent.width() / myExtent.height() >=  myQPixmap.width() / myQPixmap.height() )
(gdb) 
(gdb) 
(gdb) 
(gdb) bt
#0  0x00007ffff42f5d7c in QgsRasterLayer::previewAsPixmap (this=0x7fffb0011000, size=..., bgColor=...) at /data/src/qgis/qgis-etiennesky/src/core/raster/qgsrasterlayer.cpp:1110

I guess I will wait for your instructions on how to update plugins to work with MTR.

By the way - pretty impressive stuff!!

#15 Updated by Martin Dobias about 6 years ago

Yeah the Qt warning says it all - QPixmap should not be used in a thread. And, QgsRasterLayer::previewAsPixmap() should be ideally replaced by previewAsImage() - as pixmaps also do not work without running X server.

#16 Updated by Etienne Tourigny about 6 years ago

ok, I'll look into that within the next few weeks, if nothing has changed. Add new function QgsRasterLayer::previewAsImage() and deprecate previewAsPixmap()

#17 Updated by Jürgen Fischer about 6 years ago

Etienne Tourigny wrote:

Quick fix would be to render all plugin layers in the main thread to avoid any problems like this one.

The posted link has a solution approach for the plugin(s), doesn't it? The plugin should move it's QWebPage to the main thread and communicate with it via signals/slots. To me this is a plugin issue now.

#18 Updated by Etienne Tourigny about 6 years ago

I have added QgsRasterLayer::previewAsImage() in 75a2edb and updated the tileindexplugin (in github for now).

However, it is considerably slower than drawing on a pixmap (because it doesn't use hardware acceleration).

SO I didn't deprecate previewAsPixmap() but added a note to use previewAsImage() when rendering.

#19 Updated by Etienne Tourigny about 6 years ago

Jürgen Fischer wrote:

Etienne Tourigny wrote:

Quick fix would be to render all plugin layers in the main thread to avoid any problems like this one.

The posted link has a solution approach for the plugin(s), doesn't it? The plugin should move it's QWebPage to the main thread and communicate with it via signals/slots. To me this is a plugin issue now.

can this issue be moved over to the OpenLayers plugin tracker?

#20 Updated by Etienne Tourigny about 6 years ago

Martin Dobias wrote:

Etienne: I believe there is no need to look for alternatives of QWebPage - it is just about changing how the QWebPage is being used. I see no problem in doing all communication with QWebPage in main thread using a helper object that would then pass the rendered image through queued signal to the worker thread (which would start its event loop to receive the signal).

seems like the right approach - and you seem best suited to implement that (I have no idea how do do this). Please let me know if/when you fix this, I'll see if I can use the same approach for tileindex.

#21 Updated by Etienne Tourigny about 6 years ago

Martin - sorry for more noise... but I have just realized that the rendering of tileindex (basically a custom renderer and symbol layer that render a QImage on each polygon feature) is not done in parallel, even though I activated the parallel option. And there are 6 threads at a time that are started.

Is this expected? I imagine that renderers have to be updated for MTR to work effectively? In that case I'll wait for instructions on the ML. cheers!

#22 Updated by Etienne Tourigny about 6 years ago

I just realized the MTR allots one thread per layer, it does not allocate various threads for layers features - so disregard previous comment.

#23 Updated by Etienne Tourigny about 6 years ago

  • Status changed from Closed to Reopened

#24 Updated by Martin Dobias about 6 years ago

Why reopen? The original bug was fixed (thanks Jurgen!), the openlayers crash cannot be fixed in QGIS.

#25 Updated by Etienne Tourigny about 6 years ago

  • Status changed from Reopened to Closed

sorry - I reopened because the bug is still present.

I tried to move the ticket to the openlayers plugin tracker following http://www.redmine.org/boards/2/topics/16789

but it seems I can only move it to the trackers in which I am dev. Can someone with proper privileges (say Pirmin) transfer it?

#26 Updated by Jürgen Fischer about 6 years ago

Etienne Tourigny wrote:

sorry - I reopened because the bug is still present.

I tried to move the ticket to the openlayers plugin tracker following http://www.redmine.org/boards/2/topics/16789

but it seems I can only move it to the trackers in which I am dev. Can someone with proper privileges (say Pirmin) transfer it?

Why not file another bug there? This ticket was pointing at a problem is QGIS and that was fixed.

#27 Updated by Etienne Tourigny about 6 years ago

I already did that - #9649
cheers!

#28 Updated by Jürgen Fischer about 6 years ago

  • Subject changed from QGIS 2.3.0 Master crashes when adding OpenLayers layer to Crash when adding OpenLayers layer
  • Affected QGIS version changed from 2.0.1 to master

#29 Updated by Tobias Preuss about 6 years ago

I experience the same problem with 2.3.0-Master. Here is the console output attached as a text file.

#30 Updated by Etienne Tourigny about 6 years ago

  • Status changed from Reopened to Closed

This bug was closed intentionally, because it is a problem with the plugin not the main application. See bug #9649

#31 Updated by Pirmin Kalberer almost 6 years ago

  • Status changed from Closed to Reopened

After our discussions in Vienna, Martin offered to provide a reference implementation for Python plugins, which can be used as a guideline for the OL plugin implementation. Since we're now in feature freeze, having a MT compatible OL implementation gets high priority.

#32 Updated by Martin Dobias almost 6 years ago

  • Status changed from Reopened to Closed

A simple example plugin with QWebPage is here:
https://github.com/wonder-sk/qgis-mtr-example-plugin

Also available in: Atom PDF