QGIS » QGIS Application

QGIS code revision: bc87d51

Affected version should be QGIS 2.3.0-Master

confirmed. Here is debug output.

CANVAS refresh!
CACHE VALID: 0
src/core/qgsmaprendererjob.cpp: 546: (prepareJobs) Rendering at layer item OpenLayers_plugin_layer20140223184848905
src/core/qgsmaprendererjob.cpp: 563: (prepareJobs) layer Google Streets:  minscale:0  maxscale:1e+08  scaledepvis:0  extent:-20037508.3399999998509884,-20037508.3399999998509884 : 20037508.3399999998509884,20037508.3399999998509884  blendmode:0
src/core/qgsmaprendererjob.cpp: 579: (prepareJobs)   extent 1: -35297280.4943747147917747,-21039383.7569999992847443 : 35297280.4943747147917747,21039383.7569999992847443
src/core/qgsmaprendererjob.cpp: 580: (prepareJobs)   extent 2: Empty
src/core/qgsmaprendererjob.cpp: 891: (renderLayerStatic) job 81b3720 start
OpenlayersLayer draw
page file: file:////home/tourigny/.qgis2/python/plugins/openlayers_plugin/html/google_streets.html
QGIS died on signal 11CANVAS update timer!
CANVAS update timer!
CANVAS update timer!
CANVAS update timer!
CANVAS update timer!
Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No thread selected
No stack.
gdb returned 0
Aborted

Jürgen Fischer wrote:

Fixed in changeset 4b820362c2eb3529bd19e37e40210233b03c92b6.

Still crashes - but now in QWebPage. I guess it's invoked by the plugin from a rendering thread, which QWebPage doesn't support (see QThreadPool , QRunnable and objects that need to be created in main thread).

Quick fix would be to render all plugin layers in the main thread to avoid any problems like this one.

Ideally, plugin layers should have a variable/function canRenderMultiThread() to let simple plugin layers be rendered in a separate thread (but by default, rendered in the main thread).

For this particular plugin, perhaps it can be tweaked following this : http://qt-project.org/doc/qt-4.8/qwebpage.html#using-qwebpage-in-a-widget-less-environment

Actually it is not a general problem of plugin layers. For example, the Crayfish plugin (which also uses QgsPluginLayer) works fine without any updates.

I strongly prefer not to have optional rendering of plugin layers in main thread as this would make everything much more complex - imagine that if QGIS wants to render 5 layers and the plugin layer is in the middle of the list, in sequential rendering you would need to: render two layers in a worker thread, when it is done render plugin layer in main thread and then continue in the worker thread.

My preferred solution would be to adapt plugins that have this kind of issue. In addition to implementing QgsMapLayer::draw() they can use new API for map layer rendering:
- implement QgsMapLayer::createMapRenderer() that will return a new QgsMapLayerRenderer object. This method is called in main thread at the beginning of the rendering. Ideally the method should not do any heavy work as this would block the main thread
- implement render() method in a subclass of QgsMapLayerRenderer - this function is called in the worker thread and should do the rendering

Couldn't we solve this issue by exposing the QgsDataProvider stuff to Python and implement this stuff as data providers rather then plugin layers. The open layers plugin should really be done as a raster data provider, and something like crayfish done as a vector provider.

It's not going to solve it directly but I guess it moves the logic to the correct location.

Nathan: currently there is a technical problem that providers are handled by core library, but Python plugins are handled at the level of QGIS application. It also means that in custom applications the custom data providers would not be available. Of course it is nothing that couldn't be sorted out, but at the same time it is quite nice to have providers more under control (I think of them as slightly more low-level pieces of code).

In general I agree that openlayers could be done as a raster data provider (preferably inside qgis source tree) at some point.

The core problem specific to the openlayers plugin, as Jürgen wrote, is that it relies on QWebPage and QWebFrame, which not thread-safe because of GUI elements, supposedly.

That is regardless of whether or not it is implemented as a plugin layer or data provider.

Is there an alternative to QWebPage that is thread-safe?

Also, there might be other plugins with a similar problem (i.e. not thread-safe). It might prove necessary to allow those to render in the main thread, but I guess we can wait and see if any pop up.

Etienne: I believe there is no need to look for alternatives of QWebPage - it is just about changing how the QWebPage is being used. I see no problem in doing all communication with QWebPage in main thread using a helper object that would then pass the rendered image through queued signal to the worker thread (which would start its event loop to receive the signal).

Martin - I have just tested my plugin tileindex, which works with qgis 2.2, and it crashes also, because it uses QPixMap to generate a preview pixmap using QgsRasterLayer::previewAsPixmap.

TileIndex plugin : reading raster /data/research/work/gls/LE72280622009192_RGB.tif
Warning: QPixmap: It is not safe to use pixmaps outside the GUI thread
Stacktrace (piped through c++filt):
/home/softdev/bin/qgis[0x55e5b7]
/home/softdev/bin/qgis(myMessageOutput(QtMsgType, char const*)+0xe2)[0x55e996]
/usr/lib/x86_64-linux-gnu/libQtCore.so.4(qt_message_output(QtMsgType, char const*)+0x30)[0x7ffff2ca14d0]
/usr/lib/x86_64-linux-gnu/libQtCore.so.4(+0x71938)[0x7ffff2ca1938]
/usr/lib/x86_64-linux-gnu/libQtCore.so.4(qWarning(char const*, ...)+0x94)[0x7ffff2ca2084]
/usr/lib/x86_64-linux-gnu/libQtGui.so.4(+0x29f28c)[0x7ffff222228c]
/usr/lib/x86_64-linux-gnu/libQtGui.so.4(QPixmap::QPixmap(QSize const&)+0x34)[0x7ffff2223264]
/home/softdev/lib/libqgis_core.so.2.3.0(QgsRasterLayer::previewAsPixmap(QSize, QColor)+0x43)[0x7ffff42f5c51]
/home/softdev/share/qgis/python/qgis/core.so(+0x3dbe10)[0x7fffcd363e10]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5e75)[0x7fffceb138d5]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5c03)[0x7fffceb13663]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x81a)[0x7fffceba72fa]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0xff4a6)[0x7fffceba74a6]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x4e)[0x7fffcebd9d8e]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(+0xa106d)[0x7fffceb4906d]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyObject_Call+0x4e)[0x7fffcebd9d8e]
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47)[0x7fffcebdab37]
/usr/lib/python2.7/dist-packages/sip.so(+0x98e2)[0x7fffce8938e2]
/home/softdev/share/qgis/python/qgis/core.so(sipVH_core_45(PyGILState_STATE, void (*)(_sipSimpleWrapper*, PyGILState_STATE), _sipSimpleWrapper*, _object*, QgsFeature&)+0x70)[0x7fffcd2052ef]
/home/softdev/share/qgis/python/qgis/core.so(sipQgsFeatureRendererV2::symbolForFeature(QgsFeature&)+0x87)[0x7fffcd2f0cf1]

Program received signal SIGFPE, Arithmetic exception.
[Switching to Thread 0x7fffbb14f700 (LWP 11969)]
0x00007ffff42f5d7c in QgsRasterLayer::previewAsPixmap (this=0x7fffb0011000, size=..., bgColor=...) at /data/src/qgis/qgis-etiennesky/src/core/raster/qgsrasterlayer.cpp:1110
1110      if ( myExtent.width() / myExtent.height() >=  myQPixmap.width() / myQPixmap.height() )
(gdb) 
(gdb) 
(gdb) 
(gdb) bt
#0  0x00007ffff42f5d7c in QgsRasterLayer::previewAsPixmap (this=0x7fffb0011000, size=..., bgColor=...) at /data/src/qgis/qgis-etiennesky/src/core/raster/qgsrasterlayer.cpp:1110

I guess I will wait for your instructions on how to update plugins to work with MTR.

By the way - pretty impressive stuff!!

Yeah the Qt warning says it all - QPixmap should not be used in a thread. And, QgsRasterLayer::previewAsPixmap() should be ideally replaced by previewAsImage() - as pixmaps also do not work without running X server.

ok, I'll look into that within the next few weeks, if nothing has changed. Add new function QgsRasterLayer::previewAsImage() and deprecate previewAsPixmap()

Etienne Tourigny wrote:

Quick fix would be to render all plugin layers in the main thread to avoid any problems like this one.

The posted link has a solution approach for the plugin(s), doesn't it? The plugin should move it's QWebPage to the main thread and communicate with it via signals/slots. To me this is a plugin issue now.

I have added QgsRasterLayer::previewAsImage() in 75a2edb and updated the tileindexplugin (in github for now).

However, it is considerably slower than drawing on a pixmap (because it doesn't use hardware acceleration).

SO I didn't deprecate previewAsPixmap() but added a note to use previewAsImage() when rendering.

Jürgen Fischer wrote:

Etienne Tourigny wrote:

Quick fix would be to render all plugin layers in the main thread to avoid any problems like this one.

The posted link has a solution approach for the plugin(s), doesn't it? The plugin should move it's QWebPage to the main thread and communicate with it via signals/slots. To me this is a plugin issue now.

can this issue be moved over to the OpenLayers plugin tracker?

Martin Dobias wrote:

Etienne: I believe there is no need to look for alternatives of QWebPage - it is just about changing how the QWebPage is being used. I see no problem in doing all communication with QWebPage in main thread using a helper object that would then pass the rendered image through queued signal to the worker thread (which would start its event loop to receive the signal).

seems like the right approach - and you seem best suited to implement that (I have no idea how do do this). Please let me know if/when you fix this, I'll see if I can use the same approach for tileindex.

Martin - sorry for more noise... but I have just realized that the rendering of tileindex (basically a custom renderer and symbol layer that render a QImage on each polygon feature) is not done in parallel, even though I activated the parallel option. And there are 6 threads at a time that are started.

Is this expected? I imagine that renderers have to be updated for MTR to work effectively? In that case I'll wait for instructions on the ML. cheers!

I just realized the MTR allots one thread per layer, it does not allocate various threads for layers features - so disregard previous comment.

Why reopen? The original bug was fixed (thanks Jurgen!), the openlayers crash cannot be fixed in QGIS.

Etienne Tourigny wrote:

sorry - I reopened because the bug is still present.

I tried to move the ticket to the openlayers plugin tracker following http://www.redmine.org/boards/2/topics/16789

but it seems I can only move it to the trackers in which I am dev. Can someone with proper privileges (say Pirmin) transfer it?

Why not file another bug there? This ticket was pointing at a problem is QGIS and that was fixed.

I already did that - #9649
cheers!