Bug report #16006
Inefficient use of GDAL block cache during raster rendering
| Status: | Closed | ||
|---|---|---|---|
| Priority: | Normal | ||
| Assignee: | - | ||
| Category: | Rasters | ||
| Affected QGIS version: | 2.8.0 | Regression?: | No |
| Operating System: | Easy fix?: | No | |
| Pull Request or Patch supplied: | No | Resolution: | |
| Crashes QGIS or corrupts data: | No | Copied to github as #: | 23922 |
Description
Each time the canvas is rendered, the rendering of GDAL raster layers is done with a new GDAL dataset object. Which means the dataset must be re-opened (potentially source of slowness), but more importantly, the GDAL block cache is of no use between 2 canvas rendering. For example when panning in the same area, cached tiles could potentially be reused. One can especially observe the slowness with slow compression methods (JPEG2000, LZMA, etc)
This issue didn't exist in ancient QGIS version (it exists in 2.8 and ongoing, perhaps older). This is due to the cloning of the QgsRasterPipe object at rendering.
Presumably some pool of GDAL dataset objects should be used to avoid those constant open & close operations
Associated revisions
[GDAL provider] Use same underlying GDAL dataset for clone() (fixes #16006)
This will improve efficiency of GDAL block cache during raster rendering,
since the cached blocks are only valid during GDALDataset instance lifetime.
History
#1
Updated by Giovanni Manghi almost 7 years ago
- Easy fix? set to No
- Regression? set to No
#2
Updated by Even Rouault over 6 years ago
- Description updated (diff)
Just hit that issue again with a GDAL driver that does network accesses. Each time you scroll the canvas, the previously cached blocks are discarded.
For reference, stack trace is:
(gdb) bt
#0 GDALOpen (pszFilename=0x537abc8 "/home/even/qgis/2.18/build/eedai", eAccess=GA_ReadOnly) at gdaldataset.cpp:2568
#1 0x00007fa16a38a908 in QgsGdalProviderBase::gdalOpen (pszFilename=0x537abc8 "/home/even/qgis/2.18/build/eedai", eAccess=GA_ReadOnly) at /home/even/qgis/2.18/src/providers/gdal/qgsgdalproviderbase.cpp:272
#2 0x00007fa16a38c7d7 in QgsGdalProvider::QgsGdalProvider (this=0x52fd2b0, uri=..., update=false) at /home/even/qgis/2.18/src/providers/gdal/qgsgdalprovider.cpp:177
#3 0x00007fa16a38cca9 in QgsGdalProvider::clone (this=0x555d4d0) at /home/even/qgis/2.18/src/providers/gdal/qgsgdalprovider.cpp:192
#4 0x00007fa16a38cd31 in covariant return thunk to QgsGdalProvider::clone() const () at /home/even/qgis/2.18/src/providers/gdal/qgsgdalprovider.cpp:195
#5 0x00007fa1c5d93e4d in QgsRasterPipe::QgsRasterPipe (this=0x54a9dd0, thePipe=...) at /home/even/qgis/2.18/src/core/raster/qgsrasterpipe.cpp:34
#6 0x00007fa1c5d917fe in QgsRasterLayerRenderer::QgsRasterLayerRenderer (this=0x537aed0, layer=0x555d2c0, rendererContext=...) at /home/even/qgis/2.18/src/core/raster/qgsrasterlayerrenderer.cpp:176
#7 0x00007fa1c5d84a65 in QgsRasterLayer::createMapRenderer (this=0x555d2c0, rendererContext=...) at /home/even/qgis/2.18/src/core/raster/qgsrasterlayer.cpp:258
#8 0x00007fa1c5ad6ec3 in QgsMapRendererJob::prepareJobs (this=0x45a0d50, painter=0x45a9cf0, labelingEngine=0x0, labelingEngine2=0x447c580) at /home/even/qgis/2.18/src/core/qgsmaprendererjob.cpp:302
#9 0x00007fa1c5ad1320 in QgsMapRendererCustomPainterJob::start (this=0x45a0d50) at /home/even/qgis/2.18/src/core/qgsmaprenderercustompainterjob.cpp:100
#10 0x00007fa1c5add8b4 in QgsMapRendererSequentialJob::start (this=0x563a210) at /home/even/qgis/2.18/src/core/qgsmaprenderersequentialjob.cpp:74
#11 0x00007fa1c6840206 in QgsMapCanvas::refreshMap (this=0x2b38200) at /home/even/qgis/2.18/src/gui/qgsmapcanvas.cpp:738
#12 0x00007fa1c6920114 in QgsMapCanvas::qt_static_metacall (_o=0x2b38200, _c=QMetaObject::InvokeMetaMethod, _id=49, _a=0x7ffc3945a200) at /home/even/qgis/2.18/build/src/gui/moc_qgsmapcanvas.cxx:191
#13 0x00007fa1c52a4ddf in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#14 0x00007fa1c52accdf in QSingleShotTimer::timerEvent(QTimerEvent*) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#15 0x00007fa1c52a40ec in QObject::event(QEvent*) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#16 0x00007fa1c4622e3c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /home/even/install-qt-4.8.5/lib/libQtGui.so.4
#17 0x00007fa1c46272ba in QApplication::notify(QObject*, QEvent*) () from /home/even/install-qt-4.8.5/lib/libQtGui.so.4
#18 0x00007fa1c5978edf in QgsApplication::notify (this=0x7ffc3945b580, receiver=0x57a49a0, event=0x7ffc3945a960) at /home/even/qgis/2.18/src/core/qgsapplication.cpp:281
#19 0x00007fa1c528f9ee in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#20 0x00007fa1c52c0712 in QTimerInfoList::activateTimers() () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#21 0x00007fa1c52bd704 in timerSourceDispatch(_GSource*, int ()(void), void*) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#22 0x00007fa1bc21c197 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#23 0x00007fa1bc21c3f0 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#24 0x00007fa1bc21c49c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#25 0x00007fa1c52bdf16 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#26 0x00007fa1c46c319e in QGuiEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/even/install-qt-4.8.5/lib/libQtGui.so.4
#27 0x00007fa1c528e69f in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#28 0x00007fa1c528e928 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#29 0x00007fa1c5293668 in QCoreApplication::exec() () from /home/even/install-qt-4.8.5/lib/libQtCore.so.4
#30 0x000000000040ae18 in main (argc=2, argv=0x7ffc3945b9e8) at /home/even/qgis/2.18/src/app/main.cpp:1287
#3
Updated by Even Rouault over 6 years ago
- % Done changed from 0 to 100
- Status changed from Open to Closed
Applied in changeset qgis|3b2b98b012fdfda14761914f01318d079aeaa409.