Bug report #17523

Raster calculator cannot handle huge raster files

Added by Nicolas Cadieux over 6 years ago. Updated almost 6 years ago.

Status:Closed
Priority:High
Assignee:-
Category:Raster Calculator
Affected QGIS version:master Regression?:No
Operating System:Windows 10_64 Easy fix?:No
Pull Request or Patch supplied:No Resolution:duplicate
Crashes QGIS or corrupts data:Yes Copied to github as #:25420

Description

Hi,
QGis 2.99 crashes when trying to use the raster calculator ("Dem Sol 151013@1" * 2) with very large rasters (84GB). System memory is 64GB with 64GB of virtual memory. Calculator works when the file is smaller. Same result if I use the QGIS raster calculator in processing. I reported this issues a while ago on earlier version of Qgis (Bug report #13336) but was hoping for a fixed in version 3 as this is a game changer for anyone using Lidar files. Hope this helps!

Keep up the good work!
Nicolas Cadieux

User Feedback
h2. Report Details
Crash ID: bec0fb0ea046ec2cc4d8f538117c672a069364b0

Stack Trace

memmove :
GDALRasterBand::IRasterIO :
GDALRegister_GTiff :
GDALRasterBand::RasterIO :
OGRCoordinateTransformation::~OGRCoordinateTransformation :
OGRCoordinateTransformation::~OGRCoordinateTransformation :
OGRCoordinateTransformation::~OGRCoordinateTransformation :
QgsRasterCalculator::processCalculation :
QgsDecorationCopyright::qt_metacast :
QgsDecorationCopyright::qt_metacast :
QgsDecorationCopyright::qt_metacast :
QgsDecorationCopyright::qt_metacast :
QMetaObject::activate :
QAction::activate :
QMenu::actionGeometry :
QMenu::actionGeometry :
QMenu::mouseReleaseEvent :
QWidget::event :
QMenu::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsComposerModel::data :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsComposerModel::data :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
CallWindowProcW :
DispatchMessageW :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
QgsDecorationCopyright::qt_metacast :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info
QGIS Version: 2.99.0-Master
QGIS code revision: 95cf299099
Compiled against Qt: 5.9.2
Running against Qt: 5.9.2
Compiled against GDAL: 2.2.2
Running against GDAL: 2.2.2

System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.16299

simulateRaster.7z - This small script will help you create a big raster for testing (648 Bytes) Nicolas Cadieux, 2017-11-22 06:37 PM

Related issues

Related to QGIS Application - Bug report #13336: Allow raster calculator with huge raster inputs Closed 2015-09-09

History

#1 Updated by Alessandro Pasotti over 6 years ago

  • Assignee set to Alessandro Pasotti

looking ...

#2 Updated by Alessandro Pasotti over 6 years ago

  • Related to Bug report #13336: Allow raster calculator with huge raster inputs added

#3 Updated by Alessandro Pasotti over 6 years ago

  • Assignee deleted (Alessandro Pasotti)

too slow for my machine sorry, I cannot wait 20 minutes to load the raster and another 20 if I scroll the mouse wheel ...

#4 Updated by Giovanni Manghi over 6 years ago

  • Status changed from Open to Feedback

I'm running your script to create the raster to test this issue... the asc file is at 105GB and has not yet finished... how big it is?

#5 Updated by Alexander Bruy over 6 years ago

  • Status changed from Feedback to Closed
  • Resolution set to duplicate

Duplicates #13336

#6 Updated by Nicolas Cadieux over 6 years ago

Giovanni Manghi wrote:

I'm running your script to create the raster to test this issue... the asc file is at 105GB and has not yet finished... how big it is?

r: Nicolas
"Did it work for you? I responded via email but I cannot see the answer here."

#7 Updated by Giovanni Manghi over 6 years ago

Nicolas Cadieux wrote:

Giovanni Manghi wrote:

I'm running your script to create the raster to test this issue... the asc file is at 105GB and has not yet finished... how big it is?

r: Nicolas
"Did it work for you? I responded via email but I cannot see the answer here."

I was able to create the asc, transform to tif and create overviews, didn«t had the time to test the calculator yet.

#8 Updated by Nicolas Cadieux almost 6 years ago

  • Status changed from Closed to Reopened

Hi,
I see the problem is still present in Qgis3... Any plans on fixing this issue?
Nicolas

#9 Updated by Alexander Bruy almost 6 years ago

  • Status changed from Reopened to Closed

Please don't open duplicate tickets.

Also available in: Atom PDF