Bug report #4421
Raster are very slow to load when calculating stats
|Affected QGIS version:||master||Regression?:||No|
|Operating System:||Easy fix?:||No|
|Pull Request or Patch supplied:||No||Resolution:||worksforme|
|Crashes QGIS or corrupts data:||No||Copied to github as #:||14353|
Rasterlite layers, e.g. http://download.gfoss.it/TrueMarble/TrueMarble-2km.sqlite are extremely slow to open (I remember a while ago they were surprisingly fast); I noticed it uses only one CPU out of four, BTW
#11 Updated by Giovanni Manghi over 7 years ago
- Priority changed from Normal to 6
Giovanni Manghi wrote:
Paolo Cavallini wrote:
I can open it with c25ae56, it's just very slow.
On another machine it works (forget it).
The source of slowness seems to be if you some standard deviation value active in the raster properties.
Ok, I found the pattern.
In the raster properties, style tab, there are two properties that is possible to save (diskette icon) in order to make them persistent between QGIS sessions. Those two properties are "contrast enhancement" and "use standard deviation".
First Note) the two are saved to "no stretch" and the other to not use standard deviation ----> the raster (tested with a TIFF and a Rasterlite, both 3 bands rasters) is loaded very fast as the rasters statistics are not computed and the ".xml" file (the statistic raster file) is not created.
Second Note) the standard deviation parameter alone cannot be saved (when "contrast enhancement" is set to "no stretch"). Try choose it, set it to a certain value, save clicking on the diskette icon and load again a raster, you will see that it will show always unselected.
Third Note) if the "contrast enhancement" parameter is set and saved to something else than "no stretch" then raster statistics are computed on raster load taking a long time depending on raster size (the ".xml" will be also created). When doing this the "use standard deviation" parameter is set (automatically and wrongly) to "2" leading also the raster to load with a wrong colortable. In this case it is also not possible to override the "use standard deviation" configuration as it will alwaus revert to a checked state with a value of "2". Once the ".xml" file is created the raster will load fast, but with the already "use standard deviation" issue.
#15 Updated by Giovanni Manghi over 7 years ago
- Priority changed from 6 to High
I have made a series of tests and took a series of notes that right now I'm not able to translate in english.
From what I see there seems to be a few facts:
a) calculating stats is very slow for the attached rasterlite example (and probably other RL datasets), not for rasters in general
b) calculatings stats can take a while depending on raster size, but after the .xml file is created loading is lightning fast
c) stats are always computed for GRASS rasters (when adding them to the canvas), this because for this format the stretching is always enabled and when is enabled then QGIS computes stats. Moreover stats for GRASS rasters are apparently not saved anywhere.
d) it is not clear why it should be necessary to compute stats/stretching to show rasters "as they are" -> stats/stretching should be an optional choice of the user in case he/she needs to have a more detailed colormap.
#17 Updated by Giovanni Manghi over 7 years ago
- Resolution set to worksforme
- Status changed from Feedback to Closed
Probably the changes made by Alexander recently have provided benefit for this issue as also for #4862.
The overall behaviour when loading a raster is much more better now. Stats are not forced to be computed on load even when stretching is enabled, as no standard deviation is used by default (that caused rasters to load with a "funny" colormap).
Loading the raster attached in this ticket is fast (now).
Computing histograms is kind of slow (but acceptable), but this seems an overall issue for all rasters but not a "blocker" issue.