Bug report #9081
Raster calculator result is not accurate
| Status: | Closed | ||
|---|---|---|---|
| Priority: | Normal | ||
| Assignee: |  Marco Hugentobler | ||
| Category: | Raster Calculator | ||
| Affected QGIS version: | 2.0.1 | Regression?: | No |
| Operating System: | OSX | Easy fix?: | No |
| Pull Request or Patch supplied: | No | Resolution: | |
| Crashes QGIS or corrupts data: | No | Copied to github as #: | 17726 |
Description
When using the raster calculator with this simple example:
@layer1 / 10
the results do not come out as expected. We can verify this by using the identify tool and clicking on a pixel e.g.
Original Layer: 844.1530151367188
Calculated layer: 84.41529846191406
In other cases the difference can be more extreme.
It appears the difference may be due to resampling during the raster calculation process. It would be good so convey to the user the fact that resampling takes place during calculation, or to have an option to prevent resampling during recalculation.
Associated revisions
[rastercalc] Switch all internal calculations to doubles
...for more accurate calculations (fix #9081)
Also:
- Add test suite for raster calculator
- Fix errors with log/log10 calculation and inputs <= 0
- Fix loss of nodata values in certain circumstances
History
#1
Updated by Nyall Dawson almost 9 years ago
- Status changed from Open to Closed
Fixed in changeset f42f640d3025742911e6edb1d018945f67a38438.