Bug report #3840

"additional no data value" has no effect (identify, raster calculator, histograms, etc.)

Added by Giovanni Manghi over 8 years ago. Updated almost 6 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Rasters
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:13898

Description

This is probable due to the changes in the raster provider.

Using trunk under Ubuntu.

osgeo4w-gdal-1.10.1-qgis-dev-nodata.png - gdalinfo and QGIS stats (175 KB) Mikhail Titov, 2013-12-06 07:58 AM

osgeo4w-gdal-1.10.1-qgis-dev-nodata-extra.png - Then we set Additional NODATA to 9999 presuming QGIS thinks it is (31.8 KB) Mikhail Titov, 2013-12-06 07:58 AM

osgeo4w-gdal-1.10.1-qgis-dev-nodata-extra-range-fail.png - And actual range shows real NODATA as min value (42.4 KB) Mikhail Titov, 2013-12-06 07:58 AM

dem.tif - GeoTIFF with NODATA=-999.9 to test on rounding (1.89 MB) Mikhail Titov, 2013-12-06 08:01 AM

Associated revisions

Revision c4b24806
Added by Radim Blazek almost 6 years ago

convert GDAL no data value to a value representable by data type, fixes #3840

History

#1 Updated by Paolo Cavallini about 8 years ago

  • Tracker changed from Bug report to 4
  • Start date set to 2011-07-25
  • Pull Request or Patch supplied set to No

#2 Updated by Giovanni Manghi almost 8 years ago

  • Target version changed from Version 1.7.0 to Version 1.7.4

#3 Updated by Radim Blazek almost 8 years ago

Setting "No data value" has effect on rendering - pixels are rendered transparent.

It does not have effect on identify tool (and similar) and it cannot be easily fixed. While the no data value is set on layer, QgsRasterLayer::identify() calls QgsRasterDataProvider::identify(), and the provider does not know about the no data value set by user on the raster layer. The reason why raster provider returns textual results (QMap<QString, QString>), which can hardly be modified by layer, is to allow for example WMS provider to represent various strange results, e.g. feature attributes, non just a raster value.

#4 Updated by Giovanni Manghi almost 8 years ago

  • Assignee deleted (Redmine Admin)
  • Subject changed from setting the "no data value" in raster properties has no effect to setting "no data value" in raster properties is not working
  • Priority changed from Low to 6

Actually in QGIS 1.7.3 and master setting the "no data value" has no effect on the pixel value. The only effect it has is turning the pixels with a certain value transparent, but for this we already have the "transparent pixel list".

The function of "no data value" should be to make pixels of a certain value as NULL.

I add that this field would be much better if could accept many values, separated by a comma (or with a selector that shows all the unique values).

#5 Updated by Giovanni Manghi almost 8 years ago

  • Crashes QGIS or corrupts data set to No
  • Subject changed from setting "no data value" in raster properties is not working to "No data value" in raster properties is not working
  • Affected QGIS version set to master

#6 Updated by Alexander Bruy almost 8 years ago

See also #4312 and #4313

#7 Updated by Giovanni Manghi over 7 years ago

  • Priority changed from 6 to High

#8 Updated by Giovanni Manghi over 7 years ago

  • Tracker changed from 4 to Bug report

#9 Updated by Paolo Cavallini over 7 years ago

  • Target version changed from Version 1.7.4 to Version 1.8.0

#10 Updated by Paolo Cavallini about 7 years ago

  • Target version changed from Version 1.8.0 to Version 2.0.0

#11 Updated by Giovanni Manghi about 7 years ago

  • Operating System deleted (All)
  • Status info deleted (0)
  • Subject changed from "No data value" in raster properties is not working to "additional no data value" has no effect (identify, raster calculator, etc.)

Just spoke with both Radim and Marco at the developer meeting and they agree that this ticket should be stay open.

Even after all the latest changes in rasters the original issue is still there:

manually choosing a "additional no data value" has the only effect to make that values transparent. Both the identify tools and the raster calculator tools take into account the original pixel value.

This is confusing for users as they expect to be able to turn to NULLs the values defined in "additional no data value", and have those pixels considered as NULLs by all QGIS tools in that particular project.

#12 Updated by Giovanni Manghi about 7 years ago

Giovanni Manghi wrote:

Just spoke with both Radim and Marco at the developer meeting and they agree that this ticket should be stay open.

Even after all the latest changes in rasters the original issue is still there:

manually choosing a "additional no data value" has the only effect to make that values transparent. Both the identify tools and the raster calculator tools take into account the original pixel value.

This is confusing for users as they expect to be able to turn to NULLs the values defined in "additional no data value", and have those pixels considered as NULLs by all QGIS tools in that particular project.

Also histograms are affected, as the the values defined in "additional no data value" are still computed.

#13 Updated by Giovanni Manghi about 7 years ago

  • Subject changed from "additional no data value" has no effect (identify, raster calculator, etc.) to "additional no data value" has no effect (identify, raster calculator, histograms, etc.)

#14 Updated by Giovanni Manghi about 7 years ago

see also #1380

#15 Updated by Giovanni Manghi almost 7 years ago

  • Priority changed from High to Normal

#16 Updated by Giovanni Manghi over 6 years ago

at least the identify seems fixed, histograms and raster calc are still affected.

#17 Updated by Radim Blazek over 6 years ago

Giovanni Manghi wrote:

at least the identify seems fixed, histograms and raster calc are still affected.

Statistics and histogram should be fixed for GDAL in 8e7ffd7. AFAIK, the raster calculator has to be changed to read data from QGIS providers (it is using GDAL directly) but it is not planned for 2.0. Marco, can you comment?

If the calculator is the only problem left, I would suggest to close this issue and create a new one for raster calculator.

#18 Updated by Giovanni Manghi over 6 years ago

  • Status changed from Open to Feedback

#19 Updated by Mikhail Titov almost 6 years ago

Just an update that this is still a problem with QGIS nightly build from OSGeo4W.

gdalinfo seems to report correct NODATA and stats, however QGIS-dev (1e8635c) shows bogus data. I'm attaching screenshots with correct gdalinfo output, messed up info in QGIS master, and I attach sample GeoTIFF to play with.

#20 Updated by Mikhail Titov almost 6 years ago

I forgot to attach GeoTiff

#21 Updated by Giovanni Manghi almost 6 years ago

  • Target version changed from Version 2.0.0 to Future Release - High Priority
  • Status changed from Feedback to Open

#22 Updated by Radim Blazek almost 6 years ago

The new problem is not that additional no data value is ignored but that the original data source no data value (-999.9) is ignored, right?

#23 Updated by Mikhail Titov almost 6 years ago

I would not call it entirely new problem. I feel like these are all related. While true, that original negative fractional NODATA value is not respected, I cannot override it with additional no data value setting either.

If I use identity tool on original data as is, it reports empty areas as having 9999 value (instead of NODATA=-999.9). Histogram tool is also affected. If I set additional value to 9999, identity tool says the value on one of those cells is -999.8999 alike. I suspect there is direct float comparison, (perhaps with different bit width/precision storage, otherwise it got to work if read from GDAL) and mutual exclusion for NODATA setting, so 2 values can't be respected at the same time. I did not look at the code. It is just my guess from observed behavior.

#24 Updated by Radim Blazek almost 6 years ago

The problem is that the decimal no data value -999.9 cannot be represented with sufficient precision by the raster data type float32. -999.9 converted to float32 results in -999.9000244140625. No data value is stored in the raster as double, i.e. -999.89999999999997726 which cannot be represented by float32. The correct no data value for float32 data type could only be -999.9000244140625.

A solution could be to pass no data doubles given by GDALGetRasterNoDataValue through actual data type to get a representable value.

Until we fix that, you can simply disable (uncheck) original nonrepresentable no data value in transparency tab and everything should work as expected.

#25 Updated by Radim Blazek almost 6 years ago

  • Status changed from Open to Closed

#26 Updated by Radim Blazek almost 6 years ago

Fixed.

The next bug report will be: "Why QGIS gives no data value -999.9000244140625 while GDAL -999.89999999999997726?"

Also available in: Atom PDF