Bug report #9793

raster cumulative count cut gives wrong values

Added by Giovanni Manghi over 6 years ago. Updated over 6 years ago.

Status:Closed
Priority:Normal
Assignee:Radim Blazek
Category:Rasters
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:18334

Description

See the attached raster, it is byte type with just 10 different integer values.

When using qgis to create a color ramp using the CCC (2% and 98%) the
values seems wrong for two reasons:

*) the values presented are 0.996094 and 254.004 : if 0 is the min and
255 is max then that values do not seem that have been cut at 2% on
each side

*) shouldn't the CCC use the raster min/max values, in this case 1 and
10, to compute the cut? otherwise the ramp will be not very useful, in
this case almost all black or white.

up_5m.tif (301 KB) Giovanni Manghi, 2014-03-16 04:45 PM

up_5m-int16.tif (268 KB) Etienne Tourigny, 2014-03-16 05:21 PM

History

#1 Updated by Etienne Tourigny over 6 years ago

Actually it seems there is a bug specific to Byte data, the min/max are always set to 0.996094 and 254.004 regardless of actual data.

Attaching the same file with Int16 data, and CCC min/max seem ok.

#2 Updated by Etienne Tourigny over 6 years ago

  • Status changed from Open to Closed

fixed in master 3e4a915dd239313f71d26d72b72b5d9a160d22f6.

I also added a small fix, by rounding down/up the min/max values to nearest integer. So in this case the min/max are 1 and 9 instead of 1.0 and 8.2.

Plese reopen if it's not ok with you.

Also available in: Atom PDF