Bug report #11887
minimumValue() in 2.6.x for Windows different than Linux for NULL
|Affected QGIS version:||2.6.0||Regression?:||No|
|Operating System:||Windows||Easy fix?:||No|
|Pull Request or Patch supplied:||No||Resolution:||up/downstream|
|Crashes QGIS or corrupts data:||No||Copied to github as #:||20101|
minimumValue() against the data provider or a vector layer object in Windows if a NULL value is present in the attributes for a field will return a 0 treating the NULL as the minimum value, instead of the minimum numeric value in that field (which is my case is greater than 0). In Linux for 2.6.x it works correctly and returns the minimum numeric value and does not treat NULLs as numeric.
#2 Updated by Jeff Cavner almost 6 years ago
The dataprovider from a vector layer returned from vectorLyr.dataProvider(). Also the same thing happens against just the vectorLyr, so vectorLyr.minimumValue(index) returns the same 0.0 for a null value. The layer is not in edit mode and does not contain changes.
#9 Updated by Jukka Rahkonen almost 6 years ago
By looking at this ESRI document http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=Geoprocessing_considerations_for_shapefile_output NULL values in a numeric field are really stored as zeroes into the .dbf part of shapefile. And if NULL is stored as a zero into the .dbf file there is no way for GDAL to separate it from zero that represents a real zero. I may be wrong but this ticket and the corresponding GDAL ticket feel invalid for me. But I do not understand why Linux makes a difference for you. Could you add a shapefile with a few features and some NULLs for testing?
GDAL ticket 5333 is not related, it was about creating new functions for finding min/max/avg dates from a DATE field of a shapefile which actually a string "CCYYMMDD" in the .dbf file.
#10 Updated by Matthias Kuhn almost 6 years ago
I can reproduce the problem on Linux with GDAL 1.11
Taking the shapefile lakes from here:
Minimum value for the xlabel column (values 28399 and a some NULLs):
iface.activeLayer().dataProvider().minimumValue( 3 )
Editing the attriubte table and changing a NULL value to 0 works. The change can be saved and reloaded properly, so GDAL seems to be able to distinguish 0 and NULL for shapefiles.
Upstream issue: https://trac.osgeo.org/gdal/ticket/5799
#11 Updated by Jukka Rahkonen almost 6 years ago
I downloaded the lakes shapefile and had a try with GDAL 2.0-dev on Windows 7 but this version does not show the issue with ogrinfo:
ogrinfo lakes.shp -sql "select min(xlabel) from lakes"
INFO: Open of `lakes.shp'
using driver `ESRI Shapefile' successful.
Layer name: lakes
Feature Count: 1
Layer SRS WKT:
MIN_xlabel: Integer (8.0)
MIN_xlabel (Integer) = 28399
#12 Updated by Matthias Kuhn almost 6 years ago
Hmmm, same result here with 1.11.1.
Debugging shows that ogr fails to interpret the sql qgis sends it (see below) and it's actually our fallback code that fails to properly handle NULL values.
QGIS debug output:
src/providers/ogr/qgsogrprovider.cpp: 2360: (minimumValue) [30361ms] Failed to execute SQL: SELECT MIN("xlabel") FROM "lakes"
ogrinfo lakes.shp -sql 'SELECT MIN("xlabel") FROM "lakes"' INFO: Open of `lakes.shp' using driver `ESRI Shapefile' successful. ERROR 1: Argument of column Summary Function 'MIN' should be a column.
Therefore the proper question is, should we send an unquoted column name to ogr or should that be changed on gdal side?