Bug report #11887
minimumValue() in 2.6.x for Windows different than Linux for NULL
Status: | Closed | ||
---|---|---|---|
Priority: | Normal | ||
Assignee: | - | ||
Category: | Vectors | ||
Affected QGIS version: | 2.6.0 | Regression?: | No |
Operating System: | Windows | Easy fix?: | No |
Pull Request or Patch supplied: | No | Resolution: | up/downstream |
Crashes QGIS or corrupts data: | No | Copied to github as #: | 20101 |
Description
minimumValue() against the data provider or a vector layer object in Windows if a NULL value is present in the attributes for a field will return a 0 treating the NULL as the minimum value, instead of the minimum numeric value in that field (which is my case is greater than 0). In Linux for 2.6.x it works correctly and returns the minimum numeric value and does not treat NULLs as numeric.
Associated revisions
Don't consider NULL values for minimumValue() and maximumValue()
Fix #11887
Don't quote column names for MIN and MAX for ogrsql
See https://trac.osgeo.org/gdal/ticket/5799#comment:9
Fixes #11887
History
#1 Updated by Matthias Kuhn almost 10 years ago
- Status changed from Open to Feedback
Which data provider are you using?
Is the layer in edit mode and does it contain changes when the mistake happens?
#2 Updated by Jeff Cavner almost 10 years ago
The dataprovider from a vector layer returned from vectorLyr.dataProvider(). Also the same thing happens against just the vectorLyr, so vectorLyr.minimumValue(index) returns the same 0.0 for a null value. The layer is not in edit mode and does not contain changes.
#3 Updated by Matthias Kuhn almost 10 years ago
And which data provider as in postgis/spatialite/ogr ...?
#4 Updated by Jeff Cavner almost 10 years ago
ogr -- shapefile
#5 Updated by Matthias Kuhn almost 10 years ago
Are the OGR versions on your Windows and Linux machine the same?
#6 Updated by Jeff Cavner almost 10 years ago
windows ogr 1.11.1
linux ogr 1.9.2
#7 Updated by Matthias Kuhn almost 10 years ago
- Status changed from Feedback to Open
- Resolution set to up/downstream
Can you open an issue on their tracker?
There's a similar one open that is being fixed for GDAL/OGR 2.0
https://trac.osgeo.org/gdal/ticket/5333
#8 Updated by Jeff Cavner almost 10 years ago
Yes, just created a ticket over there. Thank you for all of the help.
#9 Updated by Jukka Rahkonen almost 10 years ago
By looking at this ESRI document http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=Geoprocessing_considerations_for_shapefile_output NULL values in a numeric field are really stored as zeroes into the .dbf part of shapefile. And if NULL is stored as a zero into the .dbf file there is no way for GDAL to separate it from zero that represents a real zero. I may be wrong but this ticket and the corresponding GDAL ticket feel invalid for me. But I do not understand why Linux makes a difference for you. Could you add a shapefile with a few features and some NULLs for testing?
GDAL ticket 5333 is not related, it was about creating new functions for finding min/max/avg dates from a DATE field of a shapefile which actually a string "CCYYMMDD" in the .dbf file.
#10 Updated by Matthias Kuhn almost 10 years ago
I can reproduce the problem on Linux with GDAL 1.11
Taking the shapefile lakes from here:
https://github.com/opengisch/QGIS-Sampledata/tree/master/shapefiles
Minimum value for the xlabel column (values 28399 and a some NULLs):
iface.activeLayer().dataProvider().minimumValue( 3 )
0
Editing the attriubte table and changing a NULL value to 0 works. The change can be saved and reloaded properly, so GDAL seems to be able to distinguish 0 and NULL for shapefiles.
Upstream issue: https://trac.osgeo.org/gdal/ticket/5799
#11 Updated by Jukka Rahkonen almost 10 years ago
I downloaded the lakes shapefile and had a try with GDAL 2.0-dev on Windows 7 but this version does not show the issue with ogrinfo:
ogrinfo lakes.shp -sql "select min(xlabel) from lakes"
INFO: Open of `lakes.shp'
using driver `ESRI Shapefile' successful.
Layer name: lakes
Geometry: None
Feature Count: 1
Layer SRS WKT:
(unknown)
MIN_xlabel: Integer (8.0)
OGRFeature(lakes):0
MIN_xlabel (Integer) = 28399
#12 Updated by Matthias Kuhn almost 10 years ago
Hmmm, same result here with 1.11.1.
Debugging shows that ogr fails to interpret the sql qgis sends it (see below) and it's actually our fallback code that fails to properly handle NULL values.
QGIS debug output:
src/providers/ogr/qgsogrprovider.cpp: 2360: (minimumValue) [30361ms] Failed to execute SQL: SELECT MIN("xlabel") FROM "lakes"
ogrinfo output:
ogrinfo lakes.shp -sql 'SELECT MIN("xlabel") FROM "lakes"' INFO: Open of `lakes.shp' using driver `ESRI Shapefile' successful. ERROR 1: Argument of column Summary Function 'MIN' should be a column.
Therefore the proper question is, should we send an unquoted column name to ogr or should that be changed on gdal side?
#13 Updated by Matthias Kuhn almost 10 years ago
- Status changed from Open to Closed
Fixed in changeset 1f8dee0caf7b995b1ccef41b2c92636082557da4.