Bug report #4955
Wrong results, when calculated results are too long for DBF field
|Affected QGIS version:||master||Regression?:||No|
|Operating System:||Easy fix?:||No|
|Pull Request or Patch supplied:||No||Resolution:||duplicate|
|Crashes QGIS or corrupts data:||No||Copied to github as #:||14750|
This was tested both on 1.7.3 and 1.9.90.
A new field in a shape file was created with the field calculator (within 'Attribute table' window). An area of a polygon was calculated with $area. The returned value was absolutely wrong. It turns out that the field is too short for the result. But there were no warnings!!! Further experiments return following results:
Field type: N(10,0) Result: 727869376 — wrong result (N. B. – there are only 9 numbers)
Field type: N(11,0) Result: -2147483648 — wrong result (N. B. – the integer part of the result is 11 numbers)
Field type: N(13,1) Result: 65152378815,7 — correct result
Additional verification of field type and number should be introduce before writing out to shape file. I suppose this is similar problem to Bug #3500.
#2 Updated by alobo - over 11 years ago
Actually, it's a problem of the field type:
If Precision = 0, even if field type has been set to Float
the result is wrong, Precision must be > 0,
I think that if Precision = 0, then the type is set to integer
(even if the user has selected float) and the results are wrong.
Field width=20, real and Precision=3 gives a correct result
Field width=20, real and Precision=0 gives a wrong result