Bug report #4955

Wrong results, when calculated results are too long for DBF field

Added by zirneklitis - almost 9 years ago. Updated about 6 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Vectors
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:duplicate
Crashes QGIS or corrupts data:No Copied to github as #:14750

Description

This was tested both on 1.7.3 and 1.9.90.

A new field in a shape file was created with the field calculator (within 'Attribute table' window). An area of a polygon was calculated with $area. The returned value was absolutely wrong. It turns out that the field is too short for the result. But there were no warnings!!! Further experiments return following results:

Field type: N(10,0) Result: 727869376 — wrong result (N. B. – there are only 9 numbers)

Field type: N(11,0) Result: -2147483648 — wrong result (N. B. – the integer part of the result is 11 numbers)

Field type: N(13,1) Result: 65152378815,7 — correct result

Additional verification of field type and number should be introduce before writing out to shape file. I suppose this is similar problem to Bug #3500.

Latvia.7z - Shape file (compresed with 7z) used for the experiment. (917 Bytes) zirneklitis -, 2012-02-07 01:37 AM


Related issues

Duplicates QGIS Application - Bug report #4766: Missing warnings when inserting wrong values into Attribu... Closed 2012-01-11

History

#1 Updated by Giovanni Manghi over 8 years ago

I can confirm this issue, it can indeed lead to further errors.

#2 Updated by alobo - over 8 years ago

Actually, it's a problem of the field type:
If Precision = 0, even if field type has been set to Float
the result is wrong, Precision must be > 0,
I think that if Precision = 0, then the type is set to integer
(even if the user has selected float) and the results are wrong.
Field width=20, real and Precision=3 gives a correct result
Field width=20, real and Precision=0 gives a wrong result
Agus

#3 Updated by Giovanni Manghi over 8 years ago

Nathan, I added you as watcher to this ticket as you worked in the field calculator. Not sure if you want/can do something about this issue, eventually just remove your name from the list. Thanks.

#4 Updated by Paolo Cavallini about 8 years ago

  • Target version set to Version 2.0.0

#5 Updated by Giovanni Manghi almost 8 years ago

  • Priority changed from High to Normal

#6 Updated by Giovanni Manghi over 6 years ago

see also #10186

#7 Updated by Jürgen Fischer over 6 years ago

  • Target version changed from Version 2.0.0 to Future Release - Lower Priority

#8 Updated by Giovanni Manghi about 6 years ago

  • Resolution set to duplicate
  • Status changed from Open to Closed

duplicate of #4766

Also available in: Atom PDF