Bug report #6640

Problem with round function of the expression builder

Added by Pedro Venâncio over 11 years ago. Updated about 11 years ago.

Status:Closed
Priority:Severe/Regression
Assignee:Nathan Woodrow
Category:-
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:15820

Description

I found a strange behavior with round function of the expression builder.
When the round is done to integer, everything seems going well. However, by introducing one or more decimal places, the results begin to be strange and often appears the default value -21474836.48.
Here ([1] , [2] , [3]) is an example with rounding areas in labels. The problem happens starting from 2 decimal places. Doing the same with the perimeter, the problem happens starting from 5 decimal places.

The data I am using are the portuguese administrative boundaries [4], but I tested it with other data and also happens.

Gioseppe Sucameli has also confirmed this problem:


-2147483648 is the min integer value, so probably
the way to round number fails somewhere.

I can see that each trailing '0' is removed, in addition
incrementing the number of decimal places the value
becomes negative:

"PM"  || '   ' || round( "PM", 6 ) || '   ' || round( "PM", 12 )

outputs

218317.69947   -2147.483648   -0.002147483648

I'm using QGIS master (ubuntugis unstable) on Xubuntu 12.04 and Gioseppe is using Ubuntu 11.04.

Associated revisions

Revision eeebf5bd
Added by Jürgen Fischer about 11 years ago

fix #6640

History

#1 Updated by Giovanni Manghi over 11 years ago

  • Target version set to Version 2.0.0
  • Category deleted (Labelling)
  • Status changed from Open to Feedback
  • Assignee set to Nathan Woodrow

I agree with Giuseppe, that |number| is the min/max value for integers in a shapfile, so it must be something wrong with rounding.

Is this a regression? can anyone check with an older release?

Please note that there is another ticket asking to add a warning anytime for some reason the length limit of shapes fields is passed, in order to avoid have users guess why their columns are filled with strange values. If possible add your note there also.

I'm giving this ticket to Nathan because he worked lately in the expression builder, but I don't know if he is the right person anyway. Nathan in case just remove your name.

#2 Updated by Nathan Woodrow over 11 years ago

That's cool you can leave it assigned to me.

It's not a regression as I only added this function after 1.8.

#3 Updated by Alister Hood over 11 years ago

The status for this ticket probably shouldn't be "Feedback".

#4 Updated by Giovanni Manghi over 11 years ago

  • Status changed from Feedback to Open

#5 Updated by Anita Graser about 11 years ago

  • Priority changed from Normal to High

I think this bug is much more important. Testing Select by expression, I ran into the following odd situation: I'm
using the lakes.shp from the QGIS sample data and the following two expressions return different results:

$area > 28386358480

selects all lakes, even though only one is that big

$area > 28386358480.0

returns the correct result and only one lake is selected.

http://lists.osgeo.org/pipermail/qgis-user/2013-April/021907.html

This should really be fixed for 2.0.

#6 Updated by Jürgen Fischer about 11 years ago

  • Priority changed from High to Severe/Regression

#7 Updated by Jürgen Fischer about 11 years ago

  • Status changed from Open to Closed

Also available in: Atom PDF