Bug report #21316
Processing: precision loss on decimal number
Status: | Closed | ||
---|---|---|---|
Priority: | Normal | ||
Assignee: | Nyall Dawson | ||
Category: | Processing/Core | ||
Affected QGIS version: | 3.5(master) | Regression?: | No |
Operating System: | Microsoft Windows 7, 64-bit | Easy fix?: | No |
Pull Request or Patch supplied: | No | Resolution: | |
Crashes QGIS or corrupts data: | No | Copied to github as #: | 29134 |
Description
Bug report¶
1. create a new Temporary Scratch Layer
2. add some fieldsname: one
type: decimal number (real)
provider type: double
- length: 32
- precision: 30
name: two
type: decimal number (numeric)
provider type: numeric
- length: 20
- precision: 18
name: three
type: decimal number (decimal)
provider type: decimal
- length: 20
- precision: 18
name: four
type: decimal number (real)
provider type: real
- length: internally set to -1
- precision: internally set to -1
name: five
type: decimal number (double)
provider type: double precision
- length: internally set to -1
- precision: internally set to -1
3. add a new feature
4. add the value "0.01234567890123456789" into each field (length 22, precision 20)
5. save & check results- one : 0.012345678901234568430878013601
- two : 0.012345678901234567
- three: 0.012345678901234567
- four : 0.012345678901234568
- five : 0.012345678901234568
6. When running "Refactor Fields", "Extract by Attribute" or others in Processing, the result is:- one : 0.012345678901234568430878013601
- two : 0.012345678901234568
- three: 0.012345678901234568
- four : 0.01235 ["source field" shows, *precision gets smashed to 5*]
- five : 0.01235 ["source field" shows, *precision gets smashed to 5*]
Expected results¶
The output field should have the same precision as the input field.
Associated revisions
Fix loss of precision when converting fields to uri in memory provider
Fixes #21316
Fix loss of precision when converting fields to uri in memory provider
Fixes #21316
(cherry picked from commit f3cb3488ec5b1edca298833c6a982fe77b056e59)
History
#1 Updated by Nyall Dawson almost 6 years ago
- Assignee set to Nyall Dawson
- Status changed from Open to In Progress
#2 Updated by Nyall Dawson almost 6 years ago
- % Done changed from 0 to 100
- Status changed from In Progress to Closed
Applied in changeset qgis|f3cb3488ec5b1edca298833c6a982fe77b056e59.