Bug report #11631
field calculator update is very slow
| Status: | Closed | ||
|---|---|---|---|
| Priority: | Normal | ||
| Assignee: | - | ||
| Category: | Field calculator | ||
| Affected QGIS version: | 2.6.0 | Regression?: | No |
| Operating System: | Easy fix?: | No | |
| Pull Request or Patch supplied: | No | Resolution: | worksforme |
| Crashes QGIS or corrupts data: | No | Copied to github as #: | 19884 |
Description
I select a small number of records in a layer with a few hundred thousand records and use the field calculator to update them. What used to be (or should be) an almost instant operation takes many seconds. It is longer than I would take to update them manually in the table.
History
#1
Updated by Giovanni Manghi about 10 years ago
- Category set to Field calculator
- Status changed from Open to Feedback
do you see any difference if you use the FC with the table of attributes open or closed?
#2
Updated by Gavin Fleming about 10 years ago
540000 road LineString records, 8 core i7 with SSD, 8GB RAM
no difference updating 17 records whether table is open or not, both 30s.
Opening the table just adds another looong wait (how can this be minimised?).
#3
Updated by Giovanni Manghi about 10 years ago
Gavin Fleming wrote:
540000 road LineString records, 8 core i7 with SSD, 8GB RAM
no difference updating 17 records whether table is open or not, both 30s.
Opening the table just adds another looong wait (how can this be minimised?).
postgis?
#4
Updated by Gavin Fleming about 10 years ago
local PostgreSQL 9.3, PostGIS 2.1
#5
Updated by Giovanni Manghi about 10 years ago
Gavin Fleming wrote:
540000 road LineString records, 8 core i7 with SSD, 8GB RAM
no difference updating 17 records whether table is open or not, both 30s.
Opening the table just adds another looong wait (how can this be minimised?).
I'm really not lucky as it doesn't seem I'm able to replicate any of the issues you file :)
In this case:
- I downloaded the OSM UK roads shape, 3 million features and imported it into a local postgis installation (9.3, Linux, 4 cores, 8gb ram and SSD).
- selected a few tens of records/features (20/30) and updated an attribute.
- repeated the above after selecting ~200k features
results:
in both cases the update operation, after clicking the "ok" button in the field calculator, takes a few seconds (eventually what can be surprising is the fact that the time it takes does not seems to differ much between a few records selected and a lot of records selected).
In the past we had issues when updating (or creating) fields with the FC and at the same time having the table of attributes open. I remember also another similar issue that was related to the number of features that were visible the moment the FC was used. In both cases fixes/patches has been applied and the situation is much better now.
Opening a 3 million table of attributes table here take ~25 seconds from shapefile and ~40 from postgis.
Unchecking the "select at id" option when loading the postgis layer make a little faster the operation, ~30 seconds, but in this case there may be a bug as the operation is done 2 times (to notice is necessary to open a postgis table of at least 80/100k features). Jurgen, is this right?
To make faster opening large table is usually used the "show features visible on map" option, but this is known to be bugged for postgis layers.
#6
Updated by Giovanni Manghi over 9 years ago
- Resolution set to worksforme
- Status changed from Feedback to Closed
closing for lack of feedback.