Bug report #20008
Very Slow Table Operations while the Table is Open
| Status: | Closed | ||
|---|---|---|---|
| Priority: | Normal | ||
| Assignee: | - | ||
| Category: | Attribute table | ||
| Affected QGIS version: | 3.2.3 | Regression?: | No |
| Operating System: | Easy fix?: | No | |
| Pull Request or Patch supplied: | No | Resolution: | fixed/implemented |
| Crashes QGIS or corrupts data: | No | Copied to github as #: | 27830 |
Description
- Adding a field took 5 minutes (table now has 22 total fields, so not massive)
- Field Calculator on ~32k selected records took 8 minutes (literally all I was putting in the cells was: N)
- Field calculator on ~480k records took so many hours that I had to leave it running over the weekend and when I came back, the Field Calculator had "closed unexpectedly"
(I'm running on a linux machine with an Intel Core i7 and 32GB of memory.)
I posed this question to the users list and the final solution was to close the attribute table and perform these operations. The improvement in speed was significant (what took hours before took just minutes when I had the attribute table closed). It'd be great to be able to do this with the table open, however!
Related issues
History
#1
Updated by Luigi Pirelli over 5 years ago
confirmed on windows too with master three days ago... not checked current master
#2
Updated by Giovanni Manghi over 5 years ago
- Operating System deleted (
Linux)
This is an issue that affects QGIS since LONG ago. In fact I'm pretty sure that there should be already a ticket about it. For anyone working with large datasets this is almost a show stopper.
#3
Updated by Luigi Pirelli over 5 years ago
not sure if it is duplicate of #19771
#4
Updated by Giovanni Manghi over 5 years ago
- Related to Bug report #19771: QGIS slow commit on large tables added
#5
Updated by Luigi Pirelli over 5 years ago
related user list thread http://osgeo-org.1560.x6.nabble.com/Very-Slow-Table-Operations-in-QGIS-3-2-3-td5380521.html
#6
Updated by Antonio Viscomi over 5 years ago
This slowness is also present in QGIS 2.18
and also occurs with fewer features in shapefiles (15000-16000 are enough),
slowness increases as the number of unsaved editing operations increases.
With the "attribute table" opened, even "merge elements" or "split feature" operations,
cause some problems (i.e. the counting of the records and the selection of the same does not seem synchronized with these editing operations)
and sometimes this cause freeze of QGIS
#7
Updated by Giovanni Manghi over 5 years ago
- Status changed from Open to Feedback
Antonio Viscomi wrote:
This slowness is also present in QGIS 2.18
and also occurs with fewer features in shapefiles (15000-16000 are enough),
slowness increases as the number of unsaved editing operations increases.With the "attribute table" opened, even "merge elements" or "split feature" operations,
cause some problems (i.e. the counting of the records and the selection of the same does not seem synchronized with these editing operations)
and sometimes this cause freeze of QGIS
please try qgis master (it must contain https://github.com/qgis/QGIS/pull/8177) hopefully that patch helps.
#8
Updated by Antonio Viscomi over 5 years ago
Giovanni Manghi wrote:
Antonio Viscomi wrote:
This slowness is also present in QGIS 2.18
and also occurs with fewer features in shapefiles (15000-16000 are enough),
slowness increases as the number of unsaved editing operations increases.With the "attribute table" opened, even "merge elements" or "split feature" operations,
cause some problems (i.e. the counting of the records and the selection of the same does not seem synchronized with these editing operations)
and sometimes this cause freeze of QGISplease try qgis master (it must contain https://github.com/qgis/QGIS/pull/8177) hopefully that patch helps.
I've just tested in Master 9cae77f262,
It seems to works fine
many thanks for the explanation
Regards
#9
Updated by Giovanni Manghi over 5 years ago
- Resolution set to fixed/implemented
- Status changed from Feedback to Closed