Bug report #18406

QGIS 3 is about 3 times slower when calculating attributes.

Added by Borys Jurgiel over 2 years ago. Updated over 2 years ago.

Status:Closed
Priority:High
Assignee:-
Category:Attribute table
Affected QGIS version:3.1(master) Regression?:Yes
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:not reproducable
Crashes QGIS or corrupts data:No Copied to github as #:26296

Description

When calculating attributes, QGIS 3.0 and 3.1 are about 3 times slower than 2.18

Tested on a simple GeoPackage and non-geometry Shapefile by updating 1k, 5k and 10k features with a constant number (just updating, without saving changes).
In all tests 2.18 needed about 1 sec for 1000 rows, while QGIS 3.0 & 3.1 about 3 sec. GDAL 2.2.3 in both cases.

I was also told by a Windows user that he aborted updating 120k features (Shapefile) after 5 hours.

History

#1 Updated by Borys Jurgiel over 2 years ago

  • File deleted (qgis)

#2 Updated by Giovanni Manghi over 2 years ago

  • Priority changed from Normal to High

#3 Updated by Alessandro Pasotti over 2 years ago

Hi Borys, would you be able to add a simple python test to help reproducing this?

#4 Updated by Borys Jurgiel over 2 years ago

Sure, but later this week or even the next weekend :/ Completely out of resources at the moment.

#5 Updated by Borys Jurgiel over 2 years ago

Alessandro, when started writing a simple script, I realized adding simple (geometryless and attributeless) features is yet slower... Or something is wrong with my build.

On my laptop:
QGIS 2.18 (from Buster) ~0.6s
QGIS 3.1 (compiled) ~2.1s usually; occasionally ~1.3s

import timeit

l = QgsVectorLayer("Point?crs=epsg:4326&field=x:integer", 'test', 'memory')
pr = l.dataProvider()

def addFeature():
    pr.addFeatures([QgsFeature()])

timeit.timeit(addFeature, number=100000)

Could anyone confirm please (in the Python console)?

#6 Updated by Nyall Dawson over 2 years ago

Borys - can you confirm that your build is a release build, not debug?

#7 Updated by Borys Jurgiel over 2 years ago

  • Resolution set to not reproducable
  • Status changed from Open to Closed

Nyall - thanks, the build type was set to None and after setting to Release, it's a bit faster. QGIS 3 is still significantly slower than 2.18 when adding features from Python (over 2x for empty features and ~1,5x for features with a geometry and one attribute), however, it was a side problem and adding 100k features in 2-3 seconds isn't any tragedy I guess.

I can't reproduce the original problem (slow updating attributes by the Field Calculator). What was taking 10s in QGIS 2.18 and 30s in QGIS 3 now works immediately. I'm closing it until I have any new observations.

Also available in: Atom PDF