Bug report #19919

QGIS 2.18 faster than QGIS 3.3 DEV in processing

Added by salvatore fiandaca about 2 years ago. Updated about 2 years ago.

Status:Closed
Priority:High
Assignee:Nyall Dawson
Category:Processing/Core
Affected QGIS version:3.3(master) Regression?:Yes
Operating System:win 10 - osgeo4w Easy fix?:No
Pull Request or Patch supplied:No Resolution:invalid
Crashes QGIS or corrupts data:No Copied to github as #:27743

Description

qgis 2.18.24 is more performing than the master version 3.3-0.79

I did various tests with the processing tools and 2.18 is faster than the master, in particular: the processing 'counts points in the polygon':
QGIS 2.18.24 time 90 sec
QGIS DEV time 134 sec

I attach db sqlite with point tables (> 1M points) and polygons

https://mega.nz/#!dYxjSQCY!eP69I0MWnyPBQ0zp8aDMYVCBRXQfQEkMJAGd9Y5-3Yg

Associated revisions

Revision 93f83a02
Added by Nyall Dawson about 2 years ago

[processing] Remove unnecessary index creation in points in polygons

...and rely instead on the data provider's spatial index.

Refs #19919

Revision 17567ee5
Added by Nyall Dawson about 2 years ago

[processing] Never run feature validity check for point layers

Saves a few cycles. The feature validity checks have no meaning
for point layers.

Refs #19919

History

#1 Updated by Nyall Dawson about 2 years ago

  • Status changed from Open to Feedback

Is this running a debug build of qgis 3 (eg the nightly builds)? You need to compare against a release build.

#2 Updated by salvatore fiandaca about 2 years ago

Nyall Dawson wrote:

Is this running a debug build of qgis 3 (eg the nightly builds)? You need to compare against a release build.

I redid the test using the 2.18.24 and 3.2.3,
2.18 is much faster

regards

#3 Updated by Nyall Dawson about 2 years ago

What about disabling the geometry validation? (From processing options). Does that fix the regression?

#4 Updated by Nyall Dawson about 2 years ago

  • Assignee set to Nyall Dawson

#5 Updated by salvatore fiandaca about 2 years ago

Nyall Dawson wrote:

What about disabling the geometry validation? (From processing options). Does that fix the regression?

Yes,
now QGIS 3.X is very fast
thank you so much!

#6 Updated by Nyall Dawson about 2 years ago

Faster than 2.18? (It should be!) I've got a pr coming in that further optimises this algorithm.

#7 Updated by Giovanni Manghi about 2 years ago

Closing?

#8 Updated by Nyall Dawson about 2 years ago

  • Status changed from Feedback to Closed
  • Resolution set to invalid

Yes. But I'd still love to see results for 3.2 with that feature disabled.

#9 Updated by salvatore fiandaca about 2 years ago

Nyall Dawson wrote:

Yes. But I'd still love to see results for 3.2 with that feature disabled.

Other comments:
QGIS 3.2 is slow at first use, after a while it resumes and I can not explain why.
test results:
2.18.24 time 90 sec
3.2.3 time 90 sec

but repeating the test several times the results change little.

#10 Updated by salvatore fiandaca about 2 years ago

I redid test after PR of Nyall in 3.3 dev
time 52 sec

thanks Nyall

win 10 64 bit
AMD FX-7500 RADEON R7 - 10 Compute cores 4C+6G 2.10 GHz - 8 GB RAM

Also available in: Atom PDF