Bug report #16239

master: (much) slower rendering (and attribute table opening) time compared to 2.18.4

Added by Giovanni Manghi over 7 years ago. Updated over 6 years ago.

Status:Closed
Priority:High
Assignee:-
Category:Map Canvas
Affected QGIS version:master Regression?:Yes
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:24149

Description

On the same machine, with the same settings, the rendering time of QGIS master seems much slower if compared to (for example) 2.18.4

For a 160k polygon features shapefile (which I can provide privately to a dev if necessary) the time to render the layer on master is around 30 seconds, while is around 7/8 on 2.18.4

For what is worth also the time to open the attribute table for the same vector is much slower on master, around 15 seconds compared to around 5.

Related issues

Related to QGIS Application - Bug report #15752: Degradation of rendering performances in MSSQL provider Closed 2016-10-25
Related to QGIS Application - Bug report #16577: Extremely slower time to open attribute table in 2.18.7 c... Closed 2017-05-18
Related to QGIS Application - Bug report #16508: Opening Large Shape or oracle table (table) takes more ti... Closed 2017-05-03

Associated revisions

Revision b97a980b
Added by Nyall Dawson over 7 years ago

Greatly speed up attribute table loading

Don't advise for rows added when a model reset is in progress.
Otherwise the rows are tested for sort order, etc triggering
a bunch of useless calculations, given that the model is in
the process of being reset anyway.

Tested using a 150k point shapefile, decreased attribute table
load times from 50+ seconds to 4 seconds.

Refs #16577, #16239

Revision ebd3e0d7
Added by Nyall Dawson over 7 years ago

Greatly speed up attribute table loading

Don't advise for rows added when a model reset is in progress.
Otherwise the rows are tested for sort order, etc triggering
a bunch of useless calculations, given that the model is in
the process of being reset anyway.

Tested using a 150k point shapefile, decreased attribute table
load times from 50+ seconds to 4 seconds.

Refs #16577, #16239

(forward port from b97a980b99a32f7cbbb8cc32ac6a781246df1171)

Revision 0b95c776
Added by Nyall Dawson over 7 years ago

Don't prefetch attribute table sort values when no sorting set

Shaves some seconds off opening the attribute table in certain
circumstances (no sorting applied)

Refs #16577, #16239

Revision 4acc4805
Added by Nyall Dawson over 7 years ago

Don't prefetch attribute table sort values when no sorting set
Shaves some seconds off opening the attribute table in certain
circumstances (no sorting applied)

Drops load time for table from 100 seconds to 50 seconds for a
2.6 million feature shapefile, and from 6.5 seconds to 3.5 seconds
for a 160k feature shapefile.

Refs #16577, #16239

(cherry-picked from 0b95c77)

Revision ce74d57b
Added by Nyall Dawson over 6 years ago

[postgres] Really disable dynamic queue size

The original commit speed up some layers, but regressed
performance for others. It was decided to revert the
dynamic queue feature, but it seems this was never actually
done.

Fixes #16239, #19203

History

#1 Updated by Nyall Dawson over 7 years ago

Can you shoot me a link to the data/project?

#2 Updated by Giovanni Manghi over 7 years ago

Nyall Dawson wrote:

Can you shoot me a link to the data/project?

sent using DropBox. Cheers!

#3 Updated by Giovanni Manghi over 7 years ago

see also #15752

#4 Updated by Nyall Dawson over 7 years ago

  • Status changed from Open to Feedback

Giovanni,

Are both builds you are using for comparison debug builds? Or are you comparing a release build vs debug build?

#5 Updated by Giovanni Manghi over 7 years ago

  • Status changed from Feedback to Open

Nyall Dawson wrote:

Giovanni,

Are both builds you are using for comparison debug builds? Or are you comparing a release build vs debug build?

I used qgis-master from osgeo4w and 2.14/2.18 from the very same osgeo4w install within the same machine.

#6 Updated by Andre Jesus over 7 years ago

I guess the issue in #15752 can be merged into this one, as the render time is rising in all data sources.

Using Windows 10 64 Bits, OSGeo4W installer 64 Bits:

3x avg render time, MSSQL
2.14.13
3100 ms
2.99.0
8800 ms +64.7%

3x avg render time, Postgres
2.14.13
1300 ms
2.99.0
3600 ms +63.8%

Project info:

As said in #15752, if we compare It to 2.8.9 version the time difference is absurd

#7 Updated by Jürgen Fischer over 7 years ago

Are you comparing builds writing debugging output (qgis-ltr-dev/qgis-rel-dev/qgis-dev; see corresponding note in the about box) and builds not writing debugging output (qgis/qgis-ltr)? Writing debugging output has a performance impact.

#8 Updated by Andre Jesus over 7 years ago

Jürgen Fischer wrote:

Are you comparing builds writing debugging output (qgis-ltr-dev/qgis-rel-dev/qgis-dev; see corresponding note in the about box) and builds not writing debugging output (qgis/qgis-ltr)? Writing debugging output has a performance impact.

Yes, I was: qgis-ltr (2.14.13) vs qgis-dev (2.99.0). And you are right, the render time using qgis-ltr-dev is bigger:

2.14.13-2 (qgis-ltr-dev)
MSSQL: 5100 ms
Postgres: 3000 ms

But again, please check #15752.
Every version since 2.10.x has constantly rising the render time by a high percentage.

#9 Updated by Nyall Dawson over 7 years ago

  • Status changed from Open to Feedback

I believe this report can be closed - it seems the issue was stemming from comparing the debug vs release builds.

#15752 describes a regression in the speed of the mssql provider. I think this is a separate issue which should be left open.

#10 Updated by Giovanni Manghi over 7 years ago

Nyall Dawson wrote:

I believe this report can be closed - it seems the issue was stemming from comparing the debug vs release builds.

#15752 describes a regression in the speed of the mssql provider. I think this is a separate issue which should be left open.

before closing I can compile master with no debug output and try again. Thanks.

#11 Updated by Giovanni Manghi over 7 years ago

  • Regression? set to Yes

#12 Updated by Giovanni Manghi over 7 years ago

  • Priority changed from Severe/Regression to High

#13 Updated by Giovanni Manghi over 7 years ago

  • Easy fix? set to No

#14 Updated by Alexander Bruy over 7 years ago

  • Description updated (diff)

Probably related to #15752

#15 Updated by Giovanni Manghi over 7 years ago

Nyall Dawson wrote:

I believe this report can be closed - it seems the issue was stemming from comparing the debug vs release builds.

#15752 describes a regression in the speed of the mssql provider. I think this is a separate issue which should be left open.

see #16577 (datasouces shapefiles)

#16 Updated by Jürgen Fischer over 7 years ago

  • Related to Bug report #16577: Extremely slower time to open attribute table in 2.18.7 compared to 2.14.14 added

#17 Updated by Jürgen Fischer over 7 years ago

  • Related to Bug report #16508: Opening Large Shape or oracle table (table) takes more time then before added

#18 Updated by Giovanni Manghi over 7 years ago

  • Affected QGIS version changed from master to 2.18.7
  • Status changed from Feedback to Open

Nyall Dawson wrote:

I believe this report can be closed - it seems the issue was stemming from comparing the debug vs release builds.

#15752 describes a regression in the speed of the mssql provider. I think this is a separate issue which should be left open.

Hi Nyall,

I think that this new (among the many others) report speaks volumes:

https://lists.osgeo.org/pipermail/qgis-psc/2017-May/005258.html

#19 Updated by Giovanni Manghi over 7 years ago

  • Affected QGIS version changed from 2.18.7 to master

#20 Updated by Nyall Dawson over 6 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF