QGIS » QGIS Application

Hi Alvaro,

Thank you for looking into this issues and contributing.

The best thing to do is to open a pull request on github, so it can be reviewed, commented and eventually merged.

Ok, thank you very much

I forgot to mention that it would be best to make one pull request for each improvement. So in case one of them is not accepted, it will still be possible to merge the good parts.

ok, I will divide this issue in four single childs tasks.
Thank you!

Hi Alvaro, do you have a branch that can be tested? Cheers!

Hi Giovanni, I am testing my changes, if it successly in some days I will try a pull request.
Thank you very much!

Alvaro Huarte wrote:

I'm making progress, I think I get promising results. Attach a table of results for three tested layers.

wow, I can't wait to test your patches! :)

Alvaro Huarte wrote:

done for test.

github:
https://github.com/ahuarte47/QGIS/commit/28691ff96e0ef2a97a8f1d08898b1c7d97941f7c
+
https://github.com/ahuarte47/QGIS/commit/a8894f8df811c61fe021607d474aa8e117a5f56f

Sorry there are two commits, I forgot to add one edited file

Hi Alvaro, can you please make a Pull Request in the qgis repo? This way other will be able to see it and review and then commit. Thanks!

Done.
I'm new using gitub and I'm learning this tool, Sorry!
Thanks!

Changes goto branch:

https://github.com/ahuarte47/QGIS/tree/Issue_8725

Alvaro Huarte wrote:

I have implemented the simplification of geometries in the vector-providers. The general behavior simplifies the geometries post-fetch the feature from the provider, but OGR-provider simplifies the geometries pre-fetch the features having a extra seed improvement.

Te final result gets ~3x painting speed faster that original code

New branch:
https://github.com/ahuarte47/QGIS/tree/Issue_8725-OGR

I think ready for test! :-)

compiling right now :)

I think ready for test! :-)

got an error during make

Building CXX object src/core/CMakeFiles/qgis_core.dir/qgsmaplayer.cpp.o
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp: In function ‘QgsRectangle calculateBoundingBox(const QVector<QPointF>&)’:
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:102:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:102:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:102:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:103:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:103:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:103:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:104:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:104:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:104:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:105:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:105:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:105:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:109:5: error: ‘x’ was not declared in this scope
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:110:5: error: ‘y’ was not declared in this scope
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp: In function ‘QgsRectangle calculateBoundingBox(QGis::WkbType, unsigned char*, std::size_t)’:
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:125:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:125:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:125:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:126:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:126:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:126:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:127:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:127:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:127:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:128:18: error: ‘numeric_limits’ is not a member of ‘std’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:128:38: error: expected primary-expression before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:128:38: error: expected ‘,’ or ‘;’ before ‘double’
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:135:5: error: ‘x’ was not declared in this scope
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:136:5: error: ‘y’ was not declared in this scope
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp: At global scope:
/home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:149:20: warning: unused parameter ‘sourceWkb’ [-Wunused-parameter]
make[2]: [ 14%] *** [src/core/CMakeFiles/qgis_core.dir/qgsmaprequest.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
Building CXX object src/core/CMakeFiles/qgis_core.dir/qgsmaplayerregistry.cpp.o
make[1]: *** [src/core/CMakeFiles/qgis_core.dir/all] Error 2
make: *** [all] Error 2

Hi Giovanni, I use Visual Studio 2010 and I don't get these errors.
What compiler do you use ?

I will try fix it!

Alvaro Huarte wrote:

Hi Giovanni, I use Visual Studio 2010 and I don't get these errors.
What compiler do you use ?

I will try fix it!

Hi Alvaro, I use Ubuntu Linux, gcc 4.6.3

It's probably a simple #include <limits.h>

Done, thanks!

Alvaro Huarte wrote:

Done, thanks!

Hi Alvaro! I made a few quick tests with shapefiles, measuring just the time to open them (with the "time" command). I have not had the time to try evaluate the improvement when zooming in/out.

The shapefiles (polygons) that I tested are:

http://faunalia.pt/downloads/GTBs_Shapefile.zip
http://faunalia.pt/downloads/MAMMTERR.zip

un-patched QGIS

giovanni@sibirica ~/Desktop $ time qgis MAMMTERR.shp

user 0m10.377s
user 0m10.237s
user 0m10.429s

giovanni@sibirica ~/Desktop $ time qgis GTBs_2011.shp

user 1m2.896s
user 1m5.436s
user 1m4.120s

patched QGIS

giovanni@sibirica ~/Desktop $ time qgis MAMMTERR.shp

user 0m6.352s
user 0m6.244s
user 0m6.820s

giovanni@sibirica ~/Desktop $ time qgis GTBs_2011.shp

user 0m35.278s
user 0m33.286s
user 0m33.602s

So there is really a substantial improvement!

I tested also another shape (big, more that 1.2gb of small polygons) where things get worst(?)

http://www.igeo.pt/scrif/crif/CRIF2011shp.zip

unpatched

giovanni@sibirica ~/Desktop $ time qgis crif2011.shp

user 0m35.186s
user 0m34.870s
user 0m35.214s

patched

giovanni@sibirica ~/Desktop $ time qgis crif2011.shp

user 0m45.571s
user 0m49.311s
user 0m46.279s

Can I test also the vectors you are using to check the rendering times?

Cheers!

Hi Giovanni, Some shapes...

http://idena.navarra.es/descargas/CARTO1_Lin_6CNivelD.zip
http://idena.navarra.es/descargas/GEOLOG_Pol_Litologia.zip

I am going to test your shapes!
Thanks!

Running a rendering performance test once may be strongly biased by other processes running.
For statistically more robust numbers I recommend using qgis_bench

It's really simple to use.

Here some more information:
https://github.com/qgis/QGIS/blob/master/tests/bench/README

Hi, I made tests with 'qgis_bench' to see the speed.

Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.

My shapefiles tested:

Nice!

Alvaro Huarte wrote:

I thought a bit :-), I think more appropriate to add a new "rendering" tab with this configuration similar to global options form.

I agree

Alvaro Huarte wrote:

Hi, I made tests with 'qgis_bench' to see the speed.

Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.

very nice results indeed! as soon as I will understand how to use qgis_bench I will make tests too :)

Hi Giovanni, qgis_bench is easy :-)

I execute in a Console Window (CMD)...

{%BIN_PATH%}/qgis_bench -i 10 -p test_project.qgs >> test_project.log

-i: number of desired iterations
-p: project to test

Regards

Alvaro Huarte wrote:

Hi Giovanni, qgis_bench is easy :-)

I execute in a Console Window (CMD)...

{%BIN_PATH%}/qgis_bench -i 10 -p test_project.qgs >> test_project.log

-i: number of desired iterations
-p: project to test

Regards

I made also tests with qgis_bench

Alvaro Huarte wrote:

Hi,

I have implemented a new tab called 'Rendering' in the options form of the layer where the user can configure the drawing simplification.

Regards

Hi Alvaro,
this means that when loading a layer the user will not see any improvement, isn't it? If yes then it is a pity because for large vectors there is also a huge improvement when loading the layer. What about an option in the general qgis options and then the switch in vector properties to allow override the general configuration?

Hi Giovanni,

I propose that by default the simplification is activated (The check-box, by default, will be checked), after the user can disable it for some reason, so I have been asked several people.

About a general option, I agree, but I do not what others think about it.

Giovanni, thank you very much for your tests and support!
Regards

Hi,

This may be a stupid question, but how does the simplified rendering work when users start to edit vectors? I think that users can not be let to edit those features which are simplified for fast viewing and save edits back to the source shapefile. And what gets saved if used does not edit anything but wants to make a copy by Save as..? Is QGIS saving the simpified layer of can is somehow handle two sort of geometries, the real ones and those temporarily simplified for viewing? Even when other layers are edited the result might be unexpected if user wants to snap vertices of the layer that is edited to vertices of the layer that is rendered as simplified.

Hi @jratike80, this is a good question!, but there is not problem.

The simplification only is applied in the 'getFeatures' request when it becomes from a rendering request (... drawing the layer in the map). Fetch geometries, update geometries... are executed using distinct 'getFeatures' requests where the geometries are fetched without simplification.

Regards

Alvaro Huarte wrote:

The simplification only is applied in the 'getFeatures' request when it becomes from a rendering request (... drawing the layer in the map). Fetch geometries, update geometries... are executed using distinct 'getFeatures' requests where the geometries are fetched without simplification.

You could also combine that with the QgsVectorLayer::isEditable() method - the layer does caching in edit mode. Another potential thing to look at is snapping.

Other things to consider are unaligned accesses. Marco Bernasocchi is currently working in qgis/QGIS-Android on cleaning the code from unaligned accesses (mainly when parsing or construction WKB; ARM doesn't like those), which is basically replacing all occurrences of *foo = bar; with memcpy(foo, &bar, sizeof bar);. I see that you're introducing more of those. Please rework it to use memcpy.

And finally run the stuff through scripts/prepare-commit.sh (or scripts/astyle.sh for already committed stuff).

But enough whining - otherwise it's very nice work - thank you very much :)

Hello Jürgen, thank you very much for your comments!

I have released new commits for the branch https://github.com/ahuarte47/QGIS/compare/Issue_8725-OGR where I replace all '*foo = bar' with 'memcpy'. This has been a lesson for me.

Also I disable the drawing simplification in 'editmode', the geometries are cached for snapping, thanks.

About the scripts, I am going to study

Regards

Hi Jürgen,

After a failed performance test I wanted to do (https://github.com/qgis/QGIS/pull/927#issuecomment-27158451), I retake his message referring to scripts. I do not know very well how execute it. I must use cygwin ?

Thank you very much

To avoid confusion I created a new pull request with the last code.

New pull request:
https://github.com/qgis/QGIS/pull/980

Obsolete pull:
https://github.com/qgis/QGIS/pull/927

Regards

Hey Alvaro,

Really like your work.

I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)

Running qgis_bench has the same effect, just crashes. Very strange.

Nathan

Hi Nathan, thanks for your comments!

I have test some shapes and all works fine in 'release' build.
Can I test your data ?

Regards

Nathan Woodrow wrote:

I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)

Same here. I can reproduce this with polygon layers. Lines and points opened correctly. Also I found that QGIS compiled from old branch (https://github.com/qgis/QGIS/pull/927) works fine.

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

Marco Hugentobler wrote:

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

Hi,

speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.

By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.

Marco Hugentobler wrote:

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.

Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.

IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.

Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.

Giovanni Manghi wrote:

Marco Hugentobler wrote:

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

Hi,

speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.

By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.

Hi, I am working for build a windows installer with this patch.
I build the default installer with the script 'creatensis.pl' but it does not contains my patch. I must study how insert it.

Hi Alvaro,

The effect of disabled anti-aliasing is still perceptible here.
Which portion of the performance is gained by disabling AA actually?

The upper part is rendered without the patch, the lower part with the patch.

Alexander Bruy wrote:

Nathan Woodrow wrote:

I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)

Same here. I can reproduce this with polygon layers. Lines and points opened correctly. Also I found that QGIS compiled from old branch (https://github.com/qgis/QGIS/pull/927) works fine.

I must study it, the code is the same but in last commit I also did a merge with master branch for make the windows installer.

Nathan Woodrow wrote:

Marco Hugentobler wrote:

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.

Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.

IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.

Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.

I agree, this patch improves the 'rendering at large scales', for getting rendering feedback the user can enable this existing option in the global options form of qgis.

To avoid problems in composer I am going disable this patch in 'printing' tasks.

Also, at least windows, there is other important improvement that I want test: Support for 'FileMapping' in the gdal-ogr library (e.g. using boost library) for speed up the access to data. I have experience with it and can be 2x faster.

Nathan Woodrow wrote:

The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.

Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.

IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.

Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.

+1

Matthias Kuhn wrote:

Hi Alvaro,

The effect of disabled anti-aliasing is still perceptible here.
Which portion of the performance is gained by disabling AA actually?

The upper part is rendered without the patch, the lower part with the patch.

Hi Matthias, disable antialiasing for far geometries is perceptible, but for large datasets it save rendering time (I will make tests with some shapes and send results). ArcGis, by example, also cheats with that. I think that is more important to get fast framerates that high quality rendering, But, of course, all people has one opinion, I do what seems best to the community.

regards

Hi Matthias, we have another think coming, :-)

A few days ago, you said me... "... another minor thing is, that the simplification code should probably not be in the request, but rather in the place where the simplification is done, so in the iterator or rather in its own simplification class (maybe other parts of the code need it...)".

Still think the same? I have no problem in creating a new utility class that implements this simplification code as static methods.

Thanks for your recommendations.

Nathan Woodrow wrote:

The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.

Sorry to nitpick, but just for clarification - are you actually meaning rendering at small scales (further zoomed out) is where the performance gain is? If so, I agree that this is a significant issue which would be great to address.

Nyall Dawson wrote:

Nathan Woodrow wrote:

The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.

Sorry to nitpick, but just for clarification - are you actually meaning rendering at small scales (further zoomed out) is where the performance gain is? If so, I agree that this is a significant issue which would be great to address.

Yeah. Just stuffed up my terms.

Ok - in that case I run into this same issue almost daily, and any improvements would be greatly appreciated!

Nyall Dawson wrote:

Ok - in that case I run into this same issue almost daily, and any improvements would be greatly appreciated!

I hope that with this patch, and the improvement of "FileMapping" I want to add to gdal-ogr providers, it result a very good behavior of qgis for render large datasets.

Best regards

Alvaro Huarte wrote:

Alexander Bruy wrote:

Nathan Woodrow wrote:

I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)

Same here. I can reproduce this with polygon layers. Lines and points opened correctly. Also I found that QGIS compiled from old branch (https://github.com/qgis/QGIS/pull/927) works fine.

I must study it, the code is the same but in last commit I also did a merge with master branch for make the windows installer.

I have reverted the last commit with the merge action.
I hope that it fix this build issue.

Matthias Kuhn wrote:

Hi Alvaro,

The effect of disabled anti-aliasing is still perceptible here.
Which portion of the performance is gained by disabling AA actually?

The upper part is rendered without the patch, the lower part with the patch.

Hi, I have tests with 'qgis_bench' to see the improvement disabling 'AA' for '1-pixel geometries'.

Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.

- Patched (1): Disable 'AntiAliasing' for '1-pixel geometries' is not executed.
- Patched (2): Disable 'AntiAliasing' for '1-pixel geometries' is executed.

My shapefiles tested:

I think that disable 'AA' for far geometries is effective, but it is true that, by contrast, a lower visual quality is perceptible especially for maps with geometries isolated.

I think it's worth apply it.
Regards

Thank you for this nice table.
So, disabling AA indeed seems to result in a noticeable difference in some cases.
I would prefer to make it an optional feature the way it looks. I have seen you committed something related to AA, was this a new config option?

Matthias Kuhn wrote:

Thank you for this nice table.
So, disabling AA indeed seems to result in a noticeable difference in some cases.
I would prefer to make it an optional feature the way it looks. I have seen you committed something related to AA, was this a new config option?

Hi Matthias,

It is very appreciable when the layer is very dense with a lot of small geometries.

Referred to the last commit, I was pending disable 'AA' for '1-pixel geometries' only when the simplification of the layer is activated. Temporal 'QgsSymbolV2RenderContext' objects created in 'QgsSymbolV2' class lose its 'VectorLayer' owner, and I have fixed it.

Referred to append a new extra option for disable the 'AA' of far geometries, I think that if the user already active the 'Fast drawing' implicitly assumes that the drawing quality can be a bit lower. I think that add other option can be strange or too complex to "understand". But I accept that the community send me :-)

Best regards

Referred to append a new extra option for disable the 'AA' of far geometries, I think that if the user already active the 'Fast drawing' implicitly assumes that the drawing quality can be a bit lower. I think that add other option can be strange or too complex to "understand". But I accept that the community send me :-)

Hi Alvaro,
In fact, if AA is enabled, symboly will be altered since you can't use symbols smaller than 1 pixel. So user will have to recalibrate symbology with your option for existing projects. We should avoid that. One big point with QGIS is precisely AA rendering, and we must keep antialising enabled. If user wants a Mapinfo-Arcgis style, faster, AA option is already there in general options, and user should gain your speed improvements + AA-OFF improvement.
My two cents, Régis

regis Haubourg wrote:

Referred to append a new extra option for disable the 'AA' of far geometries, I think that if the user already active the 'Fast drawing' implicitly assumes that the drawing quality can be a bit lower. I think that add other option can be strange or too complex to "understand". But I accept that the community send me :-)

Hi Alexandro,
In fact, if AA is enabled, symboly will be altered since you can't use symbols smaller than 1 pixel. So user will have to recalibrate symbology with your option for existing projects. We should avoid that. One big point with QGIS is precisely AA rendering, and we must keep antialising enabled. If user wants a Mapinfo-Arcgis style, faster, AA option is already there in general options, and user should gain your speed improvements + AA-OFF improvement.
My two cents, Régis

Hi, I don't understand your reply very well, :-)

I may not have described right this behavior:

The 'AA' is only individually disabled for geometries of a layer so far that it fill a '1-pixel' of the map, the rest are painted using the 'AA' option as heretofore.

Alvaro Huarte wrote:

Nathan Woodrow wrote:

Marco Hugentobler wrote:

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.

Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.

IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.

Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.

I agree, this patch improves the 'rendering at large scales', for getting rendering feedback the user can enable this existing option in the global options form of qgis.

To avoid problems in composer I am going disable this patch in 'printing' tasks.

Also, at least windows, there is other important improvement that I want test: Support for 'FileMapping' in the gdal-ogr library (e.g. using boost library) for speed up the access to data. I have experience with it and can be 2x faster.

Hi, the last commit disable the 'drawing simplification' for print compositions.
Best regards

Giovanni Manghi wrote:

Marco Hugentobler wrote:

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

Hi,

speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.

By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.

Hi Giovanni, now I am going to make a Windows Installer with the current patch state.
Telling you as I have!

Alvaro,

I can imagine to enable this functionality for screen rendering by default sooner or later, but only if all optimization are non-visible. (Sub-pixel simplification / no AA)

On the other hand it seems, that there is the possibility to improve performance even further but at the expense of visible graphics quality degradation. This can be ok for some situations / systems, but needs to be opt-in.

So I would welcome to have the two nicely separated already. Do you think this is a road we can take?

Hi Matthias, I can add other option to the new 'fast drawing' box in the 'rendering' options panel for the 'AA-disabling' advising that it is possible a graphics quality degradation.

If you like well I want to try before prepare the windows installer with this patch Giovanni asked for days.

Hi Giovanni, now I am going to make a Windows Installer with the current patch state.
Telling you as I have!

thanks!

Alvaro Huarte wrote:

Giovanni Manghi wrote:

Marco Hugentobler wrote:

I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).

Hi,

speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.

By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.

Hi Giovanni, now I am going to make a Windows Installer with the current patch state.
Telling you as I have!

Done:
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86

It contains the Windows Installer with the qgis master branch + #8725 patch.
Regards

Alvaro -

I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)

The postgis layer works in "normal" master.
OS Version: Window 8.1

Regards Bo Victor Thomsen

Bo Victor Thomsen wrote:

Alvaro -

I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)

The postgis layer works in "normal" master.
OS Version: Window 8.1

Regards Bo Victor Thomsen

confirmed here

Done:
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86

It contains the Windows Installer with the qgis master branch + #8725 patch.
Regards

Thanks! I will make some tests!

Giovanni Manghi wrote:

Bo Victor Thomsen wrote:

Alvaro -

I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)

The postgis layer works in "normal" master.
OS Version: Window 8.1

Regards Bo Victor Thomsen

confirmed here

Sorry, now I don't have installed the dump debugging tools.
I have tested with some postgis layers and works fine for me.

The error is throw for all layers ?
Are you using the application compiled from my pull request, or from the windows installer ?
If it is throwed by one layer, can you send me as shapefile for test here ?

Anyway, at night at home, I will setup the dump debugging tools for test the error
Thanks for your support!

Note: The improvement is very perceptible?
Best Regards

Matthias Kuhn wrote:

Alvaro,

I can imagine to enable this functionality for screen rendering by default sooner or later, but only if all optimization are non-visible. (Sub-pixel simplification / no AA)

On the other hand it seems, that there is the possibility to improve performance even further but at the expense of visible graphics quality degradation. This can be ok for some situations / systems, but needs to be opt-in.

So I would welcome to have the two nicely separated already. Do you think this is a road we can take?

Hi Matthias, according your advice, the last commit implements 'simplification hints' allowing configure what drawing simplification type can be executed (Points, BBOX or AA, or any combination).

By default, the full simplification (Points+BBOX+AA) will be applied.

Note: The improvement is very perceptible?

yes :)

Giovanni Manghi wrote:

Note: The improvement is very perceptible?

yes :)

I am delighted to read that.
I hope soon to add "FileMapping" support to gdal-ogr and when approved, another speed up for qgis!

Bo Victor Thomsen wrote:

Alvaro -

I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)

The postgis layer works in "normal" master.
OS Version: Window 8.1

Regards Bo Victor Thomsen

Hello Bo Victor, you are right, qgis from my windows installer crashes when the user load a postgis layer.

But the error is not related with my patch code, the exception is thrown before load the layer in 'void QgsApplication::preNotify()' method when the 'add' button of the configuration form is clicked.

I think this error is described in #5597-27.

In my compiled version I do not get that error so it has to be what files I have replaced in the master version of qgis for make the windows installer.

I must study it for rebuild the windows installer for test this patch.
Regards

Alvaro -
Sorry for being late to the party, I was at a mapping conference. Just to answer your questions even though you seem to have found the error: QGIS crashes just after I've selected one postgis table and clicked the "Add" button in the "PostGres add layer dialog".

Question no 2: The shape file (which works fine) has a speed improvement of 50%. The layer contains 1.300.000 polygon shapes and goes from 40 sec. load to 20 sec. load on my Icore7 PC with SSD.
Regards

Bo Victor Thomsen wrote:

Alvaro -
Sorry for being late to the party, I was at a mapping conference. Just to answer your questions even though you seem to have found the error: QGIS crashes just after I've selected one postgis table and clicked the "Add" button in the "PostGres add layer dialog".

Question no 2: The shape file (which works fine) has a speed improvement of 50%. The layer contains 1.300.000 polygon shapes and goes from 40 sec. load to 20 sec. load on my Icore7 PC with SSD.
Regards

Hi,
This error is strange for me, and seems like an old error is back, Should I add a issue to the list of QGIS?

Alvaro -
I've just tried the Postgis layer on the latest 32 bit QGIS Master and Current(2.0.1-3) installed with the Osgeo4w installer. They both worked without a hitch. So the problem is isolated to the special install build of QGIS with the speed improvements. I don't think you should report it as a general QGIS issue. Is it possible for you to zip the qgis.exe or maybe the "bin" directory of your working copy of the special QGIS and upload it somewhere I could get it?
Regards

Bo Victor Thomsen wrote:

Alvaro -
I've just tried the Postgis layer on the latest 32 bit QGIS Master and Current(2.0.1-3) installed with the Osgeo4w installer. They both worked without a hitch. So the problem is isolated to the special install build of QGIS with the speed improvements. I don't think you should report it as a general QGIS issue. Is it possible for you to zip the qgis.exe or maybe the "bin" directory of your working copy of the special QGIS and upload it somewhere I could get it?
Regards

Hi, thanks for your reply!

I found my error. Sorry, rookie mistake, I built the windows installer using the QGIS desktop distribution from the OSGeo4W setup, it is 2.0.X... all wrong.

I remade the installer from my master package and gives me no errors in the postgis loading form.

The new installer is (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v20
Only bin folder (8mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-bin-x86v20

Best Regards.

Hi,

There is a bug in QgsOgrFeatureIterator::ensureRelevantFields(), the feature cursor always fetchs all attributes ignoring the relevant fields configuration of the feature request.

I am trying to improve the OGR-GDAL provider and Even Rouault (gdal-dev) help me for find this bug (#9062).

af73f30c fixes this in master!

In many cases the layers are required to be painted without any field, or one or two fields, therefore, other improvement for render vector layers!

Thanks for all!
Regards

therefore, other improvement for render vector layers!

very good news!

any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.

Giovanni Manghi wrote:

therefore, other improvement for render vector layers!

very good news!

any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.

Hi Giovanni, I prepare in brief.
Thanks for you support!

Giovanni Manghi wrote:

therefore, other improvement for render vector layers!

very good news!

any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.

Hi,

The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30

It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)

Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.

Regards

Hi Alvaro

with the new installer QGIS crashes when starting.

Cheers!

Alvaro Huarte wrote:

Giovanni Manghi wrote:

therefore, other improvement for render vector layers!

very good news!

any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.

Hi,

The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30

It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)

Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.

Regards

Giovanni Manghi wrote:

Hi Alvaro

with the new installer QGIS crashes when starting.

Cheers!

Alvaro Huarte wrote:

Giovanni Manghi wrote:

therefore, other improvement for render vector layers!

very good news!

any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.

Hi,

The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30

It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)

Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.

Regards

Sorry, I can not rebuild well the installer, but it is independent of the changes I'm making.
I should download again all OSGeo4W packages.

I try shortly.

Alvaro Huarte wrote:

Giovanni Manghi wrote:

Hi Alvaro

with the new installer QGIS crashes when starting.

Cheers!

Alvaro Huarte wrote:

Giovanni Manghi wrote:

therefore, other improvement for render vector layers!

very good news!

any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.

Hi,

The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30

It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)

Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.

Regards

Sorry, I can not rebuild well the installer, but it is independent of the changes I'm making.
I should download again all OSGeo4W packages.

I try shortly.

I have updated my OSGeo4W packages and I hope that new windows installer is fine.
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x8640

Best Regards

Hi Alvaro -

I just tried your new version of QGIS on my Postgis-based dataset (1.3 million buildings, i.e simple polygons with 4-10 nodes). This dataset crashed the first version of "your" QGIS. Now it works without a hitch. Drawing time is around 10 seconds versus 20 seconds using the ordinary version of QGIS. This is nice ;-) !!

Regards
Bo Victor Thomsen
Aests-GIS
Denmark

Bo Victor Thomsen wrote:

Hi Alvaro -

I just tried your new version of QGIS on my Postgis-based dataset (1.3 million buildings, i.e simple polygons with 4-10 nodes). This dataset crashed the first version of "your" QGIS. Now it works without a hitch. Drawing time is around 10 seconds versus 20 seconds using the ordinary version of QGIS. This is nice ;-) !!

Regards
Bo Victor Thomsen
Aests-GIS
Denmark

You have used the last window installer ( http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x8640 ) ?
if so I'm glad I seem to have fixed my problems by making the installer.

For shapefiles, although it depends on the data (#records, size of geometries...), the improvement is still greater.

Thank you very much!

Yes, I did use your latest version of QGIS. The drawing sped of QGIS with the postgis dataprovider is now compareable with MapInfo using it's native file format (Same dataset 1.3 mill. polygons, MapInfo w. tabfile: 8 seconds, QGIS w. Postgis: 10 seconds)
Regards
Bo Victor Thomsen

This up guys! :-)

My proposed pull request (http://trac.osgeo.org/gdal/ticket/5272#comment:7) in GDAL-OGR library for optimize the OGR SHP-provider was accepted and Even Rouault has commited the modification with extra contributions.

The last windows installer already contains this improvement, really it contains my porposed pull, but they are equivalents.
Thanks Even!

Alvaro Huarte wrote:

This up guys! :-)

My proposed pull request (http://trac.osgeo.org/gdal/ticket/5272#comment:7) in GDAL-OGR library for optimize the OGR SHP-provider was accepted and Even Rouault has commited the modification with extra contributions.

very nice done Alvaro!

until now the feedback I received is very good, with no noticeable major issue caused by your patch. We will keep testing.

My feedback:

OS: Windows 7 64-bits
Processor: Intel(R) Core(TM)2 Duo CPU T8300 @ 2.4GHz 2.4GHz
RAM: 4.00Gb

QGIS Version: QGIS standalone Testing of the Alvaro(32-bits)

Results:

I test two shapefiles:

-Contours of Portugal (30 meters distance) size 300mb of capacity (with a lot of table features)

Results:
With patch: 4 seconds to load / Without patch: 25 seconds to load

- Landuse Level 2 of Portugal size 25mb of capacity (also with a lot of table features)

Results:
With patch: 11 seconds to load / Without patch: more then a 1 minute to load

I think if you create a spatial index in shapefile will help this option of rendering and save a cople of seconds to load.

With simplify rendering on, is necessary notice the users that this option improve rendering but create some gaps in the visualization process that can be fix after the load of the shapefile and zooming to the desarible scale.

hope this feedback helps.

Great job Alvaro.

Regards

João Gaspar wrote:

My feedback:

OS: Windows 7 64-bits
Processor: Intel(R) Core(TM)2 Duo CPU T8300 @ 2.4GHz 2.4GHz
RAM: 4.00Gb

QGIS Version: QGIS standalone Testing of the Alvaro(32-bits)

Results:

I test two shapefiles:

-Contours of Portugal (30 meters distance) size 300mb of capacity (with a lot of table features)

Results:
With patch: 4 seconds to load / Without patch: 25 seconds to load

- Landuse Level 2 of Portugal size 25mb of capacity (also with a lot of table features)

Results:
With patch: 11 seconds to load / Without patch: more then a 1 minute to load

I think if you create a spatial index in shapefile will help this option of rendering and save a cople of seconds to load.

With simplify rendering on, is necessary notice the users that this option improve rendering but create some gaps in the visualization process that can be fix after the load of the shapefile and zooming to the desarible scale.

hope this feedback helps.

Great job Alvaro.

Regards

Thank your very much to all!
I am developing other improvement in GDAL related with this patch. Therefore I will release other windows installer and I will append a "warning" label indicating that increase factor may lead to the emergence of gaps.

But these gaps should not appear in the scroll at least, João, it is right?
Thanks!

Sorry my English sometimes tricks me. :p

I don't understand what you want to say with "not appear in the scroll at least".

João Gaspar wrote:

Sorry my English sometimes tricks me. :p

I don't understand what you want to say with "not appear in the scroll at least".

No, my English is bad :-P, I trying to say: if you view gaps when the value of simplification factor configured (slider in the fast-rendering form of the layer) is the mininum.

Ahaha. ;)

In the minimum level don't appear gaps is "clean and smooth". ;)

Only appears when i increase the slider to high values of simplify geometries.

regards

João Gaspar wrote:

Ahaha. ;)

In the minimum level don't appear gaps is "clean and smooth". ;)

Only appears when i increase the slider to high values of simplify geometries.

regards

:-), ok, That's reassuring to me, with the minimum value occurs simplification but should not leave gaps. I'll put a warning for higher values ​​because in those cases it may appear.

Regards

Hi Alvaro, I tested your windows package, this is huge! Many thanks...
As a GIS administrator, I would need a general option in QGIS settings to set default behaviour and simplification level for new layers. Is that possible? We will have to deal qgs migration, could we be sure that layers coming from older QGIS will get these default QGIS settings?
Cheers
Régis

regis Haubourg wrote:

Hi Alvaro, I tested your windows package, this is huge! Many thanks...
As a GIS administrator, I would need a general option in QGIS settings to set default behaviour and simplification level for new layers. Is that possible? We will have to deal qgs migration, could we be sure that layers coming from older QGIS will get these default QGIS settings?
Cheers
Régis

Hi Régis, thanks!
I can append a general option, I write it down on my to-do list.
Regards

Alvaro Huarte wrote:

Hi João,
I added a warning label that appears when the user increases the simplification factor above the minimum level.

To describe it a little better, check the box performs the drawing simplification and accelerates the render of geometries. If the simplification factor is the minimum value, must not appear gaps, the more you increase it, is more likely to appear and so in these cases I present the warning.

As a user I do not need to increase the default value except in extreme cases of very large datasets.

Yeah really good. Describes perfectly the situation. =)

Regards

Can I suggest some improvements in the strings?

"Fast drawing" -> "Simplify geometry"

"Note ..." -> "Note: Enabling this option simplifies geometries in this layer to improve rendering speed. The simplification applies only during rendering of the layer and does not modify the layer geometry."

"Simplification factor..." -> "Simplification factor (higher values result in more simplification):"

"Warning..." -> "Warning: Increasing this factor will further speed up rendering, but may result in gaps or topological errors in the layer display."

Hi, I've tested this on Win7 64 bit with a previous project containing several raster and vector layers. Surprinsingly the whole set got slower: even while having only 2 layers enabled it got really heavy and also affected the scroll of the layers list. For the moment I'm not able to provide the dataset I was working on but I'll try to recreate the problem with a dataset I can supply.
Trying with a clean project with just one (complex) vector layer did speed up things.

Anyone else noticed something similar?

rph - wrote:

Hi, I've tested this on Win7 64 bit with a previous project containing several raster and vector layers. Surprinsingly the whole set got slower: even while having only 2 layers enabled it got really heavy and also affected the scroll of the layers list. For the moment I'm not able to provide the dataset I was working on but I'll try to recreate the problem with a dataset I can supply.
Trying with a clean project with just one (complex) vector layer did speed up things.

Anyone else noticed something similar?

Hi, I tried to reproduce this behavior and have not succeeded. I see no connection with this patch, but anyway this QGIS has not linked all data providers, it only contains most popular for test.

Regards

Anyone else noticed something similar?

can you try remove the layers (one by one) from your project and see if you find one that can be the cause?

Anyone else noticed something similar?

I may have seen this (but not sure) with a specific vector layer, made of a lot of polygons that are not possible to simplify, because it is grid polygon vector made of perfect squares.

In this case seem that qgis takes a lot trying figure how simplify, eventually taking an overhead. But as I said is just an impression as I have not tested it seriously. Example

https://www.dropbox.com/s/2leh363cdbupg1w/grid25_clip.tar.gz

Giovanni Manghi wrote:

Anyone else noticed something similar?

can you try remove the layers (one by one) from your project and see if you find one that can be the cause?

Anyone else noticed something similar?

I may have seen this (but not sure) with a specific vector layer, made of a lot of polygons that are not possible to simplify, because it is grid polygon vector made of perfect squares.

In this case seem that qgis takes a lot trying figure how simplify, eventually taking an overhead. But as I said is just an impression as I have not tested it seriously. Example

https://www.dropbox.com/s/2leh363cdbupg1w/grid25_clip.tar.gz

What you're saying makes sense, I may be trying to simplify polygons with <=5 points (grids) which themselves are not generalizable,
I look, thanks Giovanni!!

Alvaro Huarte wrote:

Giovanni Manghi wrote:

Anyone else noticed something similar?

can you try remove the layers (one by one) from your project and see if you find one that can be the cause?

Anyone else noticed something similar?

I may have seen this (but not sure) with a specific vector layer, made of a lot of polygons that are not possible to simplify, because it is grid polygon vector made of perfect squares.

In this case seem that qgis takes a lot trying figure how simplify, eventually taking an overhead. But as I said is just an impression as I have not tested it seriously. Example

https://www.dropbox.com/s/2leh363cdbupg1w/grid25_clip.tar.gz

What you're saying makes sense, I may be trying to simplify polygons with <=5 points (grids) which themselves are not generalizable,
I look, thanks Giovanni!!

I answer to myself :-), the patch try not to simplify polygons of 5 points, or linestrings of 2 points, but try to find out if it is possible to disable the 'AntiAliasing' for render the geometry faster. If the end is not possible to disable the AA (The geometry is not so far), an overload occurs in the process.

This can be avoided disabling for this layer the drawing simplification.

Hi,

I release a new windows installer which contains:

• Patch #8725 of QGIS (https://github.com/qgis/QGIS/pull/980)
• Patch #9062 of QGIS (#9062)
• Patch N5272 of GDAL (http://trac.osgeo.org/gdal/ticket/5272)

http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v50

I made new tests with the latest version to see the speed.

Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.

My shapefiles tested:

Here's some more string updates for this latest addition:

"By default simplify geometries" -> "Simplify geometries by default"

And for the text below the checkbox:

"If checked, new vector layers added to the map will automatically use geometry simplification to speed up rendering. ... (then the rest of what you have)"

Thanks!

Alvaro - these improvements look fantastic! Your hard work is much appreciated!

One related idea I was planning on investigating was adding an option to disable rendering of features smaller than a certain size (in mm). You'll see a similar option in the labelling properties for a vector layer under "Rendering" -> "Suppress labelling of features smaller than ". Having the ability to skip rendering of small features could potentially lead to some more performance gains (as well as having cartographic benefits). Obviously, it should be disabled by default for layers as it would only be useful in some circumstances.

There was a bit of discussion on the QGIS dev list about this a month or so ago -- I had a quick initial attempt at implementing it but have got side tracked on composer work (and it's looking like this will tie me up for the whole of the 2.2 dev cycle).

If you wanted to experiment with this, there's code for checking the size of a feature in mm in QgsPalLayerSettings::checkMinimumSizeMM . I had a quick play here https://github.com/nyalldawson/QGIS/commit/27a42e11b31521a521f8435fc4e82d75568c97cb (no gui, hard coded to a minimum size of 2mm) which yielded some benefit. This was done in QgsVectorLayer::drawRendererV2, I think moving it further up the chain could improve the performance some more.

Nyall Dawson wrote:

Here's some more string updates for this latest addition:

"By default simplify geometries" -> "Simplify geometries by default"

And for the text below the checkbox:

"If checked, new vector layers added to the map will automatically use geometry simplification to speed up rendering. ... (then the rest of what you have)"

Thanks!

Done, thanks Nyall!

Nyall Dawson wrote:

Alvaro - these improvements look fantastic! Your hard work is much appreciated!

One related idea I was planning on investigating was adding an option to disable rendering of features smaller than a certain size (in mm). You'll see a similar option in the labelling properties for a vector layer under "Rendering" -> "Suppress labelling of features smaller than ". Having the ability to skip rendering of small features could potentially lead to some more performance gains (as well as having cartographic benefits). Obviously, it should be disabled by default for layers as it would only be useful in some circumstances.

There was a bit of discussion on the QGIS dev list about this a month or so ago -- I had a quick initial attempt at implementing it but have got side tracked on composer work (and it's looking like this will tie me up for the whole of the 2.2 dev cycle).

If you wanted to experiment with this, there's code for checking the size of a feature in mm in QgsPalLayerSettings::checkMinimumSizeMM . I had a quick play here https://github.com/nyalldawson/QGIS/commit/27a42e11b31521a521f8435fc4e82d75568c97cb (no gui, hard coded to a minimum size of 2mm) which yielded some benefit. This was done in QgsVectorLayer::drawRendererV2, I think moving it further up the chain could improve the performance some more.

Hi Nyall, I did not know this discussion!, I have involved in QGIS later. Coincidences of life, this patch proposes an implementation of that issue. Now, I'm a little waiting for comments and suggestions to see if it is interesting to approval.

On the subject of not paint "very small" elements seems like a good idea in case of layers as you mention, in other cases (very dense layers) would appear whole areas unpainted, but I would not mix this with this patch but on a new patch. If you want I can try to implement it.

Regards

Alvaro Huarte wrote:

Hi Nyall, I did not know this discussion!, I have involved in QGIS later. Coincidences of life, this patch proposes an implementation of that issue. Now, I'm a little waiting for comments and suggestions to see if it is interesting to approval.

I know - it's great timing! Thanks for all the work you've put in developing these much needed improvements for QGIS.

On the subject of not paint "very small" elements seems like a good idea in case of layers as you mention, in other cases (very dense layers) would appear whole areas unpainted, but I would not mix this with this patch but on a new patch. If you want I can try to implement it.

I don't want to add to your workload, but it'd be great if you could check it out. I've realized I'm not going to get a chance to tackle this for 2.2. Based on your past work you've got a much better understanding of optimising code like this then I do!

Nyall Dawson wrote:

Alvaro Huarte wrote:

Hi Nyall, I did not know this discussion!, I have involved in QGIS later. Coincidences of life, this patch proposes an implementation of that issue. Now, I'm a little waiting for comments and suggestions to see if it is interesting to approval.

I know - it's great timing! Thanks for all the work you've put in developing these much needed improvements for QGIS.

On the subject of not paint "very small" elements seems like a good idea in case of layers as you mention, in other cases (very dense layers) would appear whole areas unpainted, but I would not mix this with this patch but on a new patch. If you want I can try to implement it.

I don't want to add to your workload, but it'd be great if you could check it out. I've realized I'm not going to get a chance to tackle this for 2.2. Based on your past work you've got a much better understanding of optimising code like this then I do!

Thanks Nyall, I also really appreciate the tests, advice and opinions of the community, they serve me to keep learning every day. And your english corrections ;-)

I've just seen this pull request. Dumb question, but why would you implement the user defined
simplification setting? Seems to me it should be sorted by the software automatically, based on extent and the map to pixel setting. Is also overly complicated for the user...

Jeremy Palmer wrote:

I've just seen this pull request. Dumb question, but why would you implement the user defined
simplification setting? Seems to me it should be sorted by the software automatically, based on extent and the map to pixel setting. Is also overly complicated for the user...

Hi Jeremy, the software automatically calculates the simplification factor based on geometry and current map2pixel setting (Slider with minimum value).

I added the possibility of increase this factor to get 'extra' simplification to accelerate even more the render of very large layers.
In this case it is possible that appear gaps and one warning message is showed in the configuration panel.

As I said above the user, in most cases, do not worry about modifying this value, but in other cases it may be interesting.
Regards

Hi Alvaro,

Thanks for the reply. In that case it might be worth stating on the slider label that it is a advanced experimental option.

Cheers,
Jeremy

Jeremy Palmer wrote:

Hi Alvaro,

Thanks for the reply. In that case it might be worth stating on the slider label that it is a advanced experimental option.

Cheers,
Jeremy

Hi Jeremy, I suppose we should add an entry in the user manual and explain this configuration.
Thanks!

Alvaro

Hi all, There is a modification in behavior of this patch (https://github.com/qgis/QGIS/pull/980).

Now, it contains the advise of @nyalldawson (https://github.com/qgis/QGIS/pull/1020#issuecomment-30263200) to enable AA-disabling only when the simplification slider is greater than minimum value.

Best regards

I am pleased to announce that this patch seems to have been merged!
:-)

Thank you very much for your support and advices!

Alvaro Huarte wrote:

I am pleased to announce that this patch seems to have been merged!
:-)

Thank you very much for your support and advices!

Alvaro, many thanks to YOU :)

What about add documentation to user manual?

merged in 17cd09a8a7352d761602c547d29f660aed219adb

Hi, I committed a new version of geometry simplification to speed up the vector drawing.
https://github.com/ahuarte47/QGIS/tree/Issue_8725-revival-optA-to-B

It is inspired in this PR, but it contains several changes based on advice received ( @matthias-kuhn thanks! ) and new features:

• About code, the simplification is configured in a new QgsSimplifyMethod class which indicates how to simplify the geometries in a feature iterator.
• Now, the user can define where the simplification executes (There is a new option in settings panel), locally after fetch the geometry from provider, or simplifying it on provider side. e.g. In postgres provider, first option simplifies the geometry already fetched locally, but the second option simplifies the geometry in database using the function ST_Simplify.
• The settings panel also shows the simplification threshold in pixel units as @timlinux suggested me.

I have not created a new pull request because of I would greatly appreciate if you could test it with your data and SO's. I have tested it in windows I get similar results as old version for shapefiles (OGR-provider). The new simplification on database side for postgres provider improves the old results too. I will create a table of tests to compare data and configurations.

About postgres simplification, the ST_simplify function needs a tolerance parameter, I use map2pixel/5.0 as input value, it is experimental and I must define it better (All ideas are welcome). This simplification can be applicable to other database providers (MySQL, SQL Server, Oracle...)

Thank you very much!