Feature request #8725
Speed improvement in the render of geometries for qgis
Status: | Closed | ||
---|---|---|---|
Priority: | Normal | ||
Assignee: | Alvaro Huarte | ||
Category: | Vectors | ||
Pull Request or Patch supplied: | Yes | Resolution: | fixed/implemented |
Easy fix?: | No | Copied to github as #: | 17441 |
Description
The render of geometries for vector layers in qgis can be optimized.
There are some tricks that sure make fast the rendering of big shapes:
- Prefilter the points of the geometries to be painted using a map2pixel error tolerance (_getPolygon and _getLineString in rendererv2).
- Prevents divisions in map to devices coordinates.
- Prevents of unnecessary trim of the geometries with the canvas rect.
- Prevents of unnecessary calloc using shared memory buffer (ogr to wkb in ogrFeatureIterator).
I have implements this tricks and qgis has better performance of 300% in some map situations.
I want validate the code and I will try share it.
History
#1 Updated by Matthias Kuhn about 11 years ago
Hi Alvaro,
Thank you for looking into this issues and contributing.
The best thing to do is to open a pull request on github, so it can be reviewed, commented and eventually merged.
#2 Updated by Alvaro Huarte about 11 years ago
Ok, thank you very much
#3 Updated by Matthias Kuhn about 11 years ago
I forgot to mention that it would be best to make one pull request for each improvement. So in case one of them is not accepted, it will still be possible to merge the good parts.
#4 Updated by Alvaro Huarte about 11 years ago
ok, I will divide this issue in four single childs tasks.
Thank you!
#5 Updated by Giovanni Manghi about 11 years ago
Hi Alvaro, do you have a branch that can be tested? Cheers!
#6 Updated by Alvaro Huarte about 11 years ago
Hi Giovanni, I am testing my changes, if it successly in some days I will try a pull request.
Thank you very much!
#7 Updated by Alvaro Huarte about 11 years ago
- % Done changed from 0 to 70
- File test_data_qgis_master_RenderingStatsQGIS__Feature-8725.zip added
I'm making progress, I think I get promising results. Attach a table of results for three tested layers.
#8 Updated by Giovanni Manghi about 11 years ago
Alvaro Huarte wrote:
I'm making progress, I think I get promising results. Attach a table of results for three tested layers.
wow, I can't wait to test your patches! :)
#9 Updated by Alvaro Huarte about 11 years ago
- Resolution set to fixed
- % Done changed from 70 to 100
- Status info set to for test
#10 Updated by Giovanni Manghi about 11 years ago
Alvaro Huarte wrote:
done for test.
github:
https://github.com/ahuarte47/QGIS/commit/28691ff96e0ef2a97a8f1d08898b1c7d97941f7c
+
https://github.com/ahuarte47/QGIS/commit/a8894f8df811c61fe021607d474aa8e117a5f56fSorry there are two commits, I forgot to add one edited file
Hi Alvaro, can you please make a Pull Request in the qgis repo? This way other will be able to see it and review and then commit. Thanks!
#11 Updated by Alvaro Huarte about 11 years ago
Done.
I'm new using gitub and I'm learning this tool, Sorry!
Thanks!
#12 Updated by Alvaro Huarte about 11 years ago
Changes goto branch:
#13 Updated by Alvaro Huarte almost 11 years ago
- Pull Request or Patch supplied changed from No to Yes
- Resolution changed from fixed to fixed/implemented
I have implemented the simplification of geometries in the vector-providers. The general behavior simplifies the geometries post-fetch the feature from the provider, but OGR-provider simplifies the geometries pre-fetch the features having a extra seed improvement.
Te final result gets ~3x painting speed faster that original code
New branch:
https://github.com/ahuarte47/QGIS/tree/Issue_8725-OGR
I think ready for test! :-)
#14 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
I have implemented the simplification of geometries in the vector-providers. The general behavior simplifies the geometries post-fetch the feature from the provider, but OGR-provider simplifies the geometries pre-fetch the features having a extra seed improvement.
Te final result gets ~3x painting speed faster that original code
New branch:
https://github.com/ahuarte47/QGIS/tree/Issue_8725-OGRI think ready for test! :-)
compiling right now :)
#15 Updated by Giovanni Manghi almost 11 years ago
I think ready for test! :-)
got an error during make
Building CXX object src/core/CMakeFiles/qgis_core.dir/qgsmaplayer.cpp.o /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp: In function ‘QgsRectangle calculateBoundingBox(const QVector<QPointF>&)’: /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:102:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:102:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:102:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:103:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:103:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:103:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:104:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:104:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:104:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:105:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:105:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:105:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:109:5: error: ‘x’ was not declared in this scope /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:110:5: error: ‘y’ was not declared in this scope /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp: In function ‘QgsRectangle calculateBoundingBox(QGis::WkbType, unsigned char*, std::size_t)’: /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:125:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:125:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:125:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:126:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:126:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:126:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:127:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:127:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:127:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:128:18: error: ‘numeric_limits’ is not a member of ‘std’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:128:38: error: expected primary-expression before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:128:38: error: expected ‘,’ or ‘;’ before ‘double’ /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:135:5: error: ‘x’ was not declared in this scope /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:136:5: error: ‘y’ was not declared in this scope /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp: At global scope: /home/giovanni/QGIS-Issue_8725-OGR/src/core/qgsmaprequest.cpp:149:20: warning: unused parameter ‘sourceWkb’ [-Wunused-parameter] make[2]: [ 14%] *** [src/core/CMakeFiles/qgis_core.dir/qgsmaprequest.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... Building CXX object src/core/CMakeFiles/qgis_core.dir/qgsmaplayerregistry.cpp.o make[1]: *** [src/core/CMakeFiles/qgis_core.dir/all] Error 2 make: *** [all] Error 2
#16 Updated by Alvaro Huarte almost 11 years ago
Hi Giovanni, I use Visual Studio 2010 and I don't get these errors.
What compiler do you use ?
I will try fix it!
#17 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
Hi Giovanni, I use Visual Studio 2010 and I don't get these errors.
What compiler do you use ?I will try fix it!
Hi Alvaro, I use Ubuntu Linux, gcc 4.6.3
#18 Updated by Matthias Kuhn almost 11 years ago
It's probably a simple #include <limits.h>
#19 Updated by Alvaro Huarte almost 11 years ago
Done, thanks!
#20 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
Done, thanks!
Hi Alvaro! I made a few quick tests with shapefiles, measuring just the time to open them (with the "time" command). I have not had the time to try evaluate the improvement when zooming in/out.
The shapefiles (polygons) that I tested are:
http://faunalia.pt/downloads/GTBs_Shapefile.zip
http://faunalia.pt/downloads/MAMMTERR.zip
un-patched QGIS
giovanni@sibirica ~/Desktop $ time qgis MAMMTERR.shp
user 0m10.377s
user 0m10.237s
user 0m10.429s
giovanni@sibirica ~/Desktop $ time qgis GTBs_2011.shp
user 1m2.896s
user 1m5.436s
user 1m4.120s
patched QGIS
giovanni@sibirica ~/Desktop $ time qgis MAMMTERR.shp
user 0m6.352s
user 0m6.244s
user 0m6.820s
giovanni@sibirica ~/Desktop $ time qgis GTBs_2011.shp
user 0m35.278s
user 0m33.286s
user 0m33.602s
So there is really a substantial improvement!
I tested also another shape (big, more that 1.2gb of small polygons) where things get worst(?)
http://www.igeo.pt/scrif/crif/CRIF2011shp.zip
unpatched
giovanni@sibirica ~/Desktop $ time qgis crif2011.shp
user 0m35.186s
user 0m34.870s
user 0m35.214s
patched
giovanni@sibirica ~/Desktop $ time qgis crif2011.shp
user 0m45.571s
user 0m49.311s
user 0m46.279s
Can I test also the vectors you are using to check the rendering times?
Cheers!
#21 Updated by Alvaro Huarte almost 11 years ago
Hi Giovanni, Some shapes...
http://idena.navarra.es/descargas/CARTO1_Lin_6CNivelD.zip
http://idena.navarra.es/descargas/GEOLOG_Pol_Litologia.zip
I am going to test your shapes!
Thanks!
#22 Updated by Matthias Kuhn almost 11 years ago
Running a rendering performance test once may be strongly biased by other processes running.
For statistically more robust numbers I recommend using qgis_bench
It's really simple to use.
Here some more information:
https://github.com/qgis/QGIS/blob/master/tests/bench/README
#23 Updated by Alvaro Huarte almost 11 years ago
Hi, I made tests with 'qgis_bench' to see the speed.
Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.
My shapefiles tested:
Dataset | Value | Master | Patched | |
---|---|---|---|---|
http://faunalia.pt/downloads/GTBs_Shapefile.zip | iterations | 10 | 10 | 100% |
total_avg | 180.1015625 | 68.8765625 | 38% | |
total_max | 185.8125 | 71.484375 | 38% | |
total_maxdev | 33.3046875 | 2.60781249999999 | 7% | |
total_min | 146.796875 | 67.265625 | 45% | |
total_stdev | 11.1268459888756 | 1.27757571479394 | 11% | |
http://faunalia.pt/downloads/MAMMTERR.zip | iterations | 10 | 10 | 100% |
total_avg | 88.621875 | 9.49375 | 10% | |
total_max | 95.09375 | 10 | 10% | |
total_maxdev | 53.85625 | 1.665625 | 3% | |
total_min | 34.765625 | 7.828125 | 22% | |
total_stdev | 17.9542520316393 | 0.572148158805916 | 3% | |
http://www.igeo.pt/scrif/crif/CRIF2011shp.zip | iterations | 10 | 10 | 100% |
total_avg | 275.1109375 | 70.275 | 25% | |
total_max | 286.5625 | 71.4375 | 24% | |
total_maxdev | 93.3609375 | 6.50937500000001 | 6% | |
total_min | 181.75 | 63.765625 | 35% | |
total_stdev | 31.1253721504484 | 2.18250769614909 | 7% | |
http://idena.navarra.es/descargas/CARTO1_Lin_6CNivelD.zip | iterations | 10 | 10 | 100% |
total_avg | 8.2765625 | 1.4703125 | 17% | |
total_max | 8.90625 | 1.515625 | 17% | |
total_maxdev | 4.9796875 | 0.1109375 | 2% | |
total_min | 3.296875 | 1.359375 | 41% | |
total_stdev | 1.66037296379646 | 0.0578125 | 3% | |
http://idena.navarra.es/descargas/GEOLOG_Pol_Litologia.zip | iterations | 10 | 10 | 100% |
total_avg | 10.371875 | 1.7953125 | 17% | |
total_max | 11.03125 | 1.859375 | 16% | |
total_maxdev | 5.79375 | 0.4984375 | 8% | |
total_min | 4.578125 | 1.296875 | 28% | |
total_stdev | 1.93128792438492 | 0.166455758480294 | 8% | |
Urban parcels of cadastre of navarra (Polygon2D of 142680 records, 66mb) | iterations | 10 | 10 | 100% |
total_avg | 8.6046875 | 3.3484375 | 38% | |
total_max | 9.0625 | 3.375 | 37% | |
total_maxdev | 2.9953125 | 0.1296875 | 4% | |
total_min | 5.609375 | 3.21875 | 57% | |
total_stdev | 0.999488394520542 | 0.0458481273472538 | 4% | |
Rustic parcels of cadastre of navarra (Polygon2D of 542658 records, 458mb) | iterations | 10 | 10 | 100% |
total_avg | 57.8265625 | 12.6921875 | 21% | |
total_max | 61.859375 | 13.03125 | 21% | |
total_maxdev | 28.4671875 | 0.957812499999999 | 3% | |
total_min | 29.359375 | 11.734375 | 39% | |
total_stdev | 9.50177422268632 | 0.32983557101418 | 3% |
#24 Updated by Nathan Woodrow almost 11 years ago
Nice!
#25 Updated by Alvaro Huarte almost 11 years ago
- File sketch.png added
#26 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
I thought a bit :-), I think more appropriate to add a new "rendering" tab with this configuration similar to global options form.
I agree
#27 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
Hi, I made tests with 'qgis_bench' to see the speed.
Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.
very nice results indeed! as soon as I will understand how to use qgis_bench I will make tests too :)
#28 Updated by Alvaro Huarte almost 11 years ago
Hi Giovanni, qgis_bench is easy :-)
I execute in a Console Window (CMD)...
{%BIN_PATH%}/qgis_bench -i 10 -p test_project.qgs >> test_project.log
-i: number of desired iterations
-p: project to test
Regards
#29 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
Hi Giovanni, qgis_bench is easy :-)
I execute in a Console Window (CMD)...
{%BIN_PATH%}/qgis_bench -i 10 -p test_project.qgs >> test_project.log
-i: number of desired iterations
-p: project to testRegards
I made also tests with qgis_bench
Dataset | Value | Master | Patched | |
---|---|---|---|---|
http://faunalia.pt/downloads/MAMMTERR.zip | iterations | 10 | 10 | 100% |
total_avg | 14.6881179 | 3.616626 | 24% | |
total_max | 15.500969 | 3.816239 | 24% | |
total_maxdev | 2.3073439 | 0.204413 | 8% | |
total_min | 12.380774 | 3.412213 | 27% | |
total_stdev | 0.808622095655375 | 0.108486504445484 | 13% | |
http://faunalia.pt/downloads/GTBs_Shapefile.zip | iterations | 10 | 10 | 100% |
total_avg | 41.0249639 | 32.6264391 | 79% | |
total_max | 42.986687 | 37.038314 | 86% | |
total_maxdev | 1.9617231 | 4.41187489999999 | 224% | |
total_min | 39.566472 | 30.029877 | 75% | |
total_stdev | 1.12526313538864 | 1.95203781571303 | 173% | |
http://www.igeo.pt/scrif/crif/CRIF2011shp.zip | iterations | 10 | 10 | 100% |
total_avg | 64.6020374 | 37.1403211 | 57% | |
total_max | 65.808114 | 38.874429 | 59% | |
total_maxdev | 4.3062684 | 2.76617309999999 | 64% | |
total_min | 60.295769 | 34.374148 | 57% | |
total_stdev | 1.54006099425576 | 1.13227893163129 | 73% |
#30 Updated by Jürgen Fischer almost 11 years ago
- Subject changed from Speed improvement in the render of geometries for qgis to Speed improvement in the render of geometries for qgis
#31 Updated by Alvaro Huarte almost 11 years ago
- File cfgimage.jpg added
Hi,
I have implemented a new tab called 'Rendering' in the options form of the layer where the user can configure the drawing simplification.
Regards
#32 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
Hi,
I have implemented a new tab called 'Rendering' in the options form of the layer where the user can configure the drawing simplification.
Regards
Hi Alvaro,
this means that when loading a layer the user will not see any improvement, isn't it? If yes then it is a pity because for large vectors there is also a huge improvement when loading the layer. What about an option in the general qgis options and then the switch in vector properties to allow override the general configuration?
#33 Updated by Alvaro Huarte almost 11 years ago
Hi Giovanni,
I propose that by default the simplification is activated (The check-box, by default, will be checked), after the user can disable it for some reason, so I have been asked several people.
About a general option, I agree, but I do not what others think about it.
Giovanni, thank you very much for your tests and support!
Regards
#34 Updated by Jukka Rahkonen almost 11 years ago
Hi,
This may be a stupid question, but how does the simplified rendering work when users start to edit vectors? I think that users can not be let to edit those features which are simplified for fast viewing and save edits back to the source shapefile. And what gets saved if used does not edit anything but wants to make a copy by Save as..? Is QGIS saving the simpified layer of can is somehow handle two sort of geometries, the real ones and those temporarily simplified for viewing? Even when other layers are edited the result might be unexpected if user wants to snap vertices of the layer that is edited to vertices of the layer that is rendered as simplified.
#35 Updated by Alvaro Huarte almost 11 years ago
Hi @jratike80, this is a good question!, but there is not problem.
The simplification only is applied in the 'getFeatures' request when it becomes from a rendering request (... drawing the layer in the map). Fetch geometries, update geometries... are executed using distinct 'getFeatures' requests where the geometries are fetched without simplification.
Regards
#36 Updated by Jürgen Fischer almost 11 years ago
Alvaro Huarte wrote:
The simplification only is applied in the 'getFeatures' request when it becomes from a rendering request (... drawing the layer in the map). Fetch geometries, update geometries... are executed using distinct 'getFeatures' requests where the geometries are fetched without simplification.
You could also combine that with the QgsVectorLayer::isEditable() method - the layer does caching in edit mode. Another potential thing to look at is snapping.
Other things to consider are unaligned accesses. Marco Bernasocchi is currently working in qgis/QGIS-Android on cleaning the code from unaligned accesses (mainly when parsing or construction WKB; ARM doesn't like those), which is basically replacing all occurrences of *foo = bar;
with memcpy(foo, &bar, sizeof bar);
. I see that you're introducing more of those. Please rework it to use memcpy.
And finally run the stuff through scripts/prepare-commit.sh (or scripts/astyle.sh for already committed stuff).
But enough whining - otherwise it's very nice work - thank you very much :)
#37 Updated by Alvaro Huarte almost 11 years ago
Hello Jürgen, thank you very much for your comments!
I have released new commits for the branch https://github.com/ahuarte47/QGIS/compare/Issue_8725-OGR where I replace all '*foo = bar' with 'memcpy'. This has been a lesson for me.
Also I disable the drawing simplification in 'editmode', the geometries are cached for snapping, thanks.
About the scripts, I am going to study
Regards
#38 Updated by Alvaro Huarte almost 11 years ago
Hi Jürgen,
After a failed performance test I wanted to do (https://github.com/qgis/QGIS/pull/927#issuecomment-27158451), I retake his message referring to scripts. I do not know very well how execute it. I must use cygwin ?
Thank you very much
#39 Updated by Alvaro Huarte almost 11 years ago
To avoid confusion I created a new pull request with the last code.
New pull request:
https://github.com/qgis/QGIS/pull/980
Obsolete pull:
https://github.com/qgis/QGIS/pull/927
Regards
#40 Updated by Nathan Woodrow almost 11 years ago
Hey Alvaro,
Really like your work.
I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)
Running qgis_bench has the same effect, just crashes. Very strange.
Nathan
#41 Updated by Alvaro Huarte almost 11 years ago
Hi Nathan, thanks for your comments!
I have test some shapes and all works fine in 'release' build.
Can I test your data ?
Regards
#42 Updated by Alexander Bruy almost 11 years ago
Nathan Woodrow wrote:
I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)
Same here. I can reproduce this with polygon layers. Lines and points opened correctly. Also I found that QGIS compiled from old branch (https://github.com/qgis/QGIS/pull/927) works fine.
#43 Updated by Alexander Bruy almost 11 years ago
- Resolution deleted (
fixed/implemented)
#44 Updated by Marco Hugentobler almost 11 years ago
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
#45 Updated by Giovanni Manghi almost 11 years ago
Marco Hugentobler wrote:
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
Hi,
speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.
By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.
#46 Updated by Nathan Woodrow almost 11 years ago
Marco Hugentobler wrote:
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.
Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.
IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.
Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.
#47 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
Marco Hugentobler wrote:
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
Hi,
speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.
By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.
Hi, I am working for build a windows installer with this patch.
I build the default installer with the script 'creatensis.pl' but it does not contains my patch. I must study how insert it.
#48 Updated by Matthias Kuhn almost 11 years ago
Hi Alvaro,
The effect of disabled anti-aliasing is still perceptible here.
Which portion of the performance is gained by disabling AA actually?
The upper part is rendered without the patch, the lower part with the patch.
#49 Updated by Alvaro Huarte almost 11 years ago
Alexander Bruy wrote:
Nathan Woodrow wrote:
I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)
Same here. I can reproduce this with polygon layers. Lines and points opened correctly. Also I found that QGIS compiled from old branch (https://github.com/qgis/QGIS/pull/927) works fine.
I must study it, the code is the same but in last commit I also did a merge with master branch for make the windows installer.
#50 Updated by Alvaro Huarte almost 11 years ago
Nathan Woodrow wrote:
Marco Hugentobler wrote:
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.
Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.
IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.
Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.
I agree, this patch improves the 'rendering at large scales', for getting rendering feedback the user can enable this existing option in the global options form of qgis.
To avoid problems in composer I am going disable this patch in 'printing' tasks.
Also, at least windows, there is other important improvement that I want test: Support for 'FileMapping' in the gdal-ogr library (e.g. using boost library) for speed up the access to data. I have experience with it and can be 2x faster.
#51 Updated by Giovanni Manghi almost 11 years ago
Nathan Woodrow wrote:
The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.
Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.
IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.
Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.
+1
#52 Updated by Alvaro Huarte almost 11 years ago
Matthias Kuhn wrote:
Hi Alvaro,
The effect of disabled anti-aliasing is still perceptible here.
Which portion of the performance is gained by disabling AA actually?The upper part is rendered without the patch, the lower part with the patch.
Hi Matthias, disable antialiasing for far geometries is perceptible, but for large datasets it save rendering time (I will make tests with some shapes and send results). ArcGis, by example, also cheats with that. I think that is more important to get fast framerates that high quality rendering, But, of course, all people has one opinion, I do what seems best to the community.
regards
#53 Updated by Alvaro Huarte almost 11 years ago
Hi Matthias, we have another think coming, :-)
A few days ago, you said me... "... another minor thing is, that the simplification code should probably not be in the request, but rather in the place where the simplification is done, so in the iterator or rather in its own simplification class (maybe other parts of the code need it...)".
Still think the same? I have no problem in creating a new utility class that implements this simplification code as static methods.
Thanks for your recommendations.
#54 Updated by Nyall Dawson almost 11 years ago
Nathan Woodrow wrote:
The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.
Sorry to nitpick, but just for clarification - are you actually meaning rendering at small scales (further zoomed out) is where the performance gain is? If so, I agree that this is a significant issue which would be great to address.
#55 Updated by Nathan Woodrow almost 11 years ago
Nyall Dawson wrote:
Nathan Woodrow wrote:
The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.
Sorry to nitpick, but just for clarification - are you actually meaning rendering at small scales (further zoomed out) is where the performance gain is? If so, I agree that this is a significant issue which would be great to address.
Yeah. Just stuffed up my terms.
#56 Updated by Nyall Dawson almost 11 years ago
Ok - in that case I run into this same issue almost daily, and any improvements would be greatly appreciated!
#57 Updated by Alvaro Huarte almost 11 years ago
Nyall Dawson wrote:
Ok - in that case I run into this same issue almost daily, and any improvements would be greatly appreciated!
I hope that with this patch, and the improvement of "FileMapping" I want to add to gdal-ogr providers, it result a very good behavior of qgis for render large datasets.
Best regards
#58 Updated by Alvaro Huarte almost 11 years ago
Alvaro Huarte wrote:
Alexander Bruy wrote:
Nathan Woodrow wrote:
I seem to be having a small issue with the build though. When ever I run it in normal mode e.g just running the output qgis.exe file I get a crash as some as I open the file or one feature renders. But - and here is the fun part - attach the debugger and I don't get a crash :)
Same here. I can reproduce this with polygon layers. Lines and points opened correctly. Also I found that QGIS compiled from old branch (https://github.com/qgis/QGIS/pull/927) works fine.
I must study it, the code is the same but in last commit I also did a merge with master branch for make the windows installer.
I have reverted the last commit with the merge action.
I hope that it fix this build issue.
#59 Updated by Alvaro Huarte almost 11 years ago
Matthias Kuhn wrote:
Hi Alvaro,
The effect of disabled anti-aliasing is still perceptible here.
Which portion of the performance is gained by disabling AA actually?The upper part is rendered without the patch, the lower part with the patch.
Hi, I have tests with 'qgis_bench' to see the improvement disabling 'AA' for '1-pixel geometries'.
Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.
- Patched (1): Disable 'AntiAliasing' for '1-pixel geometries' is not executed.
- Patched (2): Disable 'AntiAliasing' for '1-pixel geometries' is executed.
My shapefiles tested:
Dataset | Value | Master (0) | Patched (1) | (1-0) | Patched (2) | (2-0) | (2-1) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
http://faunalia.pt/downloads/GTBs_Shapefile.zip | iterations | 10 | 10 | 10 | ||||||||||||
total_avg | 211,02344 | 87,79219 | 42% | 86,05781 | 41% | 2% | ||||||||||
total_max | 224,60938 | 89,62500 | 88,18750 | |||||||||||||
total_maxdev | 30,39844 | 1,83281 | 2,12969 | |||||||||||||
total_min | 180,62500 | 86,81250 | 84,03125 | |||||||||||||
total_stdev | 11,06205 | 0,86665 | 1,35855 | |||||||||||||
http://faunalia.pt/downloads/MAMMTERR.zip | iterations | 10 | 10 | 10 | ||||||||||||
total_avg | 84,65156 | 12,43438 | 15% | 10,98594 | 13% | 12% | ||||||||||
total_max | 92,84375 | 15,54688 | 11,37500 | |||||||||||||
total_maxdev | 52,21406 | 3,11250 | 1,50156 | |||||||||||||
total_min | 32,43750 | 11,93750 | 9,48438 | |||||||||||||
total_stdev | 17,42508 | 1,04493 | 0,51365 | |||||||||||||
http://www.igeo.pt/scrif/crif/CRIF2011shp.zip | iterations | 10 | 10 | 10 | ||||||||||||
total_avg | 247,60625 | 109,75469 | 44% | 77,25156 | 31% | 30% | ||||||||||
total_max | 257,84375 | 112,65625 | 85,67188 | |||||||||||||
total_maxdev | 74,05938 | 16,97344 | 8,42031 | |||||||||||||
total_min | 173,54688 | 92,78125 | 74,73438 | |||||||||||||
total_stdev | 24,70081 | 5,67215 | 3,30560 | |||||||||||||
http://idena.navarra.es/descargas/CARTO1_Lin_6CNivelD.zip | iterations | 10 | 10 | 10 | ||||||||||||
total_avg | 7,85781 | 1,94844 | 25% | 1,58438 | 20% | 19% | ||||||||||
total_max | 8,40625 | 2,85938 | 2,32813 | |||||||||||||
total_maxdev | 4,54531 | 0,91094 | 0,74375 | |
total_min | 3,31250 | 1,82813 | 1,45313 | ||||||||
total_stdev | 1,51537 | 0,30411 | 0,24992 | |||||||||||||
http://idena.navarra.es/descargas/GEOLOG_Pol_Litologia.zip | iterations | 10 | 10 | 10 | ||||||||||||
total_avg | 9,31406 | 2,19063 | 24% | 2,03750 | 22% | 7% | ||||||||||
total_max | 10,06250 | 3,07813 | 3,12500 | |||||||||||||
total_maxdev | 4,84531 | 0,88750 | 1,08750 | |||||||||||||
total_min | 4,46875 | 2,06250 | 1,89063 | |||||||||||||
total_stdev | 1,61862 | 0,29603 | 0,36304 | |||||||||||||
Urban parcels of cadastre of navarra (Polygon2D of 142680 records, 66mb) | iterations | 10 | 10 | 10 | ||||||||||||
total_avg | 8,20156 | 5,30000 | 65% | 3,75938 | 46% | 29% | ||||||||||
total_max | 8,62500 | 5,54688 | 3,82813 | |||||||||||||
total_maxdev | 3,06094 | 1,51875 | 0,15000 | |
total_min | 5,14063 | 3,78125 | 3,60938 | ||||||||
total_stdev | 1,02101 | 0,50745 | 0,05896 | |||||||||||||
Rustic parcels of cadastre of navarra (Polygon2D of 472658 records, 458mb) | iterations | 10 | 10 | 10 | ||||||||||||
total_avg | 43,41719 | 16,49219 | 38% | 11,77031 | 27% | 29% | ||||||||||
total_max | 46,25000 | 17,23438 | 12,04688 | |||||||||||||
total_maxdev | 24,57344 | 0,74219 | 0,78594 | |||||||||||||
total_min | 18,84375 | 16,32813 | 10,98438 | |||||||||||||
total_stdev | 8,19141 | 0,25610 | 0,27208 |
I think that disable 'AA' for far geometries is effective, but it is true that, by contrast, a lower visual quality is perceptible especially for maps with geometries isolated.
I think it's worth apply it.
Regards
#60 Updated by Matthias Kuhn almost 11 years ago
Thank you for this nice table.
So, disabling AA indeed seems to result in a noticeable difference in some cases.
I would prefer to make it an optional feature the way it looks. I have seen you committed something related to AA, was this a new config option?
#61 Updated by Alvaro Huarte almost 11 years ago
Matthias Kuhn wrote:
Thank you for this nice table.
So, disabling AA indeed seems to result in a noticeable difference in some cases.
I would prefer to make it an optional feature the way it looks. I have seen you committed something related to AA, was this a new config option?
Hi Matthias,
It is very appreciable when the layer is very dense with a lot of small geometries.
Referred to the last commit, I was pending disable 'AA' for '1-pixel geometries' only when the simplification of the layer is activated. Temporal 'QgsSymbolV2RenderContext' objects created in 'QgsSymbolV2' class lose its 'VectorLayer' owner, and I have fixed it.
Referred to append a new extra option for disable the 'AA' of far geometries, I think that if the user already active the 'Fast drawing' implicitly assumes that the drawing quality can be a bit lower. I think that add other option can be strange or too complex to "understand". But I accept that the community send me :-)
Best regards
#62 Updated by Regis Haubourg almost 11 years ago
Referred to append a new extra option for disable the 'AA' of far geometries, I think that if the user already active the 'Fast drawing' implicitly assumes that the drawing quality can be a bit lower. I think that add other option can be strange or too complex to "understand". But I accept that the community send me :-)
Hi Alvaro,
In fact, if AA is enabled, symboly will be altered since you can't use symbols smaller than 1 pixel. So user will have to recalibrate symbology with your option for existing projects. We should avoid that. One big point with QGIS is precisely AA rendering, and we must keep antialising enabled. If user wants a Mapinfo-Arcgis style, faster, AA option is already there in general options, and user should gain your speed improvements + AA-OFF improvement.
My two cents, Régis
#63 Updated by Alvaro Huarte almost 11 years ago
regis Haubourg wrote:
Referred to append a new extra option for disable the 'AA' of far geometries, I think that if the user already active the 'Fast drawing' implicitly assumes that the drawing quality can be a bit lower. I think that add other option can be strange or too complex to "understand". But I accept that the community send me :-)
Hi Alexandro,
In fact, if AA is enabled, symboly will be altered since you can't use symbols smaller than 1 pixel. So user will have to recalibrate symbology with your option for existing projects. We should avoid that. One big point with QGIS is precisely AA rendering, and we must keep antialising enabled. If user wants a Mapinfo-Arcgis style, faster, AA option is already there in general options, and user should gain your speed improvements + AA-OFF improvement.
My two cents, Régis
Hi, I don't understand your reply very well, :-)
I may not have described right this behavior:
The 'AA' is only individually disabled for geometries of a layer so far that it fill a '1-pixel' of the map, the rest are painted using the 'AA' option as heretofore.
#64 Updated by Alvaro Huarte almost 11 years ago
Alvaro Huarte wrote:
Nathan Woodrow wrote:
Marco Hugentobler wrote:
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
The rendering at large scales is the main issue most people have with large datasets so this is where you need the most performance gain which this patch seems to achieve. When you are at smaller scales you don't have as many geometries so normally QGIS handles things well. Some of my clients have pretty large datasets and this is a major issue for them.
Regarding exporting in the composer I would suggest just turning it off at that point and render with the full geometries. Navigating the map is what most people do and printing is normally expected to not be a super quick operation. It should be as fast as possible but if I have to wait 1 minute to get correct results in a print out I will happily, however I'm not going to wait a minute for my screen to render while I am building a map or editing.
IMO this needs to be considered. Not saying it should be merged today, or even next week but I think it's a worthwhile patch. As long as there is a way to switch it off, even as a build time option in case something really bad comes up then I think that would be fine.
Couple this kind of thing with the multithreading and we should see a nice boost in the rendering speed.
I agree, this patch improves the 'rendering at large scales', for getting rendering feedback the user can enable this existing option in the global options form of qgis.
To avoid problems in composer I am going disable this patch in 'printing' tasks.
Also, at least windows, there is other important improvement that I want test: Support for 'FileMapping' in the gdal-ogr library (e.g. using boost library) for speed up the access to data. I have experience with it and can be 2x faster.
Hi, the last commit disable the 'drawing simplification' for print compositions.
Best regards
#65 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
Marco Hugentobler wrote:
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
Hi,
speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.
By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.
Hi Giovanni, now I am going to make a Windows Installer with the current patch state.
Telling you as I have!
#66 Updated by Matthias Kuhn almost 11 years ago
Alvaro,
I can imagine to enable this functionality for screen rendering by default sooner or later, but only if all optimization are non-visible. (Sub-pixel simplification / no AA)
On the other hand it seems, that there is the possibility to improve performance even further but at the expense of visible graphics quality degradation. This can be ok for some situations / systems, but needs to be opt-in.
So I would welcome to have the two nicely separated already. Do you think this is a road we can take?
#67 Updated by Alvaro Huarte almost 11 years ago
Hi Matthias, I can add other option to the new 'fast drawing' box in the 'rendering' options panel for the 'AA-disabling' advising that it is possible a graphics quality degradation.
If you like well I want to try before prepare the windows installer with this patch Giovanni asked for days.
#68 Updated by Giovanni Manghi almost 11 years ago
Hi Giovanni, now I am going to make a Windows Installer with the current patch state.
Telling you as I have!
thanks!
#69 Updated by Alvaro Huarte almost 11 years ago
Alvaro Huarte wrote:
Giovanni Manghi wrote:
Marco Hugentobler wrote:
I've done some tests with the patch, and I have mixed feelings towards applying. The optimisation has most effect in zoomed-out situations where the data is too dense for the scale. But then, the user will still perceive it as slow performance (e.g. rendering 10 or 20 seconds without feedback is still slow). In situations where the data is appropriate for the scale, the differences are very small. On the other side, the patch adds a lot of code and still has potential to cause bugs (e.g. exporting in the composer with low resolution, say 50 dpi, shows gaps in a polygon mosaic).
Hi,
speaking as a user here. We all know that rendering performances of large vectors are poor in QGIS when compared to other packages, so I really hope that things can be sorted out. Is for this reason that I asked for a Windows installer that includes the patch (Alex is working on it, thanks!), to allow people test the patch, eventually find the issues and so allow you devs to eventually fix them.
By the way, loading time of large vectors is much better with the patch, this itself is a very welcome improvement.
Hi Giovanni, now I am going to make a Windows Installer with the current patch state.
Telling you as I have!
Done:
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86
It contains the Windows Installer with the qgis master branch + #8725 patch.
Regards
#70 Updated by Bo Victor Thomsen almost 11 years ago
Alvaro -
I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)
The postgis layer works in "normal" master.
OS Version: Window 8.1
Regards Bo Victor Thomsen
#71 Updated by Giovanni Manghi almost 11 years ago
Bo Victor Thomsen wrote:
Alvaro -
I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)
The postgis layer works in "normal" master.
OS Version: Window 8.1Regards Bo Victor Thomsen
confirmed here
#72 Updated by Giovanni Manghi almost 11 years ago
Done:
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86It contains the Windows Installer with the qgis master branch + #8725 patch.
Regards
Thanks! I will make some tests!
#73 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
Bo Victor Thomsen wrote:
Alvaro -
I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)
The postgis layer works in "normal" master.
OS Version: Window 8.1Regards Bo Victor Thomsen
confirmed here
Sorry, now I don't have installed the dump debugging tools.
I have tested with some postgis layers and works fine for me.
The error is throw for all layers ?
Are you using the application compiled from my pull request, or from the windows installer ?
If it is throwed by one layer, can you send me as shapefile for test here ?
Anyway, at night at home, I will setup the dump debugging tools for test the error
Thanks for your support!
Note: The improvement is very perceptible?
Best Regards
#74 Updated by Alvaro Huarte almost 11 years ago
Matthias Kuhn wrote:
Alvaro,
I can imagine to enable this functionality for screen rendering by default sooner or later, but only if all optimization are non-visible. (Sub-pixel simplification / no AA)
On the other hand it seems, that there is the possibility to improve performance even further but at the expense of visible graphics quality degradation. This can be ok for some situations / systems, but needs to be opt-in.
So I would welcome to have the two nicely separated already. Do you think this is a road we can take?
Hi Matthias, according your advice, the last commit implements 'simplification hints' allowing configure what drawing simplification type can be executed (Points, BBOX or AA, or any combination).
By default, the full simplification (Points+BBOX+AA) will be applied.
#75 Updated by andre mano almost 11 years ago
- Operating System set to Windows 7
- OS version set to 64bits
I tested with a shapefile containing 75k polygons and although I don't have a quantitative analysis, I can say that the rendering (even without going to quality threshold) it's at least twice as fast as QGIS 2.0.
I repeated the test with another shapefile containing 1000000 + polygons and the loading and pan/zoom operations became fast enough to make this layer usable in QGIS without loading it first into a spatial DB, specially if you give a high quality threshhold value.
A great add to QGIS!
#76 Updated by Giovanni Manghi almost 11 years ago
Note: The improvement is very perceptible?
yes :)
#77 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
Note: The improvement is very perceptible?
yes :)
I am delighted to read that.
I hope soon to add "FileMapping" support to gdal-ogr and when approved, another speed up for qgis!
#78 Updated by Alvaro Huarte almost 11 years ago
- Operating System deleted (
Windows 7) - OS version deleted (
64bits)
Alvaro Huarte wrote:
Giovanni Manghi wrote:
Note: The improvement is very perceptible?
yes :)
I am delighted to read that.
I hope soon to add "FileMapping" support to gdal-ogr and when approved, another speed up for qgis!
#79 Updated by Alvaro Huarte almost 11 years ago
Bo Victor Thomsen wrote:
Alvaro -
I've tried your patched version of QGIS. Works nicely with shape files and with large speed improvements. However, QGIS crashes when I'm trying to open a Postgis based layer. Mini dump file available at: https://docs.google.com/file/d/0B_386CU-PnW0cndaZXZOT1FWM2c/edit?usp=docslist_api (It's 20MB)
The postgis layer works in "normal" master.
OS Version: Window 8.1Regards Bo Victor Thomsen
Hello Bo Victor, you are right, qgis from my windows installer crashes when the user load a postgis layer.
But the error is not related with my patch code, the exception is thrown before load the layer in 'void QgsApplication::preNotify()' method when the 'add' button of the configuration form is clicked.
I think this error is described in #5597-27.
In my compiled version I do not get that error so it has to be what files I have replaced in the master version of qgis for make the windows installer.
I must study it for rebuild the windows installer for test this patch.
Regards
#80 Updated by Bo Victor Thomsen almost 11 years ago
Alvaro -
Sorry for being late to the party, I was at a mapping conference. Just to answer your questions even though you seem to have found the error: QGIS crashes just after I've selected one postgis table and clicked the "Add" button in the "PostGres add layer dialog".
Question no 2: The shape file (which works fine) has a speed improvement of 50%. The layer contains 1.300.000 polygon shapes and goes from 40 sec. load to 20 sec. load on my Icore7 PC with SSD.
Regards
#81 Updated by Alvaro Huarte almost 11 years ago
Bo Victor Thomsen wrote:
Alvaro -
Sorry for being late to the party, I was at a mapping conference. Just to answer your questions even though you seem to have found the error: QGIS crashes just after I've selected one postgis table and clicked the "Add" button in the "PostGres add layer dialog".Question no 2: The shape file (which works fine) has a speed improvement of 50%. The layer contains 1.300.000 polygon shapes and goes from 40 sec. load to 20 sec. load on my Icore7 PC with SSD.
Regards
Hi,
This error is strange for me, and seems like an old error is back, Should I add a issue to the list of QGIS?
#82 Updated by Bo Victor Thomsen almost 11 years ago
Alvaro -
I've just tried the Postgis layer on the latest 32 bit QGIS Master and Current(2.0.1-3) installed with the Osgeo4w installer. They both worked without a hitch. So the problem is isolated to the special install build of QGIS with the speed improvements. I don't think you should report it as a general QGIS issue. Is it possible for you to zip the qgis.exe or maybe the "bin" directory of your working copy of the special QGIS and upload it somewhere I could get it?
Regards
#83 Updated by Alvaro Huarte almost 11 years ago
Bo Victor Thomsen wrote:
Alvaro -
I've just tried the Postgis layer on the latest 32 bit QGIS Master and Current(2.0.1-3) installed with the Osgeo4w installer. They both worked without a hitch. So the problem is isolated to the special install build of QGIS with the speed improvements. I don't think you should report it as a general QGIS issue. Is it possible for you to zip the qgis.exe or maybe the "bin" directory of your working copy of the special QGIS and upload it somewhere I could get it?
Regards
Hi, thanks for your reply!
I found my error. Sorry, rookie mistake, I built the windows installer using the QGIS desktop distribution from the OSGeo4W setup, it is 2.0.X... all wrong.
I remade the installer from my master package and gives me no errors in the postgis loading form.
The new installer is (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v20
Only bin folder (8mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-bin-x86v20
Best Regards.
#84 Updated by Alvaro Huarte almost 11 years ago
Hi,
There is a bug in QgsOgrFeatureIterator::ensureRelevantFields(), the feature cursor always fetchs all attributes ignoring the relevant fields configuration of the feature request.
I am trying to improve the OGR-GDAL provider and Even Rouault (gdal-dev) help me for find this bug (#9062).
af73f30c fixes this in master!
In many cases the layers are required to be painted without any field, or one or two fields, therefore, other improvement for render vector layers!
Thanks for all!
Regards
#85 Updated by Giovanni Manghi almost 11 years ago
therefore, other improvement for render vector layers!
very good news!
any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.
#86 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
therefore, other improvement for render vector layers!
very good news!
any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.
Hi Giovanni, I prepare in brief.
Thanks for you support!
#87 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
therefore, other improvement for render vector layers!
very good news!
any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.
Hi,
The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30
It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)
Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.
Regards
#88 Updated by Giovanni Manghi almost 11 years ago
Hi Alvaro
with the new installer QGIS crashes when starting.
Cheers!
Alvaro Huarte wrote:
Giovanni Manghi wrote:
therefore, other improvement for render vector layers!
very good news!
any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.
Hi,
The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.Regards
#89 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
Hi Alvaro
with the new installer QGIS crashes when starting.
Cheers!
Alvaro Huarte wrote:
Giovanni Manghi wrote:
therefore, other improvement for render vector layers!
very good news!
any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.
Hi,
The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.Regards
Sorry, I can not rebuild well the installer, but it is independent of the changes I'm making.
I should download again all OSGeo4W packages.
I try shortly.
#90 Updated by Alvaro Huarte almost 11 years ago
Alvaro Huarte wrote:
Giovanni Manghi wrote:
Hi Alvaro
with the new installer QGIS crashes when starting.
Cheers!
Alvaro Huarte wrote:
Giovanni Manghi wrote:
therefore, other improvement for render vector layers!
very good news!
any chance for an updated windows installer? I don't need it as I can compile on Linux, it is just to allow the many Windows user to keep testing your (updated) work.
Hi,
The new installer (195mb):
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v30It also contains a proposed improvement in GDAL ( https://github.com/OSGeo/gdal/pull/26 ).
I hope to be accepted :-)Note:
This installer does not contain any plugin of python, it is a subset of the QGIS 2.1 release for testing proposes.Regards
Sorry, I can not rebuild well the installer, but it is independent of the changes I'm making.
I should download again all OSGeo4W packages.I try shortly.
I have updated my OSGeo4W packages and I hope that new windows installer is fine.
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x8640
Best Regards
#91 Updated by Bo Victor Thomsen almost 11 years ago
Hi Alvaro -
I just tried your new version of QGIS on my Postgis-based dataset (1.3 million buildings, i.e simple polygons with 4-10 nodes). This dataset crashed the first version of "your" QGIS. Now it works without a hitch. Drawing time is around 10 seconds versus 20 seconds using the ordinary version of QGIS. This is nice ;-) !!
Regards
Bo Victor Thomsen
Aests-GIS
Denmark
#92 Updated by Alvaro Huarte almost 11 years ago
Bo Victor Thomsen wrote:
Hi Alvaro -
I just tried your new version of QGIS on my Postgis-based dataset (1.3 million buildings, i.e simple polygons with 4-10 nodes). This dataset crashed the first version of "your" QGIS. Now it works without a hitch. Drawing time is around 10 seconds versus 20 seconds using the ordinary version of QGIS. This is nice ;-) !!
Regards
Bo Victor Thomsen
Aests-GIS
Denmark
You have used the last window installer ( http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x8640 ) ?
if so I'm glad I seem to have fixed my problems by making the installer.
For shapefiles, although it depends on the data (#records, size of geometries...), the improvement is still greater.
Thank you very much!
#93 Updated by Bo Victor Thomsen almost 11 years ago
Yes, I did use your latest version of QGIS. The drawing sped of QGIS with the postgis dataprovider is now compareable with MapInfo using it's native file format (Same dataset 1.3 mill. polygons, MapInfo w. tabfile: 8 seconds, QGIS w. Postgis: 10 seconds)
Regards
Bo Victor Thomsen
#94 Updated by Alvaro Huarte almost 11 years ago
This up guys! :-)
My proposed pull request (http://trac.osgeo.org/gdal/ticket/5272#comment:7) in GDAL-OGR library for optimize the OGR SHP-provider was accepted and Even Rouault has commited the modification with extra contributions.
The last windows installer already contains this improvement, really it contains my porposed pull, but they are equivalents.
Thanks Even!
#95 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
This up guys! :-)
My proposed pull request (http://trac.osgeo.org/gdal/ticket/5272#comment:7) in GDAL-OGR library for optimize the OGR SHP-provider was accepted and Even Rouault has commited the modification with extra contributions.
very nice done Alvaro!
until now the feedback I received is very good, with no noticeable major issue caused by your patch. We will keep testing.
#96 Updated by João Gaspar almost 11 years ago
My feedback:
OS: Windows 7 64-bits
Processor: Intel(R) Core(TM)2 Duo CPU T8300 @ 2.4GHz 2.4GHz
RAM: 4.00Gb
QGIS Version: QGIS standalone Testing of the Alvaro(32-bits)
Results:
I test two shapefiles:
-Contours of Portugal (30 meters distance) size 300mb of capacity (with a lot of table features)
Results:
With patch: 4 seconds to load / Without patch: 25 seconds to load
- Landuse Level 2 of Portugal size 25mb of capacity (also with a lot of table features)
Results:
With patch: 11 seconds to load / Without patch: more then a 1 minute to load
I think if you create a spatial index in shapefile will help this option of rendering and save a cople of seconds to load.
With simplify rendering on, is necessary notice the users that this option improve rendering but create some gaps in the visualization process that can be fix after the load of the shapefile and zooming to the desarible scale.
hope this feedback helps.
Great job Alvaro.
Regards
#97 Updated by Alvaro Huarte almost 11 years ago
João Gaspar wrote:
My feedback:
OS: Windows 7 64-bits
Processor: Intel(R) Core(TM)2 Duo CPU T8300 @ 2.4GHz 2.4GHz
RAM: 4.00GbQGIS Version: QGIS standalone Testing of the Alvaro(32-bits)
Results:
I test two shapefiles:
-Contours of Portugal (30 meters distance) size 300mb of capacity (with a lot of table features)
Results:
With patch: 4 seconds to load / Without patch: 25 seconds to load- Landuse Level 2 of Portugal size 25mb of capacity (also with a lot of table features)
Results:
With patch: 11 seconds to load / Without patch: more then a 1 minute to loadI think if you create a spatial index in shapefile will help this option of rendering and save a cople of seconds to load.
With simplify rendering on, is necessary notice the users that this option improve rendering but create some gaps in the visualization process that can be fix after the load of the shapefile and zooming to the desarible scale.
hope this feedback helps.
Great job Alvaro.
Regards
Thank your very much to all!
I am developing other improvement in GDAL related with this patch. Therefore I will release other windows installer and I will append a "warning" label indicating that increase factor may lead to the emergence of gaps.
But these gaps should not appear in the scroll at least, João, it is right?
Thanks!
#98 Updated by João Gaspar almost 11 years ago
Sorry my English sometimes tricks me. :p
I don't understand what you want to say with "not appear in the scroll at least".
#99 Updated by Alvaro Huarte almost 11 years ago
João Gaspar wrote:
Sorry my English sometimes tricks me. :p
I don't understand what you want to say with "not appear in the scroll at least".
No, my English is bad :-P, I trying to say: if you view gaps when the value of simplification factor configured (slider in the fast-rendering form of the layer) is the mininum.
#100 Updated by João Gaspar almost 11 years ago
Ahaha. ;)
In the minimum level don't appear gaps is "clean and smooth". ;)
Only appears when i increase the slider to high values of simplify geometries.
regards
#101 Updated by Alvaro Huarte almost 11 years ago
João Gaspar wrote:
Ahaha. ;)
In the minimum level don't appear gaps is "clean and smooth". ;)
Only appears when i increase the slider to high values of simplify geometries.
regards
:-), ok, That's reassuring to me, with the minimum value occurs simplification but should not leave gaps. I'll put a warning for higher values because in those cases it may appear.
Regards
#102 Updated by Regis Haubourg almost 11 years ago
Hi Alvaro, I tested your windows package, this is huge! Many thanks...
As a GIS administrator, I would need a general option in QGIS settings to set default behaviour and simplification level for new layers. Is that possible? We will have to deal qgs migration, could we be sure that layers coming from older QGIS will get these default QGIS settings?
Cheers
Régis
#103 Updated by Alvaro Huarte almost 11 years ago
- File WarningLabel.jpg added
Alvaro Huarte wrote:
João Gaspar wrote:
My feedback:
OS: Windows 7 64-bits
Processor: Intel(R) Core(TM)2 Duo CPU T8300 @ 2.4GHz 2.4GHz
RAM: 4.00GbQGIS Version: QGIS standalone Testing of the Alvaro(32-bits)
Results:
I test two shapefiles:
-Contours of Portugal (30 meters distance) size 300mb of capacity (with a lot of table features)
Results:
With patch: 4 seconds to load / Without patch: 25 seconds to load- Landuse Level 2 of Portugal size 25mb of capacity (also with a lot of table features)
Results:
With patch: 11 seconds to load / Without patch: more then a 1 minute to loadI think if you create a spatial index in shapefile will help this option of rendering and save a cople of seconds to load.
With simplify rendering on, is necessary notice the users that this option improve rendering but create some gaps in the visualization process that can be fix after the load of the shapefile and zooming to the desarible scale.
hope this feedback helps.
Great job Alvaro.
Regards
Thank your very much to all!
I am developing other improvement in GDAL related with this patch. Therefore I will release other windows installer and I will append a "warning" label indicating that increase factor may lead to the emergence of gaps.But these gaps should not appear in the scroll at least, João, it is right?
Thanks!
Hi João,
I added a warning label that appears when the user increases the simplification factor above the minimum level.
Regards
#104 Updated by Alvaro Huarte almost 11 years ago
regis Haubourg wrote:
Hi Alvaro, I tested your windows package, this is huge! Many thanks...
As a GIS administrator, I would need a general option in QGIS settings to set default behaviour and simplification level for new layers. Is that possible? We will have to deal qgs migration, could we be sure that layers coming from older QGIS will get these default QGIS settings?
Cheers
Régis
Hi Régis, thanks!
I can append a general option, I write it down on my to-do list.
Regards
#105 Updated by Alvaro Huarte almost 11 years ago
Alvaro Huarte wrote:
Hi João,
I added a warning label that appears when the user increases the simplification factor above the minimum level.
To describe it a little better, check the box performs the drawing simplification and accelerates the render of geometries. If the simplification factor is the minimum value, must not appear gaps, the more you increase it, is more likely to appear and so in these cases I present the warning.
As a user I do not need to increase the default value except in extreme cases of very large datasets.
#106 Updated by João Gaspar almost 11 years ago
Yeah really good. Describes perfectly the situation. =)
Regards
#107 Updated by Nyall Dawson almost 11 years ago
Can I suggest some improvements in the strings?
"Fast drawing" -> "Simplify geometry"
"Note ..." -> "Note: Enabling this option simplifies geometries in this layer to improve rendering speed. The simplification applies only during rendering of the layer and does not modify the layer geometry."
"Simplification factor..." -> "Simplification factor (higher values result in more simplification):"
"Warning..." -> "Warning: Increasing this factor will further speed up rendering, but may result in gaps or topological errors in the layer display."
#108 Updated by Alvaro Huarte almost 11 years ago
- File WarningLabel2.jpg added
Nyall Dawson wrote:
Can I suggest some improvements in the strings?
"Fast drawing" -> "Simplify geometry"
"Note ..." -> "Note: Enabling this option simplifies geometries in this layer to improve rendering speed. The simplification applies only during rendering of the layer and does not modify the layer geometry."
"Simplification factor..." -> "Simplification factor (higher values result in more simplification):"
"Warning..." -> "Warning: Increasing this factor will further speed up rendering, but may result in gaps or topological errors in the layer display."
Done, thank you very much Nyall!
#109 Updated by Rui Pedro Henriques almost 11 years ago
Hi, I've tested this on Win7 64 bit with a previous project containing several raster and vector layers. Surprinsingly the whole set got slower: even while having only 2 layers enabled it got really heavy and also affected the scroll of the layers list. For the moment I'm not able to provide the dataset I was working on but I'll try to recreate the problem with a dataset I can supply.
Trying with a clean project with just one (complex) vector layer did speed up things.
Anyone else noticed something similar?
#110 Updated by Alvaro Huarte almost 11 years ago
rph - wrote:
Hi, I've tested this on Win7 64 bit with a previous project containing several raster and vector layers. Surprinsingly the whole set got slower: even while having only 2 layers enabled it got really heavy and also affected the scroll of the layers list. For the moment I'm not able to provide the dataset I was working on but I'll try to recreate the problem with a dataset I can supply.
Trying with a clean project with just one (complex) vector layer did speed up things.Anyone else noticed something similar?
Hi, I tried to reproduce this behavior and have not succeeded. I see no connection with this patch, but anyway this QGIS has not linked all data providers, it only contains most popular for test.
Regards
#111 Updated by Giovanni Manghi almost 11 years ago
Anyone else noticed something similar?
can you try remove the layers (one by one) from your project and see if you find one that can be the cause?
Anyone else noticed something similar?
I may have seen this (but not sure) with a specific vector layer, made of a lot of polygons that are not possible to simplify, because it is grid polygon vector made of perfect squares.
In this case seem that qgis takes a lot trying figure how simplify, eventually taking an overhead. But as I said is just an impression as I have not tested it seriously. Example
https://www.dropbox.com/s/2leh363cdbupg1w/grid25_clip.tar.gz
#112 Updated by Alvaro Huarte almost 11 years ago
Giovanni Manghi wrote:
Anyone else noticed something similar?
can you try remove the layers (one by one) from your project and see if you find one that can be the cause?
Anyone else noticed something similar?
I may have seen this (but not sure) with a specific vector layer, made of a lot of polygons that are not possible to simplify, because it is grid polygon vector made of perfect squares.
In this case seem that qgis takes a lot trying figure how simplify, eventually taking an overhead. But as I said is just an impression as I have not tested it seriously. Example
https://www.dropbox.com/s/2leh363cdbupg1w/grid25_clip.tar.gz
What you're saying makes sense, I may be trying to simplify polygons with <=5 points (grids) which themselves are not generalizable,
I look, thanks Giovanni!!
#113 Updated by Alvaro Huarte almost 11 years ago
Alvaro Huarte wrote:
Giovanni Manghi wrote:
Anyone else noticed something similar?
can you try remove the layers (one by one) from your project and see if you find one that can be the cause?
Anyone else noticed something similar?
I may have seen this (but not sure) with a specific vector layer, made of a lot of polygons that are not possible to simplify, because it is grid polygon vector made of perfect squares.
In this case seem that qgis takes a lot trying figure how simplify, eventually taking an overhead. But as I said is just an impression as I have not tested it seriously. Example
https://www.dropbox.com/s/2leh363cdbupg1w/grid25_clip.tar.gz
What you're saying makes sense, I may be trying to simplify polygons with <=5 points (grids) which themselves are not generalizable,
I look, thanks Giovanni!!
I answer to myself :-), the patch try not to simplify polygons of 5 points, or linestrings of 2 points, but try to find out if it is possible to disable the 'AntiAliasing' for render the geometry faster. If the end is not possible to disable the AA (The geometry is not so far), an overload occurs in the process.
This can be avoided disabling for this layer the drawing simplification.
#114 Updated by Alvaro Huarte almost 11 years ago
- File DefaultSimplifyConfig.jpg added
regis Haubourg wrote:
Hi Alvaro, I tested your windows package, this is huge! Many thanks...
As a GIS administrator, I would need a general option in QGIS settings to set default behaviour and simplification level for new layers. Is that possible? We will have to deal qgs migration, could we be sure that layers coming from older QGIS will get these default QGIS settings?
Cheers
Régis
Done!
Regards
#115 Updated by Alvaro Huarte almost 11 years ago
- File deleted (
sketch.png)
#116 Updated by Alvaro Huarte almost 11 years ago
Hi,
I release a new windows installer which contains:
- Patch #8725 of QGIS (https://github.com/qgis/QGIS/pull/980)
- Patch #9062 of QGIS (#9062)
- Patch N5272 of GDAL (http://trac.osgeo.org/gdal/ticket/5272)
http://www.filedropper.com/qgis-osgeo4w-210-f8725-setup-x86v50
I made new tests with the latest version to see the speed.
Soft/hard: WindowsXP+SP3, 32bits, 4gb RAM.
My shapefiles tested:
Dataset | Value | Master | Patched | |||||||
---|---|---|---|---|---|---|---|---|---|---|
http://faunalia.pt/downloads/GTBs_Shapefile.zip | iterations | 5 | 5 | |||||||
total_avg | 214,14063 | 84,29375 | 39% | |||||||
total_max | 222,53125 | 85,39063 | ||||||||
total_maxdev | 29,60938 | 1,21563 | ||||||||
total_min | 184,53125 | 83,07813 | ||||||||
total_stdev | 14,81578 | 0,93151 | ||||||||
http://faunalia.pt/downloads/MAMMTERR.zip | iterations | 5 | 5 | |||||||
total_avg | 86,60938 | 10,60625 | 12% | |||||||
total_max | 98,96875 | 13,39063 | ||||||||
total_maxdev | 46,48438 | 2,78438 | ||||||||
total_min | 40,12500 | 9,81250 | ||||||||
total_stdev | 23,24727 | 1,39311 | ||||||||
http://www.igeo.pt/scrif/crif/CRIF2011shp.zip | iterations | 5 | 5 | |||||||
total_avg | 287,47500 | 67,36250 | 23% | |||||||
total_max | 307,04688 | 77,32813 | ||||||||
total_maxdev | 75,53750 | 9,96563 | ||||||||
total_min | 211,93750 | 64,78125 | ||||||||
total_stdev | 37,77141 | 4,98346 | ||||||||
https://www.dropbox.com/s/2leh363cdbupg1w/grid25_clip.tar.gz | iterations | 5 | 5 | |||||||
total_avg | 100,86250 | 54,27500 | 54% | |||||||
total_max | 101,20313 | 55,20313 | ||||||||
total_maxdev | 0,72187 | 2,93125 | ||||||||
total_min | 100,14063 | 51,34375 | ||||||||
total_stdev | 0,38151 | 1,46910 | ||||||||
http://idena.navarra.es/descargas/CARTO1_Lin_6CNivelD.zip | iterations | 5 | 5 | |||||||
total_avg | 7,98125 | 1,30000 | 16% | |||||||
total_max | 9,35938 | 2,14063 | ||||||||
total_maxdev | 4,30938 | 0,84063 | ||||||||
total_min | 3,67188 | 1,06250 | ||||||||
total_stdev | 2,16033 | 0,42175 | ||||||||
http://idena.navarra.es/descargas/GEOLOG_Pol_Litologia.zip | iterations | 5 | 5 | |||||||
total_avg | 9,97188 | 1,79063 | 18% | |||||||
total_max | 11,26563 | 2,84375 | ||||||||
total_maxdev | 4,95625 | 1,05313 | ||||||||
total_min | 5,01563 | 1,50000 | ||||||||
total_stdev | 2,47837 | 0,52697 | ||||||||
Urban parcels of cadastre of navarra (Polygon2D of 142680 records, 66mb) | iterations | 5 | 5 | |
total_avg | 50,28438 | 10,01563 | 20% | ||
total_max | 55,93750 | 14,60938 | ||||||||
total_maxdev | 22,23750 | 4,59375 | ||||||||
total_min | 28,04688 | 8,81250 | ||||||||
total_stdev | 11,11898 | 2,29725 | ||||||||
Rustic parcels of cadastre of navarra (Polygon2D of 542658 records, 458mb) | iterations | 5 | 5 | |||||||
total_avg | 10,17500 | 2,65938 | 26% | |||||||
total_max | 11,04688 | 2,68750 | ||||||||
total_maxdev | 3,37813 | 0,02813 | ||||||||
total_min | 6,79688 | 2,64063 | ||||||||
total_stdev | 1,68913 | 0,01822 |
#117 Updated by Nyall Dawson almost 11 years ago
Here's some more string updates for this latest addition:
"By default simplify geometries" -> "Simplify geometries by default"
And for the text below the checkbox:
"If checked, new vector layers added to the map will automatically use geometry simplification to speed up rendering. ... (then the rest of what you have)"
Thanks!
#118 Updated by Nyall Dawson almost 11 years ago
Alvaro - these improvements look fantastic! Your hard work is much appreciated!
One related idea I was planning on investigating was adding an option to disable rendering of features smaller than a certain size (in mm). You'll see a similar option in the labelling properties for a vector layer under "Rendering" -> "Suppress labelling of features smaller than ". Having the ability to skip rendering of small features could potentially lead to some more performance gains (as well as having cartographic benefits). Obviously, it should be disabled by default for layers as it would only be useful in some circumstances.
There was a bit of discussion on the QGIS dev list about this a month or so ago -- I had a quick initial attempt at implementing it but have got side tracked on composer work (and it's looking like this will tie me up for the whole of the 2.2 dev cycle).
If you wanted to experiment with this, there's code for checking the size of a feature in mm in QgsPalLayerSettings::checkMinimumSizeMM . I had a quick play here https://github.com/nyalldawson/QGIS/commit/27a42e11b31521a521f8435fc4e82d75568c97cb (no gui, hard coded to a minimum size of 2mm) which yielded some benefit. This was done in QgsVectorLayer::drawRendererV2, I think moving it further up the chain could improve the performance some more.
#119 Updated by Alvaro Huarte almost 11 years ago
Nyall Dawson wrote:
Here's some more string updates for this latest addition:
"By default simplify geometries" -> "Simplify geometries by default"
And for the text below the checkbox:
"If checked, new vector layers added to the map will automatically use geometry simplification to speed up rendering. ... (then the rest of what you have)"
Thanks!
Done, thanks Nyall!
#120 Updated by Alvaro Huarte almost 11 years ago
Nyall Dawson wrote:
Alvaro - these improvements look fantastic! Your hard work is much appreciated!
One related idea I was planning on investigating was adding an option to disable rendering of features smaller than a certain size (in mm). You'll see a similar option in the labelling properties for a vector layer under "Rendering" -> "Suppress labelling of features smaller than ". Having the ability to skip rendering of small features could potentially lead to some more performance gains (as well as having cartographic benefits). Obviously, it should be disabled by default for layers as it would only be useful in some circumstances.
There was a bit of discussion on the QGIS dev list about this a month or so ago -- I had a quick initial attempt at implementing it but have got side tracked on composer work (and it's looking like this will tie me up for the whole of the 2.2 dev cycle).
If you wanted to experiment with this, there's code for checking the size of a feature in mm in QgsPalLayerSettings::checkMinimumSizeMM . I had a quick play here https://github.com/nyalldawson/QGIS/commit/27a42e11b31521a521f8435fc4e82d75568c97cb (no gui, hard coded to a minimum size of 2mm) which yielded some benefit. This was done in QgsVectorLayer::drawRendererV2, I think moving it further up the chain could improve the performance some more.
Hi Nyall, I did not know this discussion!, I have involved in QGIS later. Coincidences of life, this patch proposes an implementation of that issue. Now, I'm a little waiting for comments and suggestions to see if it is interesting to approval.
On the subject of not paint "very small" elements seems like a good idea in case of layers as you mention, in other cases (very dense layers) would appear whole areas unpainted, but I would not mix this with this patch but on a new patch. If you want I can try to implement it.
Regards
#121 Updated by Nyall Dawson almost 11 years ago
Alvaro Huarte wrote:
Hi Nyall, I did not know this discussion!, I have involved in QGIS later. Coincidences of life, this patch proposes an implementation of that issue. Now, I'm a little waiting for comments and suggestions to see if it is interesting to approval.
I know - it's great timing! Thanks for all the work you've put in developing these much needed improvements for QGIS.
On the subject of not paint "very small" elements seems like a good idea in case of layers as you mention, in other cases (very dense layers) would appear whole areas unpainted, but I would not mix this with this patch but on a new patch. If you want I can try to implement it.
I don't want to add to your workload, but it'd be great if you could check it out. I've realized I'm not going to get a chance to tackle this for 2.2. Based on your past work you've got a much better understanding of optimising code like this then I do!
#122 Updated by Alvaro Huarte almost 11 years ago
Nyall Dawson wrote:
Alvaro Huarte wrote:
Hi Nyall, I did not know this discussion!, I have involved in QGIS later. Coincidences of life, this patch proposes an implementation of that issue. Now, I'm a little waiting for comments and suggestions to see if it is interesting to approval.
I know - it's great timing! Thanks for all the work you've put in developing these much needed improvements for QGIS.
On the subject of not paint "very small" elements seems like a good idea in case of layers as you mention, in other cases (very dense layers) would appear whole areas unpainted, but I would not mix this with this patch but on a new patch. If you want I can try to implement it.
I don't want to add to your workload, but it'd be great if you could check it out. I've realized I'm not going to get a chance to tackle this for 2.2. Based on your past work you've got a much better understanding of optimising code like this then I do!
Thanks Nyall, I also really appreciate the tests, advice and opinions of the community, they serve me to keep learning every day. And your english corrections ;-)
#123 Updated by Jeremy Palmer almost 11 years ago
I've just seen this pull request. Dumb question, but why would you implement the user defined
simplification setting? Seems to me it should be sorted by the software automatically, based on extent and the map to pixel setting. Is also overly complicated for the user...
#124 Updated by Alvaro Huarte almost 11 years ago
Jeremy Palmer wrote:
I've just seen this pull request. Dumb question, but why would you implement the user defined
simplification setting? Seems to me it should be sorted by the software automatically, based on extent and the map to pixel setting. Is also overly complicated for the user...
Hi Jeremy, the software automatically calculates the simplification factor based on geometry and current map2pixel setting (Slider with minimum value).
I added the possibility of increase this factor to get 'extra' simplification to accelerate even more the render of very large layers.
In this case it is possible that appear gaps and one warning message is showed in the configuration panel.
As I said above the user, in most cases, do not worry about modifying this value, but in other cases it may be interesting.
Regards
#125 Updated by Jeremy Palmer almost 11 years ago
Hi Alvaro,
Thanks for the reply. In that case it might be worth stating on the slider label that it is a advanced experimental option.
Cheers,
Jeremy
#126 Updated by Alvaro Huarte almost 11 years ago
Jeremy Palmer wrote:
Hi Alvaro,
Thanks for the reply. In that case it might be worth stating on the slider label that it is a advanced experimental option.
Cheers,
Jeremy
Hi Jeremy, I suppose we should add an entry in the user manual and explain this configuration.
Thanks!
Alvaro
#127 Updated by Alvaro Huarte almost 11 years ago
Hi all, There is a modification in behavior of this patch (https://github.com/qgis/QGIS/pull/980).
Now, it contains the advise of @nyalldawson (https://github.com/qgis/QGIS/pull/1020#issuecomment-30263200) to enable AA-disabling only when the simplification slider is greater than minimum value.
Best regards
#128 Updated by Alvaro Huarte almost 11 years ago
- File BadSimplificationWithOffset.jpg added
#129 Updated by Alvaro Huarte almost 11 years ago
I am pleased to announce that this patch seems to have been merged!
:-)
Thank you very much for your support and advices!
#130 Updated by Giovanni Manghi almost 11 years ago
Alvaro Huarte wrote:
I am pleased to announce that this patch seems to have been merged!
:-)Thank you very much for your support and advices!
Alvaro, many thanks to YOU :)
#131 Updated by Giovanni Manghi almost 11 years ago
- Status changed from Open to Closed
- Resolution set to fixed/implemented
#132 Updated by Alvaro Huarte almost 11 years ago
What about add documentation to user manual?
#133 Updated by Jürgen Fischer almost 11 years ago
merged in 17cd09a8a7352d761602c547d29f660aed219adb
#134 Updated by Alvaro Huarte almost 11 years ago
Hi, I committed a new version of geometry simplification to speed up the vector drawing.
https://github.com/ahuarte47/QGIS/tree/Issue_8725-revival-optA-to-B
It is inspired in this PR, but it contains several changes based on advice received ( @matthias-kuhn thanks! ) and new features:
- About code, the simplification is configured in a new QgsSimplifyMethod class which indicates how to simplify the geometries in a feature iterator.
- Now, the user can define where the simplification executes (There is a new option in settings panel), locally after fetch the geometry from provider, or simplifying it on provider side. e.g. In postgres provider, first option simplifies the geometry already fetched locally, but the second option simplifies the geometry in database using the function ST_Simplify.
- The settings panel also shows the simplification threshold in pixel units as @timlinux suggested me.
I have not created a new pull request because of I would greatly appreciate if you could test it with your data and SO's. I have tested it in windows I get similar results as old version for shapefiles (OGR-provider). The new simplification on database side for postgres provider improves the old results too. I will create a table of tests to compare data and configurations.
About postgres simplification, the ST_simplify function needs a tolerance parameter, I use map2pixel/5.0 as input value, it is experimental and I must define it better (All ideas are welcome). This simplification can be applicable to other database providers (MySQL, SQL Server, Oracle...)
Thank you very much!