Bug report #7822
Export to Spatialite is really slow, compared to commandline ogr2ogr
| Status: | Closed | ||
|---|---|---|---|
| Priority: | Severe/Regression | ||
| Assignee: | - | ||
| Category: | Vectors | ||
| Affected QGIS version: | master | Regression?: | No |
| Operating System: | Easy fix?: | No | |
| Pull Request or Patch supplied: | Yes | Resolution: | fixed/implemented |
| Crashes QGIS or corrupts data: | No | Copied to github as #: | 16707 |
Description
Testdata: http://gis-lab.info/forum/download/file.php?id=6540
Save as Spatialite takes more than an hour.
ogr2ogr -f SQLite testspatialite.sqlite geocoded.shp -dsco SPATIALITE=YES
Produces result in 10 seconds.
Checked on 1fce8a3, GDAL 1.9.2
qgsvectorfilewriter.cpp.diff 
- patch file for cpp
(574 Bytes)
qgsvectorfilewriter.h.diff
- patch file for header
(143 Bytes)
Associated revisions
History
#1
Updated by Giovanni Manghi over 11 years ago
- Status changed from Open to Feedback
confirmed. How was speed in 1.8?
#2
Updated by Maxim Dubinin over 11 years ago
it is equally slow in 1.8
#3
Updated by Larry Shaffer over 11 years ago
Confirmed here on Mac with latest master. I thought the app had crashed. It was not a large layer that was being saved: ~100 features.
#4
Updated by Daniel Vaz over 11 years ago
- File qgsvectorfilewriter.cpp.diff added
- File qgsvectorfilewriter.h.diff added
I read the OGR documentation and I suppose that it problem would be related with transaction system which isn't being used.
I create the patches for qgsvectorfilewriter class (see files attached). The patches were created from qgis-1.8.0.tar.bz2 downloaded from project site.
I ran QGis with Testdata and QGis takes minus than ten seconds to execute (Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz - 8GB RAM)
#5
Updated by Daniel Vaz over 11 years ago
The issue #3002 reports the same problem described here.
#6
Updated by Giovanni Manghi over 11 years ago
- Pull Request or Patch supplied changed from No to Yes
Daniel Vaz wrote:
I read the OGR documentation and I suppose that it problem would be related with transaction system which isn't being used.
I create the patches for qgsvectorfilewriter class (see files attached). The patches were created from qgis-1.8.0.tar.bz2 downloaded from project site.
I ran QGis with Testdata and QGis takes minus than ten seconds to execute (Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz - 8GB RAM)
Hi!
would you mind make a pull request on github with your patch? otherwise the devs are likely not to see this patch, review it and eventually commit it. Thanks!
#7
Updated by Daniel Vaz over 11 years ago
Hi, I will do at home as soon as possible. Thanks for the answer and the directions. Best regards.
#8
Updated by Nathan Woodrow over 11 years ago
- Status changed from Feedback to Closed
Fixed in changeset e491acb2aa6e9351937d42944ebbe5d7573b5ca8.
#9
Updated by Maxim Dubinin almost 11 years ago
- Status changed from Closed to Reopened
The bug resurfaced and export in the latest nighly build is very slow. Reported both on Windows 7 and Ubuntu.
#10
Updated by Giovanni Manghi almost 11 years ago
- Priority changed from Normal to Severe/Regression
- Target version set to Future Release - High Priority
#11
Updated by Alexander Bruy almost 11 years ago
- Status changed from Reopened to Feedback
Works like a charm in latest master (021441b361). Export of the attached file took about several seconds. Tested under Linux and Windows. I also tested larger file (~30 Mb point shapefile) and it also exported very fast.
#12
Updated by Alexander Bruy almost 11 years ago
Still cannot confirm. Export takes several seconds.
#13
Updated by Giovanni Manghi almost 11 years ago
- Status changed from Feedback to Open
Alexander Bruy wrote:
Still cannot confirm. Export takes several seconds.
Hi Alex,
I can confirm the issue at some extent, the cause seems the same (already fixed by you) that slowed down a lot the process of placing off-line a postgis table with the off-line editing plugin. I tested on both Linux and Windows with qgis master.
It takes many minutes to export as SL the provided (~40000) point layer. The curios thing is that now it takes that long also from the CLI with ogr2ogr, so may be the issue is not QGIS after all. Maybe it is now necessary to use different parameters in og2ogr when converting?
#14
Updated by Giovanni Manghi almost 11 years ago
- Status changed from Open to Closed
- Resolution set to fixed/implemented
Please reopen if I'm wrong:
right now (on Ubuntu, GDAL/OGR 1.10.0) the operation is slow also from the command line.
For what I can see here
and here
http://trac.osgeo.org/gdal/ticket/5270
it seems that the upstream fix is available in gdal/ogr 1.10.2