Bug report #15292
Updated by Jürgen Fischer almost 7 years ago
See #15292-3
See: https://issues.qgis.org/issues/15292#note-3
---------------
I had a shapefile with 481k features and a half gigabyte attribute table. I did not require the majority of the data included, and so in order to make the file more manageable (less storage, faster to work with), I used the editor to delete the unwanted columns. To its credit QGIS did complete the task, unlike LibreOffice which simply crashed, however it took 36 hours* to do so! During the process I realized that part of the performance problem stems from the approach QGIS used to delete the specified attributes i.e; removing them column-by-column, then rewriting and reloading the database each time. For shapefiles where there are a large number of records or attributes, or the majority of such are being shed, it seems it would be much more efficient to create a temporary shapefile and copy over the data the user wishes to preserve, and then replace the original file with the new version.
Similarly, if not used, memory mapping large files could increase performance.
*2.5GHz Core i5-5300 with 8GB RAM on Windows 7
See: https://issues.qgis.org/issues/15292#note-3
---------------
I had a shapefile with 481k features and a half gigabyte attribute table. I did not require the majority of the data included, and so in order to make the file more manageable (less storage, faster to work with), I used the editor to delete the unwanted columns. To its credit QGIS did complete the task, unlike LibreOffice which simply crashed, however it took 36 hours* to do so! During the process I realized that part of the performance problem stems from the approach QGIS used to delete the specified attributes i.e; removing them column-by-column, then rewriting and reloading the database each time. For shapefiles where there are a large number of records or attributes, or the majority of such are being shed, it seems it would be much more efficient to create a temporary shapefile and copy over the data the user wishes to preserve, and then replace the original file with the new version.
Similarly, if not used, memory mapping large files could increase performance.
*2.5GHz Core i5-5300 with 8GB RAM on Windows 7