Bug report #15407
Saving Shapefile edits corrupts file
Status: | Closed | ||
---|---|---|---|
Priority: | Normal | ||
Assignee: | Even Rouault | ||
Category: | Data Provider/OGR | ||
Affected QGIS version: | 2.14.5 | Regression?: | No |
Operating System: | Easy fix?: | No | |
Pull Request or Patch supplied: | No | Resolution: | fixed/implemented |
Crashes QGIS or corrupts data: | Yes | Copied to github as #: | 23337 |
Description
We have a number of shapefiles that have been created/edited in previous versions of QGIS where deleted features remained as phantom features (reportedly now fixed as per Issue 11007).
In the current LTR version (2.14.5) if these files are edited (feature deleted, split, merged or vertex added) then upon saving the edits, the entire file is corrupted - deleted features reappear, others disappear and attributes become completely reassigned to different features.
I have attached an example file that this occurs to.
I know this file suffers from the 'phantom' deleted features, because the feature count in the layers menu is higher than the feature count in the attribute table.
If a save-as of the file is done, the feature count of the new file matches that of the attribute table. This re-saved file can also have any edits done without resulting in corruption.
Not all files with these 'phantom' features behave this way - some can be edited and saving edits behaves exactly as expected, with the phantom features being cleaned out and all other data remaining as it should.
This file has had significant edits with large areas deleted, etc. so the resulting corruption is very obvious immediately.
Other files may be less obvious - how is a user to know when this will occur and ensure the file will not become corrupted after they save edits?
I had hoped that by making some edits that did not result in a change of object FID's (such as a simple attribute update) and saving those edits might result in a clean out ('packing') of the file, however this is not the case. It appears that perhaps the 'repack' function is only called upon saving edits such as deleting, merging or splitting features as well as adding a vertex to an existing feature. Even the addition of a new object does not seem to result in the cleanse of the layer :(
Related issues
Associated revisions
[OGR provider] Force REPACK at the first edit action.
In the case where we deal with a shapefile, it is possible that it has
pre-existing holes in the DBF (see #15407), so if using a GDAL version
recent enough (>=2.1.2) to have reliable packing, do a packing at the
first edit action.
Fixes #15407
Conflicts:
src/providers/ogr/qgsogrprovider.cpp
History
#1 Updated by Saber Razmjooei over 8 years ago
#2 Updated by Saber Razmjooei over 8 years ago
- Status changed from Open to Feedback
#3 Updated by Jamie Portman over 8 years ago
Should I be doing that, or does an administrator need to?
I feel so bad... they were so thrilled to have finally resolved that issue after two years of feedback... :(
#4 Updated by Saber Razmjooei over 8 years ago
The ticket has been re-opened.
I suggest to try editing the file in another GIS package and see if the corruption happens there too.
#5 Updated by Jamie Portman over 8 years ago
MapInfo can't open a Shapefile that has 'phantom' features, the Universal Translator won't convert it and if you open in ArcGIS all the 'deleted' features reappear as it doesn't recognise the 'deleted' flag in the DBF!
#6 Updated by Danny Duong about 8 years ago
Does this issue occur in LTR 2.14.6?
Also, is it easily repeatable?
I don't seem to have the phantom deleted features anymore and haven't really noticed any discrepancies.
#7 Updated by Amy Taylor about 8 years ago
This is a problem we are seeing as well, and with shapefiles which have been created in QGIS as well as those older ones created using other software.
It seems to be that the split tool is the one which most commonly causes problems, where a shapefile will look ok until you hit the save button, then half of the split feature will disappear and attributes will become assigned to the wrong feature. Editing individual nodes is also flagging up problems, where if a node is deleted then sometimes the remaining adjacent nodes connect to a node on a totally separate and unconnected feature.
Unfortunately the problem isn't easily repeatable so we haven't been able to get an example shapefile which we can post on here.
#8 Updated by Even Rouault about 8 years ago
- Status changed from Feedback to Closed
Fixed in changeset 3a906a188cf2f238003a15f56b0b5177dcf87a2c.
#9 Updated by Even Rouault about 8 years ago
- Resolution set to fixed/implemented
- Category set to Data Provider/OGR
- Assignee set to Even Rouault
- Target version set to Version 2.14