Bug report #15962

Union & Intersection results are Incorrect

Added by Devdatta Tengshe almost 3 years ago. Updated about 2 years ago.

Status:Closed
Priority:High
Assignee:-
Category:Processing/QGIS
Affected QGIS version:2.18.11 Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:duplicate
Crashes QGIS or corrupts data:Yes Copied to github as #:23878

Description

I found some strange issues when running Intersect & Union from the Vector>>GeoProcessing Menu. I've reproduced this on QGIS 2.14.4 as well as 2.18.0 on Windows 8

I observed, that for some datasets, the results of Overlay analysis are strange, and not at all what you expect. I've attached two sample datasets which show this.
1)Sample 1 consists of 2 shapefiles and the results of Union & Intersection. As you can see from the Image, this is not the results one should be getting.!Intersect1!

2)Sample 2 consists of the data where I saw the issue for the first time. When you Union 'dissolved.shp' & 'wards.shp' you expect to get an layer with unioned shapes of all features , but you get some kind of data which is closer to a 'Difference'; But if you closely look, there are more artifacts present in the data.

This might be related to Bugs #14846 & #8456 ,bu I'm not sure.

It should also be noted that the input shapefiles were created in QGIS itself, so if there is any issue with incorrect Geometries, there might be a bug in the module which created these shapefiles as well.

Sample1.zip - Sample 1 (7.29 KB) Devdatta Tengshe, 2016-12-09 12:01 AM

Sample2.zip - Sample 2 (254 KB) Devdatta Tengshe, 2016-12-09 12:01 AM

Intersect.png - Intersect1 (45.9 KB) Devdatta Tengshe, 2016-12-09 12:01 AM

Screenshot_20161212_131856.png (192 KB) Giovanni Manghi, 2016-12-12 05:17 AM

Screenshot_20161212_131355.png (244 KB) Giovanni Manghi, 2016-12-12 05:17 AM

Input_files.zip - Test input layers A and B (3.25 KB) Amy Taylor, 2017-02-20 02:13 AM

QGIS_union.zip - Resulting union (2.31 KB) Amy Taylor, 2017-02-20 02:13 AM

input_layers.PNG - screenshot input layers (30.7 KB) Amy Taylor, 2017-02-20 02:13 AM

circ_Y_union_feature.PNG - screenshot expected circ Y union shape (53.8 KB) Amy Taylor, 2017-02-20 02:13 AM

circ_Y_duplicate_feature_in_union.PNG - screenshot duplicated feature in union (54.3 KB) Amy Taylor, 2017-02-20 02:15 AM


Related issues

Related to QGIS Application - Bug report #9297: ftools: warn users in clear, meaningful, visible way when... Closed 2014-01-06
Related to QGIS Application - Bug report #8456: Union tool produces wrong result Closed 2013-08-12

History

#1 Updated by Giovanni Manghi almost 3 years ago

There are different issues here, both in QGIS (and that is very unfortunate) and in your data.

The Union between your 'dissolved.shp' & 'wards.shp'fails because the "wards" layer has a lot of issue, both from a geometry point of view but also from a topology one, in fact the polygons that you probably expected to be adjacent are not, many of them have gaps, that at the end of the operation you see as "artifacts". If you give a pass trough Processing/v.clean using also the snapping parameter then the union works well.

On the other hand the union/intersection of "areaA" and "areaB" does not works because in one of the two layers you have geometry errors (auto intersections and duplicate nodes). This errors are reported in the QGIS log after the operation runs, but of course this is NOT enough for the average user, a proper warning must be thrown. This is the error

2016-12-12T07:36:28    2    GEOS geoprocessing error: One or more input features have invalid geometry.
2016-12-12T07:36:28    0    Feature geometry error: One or more output features ignored due to invalid geometry.
2016-12-12T07:36:28    2    GEOS geoprocessing error: One or more input features have invalid geometry.
2016-12-12T07:36:28    0    Feature geometry error: One or more output features ignored due to invalid geometry.

Moreover while the intersection operation seems to work ok (after fixing those issues) the union one creates an output that smells rotten fish, in fact it seems to create duplicate geometries (see attached images).

See also: #9297

I'm tagging as SEVERE all this geoprocessing issues because they MUST be fixed once for all.

#2 Updated by Amy Taylor over 2 years ago

Re the union, I've been having issues with unexpected duplicated polygons in the results.

Attached are two input shapefiles and the result of a QGIS union between these.
The resulting union has one duplicated polygon which I wouldn't expect to be there: FID 3 (circ Y) from Test_input_B is duplicated in it's entirety as FID 9 in the union output shapefile.
The input, expected and duplicated features are shown in the attached images.

#3 Updated by Amy Taylor over 2 years ago

Sorry, forgot to attach last image, screenshot of duplicated feature attached.

#4 Updated by Giovanni Manghi over 2 years ago

  • Target version set to Version 2.18
  • Affected QGIS version changed from 2.18.0 to 2.18.4

#5 Updated by Giovanni Manghi over 2 years ago

  • Regression? set to Yes

#6 Updated by Giovanni Manghi over 2 years ago

  • Priority changed from Severe/Regression to High

#7 Updated by Giovanni Manghi over 2 years ago

  • Easy fix? set to No

#8 Updated by Giovanni Manghi over 2 years ago

  • Description updated (diff)

see also #11986 for more intersection errors!

#9 Updated by Giovanni Manghi over 2 years ago

  • Subject changed from Union & Intersection results are Incorrect to Union & Intersection results are Incorrect or borken

In 2.18.7 the union operation with the provided dataset, that usually returned a wrong result, now is BROKEN

2017-05-12T19:08:56 2 Uncaught error while executing algorithm
Traceback (most recent call last):
File "C:/OSGEO4~1/apps/qgis/./python/plugins\processing\core\GeoAlgorithm.py", line 203, in execute
self.processAlgorithm(progress)
File "C:/OSGEO4~1/apps/qgis/./python/plugins\processing\algs\qgis\Union.py", line 164, in processAlgorithm
if diff_geom.isGeosEmpty() or not diff_geom.isGeosValid():
AttributeError: 'NoneType' object has no attribute 'isGeosEmpty'

#10 Updated by Giovanni Manghi over 2 years ago

  • Subject changed from Union & Intersection results are Incorrect or borken to Union & Intersection results are Incorrect or broken

#11 Updated by Giovanni Manghi over 2 years ago

see also #8456

#12 Updated by Giovanni Manghi over 2 years ago

  • Affected QGIS version changed from 2.18.4 to 2.18.7

#13 Updated by Alexander Bruy over 2 years ago

Python error should be fixed in bc4e70a0e23af98ab9fa416856b5a44b007b23f2

#14 Updated by Giovanni Manghi over 2 years ago

  • Subject changed from Union & Intersection results are Incorrect or broken to Union & Intersection results are Incorrect

#15 Updated by Sandro Santilli over 2 years ago

Was Redmine plugin automatically showing commits referencing tickets lost ?
I guess Alexander meant bc4e70a0e23af98ab9fa416856b5a44b007b23f2 (with commit: prefix it turns to a link)

#16 Updated by Giovanni Manghi over 2 years ago

Sandro Santilli wrote:

Was Redmine plugin automatically showing commits referencing tickets lost ?
I guess Alexander meant bc4e70a0e23af98ab9fa416856b5a44b007b23f2 (with commit: prefix it turns to a link)

Jurgen and Richard updated the platform, maybe something is still not working 100%.

#17 Updated by Sandro Santilli over 2 years ago

  • Assignee set to Sandro Santilli

I'm looking at an example sent by Giovanni Manghi, will report here what I see.

#18 Updated by Sandro Santilli over 2 years ago

  • Status changed from Open to Closed
  • Resolution set to worksforme

The example sent to me by Giovanni, with the instructions provided, results in an error sent to the user about the input having invalid geometries. It seems to be the correct behavior with regard to this ticket (which is about incorrect results), so this is still in a "cannot reproduce" state for me. This is as of fd7b52d5b8e668bd5db43f81eecc60818ff490ab in release-2_18 branch (which is what "Affected QGIS version" references).

I'm closing again. Please consider filing a new ticket if it helps reducing the confusion between what was fixed and what not (as this one seems to be about many different cases)

#19 Updated by Giovanni Manghi over 2 years ago

  • Status changed from Closed to Open
  • Resolution deleted (worksforme)

#20 Updated by Giovanni Manghi over 2 years ago

Sandro Santilli wrote:

The example sent to me by Giovanni, with the instructions provided, results in an error sent to the user about the input having invalid geometries. It seems to be the correct behavior with regard to this ticket (which is about incorrect results), so this is still in a "cannot reproduce" state for me. This is as of fd7b52d5b8e668bd5db43f81eecc60818ff490ab in release-2_18 branch (which is what "Affected QGIS version" references).

I'm closing again. Please consider filing a new ticket if it helps reducing the confusion between what was fixed and what not (as this one seems to be about many different cases)

do not close important tickets that are all BUT fixed/solved, thanks.

#21 Updated by Sandro Santilli over 2 years ago

  • Assignee deleted (Sandro Santilli)

#22 Updated by Sandro Santilli over 2 years ago

Giovanni for tickets to be tractable there's a need of a recipe to reproduce the problem. This ticket is very vague in what it is saying. It's about wrong results, in its original report. Then it changed to "Wrong or Broken" later. Then was closed by a commit.

At this moment it isn't clear at all what this issue is about.
I think a proper issue must have:

1. Instructions for steps to take
2. Description of what happens by taking those steps
3. Description of what would instead be the expected behavior

#23 Updated by Giovanni Manghi over 2 years ago

Sandro Santilli wrote:

Giovanni for tickets to be tractable there's a need of a recipe to reproduce the problem. This ticket is very vague in what it is saying. It's about wrong results, in its original report. Then it changed to "Wrong or Broken" later. Then was closed by a commit.

At this moment it isn't clear at all what this issue is about.
I think a proper issue must have:

1. Instructions for steps to take
2. Description of what happens by taking those steps
3. Description of what would instead be the expected behavior

I already wrote to you the EXACT steps to replicate: take "sample1.zip", open areaA and areaB shapes. Do the intersection > wrong result. How difficult is that?

#24 Updated by Giovanni Manghi over 2 years ago

3. Description of what would instead be the expected behavior

do you mean seriously that you do not know how it should look like the result of the intersection between 3/4 simple test polygons? come on...

#25 Updated by Sandro Santilli over 2 years ago

No, I meant that I don't get a WRONG result, but rather I get a FAILURE telling me (via GUI) that the operation cannot be completed. Here's a screencast: http://strk.kbt.io/tmp/vokoscreen-2017-05-16_12-37-05.mkv
That's what happens on Linux with QGIS 2.8.17+57 commits in the release-2_18 branch, and is the reason why I closed this ticket.

#26 Updated by Giovanni Manghi about 2 years ago

  • Regression? changed from Yes to No
  • Affected QGIS version changed from 2.18.7 to 2.18.11

#27 Updated by Giovanni Manghi about 2 years ago

  • Crashes QGIS or corrupts data changed from No to Yes

#28 Updated by Giovanni Manghi about 2 years ago

  • Status changed from Open to Closed
  • Resolution set to duplicate

replaced by #17131

Also available in: Atom PDF