Bug report #4567

Ftools - Union returns Wrong Table of Attributes

Added by Enrico Fiore almost 8 years ago. Updated over 2 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Processing/QGIS
Affected QGIS version:master Regression?:
Operating System: Easy fix?:
Pull Request or Patch supplied:No Resolution:duplicate
Crashes QGIS or corrupts data:No Copied to github as #:14479

Description

Makig Union there is error in table attributes values aren't allocated correctly. Show attached files to understand the error.
Correlate at this error making multiple union there is loss of polygons create in precedent union so some intersections get lost

Union_table.zip - Tables before and after union (140 KB) Enrico Fiore, 2011-11-25 01:51 AM

vectorstest.zip - testunion = A union B (20.9 KB) alobo -, 2012-02-28 07:28 AM

Union_test_data.zip (4.43 KB) Alexandre Neto, 2012-03-20 07:14 AM

06.png (198 KB) Giovanni Manghi, 2012-03-20 09:00 AM

screenshot01.png - Screenshot 01, id's from 2 union layers in 1 id field (142 KB) Bernd Eversmann, 2012-09-07 04:53 AM

screenshot02.png - Screenshot 02, one feature gets id's from both union layers (114 KB) Bernd Eversmann, 2012-09-07 04:53 AM

uniontest_bernd.zip - zipped testdata (249 KB) Bernd Eversmann, 2012-09-07 08:57 AM

shapes_and_fake_polys.zip - zip file includes original shapefile and fake testing polygons (52.2 KB) Ivan Santiago, 2013-04-16 07:03 AM

57.png (239 KB) Giovanni Manghi, 2013-04-20 02:20 AM


Related issues

Related to QGIS Application - Bug report #7708: Union produces the wrong output (plus progress bar does n... Closed 2013-04-25

Associated revisions

Revision 0498c955
Added by Victor Axbom over 7 years ago

Fix Union tool to produce correct attribute table (fix #4567)

Revision d6b15196
Added by Victor Axbom over 7 years ago

Fix Union tool to produce correct attribute table (fix #4567)

History

#1 Updated by Giovanni Manghi almost 8 years ago

  • Assignee set to cfarmer -
  • Operating System deleted (Windows 7)

Confirmed also on linux.

#2 Updated by Giovanni Manghi almost 8 years ago

A sample dataset can be found here

http://geo.regione.emilia-romagna.it/gstatico/documenti/qgis/colli_bolognesi.zip

inside the archive you can find a shape "union_A_B_corretto" that shows the correct result produced with the old "geoprocessing" plugin, no more available in the repos.

Resuming:

if you have two polygons partially overlapping, with the following attributes

ID Name
1 one

and

ID Name
2 two

in the union A+B you should expect

ID_a Name_a ID_b Name_b
1 one 2 two
1 one 0 0
0 0 2 two

(or NULLs instead of "0")

instead you get

ID_a Name_a ID_b Name_b
1 one 2 two
1 one NULL NULL
1 one NULL NULL

#3 Updated by Giovanni Manghi almost 8 years ago

  • Subject changed from Ftools - Union Table error to Ftools - Union returns Wrong Table of Attributes

#4 Updated by Giovanni Manghi almost 8 years ago

  • Target version changed from Version 1.7.2 to Version 1.7.3

#5 Updated by Giovanni Manghi almost 8 years ago

  • Target version changed from Version 1.7.3 to Version 1.7.4

#6 Updated by Paolo Cavallini over 7 years ago

  • Crashes QGIS or corrupts data set to No
  • Priority changed from 6 to Normal
  • Affected QGIS version set to master

#7 Updated by Giovanni Manghi over 7 years ago

I would prefer to have a higher priority for this... the result of the operation is wrong, not acceptable from the user point of view.

#8 Updated by Victor Axbom over 7 years ago

Hi
This was a really annoying bug so I took a look at it and I might have found the solution. By changing "atMapB" to "atMap" at line 1170 in doGeoprocessing.py, the bug should be fixed. Line 1157 can then be removed. I hope someone with the knowledge howto patch this into master can proceed with it

#9 Updated by Victor Axbom over 7 years ago

Sorry, line 1237 is the correct

Vic - wrote:

Hi
This was a really annoying bug so I took a look at it and I might have found the solution. By changing "atMapB" to "atMap" at line 1170 in doGeoprocessing.py, the bug should be fixed. Line 1157 can then be removed. I hope someone with the knowledge howto patch this into master can proceed with it

#10 Updated by Giovanni Manghi over 7 years ago

Vic - wrote:

Sorry, line 1237 is the correct

Vic - wrote:

Hi
This was a really annoying bug so I took a look at it and I might have found the solution. By changing "atMapB" to "atMap" at line 1170 in doGeoprocessing.py, the bug should be fixed. Line 1157 can then be removed. I hope someone with the knowledge howto patch this into master can proceed with it

Hi, thanks for looking into this issue.

Can you provide a patch? I find many "atMapB" references in doGeoprocessing.py but none at the lines you say. By the way I'm looking into qgis master (development) code.

#11 Updated by Victor Axbom over 7 years ago

Thought this could be a good time to learn how github works so I made a fork on github and commit the changes. Is it just to make a pull request on the commit now?

Giovanni Manghi wrote:

Vic - wrote:

Sorry, line 1237 is the correct

Vic - wrote:

Hi
This was a really annoying bug so I took a look at it and I might have found the solution. By changing "atMapB" to "atMap" at line 1170 in doGeoprocessing.py, the bug should be fixed. Line 1157 can then be removed. I hope someone with the knowledge howto patch this into master can proceed with it

Hi, thanks for looking into this issue.

Can you provide a patch? I find many "atMapB" references in doGeoprocessing.py but none at the lines you say. By the way I'm looking into qgis master (development) code.

#12 Updated by Victor Axbom over 7 years ago

  • % Done changed from 0 to 100
  • Status changed from Open to Closed

#13 Updated by Alexander Bruy over 7 years ago

  • Resolution set to fixed

Patch applied in 0498c95546

#14 Updated by Giovanni Manghi over 7 years ago

  • Resolution deleted (fixed)
  • Status changed from Closed to Reopened

#15 Updated by alobo - over 7 years ago

I get an empty table using 1.7.4 under ubuntu 10.04 (from ubuntugis-unstable binaries).
See attached files.

#16 Updated by Alexandre Neto over 7 years ago

Is there any news about this problem?

I'm trying Union both in 1.7.4 and 1.9.90-Alpha QGIS code revision
f3b78ef, in a Windows XP computer.

Union is not working correctly, it unions the geometries but looses
some attributes in the process.

See the attached files. Has you can see in the union5.shp attributes the polygon2 remaining (the part that doesn't intersect the other) have incorrect attributes.

#17 Updated by Giovanni Manghi over 7 years ago

Can you please detail what should be the expected result with the attached sample data? thanks.

Alexandre Neto wrote:

Is there any news about this problem?

I'm trying Union both in 1.7.4 and 1.9.90-Alpha QGIS code revision
f3b78ef, in a Windows XP computer.

Union is not working correctly, it unions the geometries but looses
some attributes in the process.

See the attached files. Has you can see in the union5.shp attributes the polygon2 remaining (the part that doesn't intersect the other) have incorrect attributes.

#18 Updated by Giovanni Manghi over 7 years ago

Giovanni Manghi wrote:

Can you please detail what should be the expected result with the attached sample data? thanks.

see for example in #4567-2

#19 Updated by Giovanni Manghi over 7 years ago

  • File 06.png added
  • Status changed from Reopened to Feedback

Hi Alexandre,

the resulting table of attributes looks like the attached image (I'm using master on both linux and windows). It seems fine to me. Can you leave feedback? cheers.

Alexandre Neto wrote:

Is there any news about this problem?

I'm trying Union both in 1.7.4 and 1.9.90-Alpha QGIS code revision
f3b78ef, in a Windows XP computer.

Union is not working correctly, it unions the geometries but looses
some attributes in the process.

See the attached files. Has you can see in the union5.shp attributes the polygon2 remaining (the part that doesn't intersect the other) have incorrect attributes.

#20 Updated by Alexandre Neto over 7 years ago

I try to reinstall and update Qgis, both 1.7.4 and 1.9.90, and the problem presisted.

I uninstalled QGIS completly, erasing the OSGEO folder, the .qgis from user folder. Also delected windows registry. After that I installed QGIS again and everything seems to be working well. I guess my installation got some how corrupted, and the normal reinstallation\\update wasn't able to fix the problem.

Problem solved.

#21 Updated by Giovanni Manghi over 7 years ago

  • Status changed from Feedback to Closed
  • Resolution set to fixed

#22 Updated by alobo - over 7 years ago

  • Status changed from Closed to Reopened
  • Priority changed from Normal to 6
  • Target version changed from Version 1.7.4 to Version 1.8.0

Union should have an option to integrate those fields with the same name
in the new table (i.e., an option to do what mmqgis/Transefer/Merge layers does).
(Merge layers is an ambiguous term).
In other words, if vector layer dum1 has fields A,B,C and vector layer dum2 has fields
A,B,D, the new option should create a table with fields A,B,C,D
Note that in most cases, the user wants to make union of layers with the same fields in
their tables (i.e., both dum1 and dum2 have fields A,B,C and the result (with this option activated) should have
fields A,B,C as well (currently it is A,B,C,A1,B1,C1), which is rarely what the user wants/expects

Agus

#23 Updated by Giovanni Manghi over 7 years ago

  • Status changed from Reopened to Feedback

alobo - wrote:

Union should have an option to integrate those fields with the same name
in the new table (i.e., an option to do what mmqgis/Transefer/Merge layers does).
(Merge layers is an ambiguous term).
In other words, if vector layer dum1 has fields A,B,C and vector layer dum2 has fields
A,B,D, the new option should create a table with fields A,B,C,D
Note that in most cases, the user wants to make union of layers with the same fields in
their tables (i.e., both dum1 and dum2 have fields A,B,C and the result (with this option activated) should have
fields A,B,C as well (currently it is A,B,C,A1,B1,C1), which is rarely what the user wants/expects

Hi Agus,

so how the attributes values should be in the merged columns? concatenated? or the user should have the option to choose to keep the "A" column from a specific table?

In any case I see this request as a feature request, not as a bug, so I would open a separate ticket and re-close this one.

#24 Updated by Giovanni Manghi over 7 years ago

  • Priority changed from 6 to High

#25 Updated by Giovanni Manghi over 7 years ago

  • Priority changed from High to Normal

#26 Updated by Giovanni Manghi over 7 years ago

  • Status changed from Feedback to Closed

alobo - wrote:

Union should have an option to integrate those fields with the same name
in the new table (i.e., an option to do what mmqgis/Transefer/Merge layers does).
(Merge layers is an ambiguous term).
In other words, if vector layer dum1 has fields A,B,C and vector layer dum2 has fields
A,B,D, the new option should create a table with fields A,B,C,D
Note that in most cases, the user wants to make union of layers with the same fields in
their tables (i.e., both dum1 and dum2 have fields A,B,C and the result (with this option activated) should have
fields A,B,C as well (currently it is A,B,C,A1,B1,C1), which is rarely what the user wants/expects

Agus

Hi Agus, see my other comment. This should be filed as a feature request. Cheers.

#27 Updated by Bernd Eversmann about 7 years ago

Hi all,

the union command works fine now with small testcases, I tested with two polygons having one overlap: everything is ok.

But in a more complex scenario I run into the following problem:

1) I union two layers: buffered streams (multipart, id 9000) and buffered birdnests (id 1 and 2). Both ID fields are called "id".
2) After the union, all id values end up in the id column, the the second id field (id_1) remains empty. This way, you can't distinguish overlapping from non-overlapping features using ID's. See attached screenshot01.
3) When I convert the multipart streams layer to single part and assign different names to the id fields, things look ok, except for one overlapping feature which gets both id's. I have no clue why this happens (see screenshot02).

I have tested this on mac and windows (qgis 1.8.0). It looks to me as if the union command still has some issues, or am I doing something wrong?

Cheers, Bernd

#28 Updated by Salvatore Larosa about 7 years ago

Bernd Eversmann wrote:

2) After the union, all id values end up in the id column, the the second id field (id_1) remains empty. This way, you can't distinguish overlapping from non-overlapping features using ID's. See attached screenshot01.

3) When I convert the multipart streams layer to single part and assign different names to the id fields, things look ok, except for one overlapping feature which gets both id's. I have no clue why this happens (see screenshot02).

Hi Bernd,
On current master it seems ok here, both the IDs are not NULL for overlapped geometries!
would be interesting to test with your dataset so I can replicate the issue!
Can you share a small sample?

#29 Updated by Bernd Eversmann about 7 years ago

Hi Salvatore,
thanks for your quick reply. The test datasets are not big, please find them enclosed (the buffered streams as multi- and single part, and the buffered nests). I'm looking forward to hearing about your test results.

Cheers, Bernd

#30 Updated by Salvatore Larosa about 7 years ago

  • Status changed from Closed to Reopened
  • Assignee deleted (cfarmer -)
  • Target version changed from Version 1.8.0 to Version 2.0.0
  • Resolution deleted (fixed)

I can confirm!
The result expected of "streams_f_bufd" UNION "nests_f_buf_single" should be:

id    |    id_1
----------------
9000  |    1
9000  |    2
9000  |    NULL
1     |    NULL
2     |    NULL

#31 Updated by Enrico Fiore over 6 years ago

Hi,
Are there news about this bug? I confirm that in QGIS version 1.8.0 the bug remain.

I think that is better to indicate at the users that the tool is broken,so to prevent trouble with the elaborations.

#32 Updated by maning sambale over 6 years ago

I confirm this bug using Vector > Geoprocessing Tools > Union.
However, using the Union module in Sextante produced good result.

System: Ubuntu precise latest nightly build.

#33 Updated by Ivan Santiago over 6 years ago

QGIS continues to give overlapping polygons after a second union processing. I've tested it with fTools 0.6.2. As a curious note, neither the polygon overlapping function of sextante and saga produced acceptable results after the second running of "complete" intersection in saga plugin and union algorithm using sextante.

You can see this easily applying transparency to the resultant layer. In that way you can detect overlapping polygons in a single layer. That must not exist as a result of a union function.

Making "fake" simple polygons work fine. I made three fake polygons, very simple with some holes though. The third union process using fTools worked fine.

Enclosed are original shapefiles for anybody that would have interest in this important issue.

#34 Updated by Giovanni Manghi over 6 years ago

Ivan Santiago wrote:

QGIS continues to give overlapping polygons after a second union processing. I've tested it with fTools 0.6.2. As a curious note, neither the polygon overlapping function of sextante and saga produced acceptable results after the second running of "complete" intersection in saga plugin and union algorithm using sextante.

the tool is not wrong, in fact if you see in detail (see attached screenshot) the two polygons that seems wrong after the union are not identical in the input layers, there is a very small difference, something like 0.4 mm... but this makes them two non identical, overlapping polygons...

#35 Updated by Giovanni Manghi about 6 years ago

  • Status changed from Reopened to Closed
  • Resolution set to duplicate

merging with #7708

#36 Updated by Giovanni Manghi over 2 years ago

The "ftools" category is being removed from the tracker, changing the category of this ticket to "Processing/QGIS" to not leave the category orphaned.

Also available in: Atom PDF