Skip to content

Commit 6f47b25

Browse files
nirvnnirvn
authoredMar 2, 2018
[FEATURE][processing] one-to-many join support to the join attribute table alg. (#6499)
·
ltr-3_40final-3_2_0
1 parent 2238b42 commit 6f47b25

File tree

6 files changed

+207
-10
lines changed

6 files changed

+207
-10
lines changed
 

‎python/plugins/processing/algs/qgis/SpatialJoin.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,8 +87,8 @@ def initAlgorithm(self, config=None):
8787
'crosses': 'crosses'}
8888

8989
self.methods = [
90-
self.tr('Create separate feature for each located feature'),
91-
self.tr('Take attributes of the first located feature only')
90+
self.tr('Create separate feature for each located feature (one-to-one)'),
91+
self.tr('Take attributes of the first located feature only (one-to-many)')
9292
]
9393

9494
self.addParameter(QgsProcessingParameterFeatureSource(self.INPUT,

‎python/plugins/processing/tests/testdata/expected/join_attribute_table_all_match.gfs

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
<GMLFeatureClassList>
2+
<GMLFeatureClass>
3+
<Name>join_attribute_table</Name>
4+
<ElementPath>join_attribute_table</ElementPath>
5+
<GeometryType>1</GeometryType>
6+
<SRSName>EPSG:4326</SRSName>
7+
<DatasetSpecificInfo>
8+
<FeatureCount>10</FeatureCount>
9+
<ExtentXMin>0.00000</ExtentXMin>
10+
<ExtentXMax>8.00000</ExtentXMax>
11+
<ExtentYMin>-5.00000</ExtentYMin>
12+
<ExtentYMax>3.00000</ExtentYMax>
13+
</DatasetSpecificInfo>
14+
<PropertyDefn>
15+
<Name>id</Name>
16+
<ElementPath>id</ElementPath>
17+
<Type>Integer</Type>
18+
</PropertyDefn>
19+
<PropertyDefn>
20+
<Name>id2</Name>
21+
<ElementPath>id2</ElementPath>
22+
<Type>Integer</Type>
23+
</PropertyDefn>
24+
<PropertyDefn>
25+
<Name>id_2</Name>
26+
<ElementPath>id_2</ElementPath>
27+
<Type>Integer</Type>
28+
</PropertyDefn>
29+
<PropertyDefn>
30+
<Name>NUM_A</Name>
31+
<ElementPath>NUM_A</ElementPath>
32+
<Type>Real</Type>
33+
</PropertyDefn>
34+
<PropertyDefn>
35+
<Name>ST_A</Name>
36+
<ElementPath>ST_A</ElementPath>
37+
<Type>String</Type>
38+
<Width>8</Width>
39+
</PropertyDefn>
40+
</GMLFeatureClass>
41+
</GMLFeatureClassList>

‎python/plugins/processing/tests/testdata/expected/join_attribute_table_all_match.gml

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
<?xml version="1.0" encoding="utf-8" ?>
2+
<ogr:FeatureCollection
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation=""
5+
xmlns:ogr="http://ogr.maptools.org/"
6+
xmlns:gml="http://www.opengis.net/gml">
7+
<gml:boundedBy>
8+
<gml:Box>
9+
<gml:coord><gml:X>0</gml:X><gml:Y>-5</gml:Y></gml:coord>
10+
<gml:coord><gml:X>8</gml:X><gml:Y>3</gml:Y></gml:coord>
11+
</gml:Box>
12+
</gml:boundedBy>
13+
14+
<gml:featureMember>
15+
<ogr:join_attribute_table fid="points.0">
16+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>1,1</gml:coordinates></gml:Point></ogr:geometryProperty>
17+
<ogr:id>1</ogr:id>
18+
<ogr:id2>2</ogr:id2>
19+
<ogr:id_2>1</ogr:id_2>
20+
<ogr:NUM_A>1.100000</ogr:NUM_A>
21+
<ogr:ST_A>string a</ogr:ST_A>
22+
</ogr:join_attribute_table>
23+
</gml:featureMember>
24+
<gml:featureMember>
25+
<ogr:join_attribute_table fid="points.1">
26+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>3,3</gml:coordinates></gml:Point></ogr:geometryProperty>
27+
<ogr:id>2</ogr:id>
28+
<ogr:id2>1</ogr:id2>
29+
<ogr:id_2>2</ogr:id_2>
30+
<ogr:NUM_A>2.400000</ogr:NUM_A>
31+
<ogr:ST_A>string c</ogr:ST_A>
32+
</ogr:join_attribute_table>
33+
</gml:featureMember>
34+
<gml:featureMember>
35+
<ogr:join_attribute_table fid="points.1">
36+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>3,3</gml:coordinates></gml:Point></ogr:geometryProperty>
37+
<ogr:id>2</ogr:id>
38+
<ogr:id2>1</ogr:id2>
39+
<ogr:id_2>2</ogr:id_2>
40+
<ogr:NUM_A>2.200000</ogr:NUM_A>
41+
<ogr:ST_A>string a</ogr:ST_A>
42+
</ogr:join_attribute_table>
43+
</gml:featureMember>
44+
<gml:featureMember>
45+
<ogr:join_attribute_table fid="points.2">
46+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>2,2</gml:coordinates></gml:Point></ogr:geometryProperty>
47+
<ogr:id>3</ogr:id>
48+
<ogr:id2>0</ogr:id2>
49+
<ogr:id_2>3</ogr:id_2>
50+
<ogr:NUM_A>3.300000</ogr:NUM_A>
51+
<ogr:ST_A>string a</ogr:ST_A>
52+
</ogr:join_attribute_table>
53+
</gml:featureMember>
54+
<gml:featureMember>
55+
<ogr:join_attribute_table fid="points.3">
56+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>5,2</gml:coordinates></gml:Point></ogr:geometryProperty>
57+
<ogr:id>4</ogr:id>
58+
<ogr:id2>2</ogr:id2>
59+
<ogr:id_2>4</ogr:id_2>
60+
<ogr:NUM_A>4.400000</ogr:NUM_A>
61+
<ogr:ST_A>string b</ogr:ST_A>
62+
</ogr:join_attribute_table>
63+
</gml:featureMember>
64+
<gml:featureMember>
65+
<ogr:join_attribute_table fid="points.4">
66+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>4,1</gml:coordinates></gml:Point></ogr:geometryProperty>
67+
<ogr:id>5</ogr:id>
68+
<ogr:id2>1</ogr:id2>
69+
<ogr:id_2>5</ogr:id_2>
70+
<ogr:NUM_A>5.500000</ogr:NUM_A>
71+
<ogr:ST_A>string b</ogr:ST_A>
72+
</ogr:join_attribute_table>
73+
</gml:featureMember>
74+
<gml:featureMember>
75+
<ogr:join_attribute_table fid="points.5">
76+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>0,-5</gml:coordinates></gml:Point></ogr:geometryProperty>
77+
<ogr:id>6</ogr:id>
78+
<ogr:id2>0</ogr:id2>
79+
<ogr:id_2>6</ogr:id_2>
80+
<ogr:NUM_A>6.600000</ogr:NUM_A>
81+
<ogr:ST_A>string b</ogr:ST_A>
82+
</ogr:join_attribute_table>
83+
</gml:featureMember>
84+
<gml:featureMember>
85+
<ogr:join_attribute_table fid="points.6">
86+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>8,-1</gml:coordinates></gml:Point></ogr:geometryProperty>
87+
<ogr:id>7</ogr:id>
88+
<ogr:id2>0</ogr:id2>
89+
<ogr:id_2>7</ogr:id_2>
90+
<ogr:NUM_A>7.700000</ogr:NUM_A>
91+
<ogr:ST_A>string b</ogr:ST_A>
92+
</ogr:join_attribute_table>
93+
</gml:featureMember>
94+
<gml:featureMember>
95+
<ogr:join_attribute_table fid="points.7">
96+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>7,-1</gml:coordinates></gml:Point></ogr:geometryProperty>
97+
<ogr:id>8</ogr:id>
98+
<ogr:id2>0</ogr:id2>
99+
<ogr:id_2>8</ogr:id_2>
100+
<ogr:NUM_A>8.800000</ogr:NUM_A>
101+
<ogr:ST_A>string b</ogr:ST_A>
102+
</ogr:join_attribute_table>
103+
</gml:featureMember>
104+
<gml:featureMember>
105+
<ogr:join_attribute_table fid="points.8">
106+
<ogr:geometryProperty><gml:Point srsName="EPSG:4326"><gml:coordinates>0,-1</gml:coordinates></gml:Point></ogr:geometryProperty>
107+
<ogr:id>9</ogr:id>
108+
<ogr:id2>0</ogr:id2>
109+
</ogr:join_attribute_table>
110+
</gml:featureMember>
111+
</ogr:FeatureCollection>

‎python/plugins/processing/tests/testdata/qgis_algorithm_tests.yaml

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2772,8 +2772,9 @@ tests:
27722772
type: vector
27732773

27742774
- algorithm: native:joinattributestable
2775-
name: join the attribute table by common field
2775+
name: join the attribute table by common field (one-to-one)
27762776
params:
2777+
METHOD: 0
27772778
INPUT:
27782779
name: points.gml
27792780
type: vector
@@ -2787,9 +2788,27 @@ tests:
27872788
name: expected/join_attribute_table.gml
27882789
type: vector
27892790

2791+
- algorithm: native:joinattributestable
2792+
name: join the attribute table by common field (one-to-many)
2793+
params:
2794+
METHOD: 1
2795+
INPUT:
2796+
name: points.gml
2797+
type: vector
2798+
INPUT_2:
2799+
name: table.dbf
2800+
type: table
2801+
FIELD: id
2802+
FIELD_2: ID
2803+
results:
2804+
OUTPUT:
2805+
name: expected/join_attribute_table_all_match.gml
2806+
type: vector
2807+
27902808
- algorithm: native:joinattributestable
27912809
name: Join attributes table with subset of fields
27922810
params:
2811+
METHOD: 0
27932812
FIELD: id
27942813
FIELDS_TO_COPY:
27952814
- NUM_A

‎python/plugins/processing/tests/testdata/table.dbf

40 Bytes
Binary file not shown.

‎src/analysis/processing/qgsalgorithmjoinbyattribute.cpp

Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,10 @@ QString QgsJoinByAttributeAlgorithm::groupId() const
4646

4747
void QgsJoinByAttributeAlgorithm::initAlgorithm( const QVariantMap & )
4848
{
49+
QStringList methods;
50+
methods << QObject::tr( "Take attributes of the first matching feature only (one-to-one)" )
51+
<< QObject::tr( "Create separate feature for each matching feature (one-to-many)" );
52+
4953
addParameter( new QgsProcessingParameterFeatureSource( QStringLiteral( "INPUT" ),
5054
QObject::tr( "Input layer" ), QList< int>() << QgsProcessing::TypeVector ) );
5155
addParameter( new QgsProcessingParameterField( QStringLiteral( "FIELD" ),
@@ -61,6 +65,11 @@ void QgsJoinByAttributeAlgorithm::initAlgorithm( const QVariantMap & )
6165
QVariant(), QStringLiteral( "INPUT_2" ), QgsProcessingParameterField::Any,
6266
true, true ) );
6367

68+
addParameter( new QgsProcessingParameterEnum(
69+
QStringLiteral( "METHOD" ),
70+
QObject::tr( "Join type" ),
71+
methods, false, 0 ) );
72+
6473
addParameter( new QgsProcessingParameterFeatureSink( QStringLiteral( "OUTPUT" ), QObject::tr( "Joined layer" ) ) );
6574
}
6675

@@ -69,7 +78,8 @@ QString QgsJoinByAttributeAlgorithm::shortHelpString() const
6978
return QObject::tr( "This algorithm takes an input vector layer and creates a new vector layer that is an extended version of the "
7079
"input one, with additional attributes in its attribute table.\n\n"
7180
"The additional attributes and their values are taken from a second vector layer. An attribute is selected "
72-
"in each of them to define the join criteria." );
81+
"in each of them to define the join criteria.\n\n"
82+
"The algorithm will output one feature per matching row(s) from the second vector layer." );
7383
}
7484

7585
QgsJoinByAttributeAlgorithm *QgsJoinByAttributeAlgorithm::createInstance() const
@@ -79,6 +89,8 @@ QgsJoinByAttributeAlgorithm *QgsJoinByAttributeAlgorithm::createInstance() const
7989

8090
QVariantMap QgsJoinByAttributeAlgorithm::processAlgorithm( const QVariantMap &parameters, QgsProcessingContext &context, QgsProcessingFeedback *feedback )
8191
{
92+
int joinMethod = parameterAsEnum( parameters, QStringLiteral( "METHOD" ), context );
93+
8294
std::unique_ptr< QgsFeatureSource > input( parameterAsSource( parameters, QStringLiteral( "INPUT" ), context ) );
8395
std::unique_ptr< QgsFeatureSource > input2( parameterAsSource( parameters, QStringLiteral( "INPUT_2" ), context ) );
8496
if ( !input || !input2 )
@@ -129,7 +141,7 @@ QVariantMap QgsJoinByAttributeAlgorithm::processAlgorithm( const QVariantMap &pa
129141

130142

131143
// cache attributes of input2
132-
QHash< QVariant, QgsAttributes > input2AttributeCache;
144+
QMultiHash< QVariant, QgsAttributes > input2AttributeCache;
133145
QgsFeatureIterator features = input2->getFeatures( QgsFeatureRequest().setFlags( QgsFeatureRequest::NoGeometry ).setSubsetOfAttributes( fields2Fetch ) );
134146
double step = input2->featureCount() > 0 ? 50.0 / input2->featureCount() : 1;
135147
int i = 0;
@@ -144,7 +156,7 @@ QVariantMap QgsJoinByAttributeAlgorithm::processAlgorithm( const QVariantMap &pa
144156

145157
feedback->setProgress( i * step );
146158

147-
if ( input2AttributeCache.contains( feat.attribute( joinField2Index ) ) )
159+
if ( joinMethod == 0 && input2AttributeCache.contains( feat.attribute( joinField2Index ) ) )
148160
continue;
149161

150162
// only keep selected attributes
@@ -173,10 +185,24 @@ QVariantMap QgsJoinByAttributeAlgorithm::processAlgorithm( const QVariantMap &pa
173185

174186
feedback->setProgress( 50 + i * step );
175187

176-
QgsAttributes attrs = feat.attributes();
177-
attrs.append( input2AttributeCache.value( feat.attribute( joinField1Index ) ) );
178-
feat.setAttributes( attrs );
179-
sink->addFeature( feat, QgsFeatureSink::FastInsert );
188+
if ( input2AttributeCache.count( feat.attribute( joinField1Index ) ) > 0 )
189+
{
190+
QgsAttributes attrs = feat.attributes();
191+
192+
QList< QgsAttributes > attributes = input2AttributeCache.values( feat.attribute( joinField1Index ) );
193+
QList< QgsAttributes >::iterator attrsIt = attributes.begin();
194+
for ( ; attrsIt != attributes.end(); ++attrsIt )
195+
{
196+
QgsAttributes newAttrs = attrs;
197+
newAttrs.append( *attrsIt );
198+
feat.setAttributes( newAttrs );
199+
sink->addFeature( feat, QgsFeatureSink::FastInsert );
200+
}
201+
}
202+
else
203+
{
204+
sink->addFeature( feat, QgsFeatureSink::FastInsert );
205+
}
180206
}
181207

182208
QVariantMap outputs;

4 commit comments

Comments
 (4)

anitagraser commented on Mar 2, 2018

@anitagraser
anitagraser
on Mar 2, 2018
Member

Nice feature! Seems like the definitions of one-to-one and one-to-many are inconsistent though:

SpatialJoin.py

self.tr('Create separate feature for each located feature (one-to-one)'),
self.tr('Take attributes of the first located feature only (one-to-many)')

qgsalgorithmjoinbyattribute.cpp

QObject::tr( "Take attributes of the first matching feature only (one-to-one)" )
QObject::tr( "Create separate feature for each matching feature (one-to-many)" );

nirvn commented on Mar 2, 2018

@nirvn
nirvn
on Mar 2, 2018
ContributorAuthor

@anitagraser , you mean the matching vs. located? I think it needs to be different here, since we're speaking of spatial join vs attribute join. Or am I missing your point? Glad I'm not the only one liking this one 😄 give it a try alongside the aggregate algorithm as part of a model, that made my day.

anitagraser commented on Mar 2, 2018

@anitagraser
anitagraser
on Mar 2, 2018
Member

I meant that in SpatialJoin, "one-to-one" = "create feature for each located/matching" while in JoinByAttribute "one-to-one" is "first matching feature only".

nirvn commented on Mar 2, 2018

@nirvn
nirvn
on Mar 2, 2018
ContributorAuthor

@anitagraser , oh, doh, I see it now.

Please sign in to comment.