Skip to content

Commit

Permalink
[processing] Port delete fields algorithm to c++
Browse files Browse the repository at this point in the history 
1:1 port for speed/compile time safety, no user-facing changes

Also add tests
  • Loading branch information
@nyalldawson
nyalldawson committed Nov 23, 2020
1 parent 6a86642 commit b07a58c
Show file tree
Hide file tree
Showing 14 changed files with 526 additions and 110 deletions.
3 changes: 0 additions & 3 deletions python/plugins/processing/algs/help/qgis.yaml
View file
Expand Up @@ -62,9 +62,6 @@ qgis:definecurrentprojection: >
qgis:delaunaytriangulation: >
This algorithm creates a polygon layer with the delaunay triangulation corresponding to a points layer.

qgis:deletecolumn: >
This algorithm takes a vector layer and generates a new one that has the exact same content but without the selected columns.

qgis:deleteholes: >
This algorithm takes a polygon layer and removes holes in polygons. It creates a new vector layer in which polygons with holes have been replaced by polygons with only their external ring. Attributes are not modified.

Expand Down
105 changes: 0 additions & 105 deletions python/plugins/processing/algs/qgis/DeleteColumn.py
View file

This file was deleted.

2 changes: 0 additions & 2 deletions python/plugins/processing/algs/qgis/QgisAlgorithmProvider.py
View file
Expand Up @@ -37,7 +37,6 @@
from .ConcaveHull import ConcaveHull
from .DefineProjection import DefineProjection
from .Delaunay import Delaunay
from .DeleteColumn import DeleteColumn
from .EliminateSelection import EliminateSelection
from .ExecuteSQL import ExecuteSQL
from .ExportGeometryInfo import ExportGeometryInfo
Expand Down Expand Up @@ -107,7 +106,6 @@ def getAlgs(self):
ConcaveHull(),
DefineProjection(),
Delaunay(),
DeleteColumn(),
EliminateSelection(),
ExecuteSQL(),
ExportGeometryInfo(),
Expand Down
49 changes: 49 additions & 0 deletions python/plugins/processing/tests/testdata/expected/delete_columns_three.gml
View file
@@ -0,0 +1,49 @@
<?xml version="1.0" encoding="utf-8" ?>
<ogr:FeatureCollection
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://ogr.maptools.org/ delete_columns_three.xsd"
xmlns:ogr="http://ogr.maptools.org/"
xmlns:gml="http://www.opengis.net/gml">
<gml:boundedBy>
<gml:Box>
<gml:coord><gml:X>-1</gml:X><gml:Y>-3</gml:Y></gml:coord>
<gml:coord><gml:X>10</gml:X><gml:Y>6</gml:Y></gml:coord>
</gml:Box>
</gml:boundedBy>

<gml:featureMember>
<ogr:delete_columns_three fid="delete_columns_three.0">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>-1,-1 -1,3 3,3 3,2 2,2 2,-1 -1,-1</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>aaaaa</ogr:name>
</ogr:delete_columns_three>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_three fid="delete_columns_three.1">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>5,5 6,4 4,4 5,5</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>Aaaaa</ogr:name>
</ogr:delete_columns_three>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_three fid="delete_columns_three.2">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>2,5 2,6 3,6 3,5 2,5</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>bbaaa</ogr:name>
</ogr:delete_columns_three>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_three fid="delete_columns_three.3">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>6,1 10,1 10,-3 6,-3 6,1</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs><gml:innerBoundaryIs><gml:LinearRing><gml:coordinates>7,0 7,-2 9,-2 9,0 7,0</gml:coordinates></gml:LinearRing></gml:innerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>ASDF</ogr:name>
</ogr:delete_columns_three>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_three fid="delete_columns_three.4">
<ogr:name xsi:nil="true"/>
</ogr:delete_columns_three>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_three fid="delete_columns_three.5">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>3,2 6,1 6,-3 2,-1 2,2 3,2</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>elim</ogr:name>
</ogr:delete_columns_three>
</gml:featureMember>
</ogr:FeatureCollection>
30 changes: 30 additions & 0 deletions python/plugins/processing/tests/testdata/expected/delete_columns_three.xsd
View file
@@ -0,0 +1,30 @@
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema targetNamespace="http://ogr.maptools.org/" xmlns:ogr="http://ogr.maptools.org/" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:gml="http://www.opengis.net/gml" elementFormDefault="qualified" version="1.0">
<xs:import namespace="http://www.opengis.net/gml" schemaLocation="http://schemas.opengis.net/gml/2.1.2/feature.xsd"/>
<xs:element name="FeatureCollection" type="ogr:FeatureCollectionType" substitutionGroup="gml:_FeatureCollection"/>
<xs:complexType name="FeatureCollectionType">
<xs:complexContent>
<xs:extension base="gml:AbstractFeatureCollectionType">
<xs:attribute name="lockId" type="xs:string" use="optional"/>
<xs:attribute name="scope" type="xs:string" use="optional"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<xs:element name="delete_columns_three" type="ogr:delete_columns_three_Type" substitutionGroup="gml:_Feature"/>
<xs:complexType name="delete_columns_three_Type">
<xs:complexContent>
<xs:extension base="gml:AbstractFeatureType">
<xs:sequence>
<xs:element name="geometryProperty" type="gml:PolygonPropertyType" nillable="true" minOccurs="0" maxOccurs="1"/>
<xs:element name="name" nillable="true" minOccurs="0" maxOccurs="1">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:maxLength value="5"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:schema>
49 changes: 49 additions & 0 deletions python/plugins/processing/tests/testdata/expected/delete_columns_two.gml
View file
@@ -0,0 +1,49 @@
<?xml version="1.0" encoding="utf-8" ?>
<ogr:FeatureCollection
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://ogr.maptools.org/ delete_columns_two.xsd"
xmlns:ogr="http://ogr.maptools.org/"
xmlns:gml="http://www.opengis.net/gml">
<gml:boundedBy>
<gml:Box>
<gml:coord><gml:X>-1</gml:X><gml:Y>-3</gml:Y></gml:coord>
<gml:coord><gml:X>10</gml:X><gml:Y>6</gml:Y></gml:coord>
</gml:Box>
</gml:boundedBy>

<gml:featureMember>
<ogr:delete_columns_two fid="polys.0">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>-1,-1 -1,3 3,3 3,2 2,2 2,-1 -1,-1</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:intval>33</ogr:intval>
</ogr:delete_columns_two>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_two fid="polys.1">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>5,5 6,4 4,4 5,5</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:intval>-33</ogr:intval>
</ogr:delete_columns_two>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_two fid="polys.2">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>2,5 2,6 3,6 3,5 2,5</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:intval xsi:nil="true"/>
</ogr:delete_columns_two>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_two fid="polys.3">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>6,1 10,1 10,-3 6,-3 6,1</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs><gml:innerBoundaryIs><gml:LinearRing><gml:coordinates>7,0 7,-2 9,-2 9,0 7,0</gml:coordinates></gml:LinearRing></gml:innerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:intval>0</ogr:intval>
</ogr:delete_columns_two>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_two fid="polys.4">
<ogr:intval>120</ogr:intval>
</ogr:delete_columns_two>
</gml:featureMember>
<gml:featureMember>
<ogr:delete_columns_two fid="polys.5">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>3,2 6,1 6,-3 2,-1 2,2 3,2</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:intval>2</ogr:intval>
</ogr:delete_columns_two>
</gml:featureMember>
</ogr:FeatureCollection>
30 changes: 30 additions & 0 deletions python/plugins/processing/tests/testdata/expected/delete_columns_two.xsd
View file
@@ -0,0 +1,30 @@
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema targetNamespace="http://ogr.maptools.org/" xmlns:ogr="http://ogr.maptools.org/" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:gml="http://www.opengis.net/gml" elementFormDefault="qualified" version="1.0">
<xs:import namespace="http://www.opengis.net/gml" schemaLocation="http://schemas.opengis.net/gml/2.1.2/feature.xsd"/>
<xs:element name="FeatureCollection" type="ogr:FeatureCollectionType" substitutionGroup="gml:_FeatureCollection"/>
<xs:complexType name="FeatureCollectionType">
<xs:complexContent>
<xs:extension base="gml:AbstractFeatureCollectionType">
<xs:attribute name="lockId" type="xs:string" use="optional"/>
<xs:attribute name="scope" type="xs:string" use="optional"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<xs:element name="delete_columns_two" type="ogr:delete_columns_two_Type" substitutionGroup="gml:_Feature"/>
<xs:complexType name="delete_columns_two_Type">
<xs:complexContent>
<xs:extension base="gml:AbstractFeatureType">
<xs:sequence>
<xs:element name="geometryProperty" type="gml:PolygonPropertyType" nillable="true" minOccurs="0" maxOccurs="1"/>
<xs:element name="intval" nillable="true" minOccurs="0" maxOccurs="1">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:totalDigits value="10"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:schema>
61 changes: 61 additions & 0 deletions python/plugins/processing/tests/testdata/expected/drop_fields_one.gml
View file
@@ -0,0 +1,61 @@
<?xml version="1.0" encoding="utf-8" ?>
<ogr:FeatureCollection
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://ogr.maptools.org/ drop_fields_one.xsd"
xmlns:ogr="http://ogr.maptools.org/"
xmlns:gml="http://www.opengis.net/gml">
<gml:boundedBy>
<gml:Box>
<gml:coord><gml:X>-1</gml:X><gml:Y>-3</gml:Y></gml:coord>
<gml:coord><gml:X>10</gml:X><gml:Y>6</gml:Y></gml:coord>
</gml:Box>
</gml:boundedBy>

<gml:featureMember>
<ogr:drop_fields_one fid="drop_fields_one.0">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>-1,-1 -1,3 3,3 3,2 2,2 2,-1 -1,-1</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>aaaaa</ogr:name>
<ogr:intval>33</ogr:intval>
<ogr:floatval>44.123456</ogr:floatval>
</ogr:drop_fields_one>
</gml:featureMember>
<gml:featureMember>
<ogr:drop_fields_one fid="drop_fields_one.1">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>5,5 6,4 4,4 5,5</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>Aaaaa</ogr:name>
<ogr:intval>-33</ogr:intval>
<ogr:floatval>0</ogr:floatval>
</ogr:drop_fields_one>
</gml:featureMember>
<gml:featureMember>
<ogr:drop_fields_one fid="drop_fields_one.2">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>2,5 2,6 3,6 3,5 2,5</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>bbaaa</ogr:name>
<ogr:intval xsi:nil="true"/>
<ogr:floatval>0.123</ogr:floatval>
</ogr:drop_fields_one>
</gml:featureMember>
<gml:featureMember>
<ogr:drop_fields_one fid="drop_fields_one.3">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>6,1 10,1 10,-3 6,-3 6,1</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs><gml:innerBoundaryIs><gml:LinearRing><gml:coordinates>7,0 7,-2 9,-2 9,0 7,0</gml:coordinates></gml:LinearRing></gml:innerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>ASDF</ogr:name>
<ogr:intval>0</ogr:intval>
<ogr:floatval xsi:nil="true"/>
</ogr:drop_fields_one>
</gml:featureMember>
<gml:featureMember>
<ogr:drop_fields_one fid="drop_fields_one.4">
<ogr:name xsi:nil="true"/>
<ogr:intval>120</ogr:intval>
<ogr:floatval>-100291.43213</ogr:floatval>
</ogr:drop_fields_one>
</gml:featureMember>
<gml:featureMember>
<ogr:drop_fields_one fid="drop_fields_one.5">
<ogr:geometryProperty><gml:Polygon srsName="EPSG:4326"><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates>3,2 6,1 6,-3 2,-1 2,2 3,2</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></ogr:geometryProperty>
<ogr:name>elim</ogr:name>
<ogr:intval>2</ogr:intval>
<ogr:floatval>3.33</ogr:floatval>
</ogr:drop_fields_one>
</gml:featureMember>
</ogr:FeatureCollection>
43 changes: 43 additions & 0 deletions python/plugins/processing/tests/testdata/expected/drop_fields_one.xsd
View file
@@ -0,0 +1,43 @@
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema targetNamespace="http://ogr.maptools.org/" xmlns:ogr="http://ogr.maptools.org/" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:gml="http://www.opengis.net/gml" elementFormDefault="qualified" version="1.0">
<xs:import namespace="http://www.opengis.net/gml" schemaLocation="http://schemas.opengis.net/gml/2.1.2/feature.xsd"/>
<xs:element name="FeatureCollection" type="ogr:FeatureCollectionType" substitutionGroup="gml:_FeatureCollection"/>
<xs:complexType name="FeatureCollectionType">
<xs:complexContent>
<xs:extension base="gml:AbstractFeatureCollectionType">
<xs:attribute name="lockId" type="xs:string" use="optional"/>
<xs:attribute name="scope" type="xs:string" use="optional"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<xs:element name="drop_fields_one" type="ogr:drop_fields_one_Type" substitutionGroup="gml:_Feature"/>
<xs:complexType name="drop_fields_one_Type">
<xs:complexContent>
<xs:extension base="gml:AbstractFeatureType">
<xs:sequence>
<xs:element name="geometryProperty" type="gml:PolygonPropertyType" nillable="true" minOccurs="0" maxOccurs="1"/>
<xs:element name="name" nillable="true" minOccurs="0" maxOccurs="1">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:maxLength value="5"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="intval" nillable="true" minOccurs="0" maxOccurs="1">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:totalDigits value="10"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="floatval" nillable="true" minOccurs="0" maxOccurs="1">
<xs:simpleType>
<xs:restriction base="xs:decimal">
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:schema>

3 comments on commit b07a58c

@PedroVenancio
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nyalldawson

I came to an issue about the change from qgis:deletecolumn to native:deletecolumn.

Basically, Models created in older versions (in this case, in QGIS 3.16) works fine in newer versions (QGIS 3.18 and master), but the opposite is not true. I've created a Model in QGIS 3.18 and I can not work with it 3.16

Cannot edit model: The model you are trying to run contains an algorithm that is not available: native:deletecolumn

This algorithm cannot be run :-( The model you are trying to run contains an algorithm that is not available: native:deletecolumn

So we do not have the backward compatibility.

The workaround I see is to change manually native:deletecolumn to qgis:deletecolumn. So sharing Models with others using LTR, becomes more difficult.

Do you think this workaround is the only way to overcome this, or there is a better option?

Thank you very much Nyall!

@nyalldawson
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PedroVenancio

So we do not have the backward compatibility.

That's normal -- QGIS projects/models/styles/etc have never been backward compatible. You need to create your files using the lowest version you want to support. So if you need a model which runs on 3.16 and above then you'll need to develop it using QGIS 3.16.

@PedroVenancio
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification @nyalldawson !

Please sign in to comment.