Bug report #5066

(regression) QGIS 1.7.* misdetects EPSG::25884 datasets as EPSG::2100

Added by marisn - about 7 years ago. Updated over 6 years ago.

Status:Closed
Priority:High
Assignee:-
Category:Projection Support
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed
Crashes QGIS or corrupts data:No

Description

When adding a vector dataset (shapefile) with correct PRJ file listing it's CRS as ETRS89/TM Baltic93 (EPSG::25884), it gets set as a layer CRS a Greek grid (EPSG::2100). Reprojecting such data results in ~300m shift for all features. If user digitizes data without realizing that QGIS has misdeteced it's CRS, it results in storing of wrong data!

Workaround: when adding every vector dataset, user has to check if QGIS has detected CRS correctly and in the case of detection like Greek grid, to set correct CRS manually.

Tested versions: QGIS 1.7.2 and 1.7.4 on MS-Windows.
Current git master on Linux just detects same shapefile (created with QGIS) to have "generated" CRS and thus is reprojected correctly.

Attaching a sample shapefile and a project file where the wrong detected CRS is visible.

Baltic_93tm.zip - Sample shape and project file (4.23 KB) marisn -, 2012-02-22 06:51 AM

estonia_generalized.zip - Map of Estonia in EPSG:25884 (77.3 KB) marisn -, 2012-08-29 01:52 AM


Related issues

Related to QGIS Application - Bug report #4977: Wrong interpretation of Gauss-Krüger .prj files Closed 2012-02-10
Related to QGIS Application - Bug report #5598: CRS attribution algorithm for newly added layers broken i... Closed 2012-05-14

Associated revisions

Revision 2f135c1d
Added by Jürgen Fischer about 7 years ago

use GDAL_FIX_ESRI_WKT in QgsOgrProvider::crs (fixes #5066 with GDAL 1.9)

Revision c065736b
Added by Jürgen Fischer about 7 years ago

use GDAL_FIX_ESRI_WKT in QgsOgrProvider::crs (fixes #5066 with GDAL 1.9)

History

#1 Updated by marisn - about 7 years ago

  • Operating System set to Windows
  • Target version set to 35
  • Affected QGIS version changed from master to 1.7.4

#5066 could be a related issue.

#2 Updated by marisn - about 7 years ago

Just checked that 1.6.0 is not affected. Due this issue, we are keeping our GIS labs on 1.6.0 release, as EPSG:25884 is the most used coordinate system in our university.

#3 Updated by Giovanni Manghi about 7 years ago

  • Priority changed from High to 6
  • Subject changed from QGIS 1.7.* on Windows misdetects EPSG::25884 datasets as EPSG::2100 to (regression) QGIS 1.7.* on Windows misdetects EPSG::25884 datasets as EPSG::2100

tagging this as a regression

#4 Updated by marisn - about 7 years ago

  • Subject changed from (regression) QGIS 1.7.* on Windows misdetects EPSG::25884 datasets as EPSG::2100 to (regression) QGIS 1.7.* misdetects EPSG::25884 datasets as EPSG::2100
  • Operating System deleted (Windows)

Just tested 1.7.4 on Linux and found out that it suffers from the same issue.

#5 Updated by Jürgen Fischer about 7 years ago

marisn - wrote:

Just tested 1.7.4 on Linux and found out that it suffers from the same issue.

The problem also does relate to a change in GDAL were most of the +datum= parameters were dropped and +towgs84= parameters were inserted. QGIS now ignores +datum=, if there's no match found.

EPSG:25884 and EPSG:2100 are only different by +towgs84 parameter:

$ gdalsrsinfo -o proj4 Baltic_93TM_QGIS_test.prj 
'+proj=tmerc +lat_0=0 +lon_0=24 +k=0.9996 +x_0=500000 +y_0=0 +ellps=GRS80 +units=m +no_defs '
$ gdalsrsinfo -o proj4 epsg:25884
'+proj=tmerc +lat_0=0 +lon_0=24 +k=0.9996 +x_0=500000 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs '
$ gdalsrsinfo -o proj4 epsg:2100
'+proj=tmerc +lat_0=0 +lon_0=24 +k=0.9996 +x_0=500000 +y_0=0 +ellps=GRS80 +towgs84=-199.87,74.79,246.62,0,0,0,0 +units=m +no_defs '

WKT of the shape:

PROJCS["ETRS89_TM_Baltic93",
    GEOGCS["GCS_ETRS89",
        DATUM["ETRS_1989",
            SPHEROID["GRS_1980",6378137,298.257222101]],
        PRIMEM["Greenwich",0],
        UNIT["Degree",0.017453292519943295]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",24],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["Meter",1]]

#6 Updated by Etienne Tourigny about 7 years ago

I don't know when (and why) +datum was removed from gdal output of proj.4 strings, but the problem here is that .prj files do not have +towgs84 parameters.

In gdal 1.9 this can be fixed with the config option GDAL_FIX_ESRI_WKT=TOWGS84
see OGRSpatialReference::morphFromESRI() and GDAL #4345

$ GDAL_FIX_ESRI_WKT=TOWGS84 gdalsrsinfo -o proj4 Baltic_93TM_QGIS_test.prj
'+proj=tmerc +lat_0=0 +lon_0=24 +k=0.9996 +x_0=500000 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs '

Perhaps this config option could be added to the GGis environment? How does master deal with this problem?

#7 Updated by Jürgen Fischer about 7 years ago

  • Status changed from Open to Closed

#8 Updated by Giovanni Manghi about 7 years ago

  • Target version changed from 35 to Version 1.8.0

#9 Updated by marisn - over 6 years ago

  • Status changed from Closed to Reopened
  • Affected QGIS version changed from 1.7.4 to master
  • Target version changed from Version 1.8.0 to Version 2.0.0

Unfortunately issue is back in master.

1.9.0-Master
QGIS code revision 078f5b8

Running GDAL 1.8.1

#10 Updated by Jürgen Fischer over 6 years ago

marisn - wrote:

Unfortunately issue is back in master.

1.9.0-Master
QGIS code revision 078f5b8

Running GDAL 1.8.1

This should be fixed with GDAL 1.9 (see the commit)

#11 Updated by Giovanni Manghi over 6 years ago

  • Status changed from Reopened to Feedback

#12 Updated by marisn - over 6 years ago

It's still an issue for some datasets.
Attached dataset was created by exporting from GRASS GIS. Data are in EPSG:25884, still QGIS autodetects it as EPSG:2100

gdalsrsinfo estonia_generalized.prj 

PROJ.4 : '+proj=tmerc +lat_0=0 +lon_0=24 +k=0.9996 +x_0=500000 +y_0=0 +ellps=GRS80 +units=m +no_defs '

OGC WKT :
PROJCS["Transverse_Mercator",
    GEOGCS["GCS_grs80",
        DATUM["unknown",
            SPHEROID["Geodetic_Reference_System_1980",6378137,298.257222101]],
        PRIMEM["Greenwich",0],
        UNIT["Degree",0.017453292519943295]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",24],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["Meter",1]]

#13 Updated by marisn - over 6 years ago

Forgot to add version information:

QGIS versija 1.9.0-Master
QGIS code revision ad437bf
Compiled against GDAL/OGR 1.9.1
Running against GDAL/OGR 1.9.1

Also fix only for OGR >1.9.x is unacceptable, as due to QGIS + OGR layer encoding issues, it's not possible to read/write shapefiles with GDAL/OGR 1.9.x (yes, shapefile encoding still doesn't work at all when QGIS runs with GDAL/OGR 1.9.1!) and thus GDAL/OGR 1.8.x is the only option for many QGIS users.

#14 Updated by Giovanni Manghi over 6 years ago

marisn - wrote:

Forgot to add version information:

QGIS versija 1.9.0-Master
QGIS code revision ad437bf
Compiled against GDAL/OGR 1.9.1
Running against GDAL/OGR 1.9.1

Also fix only for OGR >1.9.x is unacceptable, as due to QGIS + OGR layer encoding issues, it's not possible to read/write shapefiles with GDAL/OGR 1.9.x (yes, shapefile encoding still doesn't work at all when QGIS runs with GDAL/OGR 1.9.1!) and thus GDAL/OGR 1.8.x is the only option for many QGIS users.

not sure, but aren't both gdal issues?

#15 Updated by marisn - over 6 years ago

Giovanni Manghi wrote:

not sure, but aren't both gdal issues?

Well. No. They both are QGIS issues triggered by changes in GDAL/OGR/Proj. GDAL/OGR doesn't mandate EPSG code to operate on datasets. I don't know if it's QGIS doing the wrong guess or GDAL/OGR, still end result is clear - QGIS uses wrong CRS, OGR tools use right one. As long as QGIS was using just datasets .prj file contents, it was fine.

Here's example how to check it:

ogr2ogr -s_srs epsg:2100 -t_srs epsg:4326 est_gre.shp estonia_generalized.shp
ogr2ogr -s_srs epsg:25884 -t_srs epsg:4326 est_bal.shp estonia_generalized.shp
ogr2ogr -t_srs epsg:4326 est_raw.shp estonia_generalized.shp

Compare output. est_bal equals est_raw, est_gre is shifted by ~200m. Conclusion - ogr2ogr uses right CRS for provided file.

As goes for encoding issues - I'm fed up with them. It's still not possible to specify correct encoding in layer preferences or when adding vector dataset AND get data in specified encoding. Pointing out that there are also issues on QGIS side is threated like blasphemy. Still most likely I will reopen bugs when GDAL/OGR 1.9.2 will come out and issue will still exist.

#16 Updated by Giovanni Manghi over 6 years ago

Pointing out that there are also issues on QGIS side is threated like blasphemy.

I'm sorry but you are plain wrong. If you check you will see I'm the top bug reporter (or the second one, anyway I'm the top one in the last 3 years) -> I have no interest in not reporting a QGIS bug, on the other hand I'm very interested in hunting any possible bug.

Here the point is to keep the tracker as clean as possible, avoiding to have open tickets that are not right/necessary and eventually redirecting the issue upstream to other projects trackers.

Still most likely I will reopen bugs when GDAL/OGR 1.9.2 will come out and issue will still exist.

I have no reason to not believe you, anyway we will see as soon as qgis will be compiled against gdal 1.9.2

In any case if the latest gdal version available has the issue described here fixed, I don't understand why this should be left open. But anyway I will leave the last word to someone else.

#17 Updated by Jürgen Fischer over 6 years ago

marisn - wrote:

As long as QGIS was using just datasets .prj file contents, it was fine.

That's part of the problem. QGIS doesn't use the .prj directly and AFAIK never did. It only uses it to find a match in the srs.db and actually use the projection defined there.

And finding that match became more complex after the GDAL change, because the datum parameter was dropped from GDAL and replaced with the towgs84 parameter. And therefore QGIS would fail to recognize those as there wasn't an exact match anymore and in turn produce a user CRS. So people started to complain about user CRSs instead of EPSG CRSs. So the detection was relaxed a bit by ignoring the +datum parameter, if there's no match otherwise.

Unfortunately there are some CRSs that are only different by the datum parameter - the above two being one of those pairs. And QGIS did pick the first hit even if there are more than one.

42a9d0193 changes that. So now, QGIS produces a user crs for estonia_generalized.shp now. BTW Baltic_93TM_QGIS_test.shp still worked for me. Still a compromise and hopefully not one that breaks other things.

As goes for encoding issues - I'm fed up with them.

Welcome to the club.

It's still not possible to specify correct encoding in layer preferences or when adding vector dataset AND get data in specified encoding. Pointing out that there are also issues on QGIS side is threated like blasphemy.

"threated" should have been "treated", shouldn't it? No one is threatening anyone, right?

#18 Updated by Giovanni Manghi over 6 years ago

  • Status changed from Feedback to Closed
  • Resolution set to fixed

Also available in: Atom PDF