Bug report #418

QGIS fails to read undefined projection from user datum in shape.prj file

Added by neteler-itc-it - almost 13 years ago. Updated about 10 years ago.

Status:Closed
Priority:Low
Assignee:Magnus Homann
Category:Projection Support
Affected QGIS version: Regression?:No
Operating System:All Easy fix?:No
Pull Request or Patch supplied: Resolution:fixed
Crashes QGIS or corrupts data: Copied to github as #:10477

Description

Hi,

when opening Italian maps, we usually have to define a "user" datum since PROJ4 doesn't contain the definitions (as there are 3 datums for the same EPSC code in Italy, depending on where you are). The QgsSpatialRefSys detection fails, however:

qgis ammprvBL.shp
Warning: QgsSpatialRefSys::getRecord failed : select * from tbl_srs where parameters='+proj=tmerc +lat_0=0 +lon_0=9 +k=0.999600 +x_0=1500000 +y_0=0 +ellps=intl +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs'

If I take out the towgs84 part from the .prj file, it works.
It would be great to make QGIS tolerant for user defined towgs84 parameters which is rather the standard in Europe (many countries, many datums).

Proposal as general solution:
Maybe the towgs84 part could be temporarily taken out for the SQL search and re-added after the getRecord request was done.

Best,
Markus

PS: Attached the .prj file content

ammprvBL.prj - offending .proj file with towgs84 string (441 Bytes) neteler-itc-it -, 2006-12-03 02:40 AM

Associated revisions

Revision 8e251248
Added by Magnus Homann about 10 years ago

Automatically adds projections previously unknown to QGIS (but still valid) as a Custom CRS. This can later be edited for name etc. No user interaction required, one new text to translate. Fixes #418.

git-svn-id: http://svn.osgeo.org/qgis/trunk/[email protected] c8812cc2-4d05-0410-92ff-de0c093fc19c

Revision 93d5bd2d
Added by Magnus Homann about 10 years ago

Automatically adds projections previously unknown to QGIS (but still valid) as a Custom CRS. This can later be edited for name etc. No user interaction required, one new text to translate. Fixes #418.

git-svn-id: http://svn.osgeo.org/qgis/[email protected] c8812cc2-4d05-0410-92ff-de0c093fc19c

History

#1 Updated by Martin Dobias almost 13 years ago

  • Resolution set to wontfix
  • Status changed from Open to Closed
This is not that trivial as it looked like. Let's explain what happens when QGIS tries to match a projection with its spatial reference database:
  • check whether projection's name is in user's or global database; if not
  • check whether the whole proj4 string is in user's or global database; if not
  • try to find heuristically by traversing database and using OGRSpatialReference::IsSame()

In your case first two options don't find equivalent in database, but third does and tells QGIS that it's the same projection as "+proj=tmerc +lat_0=0 +lon_0=9 +k=0.999600 +x_0=1500000 +y_0=0 +ellps=intl +units=m +no_defs". The problem is that OGR's IsSame() function is not robust enough to make a difference between this and the specified ones. (See ogrspatialreference.cpp in OGR if interested)

To get it working correctly you have to add it as a custom projection (menu Settings -> Custom projection). If IsSame() worked correctly, it wouldn't recognize any projection and you would be asked to select one from database or a default one would be used (depending on settings).

Thus I'm closing this ticket as it seems that we're unable to fix it. ... probably this should be reported to OGR project...?

#2 Updated by neteler-itc-it - almost 13 years ago

You said that "The problem is that OGR's IsSame() function is not robust enough to make a difference between this and the specified ones." In fact, there is close to no difference besides the towgs84 part which would "just" be needed to be transferred to the projection found by QGIS in it's DB.

I don't know if failing on that completely and not recognize any projection
is the best solution here. I think that the IsSame() heuristics work quite
well. Substracting the input and the output (from IsSame()) projection string will show just towgs84 as difference which is perfect.

Best,
Markus

#3 Updated by Martin Dobias almost 13 years ago

Well, it seems that towgs84 parameter is quite important. Maybe in this case there's really small difference. But when looking to the QGIS's database of projections, there are (many) projections that specify towgs84 parameter, so I guess that it might be quite important - removing it temporarily would be incorrect.

A solution might be to use original proj string even if another projection has been declared as the same. But I don't know if this won't break other things or create some inconsistencies.

#4 Updated by neteler-itc-it - almost 13 years ago

Hi,

I think we have a missunderstanding here. I meant to say
that the towgs84 parameter should be taken out while
parsing the string (to match the QGIS SRS DB), then
reinserted, after having found the matching EPSG (which usually lacks the towgs84 parameter.

In case that a towgs84 parameter is already present in the QGIS SRS DB, it should use the user provided one (maybe pop up a notice that it was doing so).

Hope that's clarified now what I tried to say.

Markus

#5 Updated by Martin Dobias almost 13 years ago

Replying to [comment:5 ]:

I think we have a missunderstanding here. I meant to say
that the towgs84 parameter should be taken out while
parsing the string (to match the QGIS SRS DB), then
reinserted, after having found the matching EPSG (which usually lacks the towgs84 parameter.

I understand your idea. But the problem is in the way how QGIS copes with spatial references. Once proj4 string is read, QGIS searches for a SRS ID in SRS database. SRS ID is unique for every projection. Thus even if two projection are nearly same (e.g. with or without towgs84) they must have different SRS ID to enable QGIS understand the difference.

That's why taking out towgs84 temporiraly makes no sense - another SRS ID would be chosen instead.

Rescue from this situation is to add custom projection. Once added, your projection has its own SRS ID and will be detected correctly.

Martin

#6 Updated by neteler-itc-it - almost 13 years ago

Hi Martin,

I am again falling into this trap:

Warning: QgsSpatialRefSys::getRecord failed : select * from tbl_srs where parameters='+proj=utm +zone=32 +ellps=intl +towgs84=-87.000,-98.000,-121.000 +units=m +no_defs'

Displaying these two maps with on-the-fly projection activated, they are shifted by around 100m:

PROJCS[[UTM Zone 32 Northern Hemisphere"GEOGCS["international"DATUM["D_European_1950"SPHEROID["International_1924"6378388297]TOWGS84[-87000-98000-121000]]PRIMEM["Greenwich"0]UNIT["Degree"0017453292519943295]]PROJECTION["Transverse_Mercator"]PARAMETER["latitude_of_origin"0]PARAMETER["central_meridian"9]PARAMETER["scale_factor"09996]PARAMETER["false_easting"500000]PARAMETER["false_northing"0]UNIT["Meter]]

PROJCSTransverse Mercator"GEOGCS["international"DATUM["D_Monte_Mario"SPHEROID["international"6378388297PRIMEM["Greenwich"0]UNIT["Degree"0017453292519943295]]PROJECTION["Transverse_Mercator"]PARAMETER["latitude_of_origin"0]PARAMETER["central_meridian"9]PARAMETER["scale_factor"09996]PARAMETER["false_easting"1500000]PARAMETER["false_northing"0]UNIT["Meter]]

In my opinion at least a visible warning should pop up that reading the .proj file failed.

thanks,
Markus

#7 Updated by Redmine Admin over 12 years ago

  • Resolution deleted (wontfix)
  • Status changed from Closed to Feedback

Hi

Please consider Markus request. QGIS silent failure to assign a datum shift is really bad. If, instead, a warning would pop up with info that datum was not recognized and the user is asked to define the CRS manually himself to workaround it, this would be a great help.

I just had this issue too, working with archival topo maps of Germany, for which the 584.8,67.0,400.3,0.105,0.013,-2.378,10.29 datum shift from Bessel 1841 to WGS84 was not recognized by QGIS and it silently fell to no datum shift at all. This has resulted in an over 250m error. For an unexperienced user this might mean real trouble.

Maciek

#8 Updated by Magnus Homann over 12 years ago

This can't be done unless we re-do the entire projections system, earliest at 0.9.

#9 Updated by Markus Neteler about 11 years ago

In current SVN trunk i(pre 1.0.0) t continues to fail:

qgis viapri.shp
Python support ENABLED :-)
Loading python plugins
Loaded : Plugin installer (package: plugin_installer)
Critical: [[QgsSpatialRefSys]]::createFromWkt -- theWkt is uninitialised, operation failed
Warning: [[QgsSpatialRefSys]]::getRecord failed :  select * from tbl_srs where parameters='+proj=tmerc +lat_0=0 +lon_0=9 +k                                               =0.9996 +x_0=1500000 +y_0=0 +ellps=intl +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs'
Warning: [[QgsSpatialRefSys]]::findMatchingProj will only work if prj acr ellipsoid acr and proj4string are set!...

It works when I manually remove the TOWGS84 string from the WKT (.prj) file.

Suggestion: Ignore the TOWGS84 string when matching the projection, and issue a warning (not a complete failure) that TOWGS84 differs.

Note that most (in theory all) Italian QGIS users have this problem. Likewise for other countries.

thanks,
Markus

#10 Updated by Magnus Homann almost 11 years ago

  • Status changed from Feedback to Open

Is this error still present? Another towgs bug has been fixed, I believe.

#11 Updated by Magnus Homann almost 11 years ago

  • Status changed from Open to In Progress

Talked to timlinux, and the idea with have is if the specfied projection does not have an exact match (including +towgs84= )in the database of projections, it will ask the user if he/she wants to create a custom projection.

Do you think that will solve the issue? This wont happen until 1.1.0 I'm afraid, needs GUI changes and has potential for large changes of projection code needed.

#12 Updated by Markus Neteler almost 11 years ago

Replying to [comment:14 homann]:

Talked to timlinux, and the idea with have is if the specfied projection does not have an exact match (including +towgs84= )in the database of projections, it will ask the user if he/she wants to create a custom projection.

Do you think that will solve the issue?

I think so. But:

This wont happen until 1.1.0 I'm afraid, needs GUI changes and has potential for large changes of projection code needed.

Understood. But as suggested above: Please issue at least a warning that TOWGS84 differs (if not happening already).

Markus

#13 Updated by Magnus Homann almost 11 years ago

I'm sorry, but the warning is still a GUI change. That towgs differs is just a special case of the fact that the projections differs. As soons as GUI freeze is over i'll try to fix this.

#14 Updated by Paolo Cavallini over 10 years ago

There is a problem in PROJ4: there are several datum for EPSG 3003 (and
others), but in PROJ4 the epsg file does not allow a relation 1:n with
present format (thanks Markus for clarifying this).
Currently, users need to define their special SRS, which is cumbersome
and scary for newbies. GRASS solved it recently:
http://grass.osgeo.org/screenshots/gui.php#wxpython
-> Location wizard (geodetic datum selection for EPSG 3003 - Italy mainland)
Would it be feasible to copy the approach and port it to QGIS?

#15 Updated by Magnus Homann about 10 years ago

  • Resolution set to fixed
  • Status changed from In Progress to Closed

Automatic addition of a custom CRS for unknown to QGIS - but valid - CRS:es from e.g WKT in 93d5bd2d (SVN r11367). If there are any different problems from Italian qgissers, please open a new ticket.

Also available in: Atom PDF