Bug report #11980
Failure to properly geolocate Mapinfo TIFF files calibrated by TAB files
| Status: | Closed | ||
|---|---|---|---|
| Priority: | Normal | ||
| Assignee: | - | ||
| Category: | Projection Support | ||
| Affected QGIS version: | 2.6.0 | Regression?: | No |
| Operating System: | Easy fix?: | No | |
| Pull Request or Patch supplied: | No | Resolution: | up/downstream |
| Crashes QGIS or corrupts data: | No | Copied to github as #: | 20186 |
Description
Where historical maps are calibrated within Mapinfo, these are usually output in the form of .TIF files accompanied by a .TAB file giving the georeferencing data. An example of such maps is the 1880s First Series 1:2500 scale county Ordnance Survey maps for Suffolk. At this scale it takes a large number of tiles to cover a whole county. In a sample dozen of such maps around Bury St Edmunds, only four were correctly located by QGIS. The rest disappear from view.
Two sample tiles are attached, which are adjacent to each other. One will be correctly located, while the other will not.
It seems that GDAL will not properly locate the tile if the control points do not represent a clean affine transformation for the image. It falls back to returning them as ground control points which QGIS cannot utilize directly.
This is likely to be a problem for all local authorities using old historic maps who would like to convert to QGIS.
History
#1
Updated by Giovanni Manghi over 9 years ago
- Category set to Projection Support
- Status changed from Open to Feedback
- Operating System deleted (
Windows 7 Pro)
Hi,
you attached twice the same TIFF and missed one TAB.
Anyway, the raster with the TAB seems to have the following CRS
+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +towgs84=375,-111,431,-0,-0,-0,0 +units=m +no_defs
is right? what EPSG code is?
This raster do not align correctly with any layer that has for sure a correct CRS (like the BING wms for example), so maybe the raster was given the wrong CRS?
#2
Updated by David Addy over 9 years ago
- File 33044061.TAB added
Sorry for the omission of 33044061.TAB. This is now attached.
These files were created over 20 years ago on MAPINFO. They should be using the Great Britain Ordnance Survey 1936 CRS. This is coded OSGB36, or EPSG27700 on QGIS.
They maps may not exactly match over the modern CRS as they were originally drawn in 1880, before the modern OSGB36 was created. However, on MAPINFO they do fit quite well over a modern OS map.
Thanks for your interest in looking at this.
(Incidentally, if I use GDALWARP to create a world file .TFW, these files can then be properly geo-located by QGIS, but this is a tedious work round to do for thousands of files.)
#3
Updated by Giovanni Manghi over 9 years ago
David Addy wrote:
Sorry for the omission of 33044061.TAB. This is now attached.
These files were created over 20 years ago on MAPINFO. They should be using the Great Britain Ordnance Survey 1936 CRS. This is coded OSGB36, or EPSG27700 on QGIS.
They maps may not exactly match over the modern CRS as they were originally drawn in 1880, before the modern OSGB36 was created. However, on MAPINFO they do fit quite well over a modern OS map.
Thanks for your interest in looking at this.
For me it looks like the coordinates in TAB file of 33044071.tif are just wrong. There is nothing wrong in QGIS, if you edit the TAB file and replace them with something that fits mainland UK then the rasters is placed in the right place.
(Incidentally, if I use GDALWARP to create a world file .TFW, these files can then be properly geo-located by QGIS, but this is a tedious work round to do for thousands of files.)
You can batch process the creation of worldfiles, no need to make then one by one.
#4
Updated by David Addy over 9 years ago
Giovanni - Although the TAB file for 33044071.TIF may look "wrong", tile 33044061.TIF has a .TAB file with identical CRS information, and QGIS will locate this tile in the correct position in UK grid square TL 86.
Also, both files will be located properly within MAPINFO using this information.
All I want is for QGIS to do the same as MAPINFO can do using the same data.
This issue has been known about for a couple of years, but nobody seems able to fix it. Even Chris Wesson at Ordnance Survey has raised this issue on the QGIS Users Group blog on 31st January, 2014.
Please help if you can.
#5
Updated by Giovanni Manghi over 9 years ago
- Status changed from Feedback to Closed
- Resolution set to up/downstream
David Addy wrote:
Giovanni - Although the TAB file for 33044071.TIF may look "wrong", tile 33044061.TIF has a .TAB file with identical CRS information, and QGIS will locate this tile in the correct position in UK grid square TL 86.
Also, both files will be located properly within MAPINFO using this information.
All I want is for QGIS to do the same as MAPINFO can do using the same data.
This issue has been known about for a couple of years, but nobody seems able to fix it. Even Chris Wesson at Ordnance Survey has raised this issue on the QGIS Users Group blog on 31st January, 2014.
Please help if you can.
So there is nothing wrong with the coordinates in 33044071.TIF tab file, but anyway GDAL (and not QGIS) fails to correctly read them.
gdalinfo for 33044061.TIF reports
giovanni@sibirica:~/Downloads > gdalinfo 33044061.TIF
Warning 1: TIFFFetchNormalTag:ASCII value for tag "ImageDescription" contains null byte in value; value incorrectly truncated during reading due to implementation limitations
Driver: GTiff/GeoTIFF
Files: 33044061.TIF
33044061.TAB
Size is 11335, 7594
Coordinate System is:
PROJCS["unnamed",
GEOGCS["unnamed",
DATUM["OSGB_1936",
SPHEROID["Airy 1930",6377563.396,299.3249646],
TOWGS84[375,-111,431,-0,-0,-0,0]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",49],
PARAMETER["central_meridian",-2],
PARAMETER["scale_factor",0.9996012717],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",-100000],
UNIT["Meter",1]]
GeoTransform =
582371.7853128385, 0.2128106799708868, 0.007506419931898357
265177.0453393976, 0.007597130759530216, -0.2117600570261728
Metadata:
TIFFTAG_IMAGEDESCRIPTION=
TIFFTAG_MAXSAMPLEVALUE=1
TIFFTAG_MINSAMPLEVALUE=0
TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
TIFFTAG_XRESOLUTION=300
TIFFTAG_YRESOLUTION=300
Image Structure Metadata:
COMPRESSION=CCITTFAX4
INTERLEAVE=BAND
MINISWHITE=YES
Corner Coordinates:
Upper Left ( 582371.785, 265177.045) ( 0d40'19.83"E, 52d15'14.75"N)
Lower Left ( 582428.789, 263568.939) ( 0d40'19.71"E, 52d14'22.68"N)
Upper Right ( 584783.994, 265263.159) ( 0d42'27.09"E, 52d15'14.63"N)
Lower Right ( 584840.998, 263655.053) ( 0d42'26.92"E, 52d14'22.57"N)
Center ( 583606.392, 264416.049) ( 0d41'23.39"E, 52d14'48.66"N)
Band 1 Block=512x512 Type=Byte, ColorInterp=Palette
Image Structure Metadata:
NBITS=1
Color Table (RGB with 2 entries)
0: 255,255,255,255
1: 0,0,0,255
where you can see that the corner coordinates make sense, while the same for 33044071.TIF returns
giovanni@sibirica:~/Downloads > gdalinfo 33044071.TIF
Warning 1: TIFFFetchNormalTag:ASCII value for tag "ImageDescription" contains null byte in value; value incorrectly truncated during reading due to implementation limitations
Driver: GTiff/GeoTIFF
Files: 33044071.TIF
33044071.TAB
Size is 11335, 7576
Coordinate System is `'
GCP Projection =
PROJCS["unnamed",
GEOGCS["unnamed",
DATUM["OSGB_1936",
SPHEROID["Airy 1930",6377563.396,299.3249646],
TOWGS84[375,-111,431,-0,-0,-0,0]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",49],
PARAMETER["central_meridian",-2],
PARAMETER["scale_factor",0.9996012717],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",-100000],
UNIT["Meter",1]]
GCP[ 0]: Id=Pt 1, Info=
(1,7576) -> (584841,263655,0)
GCP[ 1]: Id=Pt 2, Info=
(1,0) -> (584784,265263,0)
GCP[ 2]: Id=Pt 3, Info=
(11335,1) -> (587197,265349,0)
GCP[ 3]: Id=Pt 4, Info=
(11335,7576) -> (587254,263740,0)
Metadata:
TIFFTAG_IMAGEDESCRIPTION=
TIFFTAG_MAXSAMPLEVALUE=1
TIFFTAG_MINSAMPLEVALUE=0
TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
TIFFTAG_XRESOLUTION=300
TIFFTAG_YRESOLUTION=300
Image Structure Metadata:
COMPRESSION=CCITTFAX4
INTERLEAVE=BAND
MINISWHITE=YES
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 7576.0)
Upper Right (11335.0, 0.0)
Lower Right (11335.0, 7576.0)
Center ( 5667.5, 3788.0)
Band 1 Block=512x512 Type=Byte, ColorInterp=Palette
Image Structure Metadata:
NBITS=1
Color Table (RGB with 2 entries)
0: 255,255,255,255
1: 0,0,0,255
as you can see there is some issue with the parsing of the GCP coordinates and so the corners have coordinates that are simply relative to the number of pixels that represent the WxH of the raster.
This issue must be filed in the GDAL issue tracker, not here. Cheers!
#6
Updated by Bo Victor Thomsen over 9 years ago
David -
The problem is that your geotiff files contains some "goofiness" in the GeoTiff tags inside the tiff-file. That won't affect MapInfo, because it doesn't use these tags in the Geotiff file. MapInfo uses only the information in the corresponding tab-file to geolocate the tiff. But QGIS will try to read the GeoTag information in the tiff file and use them to place the raster correctly in the map window. That's why the files work in MapInfo but dosen't work in QGIS.
You can make a workaround by removing the geotif tags inside you tiff files. When QGIS reads a tiff - file without geotags it will look for geolocation information in a corresponding .twf file or a corresponding .tab file.
You can remove geotiff tag by using the gdal_translate utility distributed with QGIS (in the \\bin directory) like this:
gdal_translate -co PROFILE=BASELINE 33044071.tif temp.tifmove /Y temp.tif 33044071.tif
(I assume you're on windows) The two lines above can be edited to a cmd file like this
gdal_translate -co PROFILE=BASELINE %1 temp.tifmove /Y temp.tif %1
and call it gt_clean.cmd. Finally you can call the cmd file like this : call gt_clean.cmd 33044071.tif
If you want to do the same operation on every tif file in a directory: for %i in (*.tif) do call gt_clean.cmd %i
#7
Updated by Bo Victor Thomsen over 9 years ago
An update and a correction to my previous post.
- The tiff files does not contain geotiff tags. Any geo-location information comes only from the corresponding tab file.
- Some of these tab files has a problem with the affine transformation established with the 4 coordinate pairs saved inside each tab - file.
- 33044061.tab is okay, so QGIS (or rather the GDAL subsystem) can read the tab, convert the coordinate pairs to meaningfull information about the geolocation for the raster.
- 33044071.tab is not okay, so the GDAL subsystem simply uses the raster corner coordinates as world coordinates. The error is probably that the 4 possible solutions for the affine transformation differs to much from each other. (One affine transformation uses 3 coordinate pairs, so 4 coordinate pairs can be used to generate 4 different transformation functions)
The solution to this problem is:
- Get the GDAL developers to change the interpretion of the coordinate pairs, so GDAL works like MapInfo. (Probably using some Least-Squares solution to find a "correct" solution)
- In every tif/tab combination that doesn't work: Remove one coordinate pair from the tab file. But the transformation will be "hit-and-run"
- If it's possible: Get the correct world-corner coordinates for each tiff file from somewhere else. And generate new artificial tab files based on the corner coordinates and the size of the raster.
#8
Updated by Giovanni Manghi over 9 years ago
Bo Victor Thomsen wrote:
An update and a correction to my previous post.
- The tiff files does not contain geotiff tags. Any geo-location information comes only from the corresponding tab file.
that is was I also found before posting in #11980-5
#9
Updated by Bo Victor Thomsen over 9 years ago
I did a further bit of investigation:
GDAL v. 2.0 has a new config-option called "TAB_APPROX_GEOTRANSFORM (YES/NO)". If set to "YES" (Default is "NO") GDAL will accept and use GCP from a TAB file to calculate the affine transformation parameters - even if it results in high error residuals from the LS approximation (Just like MapInfo). It might be an idea to introduce this as a user-selectable option (with due warning !!) in a future QGIS version that will use GDAL 2.0 a read/write library.
This might convince a lot of existing MapInfo users to use QGIS instead. As an old MapInfo user, I can tell that there is a huge amount raster files with "dodgy" tab files out in the wild :-/
As a test, I tried to use the "gdal_translate" utility from GDAL 2.0 on the test tiff-files and with the above-mentioned config option. It worked, resulting in new geotiff based files with acceptable geolocation parameters.