Bug report #8178

Shifted raster: origin in qgis different than that reported by gdalinfo

Added by alobo - over 6 years ago. Updated over 4 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Rasters
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:invalid
Crashes QGIS or corrupts data:No Copied to github as #:16997

Description

After processing a raster layer (TTC2770v2_0_1.tif) with OTB (TTC2770v2_0_1otb.tif), I get a shifted display of
the new layer in Qgis.
While gdalinfo reports a 0,0 origin, qgis reports a 0, 1023.74 origin.
The origin of the original file is reported as 0,0 by both gdalinfo and qgis
See attached screenshot.

Also, while gdalinfo reports no projection (because there is none) Qgis reports
longlat wgs84 in the Metadata (not only in General).

Tested on qgis 1.8 (on Ubuntu) and 2.0-dev clf8230 (on Mac)

Agus

shifteddisplay.jpeg (241 KB) alobo -, 2013-06-27 04:04 AM

TTC2770v2_0_1.tif (1.91 MB) alobo -, 2013-06-27 04:04 AM

sensor_stereo_left.tif (512 KB) alobo -, 2013-07-19 03:46 AM

test.jpeg (164 KB) alobo -, 2013-07-22 02:42 AM

testgrass.jpeg (415 KB) alobo -, 2013-07-22 02:42 AM

testqgis.jpeg (145 KB) alobo -, 2013-07-22 02:42 AM

testv.zip - vector point layer created by random samplig of the raster layer in R (11.5 KB) alobo -, 2013-07-23 04:50 AM

39.png - qgis gdal 1.10 no origin (303 KB) Giovanni Manghi, 2013-08-06 10:51 AM

History

#1 Updated by alobo - over 6 years ago

  • File sensor_stereo_left.tif added
  • Subject changed from Shifted raster: wrong origin to Shifted raster: origin in qgis different than that reported by gdalinfo

Confirmed on ubuntu 1.9.0-Master 18a04b5
In the example below
According to gdalinfo Origin = (4953.000000000000000,4917.000000000000000) Pixel Size = (1.000000000000000,1.000000000000000)
According to QGIS: Origin 4953,5045 Pixel Size 1,-1

#2 Updated by Giovanni Manghi over 6 years ago

  • Priority changed from High to Low
  • Status changed from Open to Feedback

To me seems just a GUI glitch and not a serious bug, but I may be wrong obviously.

The fact that a non georefereced raster is loaded in QGIS as wgs84 it just depends on how your qgis is configured in settings -> options -> crs

This "issue" then seems to surface only when you (force to) load a non georeferenced raster, because if you load a geotiff or a tiff + worldfile it seems what you see in qgis metadata and gdalinfo is the same. Can you confirm?

#3 Updated by alobo - over 6 years ago

To me seems just a GUI glitch and not a serious bug, but I may be wrong obviously.

Yes, I think. If you take the time to check the coordinates on the display, you'll see that
according to qgis the bottom left is 0,-1024, in accordance to what the qgis GUI (properties/metadata) wrongly states
and in disagreement to what is stated by gdal (and R, OTB etc)
This bug is a serious problem to interact between other software (R, OTB) and qgis, as any vector layer will not be correctly overlaid (compare test.jpeg and testgrass.jpeg to testqgis.jpeg).

The fact that a non georefereced raster is loaded in QGIS as wgs84 it just depends on how your qgis is configured in settings -> options -> crs

which is wrong, because the raster does not have georeferencing. Qgis, like, for example grass, should let the user just leave the image in an arbitrary
cartesian system, because the image is "waiting" for being georeferenced. Actually, requiring a formal CRS is a problem for the georeferencer plugin, which
enforces the absurd requirement of setting a CRS to the input image, which is, in most cases, just not georeferenced at all.

This "issue" then seems to surface only when you (force to) load a non georeferenced raster, because if you load a geotiff or a tiff + worldfile it seems what you see in qgis metadata and gdalinfo is the same. Can you confirm?

If we use this as tfw:
1
0.00
0.00
-1
0.5
0.5

the position in qgis is the same as with no tfw, but the positions in R and grass are shifted.
Properties/metadata and gdalinfo still agree:
0,1
Pixel Size
1,-1

with sizes 1 and 1:
qgis does not display anything
R displays as the original
qgis claims
ERROR: Input raster map is flipped or rotated - cannot import. You may use 'gdalwarp' to transform the map to North-up.
at importing

So this just proofs that in qgis the tfw overrides the geoti header, but this does not solve the problem.

Agus

#4 Updated by Giovanni Manghi over 6 years ago

  • Priority changed from High to Normal

Yes, I think. If you take the time to check the coordinates on the display, you'll see that
according to qgis the bottom left is 0,-1024, in accordance to what the qgis GUI (properties/metadata) wrongly states
and in disagreement to what is stated by gdal (and R, OTB etc)
This bug is a serious problem to interact between other software (R, OTB) and qgis, as any vector layer will not be correctly overlaid (compare test.jpeg and testgrass.jpeg to testqgis.jpeg).

are we speaking about a raster images that:

  • is not a geotiff
  • does not have a world file

correct?

If yes, then as far as I know/understand in qgis is not supposed/ready to handle rasters that are completely missing any georefercing, regardless of what the user can force on load, or regardless what crs is given by qgis (by settings -> options -> crs).

Probably qgis would better handle this cases in a different way (you may want to file a feature request) but imho it is not a huge deal, layers are supposed to have a proper georeferencing (?).

which is wrong, because the raster does not have georeferencing. Qgis, like, for example grass, should let the user just leave the image in an arbitrary
cartesian system, because the image is "waiting" for being georeferenced.

feature request?

Actually, requiring a formal CRS is a problem for the georeferencer plugin, which
enforces the absurd requirement of setting a CRS to the input image, which is, in most cases, just not georeferenced at all.

it is not absurd, is a bug (see #4377) but this does not stops the georefencer to work as expected: when it asks the crs just click "cancel".

If we use this as tfw:
1
0.00
0.00
-1
0.5
0.5

the position in qgis is the same as with no tfw, but the positions in R and grass are shifted.

so, your raster does not goes in the right place with a proper worldfile?

Can you please attach sample data, the previous attachments are confusing to me. Thanks.

PS
High priority tickets are the ones causing crash or data corruption.

#5 Updated by alobo - over 6 years ago

Giovanni Manghi wrote:

If yes, then as far as I know/understand in qgis is not supposed/ready to handle rasters that are completely missing any georefercing, regardless of what the user can force on load, or regardless what crs is given by qgis (by settings -> options -> crs).

That's ostrich' strategy. Instead of fixing a bug let's just find a justification for not fixing it.
If this were your choice, qgis should either refuse displaying the layer or at least warn that the display will be wrong.
But then please note that a significant part of the work with Sextante aiming to run R & OTB commands and display the results
should better be halted to search for another visualizer.

Probably qgis would better handle this cases in a different way (you may want to file a feature request) but imho it is not a huge deal, layers are supposed to have a proper georeferencing (?).

In Remote Sensing we deal with the images for a long while before they get georeferenced (as in the case of the example). So this is a huge deal or not depending on whether you want qgis to be useful for the RS community or not.

Actually, requiring a formal CRS is a problem for the georeferencer plugin, which
enforces the absurd requirement of setting a CRS to the input image, which is, in most cases, just not georeferenced at all.

it is not absurd, is a bug (see #4377) but this does not stops the georefencer to work as expected: when it asks the crs just click "cancel".

...and the image gets the by default CRS as in any other case in qgis. And you get wrong coordinates for the input image that complicates using the
list of points in another software to, for example, apply geometric methods that are not currently implemented in qgis.

According to your statement "qgis is not supposed/ready to handle rasters that are completely missing any georefercing", #4377
would not be a bug: the entire georeferencing tool should be removed.

so, your raster does not goes in the right place with a proper worldfile?

I think it's better you make the worldfile to fit your expectations, I'm not sure of what you mean by a proper worldfile and it's just a matter of
writing few lines in a text file.

Can you please attach sample data, the previous attachments are confusing to me. Thanks.

The sample tif image and some screenshots were attached in the first post. Do yo mean the vector layer?. I attach it now,

PS
High priority tickets are the ones causing crash or data corruption.

Excuses, I don't know where to look for the definitions of the different levels and, in my naivety, thought
that placing a raster layer in a wrong place should have a high priority.

I think you are complicating this discussion far too much. qgis reports a wrong origin and places the image in a wrong origin for raster
layers (at least geotif, have not tested with other formats) that do not have CRS information.

#6 Updated by Giovanni Manghi over 6 years ago

That's ostrich' strategy. Instead of fixing a bug let's just find a justification for not fixing it.

You are wrong. Re-read my words, I said "as far as I know/understand", this means two things: I may be wrong, and that anyway this reflects how qgis works to my knowledge, it doesn't means that it is perfect or it is that way by design.

Feel free to do something, like submitting patches or supporting the qgis development, the way you think is better for you and the project.

If this were your choice, qgis should either refuse displaying the layer or at least warn that the display will be wrong.

For rasters that are completely missing any CRS info? Sounds reasonable, then see above.

But then please note that a significant part of the work with Sextante aiming to run R & OTB commands and display the results
should better be halted to search for another visualizer.

Why? Anything that enters qgis/sextante has to have crs info, as geotiff or with a worldfile, then the output also will have a proper crs.

In Remote Sensing we deal with the images for a long while before they get georeferenced (as in the case of the example). So this is a huge deal or not depending on whether you want qgis to be useful for the RS community or not.

ok, but how other software packages do show in the right place (overlapping other layers with a proper crs) those images if they have any kind of crs info (internal or worldfile)? This should be the starting point to eventually make qgis work better in such cases.

...and the image gets the by default CRS as in any other case in qgis. And you get wrong coordinates for the input image that complicates using the
list of points in another software to, for example, apply geometric methods that are not currently implemented in qgis.

According to your statement "qgis is not supposed/ready to handle rasters that are completely missing any georefercing", #4377
would not be a bug: the entire georeferencing tool should be removed.

you are mixing up things and my words. The georeferencer can certainly be fixed and made better (but this does not means that at this moment is broken), so again feel free to contribute improve it.

I think it's better you make the worldfile to fit your expectations, I'm not sure of what you mean by a proper worldfile and it's just a matter of
writing few lines in a text file.

I know how a worldfile is made, and as far as I remember they work fine in QGIS.

Excuses, I don't know where to look for the definitions of the different levels and, in my naivety, thought
that placing a raster layer in a wrong place should have a high priority.

All infos you need are in the wiki.

I think you are complicating this discussion far too much.

surprise, I think the same thing about you.

qgis reports a wrong origin and places the image in a wrong origin for raster
layers (at least geotif, have not tested with other formats) that do not have CRS information.

Unless I missed something, the info is wrong only when loading an image that has not any kind of crs info (ex: geotiff or tif+worldfile) that anyway as we already know it is not a thing that at this moment qgis is supposed to do.

#7 Updated by Giovanni Manghi over 6 years ago

The sample tif image and some screenshots were attached in the first post. Do yo mean the vector layer?. I attach it now,

the attached vector is also missing the CRS (it misses the prj file), so maybe this is what you mean: if you load any kind of layer missing the crs info, do you expect anyway to overlap?

#8 Updated by alobo - over 6 years ago

Yes, because the raster layer is also missing the crs info. The vector
layer is derived by sampling the raster layer, thus they should overlap.
Both layers are missing the CRS information, but the vector layer
is displayed correctly while the raster layer is shifted.

#9 Updated by alobo - over 6 years ago

Giovanni Manghi wrote:

But then please note that a significant part of the work with Sextante aiming to run R & OTB commands and display the results
should better be halted to search for another visualizer.

Why? Anything that enters qgis/sextante has to have crs info, as geotiff or with a worldfile, then the output also will have a proper crs.

Not true. Many OTB functions that work through Sextante are aimed to raster layers with no CRS info because there is a significant part
of the processing that has to be done before the image is georeferenced.

In Remote Sensing we deal with the images for a long while before they get georeferenced (as in the case of the example). So this is a huge deal or not depending on whether you want qgis to be useful for the RS community or not.

ok, but how other software packages do show in the right place (overlapping other layers with a proper crs) those images if they have any kind of crs info (internal or worldfile)? This should be the starting point to eventually make qgis work better in such cases.

Simply by reading the gdal info correctly. The problem here is that while gdalinfo reports a 0,0 origin, qgis reports a 0, 1023.74 origin and displays the
image with such shifted origin.

...and the image gets the by default CRS as in any other case in qgis. And you get wrong coordinates for the input image that complicates using the
list of points in another software to, for example, apply geometric methods that are not currently implemented in qgis.

According to your statement "qgis is not supposed/ready to handle rasters that are completely missing any georefercing", #4377
would not be a bug: the entire georeferencing tool should be removed.

you are mixing up things and my words. The georeferencer can certainly be fixed and made better (but this does not means that at this moment is broken), so again feel free to contribute improve it.

We rather keep the discussion on #4377 in its own place.

I think it's better you make the worldfile to fit your expectations, I'm not sure of what you mean by a proper worldfile and it's just a matter of
writing few lines in a text file.

I know how a worldfile is made, and as far as I remember they work fine in QGIS.

Then it will be very simple for you to test. I do not understand what you asked me to test with the world file, so better
you do it yourself.

I think you are complicating this discussion far too much.

surprise, I think the same thing about you.

No, what I'm saying is very simple and short: While gdalinfo reports a 0,0 origin, qgis reports a 0, 1023.74 origin and displays the image with such a shift. Apparently,
this problem only happens for raster files with no CRS information.

qgis reports a wrong origin and places the image in a wrong origin for raster
layers (at least geotif, have not tested with other formats) that do not have CRS information.

Unless I missed something, the info is wrong only when loading an image that has not any kind of crs info (ex: geotiff or tif+worldfile) that anyway as we already know it is not a thing that at this moment qgis is supposed to do.

Why is not supposed to do it? Where is that statement? Again, in that case, qgis should either refuse displaying the layer or at least warn the user. Who is "we" in
"we already know"? Certainly not the general user. And again, note a
part of the applications meant to be run through sextante will not be displayed by qgis under those conditions.

#10 Updated by Giovanni Manghi over 6 years ago

alobo - wrote:

Yes, because the raster layer is also missing the crs info. The vector
layer is derived by sampling the raster layer, thus they should overlap.
Both layers are missing the CRS information, but the vector layer
is displayed correctly while the raster layer is shifted.

So let me resume:

when loading two layers (vectors or rasters) both missing any CRS information, do you expect to overlap, right?

they should be aligned by the center? or one of the corners?

And what about if you add another similar layer, but from another completely different region, where it should be placed?

#11 Updated by alobo - over 6 years ago

Giovanni Manghi wrote:

alobo - wrote:

Yes, because the raster layer is also missing the crs info. The vector
layer is derived by sampling the raster layer, thus they should overlap.
Both layers are missing the CRS information, but the vector layer
is displayed correctly while the raster layer is shifted.

So let me resume:

when loading two layers (vectors or rasters) both missing any CRS information, do you expect to overlap, right?

if they have the same geometry, yes.

they should be aligned by the center? or one of the corners?

They should be displayed according to the coordinates of the corner reported by gdalinfo, as R (see test.jpeg) or Grass
(see testgrass.jpeg) do.

And what about if you add another similar layer, but from another completely different region, where it should be placed?

They should be displayed according to the corner coordinates reported by gdalinfo. They are not georeferenced, so if we
have a set of images acquired with the same sensor (thus the same geometry), they all will overlap despite not having been acquired in the same
scene. Once they become georeferenced, they will be located according to their new geographic coordinates.
For example, all Hyperion images Level L1R (no georeferencing) of the world will overlap, while images level L1T (terrain corrected) will be displayed at their
correct geographic places. if I have an algorithm detecting bad columns o bad pixels in L1R and make a vector of those, this vector should
overlap all L1R images. As those are sensor-related problems, detecting those problems in images of, i.e., the Lybian desert, can be applied to correct
images in i.e., the Amazon. Once these problems are solved, the images are georeferenced.

#12 Updated by Giovanni Manghi over 6 years ago

With GDAL 1.10 and a copy if QGIS compiled against GDAL 1.10 (as it is now fpr qgis master on Windows and Ubuntu), the response is the same, no origin (see below and attached image);

[email protected]$ gdalinfo TTC2770v2_0_1.tif
Driver: GTiff/GeoTIFF
Files: TTC2770v2_0_1.tif
Size is 1280, 1024
Coordinate System is `'
Image Structure Metadata:
COMPRESSION=LZW
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 1024.0)
Upper Right ( 1280.0, 0.0)
Lower Right ( 1280.0, 1024.0)
Center ( 640.0, 512.0)
Band 1 Block=1280x3 Type=UInt16, ColorInterp=Gray

#13 Updated by Giovanni Manghi over 6 years ago

Oh now I see that you are referring to TTC2770v2_0_1otb.tif, that you haven't attached...

#14 Updated by Giovanni Manghi over 6 years ago

Bottom line is that, as far as I understand how qgis works, you must file anyway also a feature request ticket because qgis does expect the user to load a layer with proper georeferencing (and this looks like pretty much a fact to me). The fact that the origin looks shifted in qgis it seems not really important at this stage because before it would be needed to tell qgis how to handle such layers.

#15 Updated by Paolo Cavallini almost 6 years ago

  • Target version changed from Version 2.0.0 to Future Release - High Priority

#16 Updated by Giovanni Manghi over 4 years ago

  • Status changed from Feedback to Closed
  • Resolution set to invalid

closing for lack of feedback and because the general notion that layers that are not georeferenced should align is not valid, at least in qgis.

Also available in: Atom PDF