Bug report #19584

Raster Clip and Merge issues (QGIS 3.2.1)

Added by Loren Amelang almost 2 years ago. Updated almost 2 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Processing/QGIS
Affected QGIS version:3.2.1 Regression?:No
Operating System:Windows 10 Creators with 3000x2000 screen and 175% scaling Easy fix?:No
Pull Request or Patch supplied:No Resolution:duplicate
Crashes QGIS or corrupts data:No Copied to github as #:27411

Description

Maybe this should be several separate issues, but it will be more understandable if I describe them in sequence...

I had three 400MB 1-meter DEM .tif files, and RiverGIS only lets you query one file name per project, so I wanted to consolidate them.

First I moved them to the project CRS with Raster -> Projections -> Warp (reproject). It altered the contrast of the visible DEM image, two got way paler, one slightly higher contrast. Is that proper?

Then I tried Raster -> Miscellaneous -> Merge, but got a Memory Error every time.

I thought I'd cut the file size, so I made a vector layer with the rectangular boundary of my project. I used Raster -> Extraction -> Clip Raster by Mask Layer, and it clipped the parts outside the boundary. BUT it made each output file the size of the full mask boundary! Is that expected? So they didn't get much smaller.

Merge still got a memory error each time. And, for each new try, I found I had to close the Merge window and start over from the QGIS menu bar. If I just edited a setting and tried again, I got "Incorrect parameter value for INPUT". It would still show the files selected, and re-selecting the input files didn't help.

Win 10 Resource Monitor showed zero memory faults, and just a tiny blip from 41% to 51% memory used, for these "Memory Error" tries, so I don't think it was a hardware limitation.

I tried Raster -> Extraction -> Clip Raster by Extent (Select extent on canvas)
First file worked, second showed correct "zoom to" extent but no image; under the Layers entry it showed:
1.79769e+308
-1.79769e+308
instead of
238.166
822.751
Again, turns out you can't edit and re-use a dialog! Must start over and re-enter everything each time.

After the manual clip, there were a couple of empty cracks near the boundaries where the segments should meet... They met perfectly on the screen before the clip. I found I has to draw my manual boundary at least 300' beyond the apparent edge of the DEM to get it all! Seems strange... Yes, all the files and the map image were in the same 2226 project CRS.

Also, if I tried to save to the same filename I had used in the last test, the .tif.aux.xml file would be saved, but not the actual new .tif! When I added a character to make a new filename, the .tif was saved with the new name, but no new .tif.aux.xml was saved under either name! (Looks like all of my .tif.aux.xml files have different timestamps from the matching .tif...)

I checked the Merge command before clicking run this time:
---
gdal_merge.bat ot Float32 -of GTiff -o "C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_2226_clip_man_merge.tif" --optfile C:/Users/loren/AppData/Local/Temp/processing_b1a1b0b73d594cc4a2747620a75d7b50\mergeInputFiles.txt
--

I was a bit suspicious of all the '/' dividers being valid in Windows... The file did exist:
C:\Users\loren\AppData\Local\Temp\processing_b1a1b0b73d594cc4a2747620a75d7b50\mergeInputFiles.txt
But it was blank before I clicked "go".

But it worked! With 225MB of input data instead of 955MB (or 1201MB input which had made an all-black 1565MB output file).

After the operation completed, the mergeInputFiles.txt file listed my input files.

One more curiosity... Each time I tried one of these operations with "Open Output File" checked it made another OGR error:
---
2018-08-09T17:56:41 WARNING Cannot open C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_47_435_to2226_clip_manual.tif.()
2018-08-09T18:01:12 WARNING Cannot open C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_47_436_to2226_clip_manual.tif.()
2018-08-09T18:11:12 WARNING Cannot open C:/Users/loren/Giant Files/HEC-RAS Dam Break/QGIS Projects/FEMA808/DEM_2226_clip_man_merge.tif.()
---
--> All of those files (and lots more before them) appeared to open normally in the Layers menu. What can this mean?


Related issues

Related to QGIS Application - Bug report #19594: "Memory Error" in Raster Merge Closed 2018-08-10
Related to QGIS Application - Bug report #19595: "Clip Raster by Mask Layer" is actually "Resize to mask l... Closed 2018-08-10
Related to QGIS Application - Bug report #19596: Can't re-use "Clip Raster by Extent" dialog; boundary off... Closed 2018-08-10
Related to QGIS Application - Bug report #19597: OGR: "WARNING Cannot open <file>" when they seem to open ... Closed 2018-08-10

History

#1 Updated by Nyall Dawson almost 2 years ago

Hi Loren,

Can you file these as separate distinct issues?

#2 Updated by Loren Amelang almost 2 years ago

Merge - #19594
Resize - #19595
Clip - #19596
OGRwarn - #19597

#3 Updated by Jürgen Fischer almost 2 years ago

#4 Updated by Jürgen Fischer almost 2 years ago

  • Related to Bug report #19595: "Clip Raster by Mask Layer" is actually "Resize to mask layer" added

#5 Updated by Jürgen Fischer almost 2 years ago

  • Related to Bug report #19596: Can't re-use "Clip Raster by Extent" dialog; boundary offset? added

#6 Updated by Jürgen Fischer almost 2 years ago

  • Related to Bug report #19597: OGR: "WARNING Cannot open <file>" when they seem to open fine? added

#7 Updated by Giovanni Manghi almost 2 years ago

  • Resolution set to duplicate
  • Status changed from Open to Closed

Separate tickets have been filed, closing this.

Also available in: Atom PDF