Bug report #5041

[PATCH] support for raster sublayers (GDAL subdatasets) broken

Added by Etienne Tourigny over 7 years ago. Updated over 7 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Rasters
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:Yes Resolution:fixed
Crashes QGIS or corrupts data:No Copied to github as #:14813

Description

Support for raster sublayers was implemented in d72cfb4b (by Jürgen Fischer over 1 year ago), but it does not work now.

Background information here: #1040

The problem is that QgsGdalProvider::QgsGdalProvider() deletes the dataset when there are no raster bands in the "main dataset":

GDALRasterBandH myGDALBand = GDALGetRasterBand( mGdalDataset, 1 ); //just use the first band
if ( myGDALBand == NULL )

And when qgisapp.cpp calls layer->subLayers() the dataset is invalid, which results in the following error:

ERROR 10: Pointer 'hObject' is NULL in 'GDALGetMetadata'.

Attaching an example file.

I have prepared a patch which fixes this issue and and also adds support for sublayers when opening raster(s) from the commandline and also from the browser dock widget.

I have also improved the appearance of the sublayer dialog, and implemented a new option for when the dialog should be opened

/qgis/promptForRasterSublayers with the following values:
  • 0 = Always -> always ask (if there are existing sublayers)
  • 1 = If needed -> ask if layer has no bands but has sublayers
  • 2 = Never

However, I am not sure how to implement this into qbrowser, may have to add some code from qgisapp.cpp.

Summary of changes in the patch:

  • qgsgdalprovider.cpp :
    • do not close the dataset if there are sublayers, so subsequent calls to subLayers() work
    • fix subLayers() crash when dataset is closed (when there are no sublayers)
  • qgisapp.cpp
    • add function shouldAskUserForGdalSublayers() which uses new config option /qgis/promptForRasterSublayers (see function for more details)
    • modify addRasterLayers() and various addRasterLayer() functions to use shouldAskUserForGdalSublayers()
  • qgsbrowserdockwidget.cpp : use QgisApp::addRasterLayer() so these modifications also work in the browser dock
  • qgsogrsublayersdialog.cpp : resize columns to content (so long sublayer names are visible)
  • qgsoptions.cpp and qgsoptionsbase.ui: add UI for /qgis/promptForRasterSublayers option

Debug output below (some superflous "TMP ET..." messages added)

Debug: src/app/qgisapp.cpp: 6398: (addRasterLayer) TMP ET addRasterLayer
Debug: src/gui/qgisgui.cpp: 38: (openFilesRememberingFilter) Opening file dialog with filters: [GDAL] All files (*);;[GDAL] Virtual Raster (*.vrt *.VRT);;[GDAL] GeoTIFF (*.tif *.tiff *.TIF *.TIFF);;[GDAL] National Imagery Transmission Format (*.ntf *.NTF);;[GDAL] Raster Product Format TOC format (*.toc *.TOC);;[GDAL] ECRG TOC format (*.xml *.XML);;[GDAL] Erdas Imagine Images  (*.img *.IMG);;[GDAL] Ground-based SAR Applications Testbed File Format  (*.gff *.GFF);;[GDAL] Arc/Info Binary Grid (hdr.adf HDR.ADF);;[GDAL] Arc/Info ASCII Grid (*.asc *.ASC);;[GDAL] SDTS Raster (*.ddf *.DDF);;[GDAL] DTED Elevation Raster (*.dt0 *.dt1 *.dt2 *.DT0 *.DT1 *.DT2);;[GDAL] Portable Network Graphics (*.png *.PNG);;[GDAL] JPEG JFIF (*.jpg *.jpeg *.JPG *.JPEG);;[GDAL] Japanese DEM  (*.mem *.MEM);;[GDAL] Graphics Interchange Format  (*.gif *.GIF);;[GDAL] Graphics Interchange Format  (*.gif *.GIF);;[GDAL] Envisat Image Format (*.n1 *.N1);;[GDAL] X11 PixMap Format (*.xpm *.XPM);;[GDAL] MS Windows Device Independent Bitmap (*.bmp *.BMP);;[GDAL] PCIDSK Database File (*.pix *.PIX);;[GDAL] PCRaster Raster File (*.map *.MAP);;[GDAL] ILWIS Raster Map (*.mpr *.mpl *.MPR *.MPL);;[GDAL] SGI Image File Format 1.0 (*.rgb *.RGB);;[GDAL] SRTMHGT File Format (*.hgt *.HGT);;[GDAL] Leveller heightfield (*.ter *.TER);;[GDAL] Terragen heightfield (*.ter *.TER);;[GDAL] GMT NetCDF Grid Format (*.nc *.NC);;[GDAL] Network Common Data Format (*.nc *.NC);;[GDAL] JPEG-2000 part 1  (*.jp2 *.j2k *.JP2 *.J2K);;[GDAL] GRIdded Binary  (*.grb *.GRB);;[GDAL] Raster Matrix Format (*.rsw *.RSW);;[GDAL] EUMETSAT Archive native  (*.nat *.NAT);;[GDAL] Idrisi Raster A.1 (*.rst *.RST);;[GDAL] Golden Software ASCII Grid  (*.grd *.GRD);;[GDAL] Golden Software Binary Grid  (*.grd *.GRD);;[GDAL] Golden Software 7 Binary Grid  (*.grd *.GRD);;[GDAL] DRDC COASP SAR Processor Raster (*.hdr *.HDR);;[GDAL] R Object Data Store (*.rda *.RDA);;[GDAL] Portable Pixmap Format  (*.pnm *.PNM);;[GDAL] ESRI .hdr Labelled (*.bil *.BIL);;[GDAL] Vexcel MFF Raster (*.hdr *.HDR);;[GDAL] VTP .bt (Binary Terrain) 1.3 Format (*.bt *.BT);;[GDAL] FARSITE v.4 Landscape File  (*.lcp *.LCP);;[GDAL] NOAA Vertical Datum .GTX (*.gtx *.GTX);;[GDAL] NTv2 Datum Grid Shift (*.gsb *.GSB);;[GDAL] ACE2 (*.ace2 *.ACE2);;[GDAL] Snow Data Assimilation System (*.hdr *.HDR);;[GDAL] Swedish Grid RIK  (*.rik *.RIK);;[GDAL] USGS Optional ASCII DEM  (*.dem *.DEM);;[GDAL] GeoSoft Grid Exchange Format (*.gxf *.GXF);;[GDAL] Hierarchical Data Format Release 5 (*.hdf5 *.HDF5);;[GDAL] Northwood Numeric Grid Format .grd/.tab (*.grd *.GRD);;[GDAL] Northwood Classified Grid Format .grc/.tab (*.grc *.GRC);;[GDAL] ARC Digitized Raster Graphics (*.gen *.GEN);;[GDAL] Standard Raster Product  (*.img *.IMG);;[GDAL] Magellan topo  (*.blx *.BLX);;[GDAL] Rasterlite (*.sqlite *.SQLITE);;[GDAL] SAGA GIS Binary Grid  (*.sdat *.SDAT);;[GDAL] ASCII Gridded XYZ (*.xyz *.XYZ);;[GDAL] HF2/HFZ heightfield raster (*.hf2 *.HF2);;[GDAL] Arc/Info Export E00 GRID (*.e00 *.E00);;[GDAL] ZMap Plus Grid (*.dat *.DAT);;[GDAL] NOAA NGS Geoid Height Grids (*.bin *.BIN);;[GDAL] MBTiles (*.mbtiles *.MBTILES)
Debug: src/gui/qgisgui.cpp: 78: (openFilesRememberingFilter) Writing last used dir: /data/research/work/gdal/gdal-netcdf/narrcap
Debug: src/app/qgisapp.cpp: 6411: (addRasterLayer) TMP ET addRasterLayer opendfiles
Debug: src/app/qgisapp.cpp: 6593: (addRasterLayers) TMP ET addRasterLayers
Debug: src/app/qgisapp.cpp: 6624: (addRasterLayers) TMP ET addRasterLayers calling isValidRasterFileName()
Debug: src/core/raster/qgsrasterlayer.cpp: 2135: (loadProviderLibrary) theProviderKey = gdal
Debug: src/core/raster/qgsrasterlayer.cpp: 2139: (loadProviderLibrary) myLibPath = /home/softdev/lib/qgis/plugins/libgdalprovider.so
Debug: src/core/raster/qgsrasterlayer.cpp: 2161: (loadProviderLibrary) Library name is /home/softdev/lib/qgis/plugins/libgdalprovider.so
Debug: src/core/raster/qgsrasterlayer.cpp: 2169: (loadProviderLibrary) Loaded data provider library
Debug: src/providers/gdal/qgsgdalprovider.cpp: 1801: (isValidRasterFileName) TMP ET isValidRasterFileName()
Debug: src/providers/gdal/qgsgdalprovider.cpp: 1250: (subLayers_) sublayers:
  NETCDF:"/data/research/work/gdal/gdal-netcdf/narrcap/orog_CRCM.nc":lon
  NETCDF:"/data/research/work/gdal/gdal-netcdf/narrcap/orog_CRCM.nc":lat
  NETCDF:"/data/research/work/gdal/gdal-netcdf/narrcap/orog_CRCM.nc":orog
Debug: src/app/qgisapp.cpp: 6636: (addRasterLayers) TMP ET addRasterLayers - creating layer
Debug: src/core/qgsmaplayer.cpp: 53: (QgsMapLayer) lyrname is 'orog_CRCM'
Debug: src/core/qgsmaplayer.cpp: 59: (QgsMapLayer) layerName is 'orog_CRCM'
Debug: src/core/raster/qgsrasterlayer.cpp: 98: (QgsRasterLayer) Entered
Debug: src/core/raster/qgsrastershader.cpp: 25: (QgsRasterShader) called.
Debug: src/core/raster/qgsrastershaderfunction.cpp: 24: (QgsRasterShaderFunction) entered.
Debug: src/core/qgsmaplayer.cpp: 104: (setLayerName) new name is 'orog_CRCM'
Debug: src/core/raster/qgsrastershader.cpp: 25: (QgsRasterShader) called.
Debug: src/core/raster/qgsrastershaderfunction.cpp: 24: (QgsRasterShaderFunction) entered.
Debug: src/core/raster/qgsrasterlayer.cpp: 2176: (loadProvider) Entered
Debug: src/core/raster/qgsrasterlayer.cpp: 2135: (loadProviderLibrary) theProviderKey = gdal
Debug: src/core/raster/qgsrasterlayer.cpp: 2139: (loadProviderLibrary) myLibPath = /home/softdev/lib/qgis/plugins/libgdalprovider.so
Debug: src/core/raster/qgsrasterlayer.cpp: 2161: (loadProviderLibrary) Library name is /home/softdev/lib/qgis/plugins/libgdalprovider.so
Debug: src/core/raster/qgsrasterlayer.cpp: 2169: (loadProviderLibrary) Loaded data provider library
Debug: src/core/raster/qgsrasterlayer.cpp: 2178: (loadProvider) Library loaded
Debug: src/core/raster/qgsrasterlayer.cpp: 2185: (loadProvider) Attempting to resolve the classFactory function
Debug: src/core/raster/qgsrasterlayer.cpp: 2193: (loadProvider) Getting pointer to a mDataProvider object from the library
Debug: src/providers/gdal/qgsgdalprovider.cpp: 92: (QgsGdalProvider) QgsGdalProvider: constructing with uri '/data/research/work/gdal/gdal-netcdf/narrcap/orog_CRCM.nc'.
Debug: src/providers/gdal/qgsgdalprovider.cpp: 118: (QgsGdalProvider) GdalDataset opened
Debug: src/app/qgscustomization.cpp: 778: (customizeWidget) objectName =  event type = 17
Debug: src/app/qgscustomization.cpp: 780: (customizeWidget) QMessageBox x QDialog
Debug: src/core/raster/qgsrasterlayer.cpp: 2207: (loadProvider) Data driver created
Debug: src/core/raster/qgsrasterlayer.cpp: 2259: (setDataProvider) Instantiated the data provider plugin with layer list of  and style list of  and format of  and CRS of 
Debug: src/providers/gdal/qgsgdalprovider.cpp: 1194: (isValid) valid = 0
Debug: src/app/qgisapp.cpp: 6638: (addRasterLayers) TMP ET addRasterLayers - calling subLayers()
Debug: src/providers/gdal/qgsgdalprovider.cpp: 1570: (subLayers) TMP ET QgsGdalProvider::subLayers()
ERROR 10: Pointer 'hObject' is NULL in 'GDALGetMetadata'.

Debug: src/providers/gdal/qgsgdalprovider.cpp: 1250: (subLayers_) sublayers:

Debug: src/app/qgisapp.cpp: 6640: (addRasterLayers) TMP ET addRasterLayers got 
Debug: src/providers/gdal/qgsgdalprovider.cpp: 329: (~QgsGdalProvider) QgsGdalProvider: deconstructing.

qgis-sublayers-1.patch Magnifier (10.7 KB) Etienne Tourigny, 2012-02-17 08:37 AM

orog_CRCM.nc (319 KB) Etienne Tourigny, 2012-02-17 08:37 AM

qgis-sublayers-2.patch Magnifier - slightly modified patch, which adds sublayers directly in browser widget (13.3 KB) Etienne Tourigny, 2012-02-18 08:50 AM

qgis-sublayers-browser.png (307 KB) Etienne Tourigny, 2012-02-18 08:59 AM

qgis-browser-short-names.png (270 KB) Etienne Tourigny, 2012-02-18 11:29 AM

qgis-sublayers-4.patch Magnifier (3.45 KB) Etienne Tourigny, 2012-02-18 11:29 AM

History

#2 Updated by Etienne Tourigny over 7 years ago

added slightly modified patch, with sublayers visible in the browser widget (screenshot attached)

  • added new function static QStringList QgsGdalProvider::subLayers( GDALDatasetH dataset )
  • modified dataItem() in qgsgdaldataitem.cpp to add sublayers as child items

It would be best to shorten layer names (both in the browser and also in the Layers dock)

For example, [NETCDF:"/data/research/work/gdal/gdal-netcdf/narrcap/orog_CRCM.nc":lon] should be shortened as [NETCDF:"orog_CRCM.nc":lon] or [orog_CRCM/lon]

#3 Updated by Jürgen Fischer over 7 years ago

  • Assignee deleted (Jürgen Fischer)

#4 Updated by Jürgen Fischer over 7 years ago

  • Resolution set to fixed
  • Status changed from Open to Closed

applied in 73afb0f2

#5 Updated by Etienne Tourigny over 7 years ago

Jurgen,

thanks for applying the patch, works great!

I have got a final (small) patch, that shortens the files names that are displayed in the Browser and Layers docks (long name seen in qgis-sublayers-browser.png)

See attached patch and screenshot.

Also a small fix for this warning:
[ 58%] Building CXX object src/app/CMakeFiles/qgis.dir/qgscustomprojectiondialog.cpp.o
/home/src/qgis-master/Quantum-GIS/src/app/qgsbrowserdockwidget.cpp: In member function ‘void QgsBrowserDockWidget::itemClicked(const QModelIndex&)’:
/home/src/qgis-master/Quantum-GIS/src/app/qgsbrowserdockwidget.cpp:129:16: warning: variable ‘layer’ set but not used [-Wunused-but-set-variable]

Many thanks!

#6 Updated by Paolo Corti over 7 years ago

Etienne, Jurgen
thanks for the patch
I hope I can give a try to this in the next days, and will keep you informed
P

#7 Updated by Jürgen Fischer over 7 years ago

unfortunately I missed that it caused #5062 - fixed in 927dcbd7 with rest "applied".

#8 Updated by Etienne Tourigny over 7 years ago

great! sorry for the oversight on the layer->providerType() != "gdal" check...

However - might there be other raster providers which could benefit from this mechanism?

Thanks

#9 Updated by Etienne Tourigny over 7 years ago

I am not familiar with WMS, but there might be something wrong with the wms raster provider, which returns 0 Bands and non-empty sublayers, which is why the dialog is triggered...

#10 Updated by Paolo Corti over 7 years ago

Etienne Tourigny wrote:

great! sorry for the oversight on the layer->providerType() != "gdal" check...

However - might there be other raster providers which could benefit from this mechanism?

Thanks

Etienne, Jürgen: many tanks, works really well now :)

Also available in: Atom PDF