Feature request #4594

identify feature on ecw extremely long

Added by Regis Haubourg almost 8 years ago. Updated about 7 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Data Provider
Pull Request or Patch supplied:No Resolution:fixed
Easy fix?:No Copied to github as #:14499

Description

Hi dev's,
I'am using big ecw file (from 500Mo to 92 Go) and an Identify action on it take ages to answer band values (5 minutes sometimes).

- First question: is there any workaround to optimize it? is that a question dedicated to gdal developpers or Qgis dev's?

- Second Question: users do often get trap interrogating an ecw when they did not want to. That could be because of a bad habit coming from mapinfo OR identify mode different from "current layer" in settings). If there's no way to optimize the identify to respond within ~ 3 seconds, is there any solution to set ecw "not identifiable" by default. It's possible in project properties by hand, but users won't do that. that's why a default "identifiable value" to 'off' in ecw addlayers method would be perfect.

All the best Régis

History

#1 Updated by Giovanni Manghi almost 8 years ago

  • Status changed from Open to Feedback

With not that big ecws (~100mb) the identify operation is normal fast. Can you test with smaller images and/or with different formats?

#2 Updated by Regis Haubourg almost 8 years ago

Hi Ghiovanni,
here are result tests:

QGIS 1.7.2, on the fly "off", Xeon Windows xp 32 bits. ecw on NAS servers via LAN 100Mbit/s:

- scan 1 / Size: 139Mo / pixels : 12150 * 11000 (sdk ecw 4.1) / identify delay: 1s / RAM consumption : opening ecw + 50Mo, each identify action + ~ 15 Mo. Memory not released after identify

- scan 2 / Size: 525Mo/ pixels : 30000*30000 (sdk ecw 4.1) / identify delay: 4s / RAM consumption : opening ecw + 10Mo, each identify action + ~ 15 Mo. Memory not released after identify

- scan 3 / Size: 870Mo/ pixels : 60000*60000 (sdk ecw 4.1) / identify delay: 6s / RAM consumption : opening ecw + 30Mo, each identify action + ~ 15 Mo. Memory not released after identify

- scan 4 / Size: 1,05Go/ pixels : 220 000* 220 000 (sdk ecw 4.1) / identify delay: 20s / RAM consumption : opening ecw + 30Mo, identify action rises once RAM to 663 Mo! / Memory not released after identify

- aerial views / Size: 74 Go/ pixels : 1 080 000* 1 090 000 (sdk ecw 4.1) / identify delay: 50s / RAM consumption : opening ecw + 30Mo, identify action rises once RAM to 663 Mo! / Memory not released after identify. Many crashes if on-the_fly reprojection is "On" (true also on 1.7.1).

I tested 1.7.1 on a windows server 2003, with gigabit access to datas, (vmware environnement). only identify delay is shortened to 30 s instead of 50s. Crashes sometimes, 660 Mo of RAM consumption.

I guess gdal or Qgis uncompress datas and also have a memory leak. I have not noticed any improvement identifying on a hih or low scale (more pixels inside cursor tolerance?)

Régis

#3 Updated by Nathan Woodrow almost 8 years ago

Just a note: The large memory usage with ecws is due to the caching the gdal driver does. We could turn this off as I don't find it improves performance a lot.

#4 Updated by Regis Haubourg almost 8 years ago

Thanks Nathan, I didn't find in source code where to toggle gdal cache off. Any idea? I also wanted to test the identify query directly in gdal command line. What is the query send by qgis to gdal?
Régis

#5 Updated by Giovanni Manghi over 7 years ago

  • Target version changed from Version 1.8.0 to Version 2.0.0

#6 Updated by Radim Blazek about 7 years ago

The problem was that identify requested single pixel by GDALRasterIO() and GDAL ECW driver reads and caches whole line if requested block height is 1.

It should be fixed in master a795a5d334.

Please test and close if it is resolved.

#7 Updated by Regis Haubourg about 7 years ago

Problem is solved for me on a795a5d. Thanks a lot guys. Can you close the bug? I don't have permissions apparently.

#8 Updated by Giovanni Manghi about 7 years ago

  • Resolution set to fixed
  • Status changed from Feedback to Closed

Also available in: Atom PDF