Bug report #17985

Huge memory leak with QgsFeature.attributes() in Python

Added by Nyall Dawson almost 3 years ago. Updated over 2 years ago.

Status:Closed
Priority:High
Assignee:Denis Rouzaud
Category:Python bindings / sipify
Affected QGIS version:master Regression?:Yes
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed/implemented
Crashes QGIS or corrupts data:Yes Copied to github as #:25881

Description

See discussion in https://github.com/qgis/QGIS/pull/6186 and https://github.com/qgis/QGIS/pull/6190

Steps to reproduce:

Load a big layer with many NULL values

Run the python code:

for f in iface.activeLayer().getFeature():
f.attributes()

And watch your memory go!

It's caused by the ConvertFromTypeCode in QgsAttributes.h/sip - commenting out the code which sets list items in the loop avoids the leak. Martin has also deduced that it's only triggered by the presence of NULL values in the list.

History

#1 Updated by Nyall Dawson almost 3 years ago

Nathan reports that the leak is also present in the QgsFeature python code - e.g. calling feature[0] or feature['my_attr'] will leak if the attribute is NULL.

To be precise - it's not technically a memory leak, it's just that Python is not garbage collecting these values.

#2 Updated by Nathan Woodrow almost 3 years ago

The following code also seems to show the issue

ids = set()
def leakme():
    count = 0
    for f in iface.activeLayer().getFeatures():
        a = f[4]
        ids.add(id(a))
        if count == 1000:
            break
        count += 1

leakme()
print(ids)

#3 Updated by Nathan Woodrow almost 3 years ago

@Nayll This one is fixed now isn't it?

#4 Updated by Nyall Dawson almost 3 years ago

Let me verify a few other things first - I did see evidence of another similar leak at one stage.

#5 Updated by Giovanni Manghi almost 3 years ago

  • Status changed from Open to Feedback

#6 Updated by Denis Rouzaud over 2 years ago

Nyall, can this be closed?

#7 Updated by Giovanni Manghi over 2 years ago

Hi, what is the status of this issue?

#8 Updated by Nyall Dawson over 2 years ago

  • Resolution set to fixed/implemented
  • Status changed from Feedback to Closed

Fixed

Also available in: Atom PDF