Bug report #17985: Huge memory leak with QgsFeature.attributes() in Python - QGIS Application - QGIS Issue Tracking

Bug report #17985

Huge memory leak with QgsFeature.attributes() in Python

Added by Nyall Dawson about 6 years ago. Updated over 5 years ago.

Status:	Closed
Priority:	High
Assignee:	Denis Rouzaud
Category:	Python bindings / sipify
Affected QGIS version:	master	Regression?:	Yes
Operating System:		Easy fix?:	No
Pull Request or Patch supplied:	No	Resolution:	fixed/implemented
Crashes QGIS or corrupts data:	Yes	Copied to github as #:	25881

Description

See discussion in https://github.com/qgis/QGIS/pull/6186 and https://github.com/qgis/QGIS/pull/6190

Steps to reproduce:

Load a big layer with many NULL values

Run the python code:

for f in iface.activeLayer().getFeature():
        f.attributes()

And watch your memory go!

It's caused by the ConvertFromTypeCode in QgsAttributes.h/sip - commenting out the code which sets list items in the loop avoids the leak. Martin has also deduced that it's only triggered by the presence of NULL values in the list.

History

#1 Updated by Nyall Dawson about 6 years ago

Nathan reports that the leak is also present in the QgsFeature python code - e.g. calling feature[0] or feature['my_attr'] will leak if the attribute is NULL.

To be precise - it's not technically a memory leak, it's just that Python is not garbage collecting these values.

#2 Updated by Nathan Woodrow about 6 years ago

The following code also seems to show the issue

ids = set()
def leakme():
    count = 0
    for f in iface.activeLayer().getFeatures():
        a = f[4]
        ids.add(id(a))
        if count == 1000:
            break
        count += 1

leakme()
print(ids)