QGIS » QGIS Application

User Feedback¶

I have tried to reproduce this in later version 3.2.2, and it occurs more frequently. Im doing the following:
I parse multiple geojson files, for each json file, I convert to QgsFeature with geometry, and add those to a memory Vector Layer / data provider. I also check for attributes/fields alignment in Vector layer and in the new data, and update it accordingly (add new attributes/fields into the VectorLayer). Maybe the crash occurs during this proccess.
Each iteration of adding attributes + feature is done in a ThreadPool of maximum 8 threads. Also after each iteration, I do layer.triggerRepaint() or mapCanvas().refresh() for rendering.

Note: this only happens when I parse large geojson file. In total it consists 140033 features with large MultiPolygon geometries.

Report Details¶

Crash ID: 0b740b1a81b6b145766026a5ee9136e0c1ec1e6f

Stack Trace

QVector<QVariant>::QVector<QVariant> :
QgsFeature::attributes :
QgsProcessingModelOutput::toVariant :
std::_String_alloc<std::_String_base_types<char,std::allocator<char> > >::_Swap_all :
PyCFunction_FastCallDict :
PyObject_GenericGetAttr :
PyEval_EvalFrameDefault :
PyErr_Occurred :
PyBytes_Resize :
PyEval_EvalFrameDefault :
PyFunction_FastCallDict :
PyObject_CallFunctionObjArgs :
PyObject_Call :
PyInit_sip :
QThreadPoolPrivate::reset :
QThread::start :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info
QGIS Version: 3.2.2-Bonn
QGIS code revision: 26842169e9
Compiled against Qt: 5.9.2
Running against Qt: 5.9.2
Compiled against GDAL: 2.2.4
Running against GDAL: 2.2.4

System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.15063

Note: this only happens when I parse large geojson file. In total it consists 140033 features with large MultiPolygon geometries.

This requires a newer version of GDAL where geojson parsing was greatly improved. I'm inclined to close this as an "upstream" issue.

Nyall Dawson wrote:

Note: this only happens when I parse large geojson file. In total it consists 140033 features with large MultiPolygon geometries.

This requires a newer version of GDAL where geojson parsing was greatly improved. I'm inclined to close this as an "upstream" issue.

I am aware of GDAL limitation in parsing large geojson. My workaround is to process geojson file to a Python dict, then parse each single dict entry to QgsFeature. Geometry is parsed using ogr.CreateGeometryFromJson(). This approach of parsing geojson is used in the original bug reports.

The geojson parsing and adding QgsFeature are realiazed in async manner (using signal/slot mechanism in Qt, AutoConnection). Before, I use nested function or lambda function for signal/slot connection. Then, I switched to use defined function for signal connection, and the error occur much less frequent. So, I am suspecting the error is caused by memory overflow and/or dangling C++ pointer/Python bound method, as sometime during debugging, I got error like QgsFeature does not have signal finished occurring inside the nested function.

Currently, I am rewriting my async code, and hopefully the error will go away. I will keep this thread updated.

I have successfully debugged the cause of this crash and resolved it.
This crash happens when I call provider.addFeatures() in worker thread and the canvas trying to render (via the UI or via layer.triggerUpdate() and canvas.refresh()). Simple solution is to call provider.addFeatures() in main thread.