Bug report #9229

Layer editing saving for a long time (memory leak?)

Added by Giedrius Vaivilavičius over 10 years ago. Updated over 10 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Digitising
Affected QGIS version:2.0.1 Regression?:No
Operating System:Debian Wheezy Easy fix?:No
Pull Request or Patch supplied:No Resolution:up/downstream
Crashes QGIS or corrupts data:No Copied to github as #:17850

Description

8-9 clicks "save layer edits, work begins to slow down. 12-14 clicks slow down completely.
Hover:

terminal htop parameters when the procedure is:

on ubuntu (version12.04) the same problem
on windows 7 works well

mem_leak.zip (1.44 KB) Giedrius Vaivilavičius, 2013-12-26 06:52 AM

editcrash.zip (3.6 KB) Richard Duivenvoorde, 2014-01-04 09:03 AM

editcrash.zip (4.75 KB) Richard Duivenvoorde, 2014-01-04 09:15 AM

console_mem (6.55 KB) Giedrius Vaivilavičius, 2014-01-04 09:17 AM

crash_ubuntu (766 Bytes) Giedrius Vaivilavičius, 2014-01-04 09:37 AM

History

#1 Updated by Matthias Kuhn over 10 years ago

Hi Giedrius

Can you please provide more information about the problems you are experiencing.

  • What type of data are you working with?
  • Do you actually change data (geometries, attributes...)?
  • How big is your dataset?
  • Are you able to provide a test dataset or a screencapture of such an edit session?

Thank you very much

#2 Updated by Matthias Kuhn over 10 years ago

  • Status changed from Open to Feedback

#3 Updated by Giovanni Manghi over 10 years ago

  • Category set to Digitising
  • Target version set to Future Release - High Priority

Matthias Kuhn wrote:

Hi Giedrius

Can you please provide more information about the problems you are experiencing.

  • What type of data are you working with?
  • Do you actually change data (geometries, attributes...)?
  • How big is your dataset?
  • Are you able to provide a test dataset or a screencapture of such an edit session?

Thank you very much

Hi Matthias, this issue has also been reported recently on the dev mailing list

http://lists.osgeo.org/pipermail/qgis-developer/2013-November/029325.html

#4 Updated by Giedrius Vaivilavičius over 10 years ago

Matthias Kuhn wrote:

Hi Giedrius

Can you please provide more information about the problems you are experiencing.

  • What type of data are you working with?
  • Do you actually change data (geometries, attributes...)?
  • How big is your dataset?
  • Are you able to provide a test dataset or a screencapture of such an edit session?

Thank you very much

Just create a new point layer with one or more fields (Spatialite, shapefile, there is no difference):
1. In edit mode I put the point. enter data into the opened form and clicked "OK"
2. I click "save layer edits"

1. and 2. repeat steps 9-11 times and the program starts to brake.

It happens on 2.0.1 version. I noticed that on 2.1 (master) version of this happens only 40-50 times. The problem persists, but has to work.

#5 Updated by Gerhard Spieles over 10 years ago

Hello,
can confirm this behavior also on windows machine.
Intensive digitizing makes work more and more slower, especially when create new geometries and save changes.
Then, best is to close QGIS and start it again.

Maybe, the "undo/redo" function handling in the memory could be a reason?
Gerhard

#6 Updated by Matthias Kuhn over 10 years ago

About 45 edit sessions resulting in a total of 93 points starting from the attached shapefile (mem_leak) don't change the responsiveness or memory consumption of a recent master build here...

Can you try the following:

  • Start QGIS from the console
  • Reproduce the problem
  • While QGIS is hanging / unresponsive, force QGIS to crash and print a stacktrace (E.g. by pressing Ctrl + \\ on the console)
  • Paste the result here (Maybe some QGIS debug libraries will need to be installed to produce a useful output, I am not sure about debian/ubuntu (maybe a qgis-dbg package?) )

#7 Updated by Richard Duivenvoorde over 10 years ago

I can reproduce this here on a self compiled 2.0 branch build on Debian.
Took me some clicks and edits, but at a certain point, clicking the save-edits button started to take some seconds.
After that every new save takes longer, used ctrl-\\ to stop and output attached.

Hope this is helpfull

#8 Updated by Richard Duivenvoorde over 10 years ago

also reproduced on a master build here (31 dec).

Again the crash output, but I took some more info from the debug output.

Will also try without plugins....

#9 Updated by Giedrius Vaivilavičius over 10 years ago

Matthias Kuhn wrote:

About 45 edit sessions resulting in a total of 93 points starting from the attached shapefile (mem_leak) don't change the responsiveness or memory consumption of a recent master build here...

Can you try the following:

  • Start QGIS from the console
  • Reproduce the problem
  • While QGIS is hanging / unresponsive, force QGIS to crash and print a stacktrace (E.g. by pressing Ctrl + \\ on the console)
  • Paste the result here (Maybe some QGIS debug libraries will need to be installed to produce a useful output, I am not sure about debian/ubuntu (maybe a qgis-dbg package?) )

#11 Updated by Salvatore Larosa over 10 years ago

Confirmed here although executing QGIS with --noplugins does disappear the issue (same OS).

Please test with --noplugins parameter and let us know if the issue still happens for you!

#12 Updated by Salvatore Larosa over 10 years ago

For me it was enough to disable the Multipart Split plugin!

#13 Updated by Richard Duivenvoorde over 10 years ago

--noplugins fixed it, but I did not have the Multipart Split plugin

so hitting ctrl-c (in terminal) when 'hanging' during a normal round, showed me de pyhon debug console, showing:

Traceback (most recent call last):
File "/home/richard/.qgis2/python/plugins/traceDigitize/traceDigitize.py", line 99, in toggle
QObject.disconnect(layer,SIGNAL"),self.toggle)
KeyboardInterrupt

So apparently with me it was traceDigitze plugin. Disabling that one worked for me...

So apparently these plugins have something in common which screws up QGIS editing... actions calling actions or so?

#14 Updated by Giedrius Vaivilavičius over 10 years ago

Salvatore Larosa wrote:

Confirmed here although executing QGIS with --noplugins does disappear the issue (same OS).

Please test with --noplugins parameter and let us know if the issue still happens for you!

with --noplugins parameter works great. checking the problem by turning off one plug. When you turn off the Rectangle Ovals Digitizing  plugin problem disappeared. Thank you!

#15 Updated by Salvatore Larosa over 10 years ago

Richard Duivenvoorde wrote:

--noplugins fixed it, but I did not have the Multipart Split plugin

so hitting ctrl-c (in terminal) when 'hanging' during a normal round, showed me de pyhon debug console, showing:

Traceback (most recent call last):
File "/home/richard/.qgis2/python/plugins/traceDigitize/traceDigitize.py", line 99, in toggle
QObject.disconnect(layer,SIGNAL"),self.toggle)
KeyboardInterrupt

So apparently with me it was traceDigitze plugin. Disabling that one worked for me...

So apparently these plugins have something in common which screws up QGIS editing... actions calling actions or so?

good catch, same problem with the same SIGNAL: layer.editingStopped.disconnect(self.toggle)

#16 Updated by Matthias Kuhn over 10 years ago

  • Status changed from Feedback to Closed
  • Resolution set to up/downstream

These are bugs in the plugins and need to be fixed in each one individually.

It seems that these plugins (verified only for Multipart Split) are connecting several times to the editingStarted signal and fail to disconnect it again properly. Because for every existing connection a new connection is made, this results in an exponential growth of connections, so the number of times the code is invoked doubles with every startEditing call.

To validate, just remove the try/except statements around the connect/disconnect calls and watch the number of exceptions increase.

#17 Updated by Giedrius Vaivilavičius over 10 years ago

Matthias Kuhn wrote:

These are bugs in the plugins and need to be fixed in each one individually.

It seems that these plugins (verified only for Multipart Split) are connecting several times to the editingStarted signal and fail to disconnect it again properly. Because for every existing connection a new connection is made, this results in an exponential growth of connections, so the number of times the code is invoked doubles with every startEditing call.

To validate, just remove the try/except statements around the connect/disconnect calls and watch the number of exceptions increase.

Rectangle Ovals Digitizing  plugin also. ctrl-c interminal:

An error has occured while executing Python code:

Traceback (most recent call last):
File "/home/giedrius/.qgis2/python/plugins/rectovalDigit/rectovaldigit.py", line 209, in toggle
QObject.disconnect(layer,SIGNAL"),self.toggle)
KeyboardInterrupt

Python version:
2.7.3 (default, Jan 2 2013, 14:09:21)
[GCC 4.7.2]

QGIS version:
2.1.0-Master Master, exported

I noticed that after restarting qgis Rectangle ovals Digitizing disappears from the menu bars, but in Plugin Manager is marked as active. When reactivating it again appears in the menu bar. After this action, the problem does not recur.

#18 Updated by Matthias Kuhn over 10 years ago

When disabling signals get disconnected, and not necessarily get reconnected when re-enabling. E.g. the Multipart Split would reconnect it on the next change of the active layer.

I would like to kindly ask you to please report a bug for any affected plugin you find, so these issues can be ironed out.

Multipart Split #9291

#19 Updated by Paolo Cavallini over 10 years ago

Agreed, very important to open specific tickets for individual plugins. Please do it; ask for help if needed.

#20 Updated by Richard Duivenvoorde over 10 years ago

Ok, done for:

traceDigitize: https://sourceforge.net/apps/trac/geoing/ticket/3
and
rectangles oval digitizing: #9293

#21 Updated by Alexandre Neto over 10 years ago

I have made a patch for the multipart split plugin. Please confirm that it's working, and report in #9291.

I have also notice the same problem in Numerical Digitize and Numerical Vertex Edit plugins. I think they are already included in this ticket

Also available in: Atom PDF