Bug report #3648

Qgis 1.6/trunk, osgeo4w/standalone hangs/crashes when adding GRASS vectors in a QGIS project

Added by Giovanni Manghi over 9 years ago. Updated about 9 years ago.

Status:Closed
Priority:High
Assignee:-
Category:GRASS
Affected QGIS version: Regression?:No
Operating System:Windows Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed
Crashes QGIS or corrupts data: Copied to github as #:13707

Description

I spent the afternoon testing after Paolo told me that the last week he had many problems with GRASS vectors under Windows.

I can confirm, under both XP and Seven 32bit, QGIS crashes when it comes to simply operations with vectors such as:

*) importing a vector (shapefile) with v.in.ogr

*) adding an imported vector by clicking "view output" or using the proper button in the GRASS (toolbox) toolbar or using the QGIS GRASS toolbar

*) sometimes adding a vector qgis does not crash, but adding a second time it does

*) it seems to happen more with lines and polygons than with points and with vectors that are "big" (I'll attach a 13mb shapefile, in epsg 3003, that can be used to replicate easily the crashes).

All it seems pretty unpredictable and it does NOT happens under qgis 1.5 standalone, like #3646

http://rapidshare.com/files/453528319/freguesias_shp.zip

USER-PC.7z - debug (83.4 KB) Bill Williamson, 2011-05-07 10:12 AM

History

#1 Updated by Giovanni Manghi over 9 years ago

This is very frustrating... because the bug it is pretty unpredictable, but definitely there. It seems to happen more on osgeo4w than in the standalone version.

#2 Updated by Giovanni Manghi over 9 years ago

it does not seems to be related to size/complexity of vectors, it just happened with a very small line vector (3 lines).

#3 Updated by Giovanni Manghi over 9 years ago

I am pretty sure that at the beginning of February this wasn't a issue with both trunk/1.6 from osgeo4w. Since then it changed the GRASS version? Any backport to 1.6?

#4 Updated by Giovanni Manghi over 9 years ago

a downgrade to the previous GRASS release available in osgeo4w proven ineffective. On standalone 1.5 it seems to really work as expected.

#5 Updated by hellik - over 9 years ago

tested with osgeo4w-qgis-trunk (1.7.0-100) and the attached shapefile.

*) importing a vector (shapefile) with v.in.ogr

I've imported freguesias.shp several times by v.in.ogr in the same qgis-grass-session and several times in different qgis-grass-sessions. the import worked all the time without any qgis-crash. the only thing is that I thought the first time that the import has already finished (just too impatient ;-)), but it wasn't.

*) adding an imported vector by clicking "view output" or using the proper button in the GRASS (toolbox) toolbar or using the QGIS GRASS toolbar

tried this in several qgis-grass-sessions, adding the vector sometimes but not always crashes in an unreproducable way (maybe some memory issues?)

*) sometimes adding a vector qgis does not crash, but adding a second time it does

confirmed.

*) it seems to happen more with lines and polygons than with points and with vectors that are "big" (I'll attach a 13mb shapefile, in epsg 3003, that can be used to replicate easily the crashes).

All it seems pretty unpredictable and it does NOT happens under qgis 1.5 standalone, like #3646

if the attribute table is opened, by scrolling in the attribute entries, sometimes the attribute table freezes for a very short time

#6 Updated by Giovanni Manghi over 9 years ago

I've imported freguesias.shp several times by v.in.ogr in the same qgis-grass-session and several times in different qgis-grass-sessions. the import worked all the time without any qgis-crash. the only thing is that I thought the first time that the import has already finished (just too impatient ;-)), but it wasn't.

really weird. I have also updated qgis-trunk to 1.7.0-100 and now it seems that this problem become really rare. I can also import now the attached vector over and over almost without any problem. I got a crash but it was one among tens of attempts...

*) adding an imported vector by clicking "view output" or using the proper button in the GRASS (toolbox) toolbar or using the QGIS GRASS toolbar

tried this in several qgis-grass-sessions, adding the vector sometimes but not always crashes in an unreproducable way (maybe some memory issues?)

as above, now it works almost without problems. Crashes doing this operation become rare. More common when adding more then one vector at the same time

*) sometimes adding a vector qgis does not crash, but adding a second time it does

confirmed.

see above.

*) it seems to happen more with lines and polygons than with points and with vectors that are "big" (I'll attach a 13mb shapefile, in epsg 3003, that can be used to replicate easily the crashes).

All it seems pretty unpredictable and it does NOT happens under qgis 1.5 standalone, like #3646

after the first tests it become clear that there is (was?) no pattern.

if the attribute table is opened, by scrolling in the attribute entries, sometimes the attribute table freezes for a very short time

yes, VERY slow to scroll the attributes table. I don't see "short time freezes", if the table has just hundreds of records it freezes for many seconds.

#7 Updated by Giovanni Manghi over 9 years ago

really weird. I have also updated qgis-trunk to 1.7.0-100 and now it seems that this problem become really rare. I can also import now the attached vector over and over almost without any problem. I got a crash but it was one among tens of attempts...

but yet it seems fragile. After having imported the attached vector it crashed the first time I attempted a dissolve operation with v.dissolve. I made further attempts and no crashes.

By the way as for v.in.ogr when it crashes it does after having completed correctly the operation.

#8 Updated by Giovanni Manghi over 9 years ago

it is really unpredictable. Sometimes seems to work overall in a acceptable way, other times is very unstable/fragile and freezes crashes when importing/adding/removing (from qgis TOC) a vector that seemed to work fine minutes before. You may also want to try the vectors attached to #3650 and/or merge the two tickets, they seems related to me.

I'll appreciate if someone other than me and hellik can give it a try.

I'm using QGIS trunk/osgeo4w under Seven 32/64 bit.

#9 Updated by marisn - over 9 years ago

Replying to [comment:7 lutra]:

By the way as for v.in.ogr when it crashes it does after having completed correctly the operation.

Hm. Sounds familiar. It's a known issue when v.in.ogr runs on Windows. There where some cleanups in DBF area, that was a source of such crashes. Please test with current SVN.

#10 Updated by hamish - over 9 years ago

Replying to [comment:9 marisn]:

There where some cleanups in DBF area, that was a source of such crashes.
Please test with current SVN.

and/or, to test that theory, try using db.connect to swap to SQLite as the DB backend instead of DBF.

Hamish

#11 Updated by Giovanni Manghi over 9 years ago

and/or, to test that theory, try using db.connect to swap to SQLite as the DB backend instead of DBF.

test done using sqlite as backend. Proven ineffective, as the crashes and general instability are confirmed.

#12 Updated by Giovanni Manghi over 9 years ago

There where some cleanups in DBF area, that was a source of such crashes. Please test with current SVN.

the fixes were backported to grass 6.4 or are just available in svn (grass 6.5?)?

#13 Updated by hamish - over 9 years ago

Replying to [comment:9 marisn]:

There where some cleanups in DBF area, that was a source of such crashes.
Please test with current SVN.

Replying to [comment:12 lutra]:

the fixes were backported to grass 6.4 or are just available in svn (grass 6.5?)?

They've been backported to the 6.4 branch and are in 6.4.1rc2.

AFAICT there were two issues:
1) .dlls do not like to free() memory which was allocated by another function in another dll. the result is a crash even if the alloc history is kosher & correct. the work around seems to be to clone pass-through wrappers around free() in each library, and to use the library-specific free()-wrapper instead of the normal one. jef supplied a large patch tending to many of those, and Martin applied that patch.

2) https://trac.osgeo.org/grass/changeset/45652
Smells like a work-around. I'm not sure what component is not robust enough to deal with the altered dirsep. If it's a GRASS component I'd like to fix it to be more "read sloppy, write exact", if it's a Microsoft component, not much we can do about it. MS support for / as the dirsep seems to be about 80% and dates back to first QD-DOS. I'm a bit fuzzy on all the Windows voodoo, but I think system() was an important unsupported fn for the alternate dirsep. But we've now replaced almost all of the system() calls in GRASS with more robust alternatives. Maybe the dbf driver in GRASS 6.x still uses one?

anyway, please try against GRASS 6.4.1rc2, and,

Replying to [comment:10 hamish]:

to test that theory, try using db.connect to swap to SQLite as the DB backend
instead of DBF.

use the db.connect module to do that, see the man page for instructions. Variable path names used by that command do not need to be expanded by the user or the shell (so you can move the data without having to edit the database path links).

Hamish

#14 Updated by Paolo Cavallini over 9 years ago

I used db.connect, moving to SQLite, but I no joy

#15 Updated by Anne Ghisla over 9 years ago

Merged with #3364.

#16 Updated by Giovanni Manghi over 9 years ago

It is probable that this has been solved in GRASS 6.4.1

Waiting for the new version to enter osgeo4w to confirm it.

#17 Updated by Giovanni Manghi over 9 years ago

Well... after a few days it is possible to try QGIS with GRASS 6.4.1 (not yet qgis trunk, as it seems there is a packaging problem that causes qgis show warnings when adding vectors and crashes when adding rasters).

The overall feeling is that it works slightly better that 6.4.0, but the problems described here are still replicable.

Just use the QGIS sample dataset or the GRASS dataset: importing vectors with v.in.ogr works most of the times, but sometime it freezes qgis. The freeze happens always after the window "c:\\windows\\system32\\cmd.exe" pops out.

The problems are not of one particular vector and regardless the complexity/geometries: it happens to import one and it goes everything fine and importing it a second time freezes qgis.

The same happens when adding already imported vectors: most of the times it works ok but sometimes it causes qgis to freeze.

If this problems will be confirmed in qgis 1.7 I would say that it is not possible to close this ticket and that this remain as a major issue as it will make almost impossible to use GRASS in QGIS/Windows.

#18 Updated by Giovanni Manghi over 9 years ago

Hi all,

I finally made further tests, using qgis/osgeo4w (both 1.6 and trunk) and trunk/standalone created with creatensis.pl

Used a clean Windows Seven environment

Results

A) qgis 1.6 osgeo4w and 1.7 standalone:

adding a GRASS vector to the qgis canvas (via the GRASS browser or using the "view output" button, for example after having imported a vector) usually works, but at some point it will happen that adding a new vector qgis will hang and you will need to take it down.

After many hours of testing I realized that it is absolutely random: it doesn't depend on the vector, or the kind of geometry or the complexity/size or its projection. It is simply random. It can happen when adding the first vector in a project or after 20 tries. Sooner or later qgis will choke.

B) qgis trunk/osgeo4w:

qgis immediately crashes (not hangs as before) when adding a GRASS vector in a qgis project.

Forget about the vector I attched to this ticket, if you want to replicate the issue just use the QGIS sample dataset,

#19 Updated by Paolo Cavallini over 9 years ago

I think this is really blocking for release, as it makes half of GRASS unusable.

#20 Updated by Jürgen Fischer over 9 years ago

Replying to [comment:18 lutra]:

adding a GRASS vector to the qgis canvas (via the GRASS browser or using the "view output" button, for example after having imported a vector) usually works, but at some point it will happen that adding a new vector qgis will hang and you will need to take it down.

dbgview output?

B) qgis trunk/osgeo4w:

qgis immediately crashes (not hangs as before) when adding a GRASS vector in a qgis project.

Not reproducable here.

#21 Updated by Jürgen Fischer over 9 years ago

Replying to [comment:13 hamish]:

2) https://trac.osgeo.org/grass/changeset/45652
Smells like a work-around.

And shouldn't that be '\\\\' instead of '\\' at the very least?

#22 Updated by Bill Williamson over 9 years ago

Me too on Win7, although I have had other versions of GRASS on this machine.
QGIS 1.8.0.2 from the osgeo install.

#23 Updated by Markus Neteler about 9 years ago

  • Pull Request or Patch supplied set to No

Jürgen Fischer wrote:

Replying to [comment:13 hamish]:

2) https://trac.osgeo.org/grass/changeset/45652
Smells like a work-around.

And shouldn't that be '\\\\' instead of '\\' at the very least?

Yes, fixed now in 6.4.svn, 6.5.svn and 7.svn.

#24 Updated by Paolo Cavallini about 9 years ago

So now the issue is to be moved to osgeo packaging (upgrading GRASS there: which rev?)

#25 Updated by Markus Neteler about 9 years ago

Paolo Cavallini wrote:

So now the issue is to be moved to osgeo packaging (upgrading GRASS there: which rev?)

In GRASS, the fixes are
http://trac.osgeo.org/grass/changeset/47517 (GRASS7-trunk)
http://trac.osgeo.org/grass/changeset/47518 (GRASS 6.5.svn)
http://trac.osgeo.org/grass/changeset/47519 (GRASS 6.4.svn)

#26 Updated by Giovanni Manghi about 9 years ago

In GRASS, the fixes are
http://trac.osgeo.org/grass/changeset/47517 (GRASS7-trunk)
http://trac.osgeo.org/grass/changeset/47518 (GRASS 6.5.svn)
http://trac.osgeo.org/grass/changeset/47519 (GRASS 6.4.svn)

Giusppe Sucameli made to compile GRASS 6.4.2 svn for OSGeo4W and we tested it with QGIS trunk: The problem is still there, not solved and a real showstopper for using GRASS trough QGIS in Windows.

#27 Updated by Giovanni Manghi about 9 years ago

  • Assignee deleted (Redmine Admin)
  • Priority changed from Low to High

Giuseppe Sucameli (Faunalia) made to compile again the necessary packages (qgis-trunk and GRASS 6.4.2svn, latest revisions) for osgeo4w and so I
made the necessary tests.

The very good news is that now this issue seems gone.

I made a few tests and I never get a freeze, where before it was almost the rule. This is indeed a very good news as it was a showstopper for QGIS/GRASS user under Windows.

Now it would "just" a matter to have GRASS 6.4.2svn shipped AS SOON AS POSSIBLE in OSGeo4W.

http://ubuntuone.com/2lRa1ReYqBIoP9daJXjaU9

GRASS 6.4.2svn

http://ubuntuone.com/78DkQJB5qmuOHogTC3aXIf

QGIS trunk compiled for GRASS svn

http://ubuntuone.com/16JW6U3PXY19n22L6NZ6MB

launcher for the above QGIS binary

#28 Updated by Jürgen Fischer about 9 years ago

Giovanni Manghi wrote:

Now it would "just" a matter to have GRASS 6.4.2svn shipped AS SOON AS POSSIBLE in OSGeo4W.

Martin Landa maintains the GRASS packages in OSGeo4W. But is it really necessary to have a SVN version there? When will 6.4.2 be released?

#29 Updated by Giovanni Manghi about 9 years ago

Martin Landa maintains the GRASS packages in OSGeo4W.

yes we know, we are in contact with him

But is it really necessary to have a SVN version there? When will 6.4.2 be released?

he plans to add GRASS daily builds to osgeo4w as we have for QGIS. And yes, it is really necessary to have ASAP GRASS svn into ogseo4w because under Windows GRASS (6.4.1) under QGIS is not working (for the reasons explained in this ticket). We are also working on other Windows issues related to vector removal from mapsets.

The urgency comes from the fact that not every analysis tool is available natively in QGIS, so for many things it is needed the GRASS toolbox.

The new SAGA toolbox is being developed but AFAIK it doesn't work yet under Windows.

#30 Updated by Giovanni Manghi about 9 years ago

  • Status changed from Open to Closed
  • Resolution set to fixed

The issue has been solved upstream in GRASS 6.4.2 svn. We (Faunalia, Giuseppe Sucameli) made the packages to test the fix in OSGeo4w and the issue is fixed indeed.

The issue can be considered fixed, but only if/when GRASS 6.4.2 (svn) will be available in OSGeo4w (and then subsequently in the standalone installer).

Also available in: Atom PDF