Bug report #19315

QGIS 3.2 clean install misses many dependencies

Added by Kim Frankcombe over 2 years ago. Updated about 2 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Build/Install
Affected QGIS version:3.2 Regression?:No
Operating System:Ubuntu 16.04 Easy fix?:No
Pull Request or Patch supplied:No Resolution:worksforme
Crashes QGIS or corrupts data:Yes Copied to github as #:27143

Description

A clean install (all previous QGis versions purged and directories manually deleted) of Bonn on Ubuntu 16.04 misses many files.
We installed using sudo apt install qgis python-gis qgis-plugin-grass with repo https://qgis.org/ubuntugis and http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu, both xenial main. The installation ran without complaint and dpkg.log showed no errors.

On starting QGIS we got a stream of errors starting with an inability to find the grass configuration file, then a failure to authenticate. There were no predefined CRSs available and no data providers. We tried a purge and install again with no change. Followed with an apt install --reinstall and a dpkg-reconfigure but no joy. Luckily we had another Ubuntu machine that had worked through from 2.18 to the early adopters release of 3 and onto 3.2 so copied files from it across to the other machine. Much to my surprise it now works fine.

Directories we copied which were missing from the clean system were /usr/share/qgis/python, /usr/share/qgis/grass and /usr/share/qgis/svg (probably not required but needed for our use in the long run anyway)
We also copied /usr/share/qgis/resources/cpt-city-qgis-min/ which was not on the clean machine - again not sure whether that was important.
The clean machine was missing qgis.db, the two srs dbs and the symbology-style.xml file from /usr/share/qgis/resources/ so copied those.
Finally half the object code from /usr/lib/qgis/plugins/ was missing including all the data providers so copied them across as well
Now everything appears to work as expected.

I wondered if it might be a permissions issue for some of the install scripts but if so they are not being reported.

dpkg.log Magnifier - part of dpkg log (39.4 KB) Kim Frankcombe, 2018-07-04 05:11 AM

History

#1 Updated by Jürgen Fischer over 2 years ago

  • Status changed from Open to Feedback

Not reproducable.

A clean install (all previous QGis versions purged and directories manually deleted) of Bonn on Ubuntu 16.04 misses many files.
We installed using sudo apt install qgis python-gis qgis-plugin-grass with repo https://qgis.org/ubuntugis and http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu, both xenial main. The installation ran without complaint and dpkg.log showed no errors.

You probably installed python-qgis.

On starting QGIS we got a stream of errors starting with an inability to find the grass configuration file, then a failure to authenticate.

Quoting the error messages would have been nice.

Directories we copied which were missing from the clean system were /usr/share/qgis/python, /usr/share/qgis/grass and /usr/share/qgis/svg (probably not required but needed for our use in the long run anyway)

We also copied /usr/share/qgis/resources/cpt-city-qgis-min/ which was not on the clean machine - again not sure whether that was important.
The clean machine was missing qgis.db, the two srs dbs and the symbology-style.xml file from /usr/share/qgis/resources/ so copied those.
Finally half the object code from /usr/lib/qgis/plugins/ was missing including all the data providers so copied them across as well

The directories you mention should be in these packages (on your machine the package versions should just be 1:3.2.0+24xenial-ubuntugis instead of 1:3.2.0+99unstable):

$ dpkg -S /usr/share/qgis/python
python-qgis-common: /usr/share/qgis/python
$ dpkg -S /usr/share/qgis/grass
qgis-plugin-grass-common: /usr/share/qgis/grass
$ dpkg -S /usr/share/qgis/svg
qgis-providers-common: /usr/share/qgis/svg
$ dpkg -S /usr/share/qgis/resources/cpt-city-qgis-min/
qgis-providers-common: /usr/share/qgis/resources/cpt-city-qgis-min
$ dpkg -S /usr/lib/qgis/plugins
qgis-plugin-grass, qgis-providers, qgis, qgis-provider-grass: /usr/lib/qgis/plugins
$ dpkg -l qgis-plugin-grass qgis-providers qgis qgis-provider-grass python-qgis-common qgis-plugin-grass-common qgis-providers-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                  Version                 Architecture            Description
+++-=====================================-=======================-=======================-================================================================================
ii  python-qgis-common                    1:3.2.0+99unstable      all                     Python bindings to QGIS - architecture-independent files
ii  qgis                                  1:3.2.0+99unstable      amd64                   Geographic Information System (GIS)
ii  qgis-plugin-grass                     1:3.2.0+99unstable      amd64                   GRASS plugin for QGIS
ii  qgis-plugin-grass-common              1:3.2.0+99unstable      all                     GRASS plugin for QGIS - architecture-independent data
ii  qgis-provider-grass                   1:3.2.0+99unstable      amd64                   GRASS provider for QGIS
ii  qgis-providers                        1:3.2.0+99unstable      amd64                   collection of data providers to QGIS
ii  qgis-providers-common                 1:3.2.0+99unstable      all                     collection of data providers to QGIS - architecture-independent files

I wondered if it might be a permissions issue for some of the install scripts but if so they are not being reported.

Me too. Can you verify which packages got installed?

#2 Updated by Kim Frankcombe over 2 years ago

Thanks Jürgen

Some answers;

On checking the CL in the terminal I see that yes it was python-qgis not python-gis that was installed. They had cut and pasted from the QGIS website so got it right whereas I had written from memory and missed it, sorry.

I did feel guilty about not quoting the error messages but hadn't written them down as they happened. However I had googled them so I got the user to go back through the Firefox history for the past couple of days google searches and list them. In order of appearance they are;

The config file (/usr/share/qgis/grass/modules/default.qgc) not found 

Authentication System: DISABLED. Resources authenticating via the system can not be accessed

ImportError: No module named 'pyplugin_installer'

No error was given for the lack of CRS's or data handlers they just didn't show when you went to the relevant dialogue.

$ dpkg -S /usr/share/qgis/python
python-qgis-common: /usr/share/qgis/python
$ dpkg -S /usr/share/qgis/grass
qgis-plugin-grass-common: /usr/share/qgis/grass
$ dpkg -S /usr/share/qgis/svg
qgis-providers-common: /usr/share/qgis/svg
$ dpkg -S /usr/share/qgis/resources/cpt-city-qgis-min/
qgis-providers-common: /usr/share/qgis/resources/cpt-city-qgis-min
$ dpkg -S /usr/lib/qgis/plugins
qgis-provider-grass, qgis-plugin-grass, qgis: /usr/lib/qgis/plugins
$ dpkg -l qgis-plugin-grass qgis-providers qgis qgis-provider-grass python-qgis-common qgis-plugin-grass-common qgis-providers-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  python-qgis-co 1:3.2.0+24xe all          Python bindings to QGIS - archite
ii  qgis           1:3.2.0+24xe amd64        Geographic Information System (GI
ii  qgis-plugin-gr 1:3.2.0+24xe amd64        GRASS plugin for QGIS
ii  qgis-plugin-gr 1:3.2.0+24xe all          GRASS plugin for QGIS - architect
ii  qgis-provider- 1:3.2.0+24xe amd64        GRASS provider for QGIS
ii  qgis-providers 1:3.2.0+24xe amd64        collection of data providers to Q
ii  qgis-providers 1:3.2.0+24xe all          collection of data providers to Q

Unfortunately this is all on the "patched" machine so we would have got a different result had I tried it first thing this morning.

Does that help?

Cheers
Kim

#3 Updated by Jürgen Fischer over 2 years ago

  • Priority changed from High to Normal

Kim Frankcombe wrote:

The config file (/usr/share/qgis/grass/modules/default.qgc) not found
Authentication System: DISABLED. Resources authenticating via the system can not be accessed
ImportError: No module named 'pyplugin_installer'

/usr/share/qgis/grass/modules/default.qgc is in qgis-plugin-grass-common and pyplugin_installer in python-qgis-common.

You should see the it in the packages:

$ dpkg -S default.qgc
qgis-plugin-grass-common: /usr/share/qgis/grass/modules/default.qgc
$ dpkg --contents /var/cache/apt/archives/qgis-plugin-grass-common_1%3a3.2.0+*all.deb | grep default.qgc 
-rw-r--r-- root/root     26758 2018-06-22 09:00 ./usr/share/qgis/grass/modules/default.qgc

So after installing they should be there. If dpkg -S has stuff that the file system doesn't have there seems to be a puzzling local instead of a packaging issue.

#4 Updated by Kim Frankcombe over 2 years ago

Remember that after the initial installation and re-installation there was no grass or python directory in /usr/share/qgis/ we had to add them manually.

$ dpkg -S default.qgc
qgis-plugin-grass-common: /usr/share/qgis/grass/modules/default.qgc
$ dpkg --contents /var/cache/apt/archives/qgis-plugin-grass-common_1%3a3.2.0+*all.deb | grep default.qgc
dpkg-deb: error: failed to read archive '/var/cache/apt/archives/qgis-plugin-grass-common_1%3a3.2.0+*all.deb': No such file or directory

Finally an error!

$ ls -l /var/cache/apt/archives/qgis*
-rw-r--r-- 1 root root  6519822 Jan 20 12:37 /var/cache/apt/archives/qgis_1%3a2.18.16+24xenial_amd64.deb
-rw-r--r-- 1 root root  6853746 May 19 09:52 /var/cache/apt/archives/qgis_1%3a3.0.3+24xenial-ubuntugis_amd64.deb
-rw-r--r-- 1 root root 10898914 May 19 09:52 /var/cache/apt/archives/qgis-common_1%3a3.0.3+24xenial-ubuntugis_all.deb
-rw-r--r-- 1 root root  3246386 May 19 09:52 /var/cache/apt/archives/qgis-providers_1%3a3.0.3+24xenial-ubuntugis_amd64.deb
-rw-r--r-- 1 root root  2527282 May 19 09:52 /var/cache/apt/archives/qgis-providers-common_1%3a3.0.3+24xenial-ubuntugis_all.deb
-rw-r--r-- 1 root root  1759874 Jan 20 12:37 /var/cache/apt/archives/qgis-server_1%3a2.18.16+24xenial_amd64.deb
-rw-r--r-- 1 root root  2094140 May 19 09:52 /var/cache/apt/archives/qgis-server_1%3a3.0.3+24xenial-ubuntugis_amd64.deb

It's not there

I'm not sure why they decided to install the server but I hav't got them to delete it yet as I wasn't sure what its dependencies were.

#5 Updated by Jürgen Fischer over 2 years ago

Kim Frankcombe wrote:

Remember that after the initial installation and re-installation there was no grass or python directory in /usr/share/qgis/ we had to add them manually.

I thought you copied the files, because the files were missing. You also said that qgis-plugin-grass was installed without an error - which implies that qgis-plugin-grass-common was also installed. Which is also what your package listing shows:

ii  qgis-plugin-gr 1:3.2.0+24xe all          GRASS plugin for QGIS - architect

That should be qgis-plugin-grass-common. If it was a freshly (re-)installed, it should still be in the cache - unless you meanwhile ran apt-get clean or the like. Odd that it isn't there anymore.

I'm not sure why they decided to install the server but I hav't got them to delete it yet as I wasn't sure what its dependencies were.

qgis and qgis-server both depend on eg. qgis-providers corresponding to their version. So installing qgis should also have initiate an upgrade of qgis-server to 3.2.

The question still is, why qgis-plugin-grass-common among others was apparently installed, although the files were not there. Please run debsums -s qgis-plugin-grass-common which will verify the the installed files are available and unmodified (without the package name it'll check all packages).

#6 Updated by Kim Frankcombe over 2 years ago

Yes we copied the whole directories from my machine because they were missing from the clean machine. Also true that no errors were reported either to the terminal or dpkg log. No they did not run apt-get clean, I've been trying to encourage them to use apt rather than apt-get. They may have run apt autoclean after installing the new kernel update this morning though. However why would that selectively remove qgis-grass-plugin-common and not older packages like the V2.18 packages in there?

Could it be a typo or broken package list on the ubuntugis xenial repo?

I'll answer the last question tomorrow when I'm back in the office. Thanks for your attention to this.

#7 Updated by Kim Frankcombe over 2 years ago

OK. debsums -s qgis-plugin-grass-common returns nothing. On checking with my machine where I worked through from 2.18 to the early adopter to 3.2 successively I get the same result.

In case it helps I've attached the section of the dpkg log from the clean machine and it shows qgis-plugin-grass as installed. apt list sees ...-common

$ sudo apt list |grep grass-common

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

qgis-plugin-grass-common/xenial,xenial,now 1:3.2.0+24xenial-ubuntugis all [installed]

#8 Updated by Jürgen Fischer over 2 years ago

Kim Frankcombe wrote:

OK. debsums -s qgis-plugin-grass-common returns nothing. On checking with my machine where I worked through from 2.18 to the early adopter to 3.2 successively I get the same result.

In case it helps I've attached the section of the dpkg log from the clean machine and it shows qgis-plugin-grass as installed. apt list sees ...-common

Not really. Except for qgis-common, which doesn't have any of the missing files, none of the -common packages are mentioned in dpkg.log - although they are installed in the current version. So it's an older problem…

#9 Updated by Jürgen Fischer about 2 years ago

  • Subject changed from QGIS 3.2 clean install misses many dependencies to QGIS 3.2 clean install misses many dependencies
  • Status changed from Feedback to Closed
  • Resolution set to worksforme

Looks like an local issue to me.

#10 Updated by Kim Frankcombe about 2 years ago

For the benefit of anyone suffering from the same symptoms. Check the ownership of /usr/lib I can not be sure that this was the cause of this particular problem but I did notice that instead of being root:root it was local_user:local_user. At the time I though the user most likely responsible but over the next 3 months all of our four similarly configured systems had the same thing happen with mine being the second. I knew I had not changed the ownership so it has something to do with one of the packages we have installed. The final system to suffer the same problem only showed up last week so I'm at a loss to understand what it is that is is modifying the ownership. Either way if you see these symptoms check that all the key subdirectories of /usr are owned by root.

HTH
Kim

Also available in: Atom PDF