Bug report #20872

QGIS 3.5 not loading or saving multi-dimensional arrays properly from PostGIS layers

Added by Cory Albrecht 12 months ago. Updated 9 months ago.

Status:Reopened
Priority:High
Assignee:Julien Cabieces
Category:Editing
Affected QGIS version:3.7(master) Regression?:Yes
Operating System: Easy fix?:No
Pull Request or Patch supplied:Yes Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:28691

Description

I have map layers stored in PostgreSQL where a field contains two dimensional arrays of values.

Alternate History=# select name,full_name,alternate_name from "Balance of Power"."timeline-countries" where full_name is not null and alternate_name is not null;
      name       |                                       full_name                                        |                            alternate_name                            
-----------------+----------------------------------------------------------------------------------------+----------------------------------------------------------------------
 New Spain       | {{en,"Viceroyalty of New Spain"}}                                                      | {{es,"Virreinato de la Nueva España"},{es,"Nueva España"}}
 Mexico          | {{es,"Imperio Mexicano"},{en,"Mexican Empire"}}                                        | {{es,"Imperio Mexicano"}}
 Mexico          | {{es,"Estados Unidos Mexicanos"},{en,"United Mexican States"}}                         | {{es,"Estados Unidos Mexicanos"}}
 Mexico          | {{es,"Estados Unidos Mexicanos"},{en,"United Mexican States"}}                         | {{es,"Estados Unidos Mexicanos"}}
 Alta California | {{es,"República de Alta California"},{en,"Republic of Alta California"}}               | {{en,"Upper California"}}
 New Spain       | {{en,"Viceroyalty of New Spain"}}                                                      | {{es,"Virreinato de la Nueva España"},{es,"Nueva España"}}
…

In QGIS 2.18.x those arrays appeared in Feature Attributes dialog in the PostgreSQL text format for specifying arrays, using curly braces as seen in the psql output fragment above, and you could add to the arrays by entering new items in that format.

But in 3.5 that is no longer the case. Instead, only a bit of the text is shown in the text widget of the dialog.

Further, when you try to update that field with as value like

{{en,"Cape Colony"}}

it gets saved in the database as

{"{{en,\"Cape Colony\"}}"}

scr_rqst.png (15.5 KB) Julien Cabieces, 2019-03-20 08:40 AM

scr_form.png (10 KB) Julien Cabieces, 2019-03-20 08:40 AM

scr_table.png (22.9 KB) Julien Cabieces, 2019-03-20 08:40 AM

Associated revisions

Revision 0753b144
Added by Julien Cabieces 10 months ago

fixes #20872 : Manage postgres multidimensionnal array

Revision 1957cf53
Added by Julien Cabieces 9 months ago

fixes #20872 : Manage postgres multidimensionnal array

Revision 2dbaa18f
Added by Julien Cabieces 9 months ago

fixes #20872 : Manage postgres multidimensionnal array

History

#1 Updated by Cory Albrecht 12 months ago

On loading the layer, the Qt messages windows displays lines like the following;

2018-12-22T19:48:10     WARNING    Cannot find separator: },{es,"Nueva España"}
2018-12-22T19:48:10     WARNING    Error parsing array: {es,"Virreinato de la Nueva España"},{es,"Nueva España"}
2018-12-22T19:48:10     WARNING    Cannot find separator: }
2018-12-22T19:48:10     WARNING    Error parsing array: {en,"Viceroyalty of New Spain"}

#2 Updated by Cory Albrecht 12 months ago

I just tried 3.4.2 and the issue is there, too.

#3 Updated by Cory Albrecht 12 months ago

Here is what the feature attributes dialog looks like in 2,18.25 for the same feature object.

#4 Updated by Cory Albrecht 11 months ago

Ping.

#5 Updated by Giovanni Manghi 11 months ago

  • Priority changed from Normal to High
  • Regression? changed from No to Yes

#6 Updated by Julien Cabieces 11 months ago

  • Assignee set to Julien Cabieces

#7 Updated by Julien Cabieces 11 months ago

  • Pull Request or Patch supplied changed from No to Yes

#8 Updated by Cory Albrecht 9 months ago

Still happening in 3.45, 3.60 and 3.7

#9 Updated by Giovanni Manghi 9 months ago

  • Affected QGIS version changed from 3.5(master) to 3.7(master)

#10 Updated by Julien Cabieces 9 months ago

Still happening in 3.45, 3.60 and 3.7

Proposed PR isn't accepted yep.

#11 Updated by Julien Cabieces 9 months ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

#12 Updated by Cory Albrecht 9 months ago

Julien Cabieces wrote:

Applied in changeset qgis|0753b144b92763b18021f1d706a1cf184323b1aa.

When will this be in a nightly? I just tried 3.7 and it's not there yet.

#13 Updated by Julien Cabieces 9 months ago

Here is some screenshots with current master

Does it look like what you're seeing on your version of QGIS?

Could you please check the QGIS version once launched in the help > About dialog.

#14 Updated by Cory Albrecht 9 months ago

Julien Cabieces wrote:

Does it look like what you're seeing on your version of QGIS?

I did not use the new list widget, just the plain old text widget, as shown in my screencaps.

Any text representation that the psql command-line client uses to either show the results of a query, or how one might type in the command by hand as a raw SQL, one should be able to simply copy-paste back and forth between a terminal and QGIS in either the Feature Attribute form's plain text widget or in the Attribute Table columns.

That way all possible PostGIS data types have some kind of minimal support, and special widgets can wait until somebody has the inclination and time to write them.

Could you please check the QGIS version once launched in the help > About dialog.

Whatever was in the PPA https://qgis.org/ubuntu-nightly Monday afternoon and evening, but I reverted to 3.6. I will check whatever is in the PPA once I get home from work.

#15 Updated by Julien Cabieces 9 months ago

  • Status changed from Closed to Reopened

I see, there is an additionnal issue when we use the plain text display (the list widget is working by the way).

I reopen the ticket, and I'll try to find a way to correct this.

#16 Updated by Julien Cabieces 9 months ago

Proposed PR : https://github.com/qgis/QGIS/pull/9610

Be aware that there is still a problem to deal with integer and float array in plain text.

Also available in: Atom PDF