QGIS » QGIS Application

This is related to definitions of field types in the QGIS provider implementation which do not match the field type which is created by the driver in the background.

Therefore it is essentially to have a list of affected providers / field type pairs.

OGR Provider : Date is fixed in d758340c
OGR Provider : DateTime is not supported by all OGR drivers. E.g. Shapefile does not support it. Therefore, this fieldtype needs special attention and we need to find eigher a way to query OGR drivers if they support this type or create and maintain a blacklist/whitelist.

OGR Provider : DateTime Blacklisted this datatype for ShapeFile in a8a1090. Please help to extend the list for other formats which do not support this field type. Querying OGR for support is unfortunately not possible according to #gdal.

For what it can matter I noticed that on a postgis layer there is no error just when adding "numeric" and "decimal" fields (cannot test real and double because of #8480) and it gives the error when choosing any other type.

In my opinion among the list of blockers this issue is the really important one as it is a regression of a pretty basic (q)qgis function. Any chance to have this fixed (in time)?

thanks in advance

Giovanni Manghi wrote:

For what it can matter I noticed that on a postgis layer there is no error just when adding "numeric" and "decimal" fields (cannot test real and double because of #8480) and it gives the error when choosing any other type.

on SL the error shows when adding integer/double column types, but not "text".

2f2e088 solves this partially

The other half of the problem is, that the [qgis] spatialite provider indicates, that you can define length and precision of these fields between 0 and 20, while sqlite does in fact not support this. Therefore afterwards any column will have a default length (0) and default precision (0) assigned.

I would propose to remove the option to define these (useless) options from the spatialite drivers.

For now: As long as you don't change them (i.e. leave them set to 0) you should not get any warnings.

Hi Matthias

For now: As long as you don't change them (i.e. leave them set to 0) you should not get any warnings.

In SL I get the error also when creating integer fields even when leaving 0 as width, does the above patch fix this case? Thanks!

Giovanni Manghi wrote:

In SL I get the error also when creating integer fields even when leaving 0 as width, does the above patch fix this case? Thanks!

Exactly

Matthias Kuhn wrote:

Giovanni Manghi wrote:

In SL I get the error also when creating integer fields even when leaving 0 as width, does the above patch fix this case? Thanks!

Exactly

sweet. I guess that fixing postgresql is a different beast(?). thanks again.

Fixed the trivial ones in a9c05d7

• Postgres: Field length and precision are not correctly read from the database. (For char type with configurable length)
• Spatialite: Field length and precision are configurable although they should not be. (Easy fix, but I need confirmation that the length is really not used by anything)

Giovanni, could you also test the other providers, that the changes did not introduce any errors which have been masqueraded by the add attribute dialog behavior before. Thank you very much.

Giovanni, could you also test the other providers, that the changes did not introduce any errors which have been masqueraded by the add attribute dialog behavior before. Thank you very much.

Hi Matthias, for the moment I re-tested postgis, SL and shapes.

Postgis:
now I see the error only when adding a char (text, fixed length) column

SL:
I see no errors when adding columns and leave width/precision to 0, but when adding an integer column I see that the values stored are still decimal

SHP:
no problems

Fixed the spatialite problems in 249526b

No other value than 0 is allowed for integer and real columns.
I'm pretty confident, that this is ok. Length / precision settings will be lost anyway when saving the fields to the provider according to https://www.sqlite.org/datatype3.html#storageclasses

I was hesitating to backport this to the 2.0 fixes but if there is demand, I can do so. The task itself is easy.

Matthias Kuhn wrote:

Is this still an issue?

Looking at my earlier comments, I didn't take care of this one: "Postgres: Field length and precision are not correctly read from the database. (For char type with configurable length)"

Can somebody confirm this (or are there others affected)

when I try to add a "char" column on a PostGIS layer, on save I get

Could not commit changes to layer sele

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 53 is not the same!

Giovanni Manghi wrote:

Matthias Kuhn wrote:

Is this still an issue?

Looking at my earlier comments, I didn't take care of this one: "Postgres: Field length and precision are not correctly read from the database. (For char type with configurable length)"

Can somebody confirm this (or are there others affected)

when I try to add a "char" column on a PostGIS layer, on save I get

Could not commit changes to layer sele

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 53 is not the same!

PostGIS:

still getting error when saving a newly created char column, example

Could not commit changes to layer evora

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 8 is not the same!

after discarding the edits the column seems that can be updated without errors

Giovanni Manghi wrote:

I have seen this again, on master, working on a shapefile

Errors: SUCCESS: 1 attribute(s) added.

ERROR: field with index 8 is not the same!

And which types did you use?

And which types did you use?

will try to replicate again and let you know.

And which types did you use?

just got it when after adding an integer column to a point file, and filling it with field calculator and the rand function.

and just seen this on a PostGIS layer, with QGIS master

Errors: SUCCESS: 1 attribute(s) added.

ERROR: the count of fields is incorrect after addition/removal of fields!

When updating this issue, please provide details not only about the provider, but also the data type you used, this considerably increases the chances of having the issue fixed ;-)

Matthias Kuhn wrote:

When updating this issue, please provide details not only about the provider, but also the data type you used, this considerably increases the chances of having the issue fixed ;-)

Hi Matthias,

will do it. Anyway I have already done it (sometimes), see for example in #8481-22

cheers!

PS
the problem with this issue is that is not easily replicable.

Giovanni Manghi wrote:

will do it. Anyway I have already done it (sometimes), see for example #8481-22

Well, the problem is the mapping between QgsField (name, type, precision) to a provider specific type on creationg of the field and the retrieval of field definitions from a provider and their mapping back to QgsField. After the field is created, the provider retrieves the new field definitions and checks it the QgsField created from that, matches what you originally entered. A mismatch produces the error message.

For instance the postgres provider didn't care to retrieve the length of character fields and therefore each character field with a given length constraint would produce that error - because the provider wouldn't reflect the length constraint in the QgsField it reported (and in turn not limit edit widget in forms to that given length later and lead to a commit error if the user entered a value exceeding that length). That's what the related commit above fixes.

The spatialite fix however removes the ability to specify a length constraint for float and integer field in spatialite databases as spatialite would ignore and drop constraint anyway and lead to the same error message - although it's actually a different problem.

With other types and/or providers there might be other different mismatches (the name might change case or be truncated to a certain length, types might change (eg. when a integer width exceeds some limit, it might be turned into a real type). So to reproduce we need to know what combination was actually used (point file and integer doesn't really cut it ;)).

But reporting and fixing another combination that produces this error wouldn't 'fix' this ticket. Maybe we should close it in favor of individual tickets for each failing combination we find.

maybe we should put some more details into the log containing dumps of the specified and provider reported field definitions, so there is a bit more information about what failed.

Giovanni Manghi wrote:

It is a few days now that I don't see anymore this kind of messages, before it was quite common.

Maybe we can lower the priority of this ticket?

As said to me this are individual issues - although there is always the same error message popping up. So I close this and open new ones when a new combinations where creation and retrieval of types disagree is identified.