Bug report #19071

Discard Empty Fields: behaviour, default status, name

Added by Simon Dedman about 2 years ago. Updated over 1 year ago.

Category:Data Provider/Delimited Text
Affected QGIS version:3.0.3 Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:26901


Imagine a csv with columns:
lat, long, 2016_data, 2017_data, 2018_data, Total
Where a selection of the 2016:18 cells are blank.
If one imports this as a layer and doesn't notice the 'discard empty fields' option is on (by default, 2.18.20 & 3.0.3), Qgis will not discard empty FIELDS, as defined elsewhere (e.g. layer properties) as being the columns of data, but will discard empty CELLS.... and then slide all cells left to fill the blanks, such that e.g.:

lat, long, 2016_data, 2017_data, 2018_data, Total
la1 lo1 2 =blank= 3 5

would become
lat, long, 2016_data, 2017_data, 2018_data, Total
la1 lo1 2 3 5 =blank=

and therefore plotting Total would show you that there are no data when you can see from your original csv that there certainly are.
This may be the correct implementation for fields for other file types but is very much counterintuitive and bad for csvs. Please could:
1. 'discard empty fields' be turned off by default, either for csvs or all
2. 'discard empty fields' be renamed to e.g. 'remove blank cells', ideally with explanatory tooltip
3. This behaviour be corrected so blank fields i.e. columns of data are removed but not cells, or rather not with the post-removal rearrangement of other fields' data.



#1 Updated by Alexander Bruy over 1 year ago

  • Operating System deleted (Win10)
  • Category changed from Processing/Core to Data Provider/Delimited Text

Also available in: Atom PDF