Bug report #14078

Incorrect loading of csv files

Added by Paolo Cavallini about 4 years ago. Updated about 4 years ago.

Status:Closed
Priority:Normal
Assignee:Sebastian Dietrich
Category:Data Provider
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:up/downstream
Crashes QGIS or corrupts data:No Copied to github as #:22088

Description

The first row of the attached file is not interpreted as column headings.
Other, apparently similar, files are loaded fine.

test0.csv Magnifier (223 Bytes) Paolo Cavallini, 2016-01-08 08:17 AM

QgisLoadCsvWithHeaders.png - Screenshot dialog "Create CSV layer" (34.6 KB) Sebastian Dietrich, 2016-01-09 01:17 PM

QgisCsvLayerFields.png - Screenshot attribute table of created layer (25.5 KB) Sebastian Dietrich, 2016-01-09 01:17 PM

test1.csv Magnifier (517 Bytes) Paolo Cavallini, 2016-01-10 11:39 PM


Related issues

Related to QGIS Application - Feature request #14093: Prefer the QGIS-providers over the OGR-provider Open 2016-01-12

History

#2 Updated by Nyall Dawson about 4 years ago

  • Status changed from Open to Feedback

#3 Updated by Paolo Cavallini about 4 years ago

  • Status changed from Feedback to Open

Loading through the text delimited plugin is OK, but simply dragging it into the cnavas shows the issue reported. Similar files load fine.

#4 Updated by Sebastian Dietrich about 4 years ago

OK, now I could reproduce in master. Turns out this is a GDAL/OGR issue.

Note that files are loaded through GDAL/OGR when
  • dropping them onto QGIS
  • specifying them on the commandline
  • double-clicking them on MAC

The meta-data will show OGR data provider.

This is different from opening a file using the Create a Layer from a Delimited Text File dialog where the provider will be Delimited text data provider.

The GDAL/OGR .csv-driver uses some logic to detect whether the first line is a header or not. In your case it finds some all numeric values in the first row (D1 to D31, D is considered a hex-digit) and treats it as a data record instead of as a header.

The easiest solution would be to put the column names within double quotes, at least the offending ones.
Bad:

D1,D2
10,20

Good:
"D1","D2" 
10,20

Can you confirm the similar files that load fine have different, non-numeric column names?

#5 Updated by Paolo Cavallini about 4 years ago

The attached test1 loads fine.
Thanks for your feedback.
Perhaps worth opening a ticket on GDAL.

#6 Updated by Giovanni Manghi about 4 years ago

  • Category changed from Data Provider/Delimited Text to Data Provider

#7 Updated by Sebastian Dietrich about 4 years ago

  • Status changed from Open to Closed
  • Resolution set to up/downstream

Paolo Cavallini wrote:

The attached test1 loads fine.
Thanks for your feedback.
Perhaps worth opening a ticket on GDAL.

You're welcome.

You might consider opening a feature request to prefer the QGIS-providers over the OGR-provider for files that are supported by both, like CSVs.

#8 Updated by Sebastian Dietrich about 4 years ago

  • Assignee set to Sebastian Dietrich

#9 Updated by Paolo Cavallini about 4 years ago

Done both, thanks.

#10 Updated by Even Rouault about 4 years ago

Fixed in GDAL per https://trac.osgeo.org/gdal/ticket/6305. I'd note that the OGR CSV driver has gained editing capabilities in GDAL 2.1dev (specificaly to be used through QGIS), whereas AFAICS the QGIS CSV provider is read-only. It would be a shame that using the OGR CSV driver is no longer possible.

Also available in: Atom PDF