Feature request #13814

Processing: use sqlite/spatialite as intermediate format

Added by barryrowlingson - over 4 years ago. Updated almost 2 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Processing/Core
Pull Request or Patch supplied:No Resolution:fixed/implemented
Easy fix?:No Copied to github as #:21839

Description

In Processing, when a layer is passed to an R script, it is first written to a shapefile and then read into R as a SpatialDataFrame before running the user script. In this process, attribute names are truncated to 10 characters.

Attribute names provided to the script from the script parameters are the full-length names and so extracting the values from the SpatialDataFrame fails if the attribute names are longer than ten characters.

Fix would seem to be either use Spatialite as an intermediate format (or anything else that can handle longer names) or write the full attribute names to some metadata file, then after the pre-script code loads the shapefile into the SpatialDataFrame, to read that metadata and assign the original names over the possibly munged ones from the shapefile.

The following test script illustrates the problem when called with an attribute name longer than ten chars:

##Vector Processing=group
  1. showplots
    ##Layer=vector
    ##X=Field Layer

print(names(Layer))
if(! X in names(Layer)){
message("Error - attribute ",X," not in ",paste(names(Layer),collapse=","))
}
print(X)
hist(LayerX)

History

#1 Updated by barryrowlingson - over 4 years ago

Properly formatted test script:

##Vector Processing=group
## showplots
##Layer=vector
##X=Field Layer

print(names(Layer))
if(! X %in% names(Layer)){
 message("Error - attribute ",X," not in ",paste(names(Layer),collapse=","))
 }
print(X)
hist(Layer[[X]])

#2 Updated by Giovanni Manghi over 4 years ago

  • Status changed from Open to Feedback
  • Category set to Processing/Core

this ticket should change title and description to make it more general, it is a fact that Processing uses shapes as output/intermediate format.

#3 Updated by Giovanni Manghi over 4 years ago

  • Status changed from Feedback to Open
  • Subject changed from Attribute names truncated to 10 chars when passed to R scripts to Processing: use sqlite/spatialite as intermediate format
  • Tracker changed from Bug report to Feature request

#4 Updated by Alexander Bruy over 3 years ago

We stick to Shapefiles because this is format which supported by all 3rd party tools (e.g. SAGA, GRASS, OTB etc). Using SpatiaLite will break some of them.

#5 Updated by Giovanni Manghi almost 3 years ago

  • Easy fix? set to No

#6 Updated by Nyall Dawson almost 2 years ago

  • Description updated (diff)
  • Status changed from Open to Closed
  • Resolution set to fixed/implemented

GPKG is used now

Also available in: Atom PDF