Bug report #19490

Strange problems exporting a layer to file, restoring it to map

Added by Loren Amelang over 5 years ago. Updated about 5 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Project Loading/Saving
Affected QGIS version:3.2 Regression?:No
Operating System:Windows 10 Creators with 3000x2000 screen and 175% scaling Easy fix?:No
Pull Request or Patch supplied:No Resolution:no timely feedback
Crashes QGIS or corrupts data:No Copied to github as #:27318

Description

After yesterday's file corruption debacle, I'm trying to make backups of complex creations.

Right-clicked a layer XSCutLines, Export -> Save Features As... (with load to map checked)

After Exporting a copy to shapefile...

Attribute Table data looked good:
---
wkt_geom XsecID ReachID Station RiverCode ReachCode LeftBank RightBank LLength ChLength RLength NodeName DtmID
MultiLineString ((6186583.15544720087200403 2243164.85563876014202833, 6186209.05874442309141159 2243863.58171171601861715)) 69 10 4253.06211053154 WALKER GULCH 2
...
---

Attribute Tables and data match, copy and original, include stationing
Original draws stationing
Copy draws nothing onscreen, despite showing 71 items

Removed layer and reloaded from shapefile, it shows zero items, but now draws properly
But mousing over its layer name shows the original RiverGIS PostGIS table as source, not the shapefile!

Removed again, and now nothing happens when trying every route to Add it back...

Saved project, closed and restarted QGIS
Still won't load saved file

--> It saved only XSCutLines Gulch 180725 OK.shp and .dbf, not all six files!

Layer -> SaveAs again, same dialog, same filename, this time it saved six files
Now it can be Added or drug into QGIS, shows shapefile as source
and it draws the lines, but not the station numbers
Numbers are clearly in the table...

Maybe there is some detail in the PostGIS db that doesn't export to a shapefile?
Database -> DB Manager -> Table -> Export to vector file -> GeoPackage
Seemed to work, could load it, lines but no station numbers drawn

Is there some way to make backup copies of PostGIS tables? I'm not finding anything even in DB Manager...

Maybe the RiverGIS plugin is actively drawing those station numbers for the files it remembers it wrote?
I'd assumed it was only active when I asked it to do something.
Its interface isn't even open now, and the numbers are still drawn for the file it made.

How can I get a backup of that file that works like the original?
(See #19489 for the reason I'm paranoid...)

Backups Attributes.JPG - First four Attribute Tables aligned (333 KB) Loren Amelang, 2018-07-27 03:35 AM

History

#1 Updated by Jürgen Fischer over 5 years ago

  • Description updated (diff)

#2 Updated by Giovanni Manghi over 5 years ago

  • Status changed from Open to Feedback
  • Description updated (diff)

I cannot replicate (trying also from PostGIS to Shapefile), seems likely a local issue of your QGIS installation or maybe something with your database.

There are tons of ways you can make copies (in different formats) of your PostGIS tables: "save features as..." is one, the DB manager had an export (and import) functionality. In Processing you can use "convert format" that makes use of ogr2ogr under the hood. And I'm sure there are probably other ways.

#3 Updated by Loren Amelang over 5 years ago

I have not duplicated the early problems with saving only .shp and .dbf, instead of all six files. But I have noticed several problems with my later efforts. The Attribute Table views are overlaid in the screenshot, with matching data rows highlighted and copied below.

Comparing the copied data from the Attribute Tables in the screenshot to the original file:

XSCutlines33 multiline:

(Original did not have "Station" when this was saved)

wkt_geom    XsecID    ReachID    Station    RiverCode    ReachCode    LeftBank    RightBank    LLength    ChLength    RLength    NodeName    DtmID
MultiLineString ((6182445.5257692439481616 2245399.48681368958204985, 6182721.15499151404947042 2245816.65408449433743954))    2    10        WALKER GULCH    2                            
MultiLineString ((6182464.27876121643930674 2245387.57281608274206519, 6182762.44966983143240213 2245788.93812257470563054))    1    10        WALKER GULCH    2                            

Changed to MultiLineString!
Rows re-ordered
Remaining data blank, not zero

Gulch multiline:

wkt_geom    XsecID    ReachID    Station    RiverCode    ReachCode    LeftBank    RightBank    LLength    ChLength    RLength    NodeName    DtmID
MultiLineString ((6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513))    1    10    6998.78176718143    WALKER GULCH    2                            
MultiLineString ((6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848))    2    10    7144.87310829749    WALKER GULCH    2                            

Changed to MultiLineString!
Rows re-ordered
Remaining data blank, not zero

Gulch gpkg:

wkt_geom    fid    XsecID    ReachID    Station    RiverCode    ReachCode    LeftBank    RightBank    LLength    ChLength    RLength    NodeName    DtmID
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    1    10    6999    0    2                                
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)    3    10    7378    0    2                                
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848)    2    10    7145    0    2                                

Added the "fid" label, didn't seem to add a data column
Reduced float "Station" to integer
"RiverCode" text changed to zeros
Remaining data blank, not zero

Original:
(I spaced out and edited labels here to fit columns - but your "pre" is different from my TextPad!)

wkt_geom                                                                                                             XsecID    ReachID    Station              RiverCode    RchCode    LftBank    RgtBank    LLength    ChLngth    RLength    NdeName    DtmID
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    1    10    6998.78176718143    WALKER GULCH    2    0    0    0    0    0        0
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)    3    10    7378.2609221161     WALKER GULCH    2    0    0    0    0    0        0
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848)    2    10    7144.87310829749    WALKER GULCH    2    0    0    0    0    0        0

I'm not positive changes in file details are the explanation, but I get errors from RiverGIS when I try to use the changed backup files:

<https://postgis.net/docs/manual-2.4/ST_LineLocatePoint.html>
ST_LineLocatePoint — Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point, as a fraction of total 2d line length.
Synopsis
float8 ST_LineLocatePoint(geometry a_linestring, geometry a_point);

Here's the error message:

ST_LineLocatePoint(path.geom, ST_Intersection(r.geom, path.geom))

2st arg isn't a point

So the first path.geom didn't error, thus ST_Intersection(r.geom... must be the problem

<https://postgis.net/docs/manual-2.4/ST_Intersection.html>

ST_Intersection — (T) Returns a geometry that represents the shared portion of geomA and geomB.
SELECT ST_AsText(ST_Intersection('POINT(0 0)'::geometry, 'LINESTRING ( 2 0, 0 2 )'::geometry));
st_astext

--> So it can return a point, but also a point set, or other geometry...

I'm way out of my depth here, but it seems like sending it a MultiLineString triggers the error, even if the actual string seems to have only two points in the data. (A lot of other quirks in the source file also trigger the same error, so it is hard to be sure...)

So I just installed QGIS 3.2.1. Let's see what it makes...

Select Original file
Export -> Save Features As
Make new folder, open it, enter file name, ok
Saved six files, shows on map, but no station numbers drawn, changed to MultiLineString
Same data as before

Database -> DB Manager -> Table -> Export to vector file -> GeoPackage
Shows on map, still LineString, but station numbers all zero in Table, integer in actual data
Numbers not drawn
Same data as before
Maybe the function that draws Station numbers insists on a float value, not integer?

Processing -> Toolbox -> GDAL -> Vector conversion -> Convert format -> .gpkg
Hmmm... It shows three listings for my XSCutLines source file... They have identical popup descriptions, in the Postgre database.
Picked first one, chose .gpkg output file, ok...
Yet a different row order
Hey - it saved Station as a float, in a gpkg! And they show in the Attribute Table!
But the numbers still are not drawn on the screen:

wkt_geom    fid    XsecID    ReachID    Station    RiverCode    ReachCode    LeftBank    RightBank    LLength    ChLength    RLength    NodeName    DtmID
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848)    49    2    10    7144.87310829749    WALKER GULCH    2                            
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)    48    3    10    7378.2609221161    WALKER GULCH    2                            
LineString (6182872.53668023273348808 2245141.03621458914130926, 6182984.4070006487891078 2245638.22226869175210595)    51    14    10    8625.57504443754    WALKER GULCH    2                            
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    50    1    10    6998.78176718143    WALKER GULCH    2                            

Same but export to .sqlite:
Much smaller file, column names all lowercase, but Station still float, shows in Table
Same data order as gpkg export.
Still no numbers drawn.

wkt_geom    ogc_fid    xsecid    reachid    station    rivercode    reachcode    leftbank    rightbank    llength    chlength    rlength    nodename    dtmid
LineString (6183893.3692371491342783 2244500.47361892322078347, 6184310.32890845648944378 2244814.4953377665951848)    49    2    10    7144.87310829749    WALKER GULCH    2                            
LineString (6183766.44361504912376404 2244575.61532213492318988, 6184043.07167843915522099 2244992.12092179106548429)    48    3    10    7378.2609221161    WALKER GULCH    2                            
LineString (6182872.53668023273348808 2245141.03621458914130926, 6182984.4070006487891078 2245638.22226869175210595)    51    14    10    8625.57504443754    WALKER GULCH    2                            
LineString (6183983.31402974110096693 2244442.10679816640913486, 6184427.51780712697654963 2244671.6345833158120513)    50    1    10    6998.78176718143    WALKER GULCH    2                            

So it looks like
Processing -> Toolbox -> GDAL -> Vector conversion -> Convert format -> .gpkg
is the clear winner, avoiding most of the problems all the other routes create.

But the Station numbers still don't get drawn on the map. How do I discover why?

Does nobody else care about all the other "little glitches" in reproducing the original file? Is preparing input files for HEC-RAS via RiverGIS really so much more picky than what other people use QGIS for?

#4 Updated by Jürgen Fischer over 5 years ago

  • Description updated (diff)

#5 Updated by Jürgen Fischer about 5 years ago

  • Resolution set to no timely feedback
  • Status changed from Feedback to Closed

Bulk closing 82 tickets in feedback state for more than 90 days affecting an old version. Feel free to reopen if it still applies to a current version and you have more information that clarify the issue.

Also available in: Atom PDF