Import/Queensland Waterways

Import Queensland Waterways is a manual import of selected Surface Hydrology Lines (Regional) dataset which are waterways covering Australia.

Goals

Import mainly named waterways in Queensland.

Schedule

Will import the waterways on an irregular basis.

Import data

Background

Data source and license info: https://ecat.ga.gov.au/geonetwork/srv/eng/catalog.search?node=srv#/metadata/83107

Type of license: Creative Commons Attribution 4.0 International Licence

Link to permission: Surface Hydrology Lines https://wiki.openstreetmap.org/wiki/Australian_data_catalogue

OSM changeset source attribution: Geoscience Australia Surface Hydrology Lines (Regional)

Data resolution varies greatly in this dataset so check the data for the areas you wish to add before adding to OSM and be prepared to align to satellite imagery.

OSM data files

Link to the .osm data file example prepared for import: Pine River example

Import type

This is a gradual import using a script for tag transformation and then manually merge selected waterways into the OSM using JOSM.

Data preparation

Waterways preparation method

QGIS:

Add Vector Layer > SurfaceHydrologyLinesRegional.gdb folder to QGIS. Zoom to an area of interest, Select Features by drag-select across the area to process, Edit > Copy Features and Edit > Paste Features as a New Vector Layer. Choose ESRI Shapefile format, a filename and browse to a save directory, change the CRS to EPSG:4326 - WGS 84, rest of the defaults should be ok. QGIS will save the shapefile and load as a separate layer. Close or hide or remove the SurfaceHydrologyLinesRegional.gdb layer.

Optional: River Region boundary polygon:

I find it best to use a polygon to generally define the river region/catchment area of streams I wish to process as it allows the entire waterway/s to be completed with a single extract. The river region boundary polygon is not uploaded to OSM. The polygon is only used to clip the area of streams to a more manageable boundary size.

I used the Geofabric site data http://www.bom.gov.au/water/geofabric/ I downloaded from http://www.bom.gov.au/water/geofabric/download.shtml Specifically Version 2 product File:HR_Regions_GDB_V2_1_1.zip

QGIS: Add vector layer > HR_Regions.gdb > RiverRegion layer only Edit > select > select features > Select one or more polygons to suit the area you wish to work on Edit > copy features, Edit > Paste features as a new vector layer > Choose format ESRI shapefile, your directory and filename to save to, CRS EPSG:4326 - WGS 84, rest of the form should be fine with ticks in all the fields, geometry type automatic, OK.

On zoom in to edges of the river region polygon the edges will appear angular and this causes the ends of upper streams to be clipped off so it is best to add a buffer to the River Region polygon. Place a check next to the View>Panels>Processing Toolbox, select Buffer under the Vector geometry in the Processing Toolbox. Select the River Region/s polygon as the input layer, Distance of 0.001 or 0.002 degrees (aprox 100-200m)seem reasonable choices, other defaults are fine, tick the Dissolve result box if you have chosen multiple river regions and want only one outer polygon encompassing them. Choose the directory and filename to save to. Qgis may find errors in the polygon when clipping the waterways to the polygon and a buffer seems to fix this.

Clip waterways to River Region boundary polygon extent:

From the Vector>Geoprocessing Tools>Clip or from the Processing Toolbox>Vector overlay>Clip. enter the waterways as input layer and the River Region boundary polygon as the overlay layer, choose a directory and filename and save.

Tagging plans

Translate the shapefile to .osm xml file with correct tags.

Use a ogr2osm translation file on the shapefile to prepare suitable tags in an .osm file using https://github.com/roelderickx/ogr2osm

My SfcHydroLinesReg_tags.py translation file:

import ogr2osm

class SfcHydroLinesReg_tagsTranslation(ogr2osm.TranslationBase):

    def filter_tags(self, attrs):
        if not attrs:
            return
        tags = {}

        if 'FEATURETYP' in attrs:
            if attrs['FEATURETYP'].strip() == 'Canal Line':
                tags['waterway'] = 'canal'
            elif attrs['FEATURETYP'].strip() == 'Water Pipeline':
                tags['man_made'] = 'pipeline'
                tags['substance'] = 'water'

        if 'RELATIONSH' in attrs:
            if attrs['RELATIONSH'].strip() in ['Above Ground', 'Above ground']:
                tags['location'] = 'overground'
            elif attrs['RELATIONSH'].strip() == 'Elevated':
                tags['location'] = 'overhead'
            elif attrs['RELATIONSH'].strip() == 'Underground':
                tags['location'] = 'underground'
            elif attrs['RELATIONSH'].strip() == 'Submerged':
                tags['location'] = 'submerged'

        if 'PERENNIALI' in attrs:
            if attrs['PERENNIALI'].strip() in ['Non Perennial','Non-perennial','Non-Perennial']:
                tags['intermittent'] = 'yes'

        if 'NAME' in attrs:
            if 'River' in attrs['NAME']:
                tags.update({'waterway':'river'})

        if 'NAME' in attrs:
            if 'Creek' in attrs['NAME']:
                tags.update({'waterway':'stream'})

        if 'FEATURETYP' in attrs:
            if ('River' not in attrs['NAME']) and ('Canal' not in attrs['NAME']) and ('Canal Line' not in attrs['FEATURETYP']) and (attrs['FEATURETYP'].strip() in ['Connector','Watercourse','DEMConnector','Rapid Line']):
                tags.update({'waterway':'stream'})

        if attrs['NAME']:
            tags.update({'name': attrs['NAME'].title()})

        return tags

Mac Terminal command (or Windows command prompt)

/Library/Frameworks/Python.framework/Versions/3.9/bin/ogr2osm /path/waterways.shp --translation=/path//translation_file_name.py --never-upload --output=/path/waterways.osm

Drag the path to your ogr2osm from your file manager to the Mac Terminal prompt or Windows Command Prompt Similarly drag the path for the .shp file and .osm file

Data Reduction apart from only importing necessary fields or simplification hasn't been necessary as the waterways are not overnoded.

Data merge

Add the waterways.osm file to JOSM, then download OSM data for the area you want to work on into a new data layer. I select the streams I wish to add to osm then Edit>'Merge Selection' into the data layer I have downloaded from osm.
If the data amount is large I will select the ways plus Selection>'Select Way Nodes' then check the number of nodes using Windows>Selection and try to limit the size to well below 10,000 nodes as very large changesets often fail.

Information that is already contained in OSM is evaluated and the better geometry is used. If the data in osm is best the dataset data is discarded. If dataset data is preferred, the original waterway is replaced using 'More Tools' > 'Replace Geometry' so that the history is maintained wherever possible. Culverts are not included in the dataset so those in osm need to be added. Satellite imagery is rarely enough to evaluate if there is a bridge, culvert or ford at waterway crossings so these are retained if already mapped but not usually added from new dataset crossings.

Select upload and cancel, then fix the Josm validator alerts for your new data. This gives the opportunity to more accurately align the waterways to satellite imagery at highways and barriers and check for bridges, culverts and fords. All the Josm validator alerts pertaining to the upload are fixed.
If you find the upload size is too big, you can zoom in to an area you expect would be a better size then Edit>Search>for edit string "new inview " then File>"Upload selection". Next do another Edit>Search>for edit string "new" to do the remainder.

Changeset tags

Next I select Edit>Search>new then Windows>Tags/Memberships>right-click on name and copy value, then paste those waterway names in the changeset comment on upload. Source is added on upload... Geoscience Australia - Surface Hydrology Lines (Regional); any satellite imagery used; or other source like Mapillary.

If the upload begins but fails to finish, I cancel the upload window, then select upload again until all is completed. This leaves problems in the osm database to fix... I delete the osm data layer and download the area again into a new layer, select validate and there will be a large amount of duplicated nodes to delete. Select that alert in the validator and Josm allows you to select 'fix' to delete the all the duplicates. All or most of the remaining alerts will not relate to your changeset so you can select upload again to correct the osm database. You could search the downloaded area for useriD:YourUserName and then run the validator only on your edits and the validator won't alert on other stuff.

I have been doing this solo but all are welcome to use this method.

Post import

Will change waterway=river and waterway=stream to comply with OSM method of categorising these at a later date.

The post to the community forum was sent on 2025-06-25 and can be found here {https://community.openstreetmap.org/t/queensland-waterways/131846]