Adding functions to the QGIS toolbox

The graphical modeler makes it easy to create models by chaining together different operations/algorithms available in the toolbox. But what if a function is not available in the toolbox? In that case you can write your own script and add this as a new function to the toolbox.

For example, I want to create a model to download zipped ascii raster layers, unzip them, convert them to geotif files (if not already) and store them on my computer. The first step, downloading the data, can be automated using the Download function in the toolbox (Figure 1).

Extract the rasters

For the next step, unzipping the files, no function is available in the toolbox. So apart from doing this part manually, the option is to write a small script to extract raster layers from a zip file. The tricky part is to write a function that can be used in the QGIS toolbox. This means amongst others that the function’s input parameters need to be defined with the initAlgorithm function and the addParameter function. This is illustrated with the code snippet below, which defines the input and output parameters for the unzip functions.

Code

def initAlgorithm(self, config=None):
'INPUT',
'INPUT',
optional=True,
behavior=QgsProcessingParameterFile.File,
fileFilter='All files (*.*)',
defaultValue=None))
'OUTPUT',
'OutputFolder',
optional=True,
behavior=QgsProcessingParameterFile.Folder,
fileFilter='All files (*.*)',
defaultValue=None))

With the input parameters defined, the code snippet below gets a list of the files names in the zipfile, selects the raster layers (based on their file extension) and extracts them.

Code

# Output folder
out_dir = parameters['OUTPUT']
# Input zip file
yzf = parameters['INPUT']
temp_dir = tempfile.mkdtemp()

with zipfile.ZipFile(yzf) as zf:
# Get list with file names in zipfile
files = zf.namelist()
# Get list with all raster layers in zip file (ascii and tif rasters)
lrf = ('.asc', '.tif', '.bil', '.dwg', '.hdr')
filesasc = [i for i in files if i.endswith(lrf)]
# Extract the raster layers to the user defined folder
for asciifile in filesasc:
file_path = os.path.join(temp_dir, asciifile)
f = open(file_path, 'wb')
f.close()

You can check out the complete script here. Besides the code snippets above, it contains a header section, it loads the required libraries, and it contains some extra (html formatted) information that will be shown as help when running the script from the toolbox. After loading the script in the QGIS toolbox, it can be used like any other function.

Convert the rasters

The translate function can be used to convert raster layers in another format to geotifs. My initial idea was to import the above script in the toolbox, and build a model in which the output of this particular function, the downloaded and unzipped raster file, would be used as input in the translate function. But the downloaded zip files can contain more than one raster file, complicating things. I therefore decided to include the raster conversion part in the script. The code snippet below shows the part where the raster file is converted using the gdal::translate function.

Code

outputfiles = []
for rasterfile in myfiles:
# If other than geotif, convert raster to tif
if pathlib.Path(rasterfile).suffix != '.tif':
# Replace suffix with tif and temp folder path for path output folder
tmpfilename = pathlib.Path(rasterfile).with_suffix('.tif')
transfile = os.path.join(out_dir, ntpath.basename(tmpfilename))
# Translate (convert format)
alg_params = {
'COPY_SUBDATASETS': False,
'DATA_TYPE': 0,
'EXTRA': '',
'INPUT': rasterfile,
'NODATA': None,
'OPTIONS': '',
'TARGET_CRS': None,
'OUTPUT': transfile
}
outputs['TranslateConvertFormat'] = processing.run(
'gdal:translate',
alg_params, context=context,
feedback=feedback,
is_child_algorithm=True)

You can find the complete script here. What the script does is to download a zip file, extract the raster(s), and, if not already in geotif format, convert the raster layer(s) to a geotif raster layer(s). The latter is, obviously, skipped, if the downloaded file is already in geotif format. And lastly, the intended output format can be changed by the user. You can load the script in your QGIS toolbox and run it (Figure 3).

Combining the different steps

To create one combined workflow from download to conversion, I created a simple model in which a zip file is downloaded using the Download file function, and the raster layers it contains are extracted and converted to a geotif format using the newly created unzipconvert function (Figure 4). You can get the model here. Load it in the QGIS toolbox and try it out. Or download the Python script here and load that in your toolbox instead.