NDR and Route DEM

Hi all,
I am running the NDR. The NDR model is running to completion for me but it does not generate any nutrient discharge At first I thought it might have something to do with my nutrient loading values but I did sensitivity test and loaded all the land use codes with Nitrogen loads of 200 and still no discharge so it must be some other problem. 

I immediately suspected my DEM.
I am not entirely sure what I am looking for. How do I find the hydrological "pits". I ran my DEM through the Route DEM routine but the documentation advises users to "define file names" I am not well versed in coding and I have NO CLUE how to define file names . . should I assume that the file labeled Stream Mask is the "plateau resolved DEM" ? The only tif my model runs produced that even vaguely resembles a DEM is the slope map. 
Once I get a plateau resolved DEM out of the route model can I use that in the NDR ?

Also, the documentation for the NDR suggests that the Nutrient runoff proxy can be an annual precipitation raster . . Do I have to "normalize the raster or will the model do this for me ?

No more questions for now, Can't Wait to hear from You all !
HDL

Comments

  • RichRich Administrator, NatCap Staff
    Hello, an easy way to see if the issue might be your dem: there should be a flow accumulation raster in your intermediate outputs. If you load that into a GIS desktop tool, you should see long smooth continuous flows. If instead you see a bunch of short stop and start flows, you're probably looking at hydrological pits.

    RouteDEM does not yet fill pits so you'll need to use an external tool. I use the Wang & Liu algoritm in SAGA-GIS but I've also heard that ArcHydro has a model to do DEM fills.
  • swolnyswolny Member, NatCap Staff
    ... and no, you don't need to normalize the runoff proxy raster, just enter the annual precipitation raster directly and the model will do it for you.

    ~ Stacie
  • Great thanks,
    Once all the pits are filled which of the files is the plateau resolved DEM" I can use in the NDR model ? is it the stream mask ?
    Big time thanks again for the speedy response.
    HDL
  • swolnyswolny Member, NatCap Staff
    Sorry, the User Guide section for RouteDEM hasn't been updated to reflect how the current version (3.4.2) works, so it's confusing.

    RouteDEM doesn't currently output a resolved DEM, it outputs flow accumulation, direction and streams, which you can use to see if the DEM is producing a good, connected stream layer. What you should do is run your DEM through the QGIS Wang and Liu pit-filling tool, and use the result from that in NDR as the DEM. Or, to pre-check your streams without having to run NDR, you can put the pit-filled DEM into RouteDEM and check out the resulting streams and flow accumulation. If the streams are still not connected, run the pit-filled DEM through Wang and Liu again and repeat the process.

    ~ Stacie

  • Great input.
    Am trying to use the Wang and Liu fill sinks tool and I keep getting the 'str' object has no attribute 'bandCount' message. Seems like I need to reformat my dem ? it is currently projected for use in the NDR tool. It ran well enough to identify pits in the routing tool what else do I need to do to it ?
    This is incredibly helpfull.
    HDL
  • RichRich Administrator, NatCap Staff
    Hi, "'str' object has no attribute 'bandCount'" sounds like a Python program is trying to treat a filename like a GDAL object. Does that context make sense for whatever software you're trying to use W&L?
  • I am using Qgis
  • Can you recommend a fix for this? Is it expecting a projected Tiff as a DEM ?
  • Why would a file that worked for InVest not work for Qgis or Saga gis ?
  • swolnyswolny Member, NatCap Staff
    Both the QGIS Wang and Liu tool and InVEST expect a DEM that's a raster with a projected coordinate system. TIFFs are pretty standard, and tend to work well in both environments. I do this particular DEM-filling process a lot with TIFFs and it works fine. Make sure that you're entering the file that ends in ".tif" and not ".tif.xml" or any other extension.

    ~ Stacie
  • Hi and thanks. I checked and I could not see any other extensions both the DEMs I have tried to run in QGIS with the Wang and Liu tool show up as" TIF files"  is it possible that there are naming conventions ?

    All of the DEMs I am working with are too large to share even in compressed form

    HDL

  • Here is the error message

    An error has occurred while executing Python code:

    AttributeError: 'str' object has no attribute 'bandCount'

    Traceback (most recent call last):
      File "C:/PROGRA~1/QGIS3~1.0/apps/qgis/./python/plugins\processing\gui\AlgorithmDialog.py", line 183, in accept
        ok, msg = self.algorithm().checkParameterValues(parameters, context)
      File "C:/PROGRA~1/QGIS3~1.0/apps/qgis/./python/plugins\processing\algs\saga\SagaAlgorithm.py", line 428, in checkParameterValues
        if layer.bandCount() > 1:
    AttributeError: 'str' object has no attribute 'bandCount'


    Python version: 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)]
    QGIS version: 3.0.1-Girona Girona, a86bec25eb

  • Hi Nat Cap Gurus, Stacie and Rich,

    I think I have figured out how to make it run Yay !  Can you direct me to guidance on minimum slopes.

    HDL

  • Hi All,
    I have run the W&L pit filling tool on my DEM several times now and put the resulting DEM thru the Route DEM tool. The flow accumulation lines on the resulting map still look like dotted and dashed lines. Is it possible that there is a problem with the DEMs I mosaiced together to make the DEM I am trying to use ?
    can you make any suggestions ?
    HDL
  • swolnyswolny Member, NatCap Staff
    What process did you use to create the DEM?

    One thing that often happens when mosaicking, then re-projecting is that it can produce a grid pattern. I tend to take the raw DEM tiles, mosaick them, then reproject using the "bilinear" interpolation method. Using "nearest neighbor" often leads to weird results.

    ~ Stacie

  • Hi Stacie,
    I actually ran W&L and then Route DEM on the individual DEMS and the accumulation lines were still very faint dotted and dashed lines. I think I got the DEMS from USGS
    HDL
  • swolnyswolny Member, NatCap Staff
    So you didn't do any reprojection or anything on the raw tiles before putting them through the fill and RouteDEM? Can you post an image of the dotted/dashed output?

    For what it's worth, it really is best to mosaic and reproject before doing the fill etc when working with multiple tiles. You could also try doing that and see if it helps.

    ~ Stacie
  • Hi Stacie,

    I downloaded DEMS for all of the counties in my study area and mosaiced them together, extracted just the study area I'm interested in and then reprojected it.

    Here is what the flow accumulation looks like. I tried it with the NDR and the model generated 0 nutrient discharge. I am back on the USGS rextracting DEM data. Can you suggest the minimum acceptable resolution.

    HDL

  • swolnyswolny Member, NatCap Staff
    That flow accumulation looks pretty good to me at that scale. Have you zoomed in on the streams? They often look cut up zoomed out when really they aren't, it's just a display issue at that scale.

    Looking at the NDR output, what does the stream.tif look like? Is it generating streams? If there are no streams generated, you won't get any nutrient export, because the model routes nutrient until it hits a stream. In that case, you'll need to choose a different threshold flow accumulation value that creates a stream network that looks correct.

    There's no particular recommended resolution, it's a matter of what's available, what's appropriate for the level of detail needed for your study, and computing resources. If you're working in a smaller area, you'd want finer resolution, if working on a very large area it can be coarser, and might need to be computation-wise. 30m DEMs are pretty standard for smallish-medium-scale watersheds, 10m is common in the US, 90m or more for covering large countries/watersheds/etc.

    ~ Stacie
  • OK Wow ! thanks, now I am feeling considerably less stupid ! :-)
    no there are no streams. the I am using a TFA of 1. I assumed 1 was the smallest. I also assumed that the smaller the TFA the more likely to have nutrient export
  • I am currently running w/ TFA=.5, the model seems happier
  • still no streams 
  • OK so I boosted the TFA to 10 and I have some nutrient export Yay !
    I have a couple of quasi-hydro questions relating to W&L and the NDR model

    Wang & Liu - there is a " minimum slope" input for Wang and Liu. I briefly read over the W&L article explicating their system. Is there a discussion of minimum slope in that article or can you direct me to guidance on minimum slopes elsewhere ? I am currently using the default of .01. I noticed as I increased minimum slope the definition of the pitfilled DEM decreased

    NDR - The abstraction landscape I am working with is managed to generate zero nutrient export. Do you think it is reasonable to adjust the TFA to back into zero export and then use that TFA in subsequent simulations ?
    Thanks for everything Stacie !
    HDL
  • swolnyswolny Member, NatCap Staff
    It's important to understand what the threshold flow accumulation does, because selecting the correct value is really important. The User Guide says this: "Integer value defining the number of upstream pixels that must flow into a pixel before it’s considered part of a stream. This is used to generate a stream layer from the DEM (see RouteDEM documentation of the InVEST manual). This threshold expresses where hydrologic routing is discontinued, i.e. where retention stops and the remaining pollutant will be exported to the stream. The default is 1 over the pixel area (in km2), i.e. ~1000 for 30m resolution. If the user has a map of stream lines in the watershed of interest, he/she should “calibrate” the threshold value by comparing the map with the stream.tif map output by the model."

    And I explained it this way on another forum thread recently, "The threshold flow accumulation (TFA) value is used to create the stream network from the DEM. Higher values of TFA create a stream network with fewer tributaries, lower values of TFA produce more tributaries. The "correct" value is different for each place. It's the value that creates a stream network that is as close as possible to the real-world stream network in your area of interest. So you'll need to find a real-world stream map, compare it with the stream.tif output from the model, and adjust the TFA until the streams come close." 

    So really, you should probably have a TFA that's much higher than 10. Do the exercise of adjusting TFA until you get a stream network that looks as close to the real world as possible, and let me know how that goes.

    As for Wang & Liu, I've never changed the minimum slope value, so can't offer any advice there.

    ~ Stacie






  • you rock ! have a great Friday nite !
  • Hi all,

    I am running the NDR with several different precipitation rasters. The one with the larger precipitation values actually generates a nitrogen export value that is smaller than the raster with smaller precipitation values. The raster with the larger precipitation values is lower resolution. It is possible that the large resolution raster actually covers a smaller surface area and therefore generates a smaller export number ?

  • Next question,

    In all of my work I have mask extracted the various rasters to the contours of my area of interest. The current data set I am working with is fairly low resolution and when I mask extract the raster, some of my area of interest does not have data associated with it. Finer data sets exist but they do not exist for all of the variables I am working with. My solution was to run the NDR model without mask extracting the low resolution data raster. The nutrient export total makes more sense, What do you think ?

    HDL

  • swolnyswolny Member, NatCap Staff
    Hi Hildegaard -

    It's common for some of the inputs (especially climate) to be of lower resolution, and what I usually do is create a buffer around my area of interest, clip the lower-res inputs to that, and use the buffer-clipped result as an input to InVEST. You definitely want complete coverage of your watershed, or you'll end up with NoDatas, and incomplete results.

    I'm not sure what to say about the different precip rasters giving different results, without looking at the data. Within the model, InVEST will resample all of your inputs to be the same resolution as your DEM before doing calculations. As long as each of the precip inputs that you give covers the same AOI as your other inputs, the resolution itself shouldn't really matter, just the values. If you gave the model a coarse precip map that doesn't cover your whole AOI, then I would think that total export could be lower, simply because you're missing export data, which leads to NoDatas in the resulting n_export.tif raster. The model can only calculate n_export values per pixel where there is data for all inputs. And the total export value of the watershed is simply the sum of all of the n_export.tif pixel values within the watershed.

    ~ Stacie
  • Thanks !
    I am filling out the Phosphorus BioFizz table.  Below is what the users guide says about the Critical length.
     My question is 2 fold . .  are the Nitrogen  critical lengths OK for phosphorus ? if no, the when you take the critical length equal to the pixel size should I take one side of the pixel ie 30 meters for a 30x30 pixel or the hypotenuse which would be 42 meters and this means that the maximum retention of the nutrend possible is aomplished in each pixel ?
    "
    1. Retention length
    This value represents the typical distance necessary to reach the maximum retention efficiency. It was introduced in the model to remove any sensitivity to the resolution of the LULC raster. The literature on riparian buffer removal efficiency suggests that retention lengths range from 10 to 300 m (Mayer et al., 2007; Zhang et al., 2009). In the absence of local data for land uses that are not forest or grass, one can simply set the retention length constant, equal to the pixel size: this will result in the maximum retention efficiency being reached within a distance of one pixel only.
Sign In or Register to comment.