Memory error routing DEM

Hi,

I am trying to process a 1.6m per pixel DEM for use with the Nutrient Delivery Ratio model. I am trying to run it in RouteDEM, but it keeps returning a "memory error". The DEM is filled and has no sinks left... And If I resample it to 5m per pixel it works fine. Not sure about the accuracy of resampling a DEM though... I have attached the log file. Can you please help?

Also, I am trying to find the right threshold for the streams (with the 5m DEM) but what is working for me is somewhere around 60.000, even though there are some streams that have gaps for no apparent reason. Why is the threshold so different from the default? Are the streams drawn with the threshold supposed to match only the rivers and small streams or are they supposed to branch out a little? Another problem with my DEM is that it has a lot of drainage canals and it is really flat (Denmark)... So I am guessing that may be the problem... but how to solve it?

Well, sorry for all the questions! :) Any help will be much appreciated. 

Best regards,
Raphael
Tagged:

Comments

  • RichRich Administrator, NatCap Staff
    Hi Raphael,

    A few things to try.  First if you're willing to try another 4 hour run (yikes!) you could try our development version of InVEST.  It's fixed a load of bugs since the last 3.2.0 release and may even have addressed this routing issue (I vaguely remember working on something similar this summer).  It's possible it still might memory error though in which case the only way I can debug and fix is to have your data stack.  Could you pass that to me via dropbox at richsharp@stanford.edu?  Also here's the latest development version of InVEST if you fancy trying that first: http://data.naturalcapitalproject.org/invest-releases/3.3.0a1.post60+n8a005d8be224/InVEST_3.3.0a1.post60+n8a005d8be224_x86_Setup.exe

    And in terms of the stream threshold, that's a little confusing because it's represented in pixels which makes it dependent on your raster resolution.  Just because the default is 1000 doesn't mean anything for your data.  In the future I think we should change that number to be the upstream area, but that's a future issue. :)

    For your last issue with gaps, that may be because your DEM has hydrological pits that can't be drained to a watershed outlet using the surface height alone.  I know you said it was a filled DEM, but still I wonder... If it's not the issue I'd have to debug it directly given the data.

    And just a warning, this might be a BIG deal for us to solve so if you're time constrained and 5m resolution is good enough you may want to follow that path for a while.  We'd still like your data though if you don't mind passing it along.
  • Hi Rich,

    Thank you for the quick reply!
    I spent the weekend and today trying to make it work... loooong runs :). I will try to explore all my options until tomorrow and then I will post here again. And of course I can share my data with you. I will send you a link tomorrow.  
  • Hi again, Rich

    I got it! Well, kinda... I had the brilliant idea to clip the DEM to a subwatershed and it worked. I only did it with one so far, so it might be that the problem was in another region, but I think the real issue was the size of the file (around 4GB). I will keep you posted. Oh, the developer's version crashed :). I did it with the normal version. 

    Anyway, I just shared a dropbox folder with you with the raw DEM, a shapefile of the "official" watershed provided by the Danish Government and a "watermerge" shapefile of the "official" rivers and waterways (not 100% accurate). If you want to have a look at it out of curiosity... It is an extremely flat terrain and it has a lot of drainage canals... That's also why I want to use the finer DEM so it is easier to catch those canals... I hope.  

    Well, thank you very much for offering your help! I am going to try to use your "under development" Nutrient Delivery Model, so you might hear from me again in the near future :)

    Here is the link to make it easier for you:

    Cheers,
    Raphael

  • RichRich Administrator, NatCap Staff
    Thanks Raphel, I'm downloading the dataset now and will hopefully patch a bug in the next release!
  • Hi Rich,

    Great! 
    Just to give you an update, I tried to process the next subwatershed and I think I found the problematic region (at least one of them). It is a flat region around 010.40E 55.47N.

    I just uploaded a clipped DEM of that region if it makes your life easier :)

    I am sticking to the 5 meters DEM for now. Please let me know if you have some news.

    Cheers,
    Raphael
Sign In or Register to comment.