Running into error in seasonal water yield model

Hi there,

I'm still working on running the seasonal water yield model since my last thread. My partner and I are running the model and after a couple hours of geoprocessing we get this error:
  • Nonetype error: invalid value encountered in divide
I'm not sure what this means and have tried altering some of the data with no luck. I'd be happy to drop someone the log file or data if you have time to help.

Thank you!

-Pete


Comments

  • RichRich Administrator, NatCap Staff
    Hi Pete, that's a frustrating error.  Looks like a divide by zero, inf, or NaN, in the Vri calculation.  That can happen for a number of reasons, but a common one might be that your DEM has hydrological pits in it and isn't getting flow paths long enough to calculate streams.  You can get a good sense of that by inspecting the flow accumulation raster in the intermediate output.  If you see flows starting and stopping all over that's a good sign there are pits.  If so, I'd suggest using SAGA's "Terrain Analysis - Preprocessing -> Fill Sinks (Wang & Liu)"; it comes with QGIS these days to fill your DEM and try again.

    If that's not the issue, can you post a follow up here?
  • ptelarolptelarol Member
    Hi Rich,

    I looked at the flow accumulation raster and it did looked like as you said it might, so I used Wang & Liu in QGIS to fill the DEM, but am still running into the same error. After filling the DEM, INVEST ran much quicker, but I still received the same error in the same lines. Log file is attached. 


  • RichRich Administrator, NatCap Staff
    Dang.  Can you dropbox me your data to richsharp@stanford.edu?  I'll try to run it on my end and see if I can recreate your issue.
  • ptelarolptelarol Member
    Just shared the data. Thank you for the help!
  • RichRich Administrator, NatCap Staff
    Hi, sorry I'm not seeing a dropbox invitation from you.  Can you verify that you sent it to richsharp@stanford.edu?
  • ptelarolptelarol Member
    Hi Rich - I just resent it. Let me know if you don't get an invitation. 
  • RichRich Administrator, NatCap Staff
    Hi, I'm getting back to your issue now.  I'm getting the same error on my end, but am also seeing a bunch of nodata holes on the routing step.  Clicking around, I see those are centered on hydrological pits.  So it seems this error is caused by hydrological pits in the DEM.  For the moment, the only way to get around this is to get the DEM filled & routed. I'll try filling w/ Wang & Liu on my end, but wanted to let you know I was in the midst of that since your run takes a few hours to get through.

    Also, when looking at the slope of your DEM I can see a bunch of artifacts that look like stitching artifacts (they're all grid aligned slope lines).  Just an FYI in case you weren't aware of that.

    I'll post again after I do a pit filling pass to see if I can get a complete run going.
  • RichRich Administrator, NatCap Staff
    Oh boy, I see i ran it on your original unfilled DEM.  Anyway, I'll still try what I said above and post back when I know more.
  • RichRich Administrator, NatCap Staff
    Hi, after filling sinks, I can see another problem.  The SWY model, for the moment, requires all the watersheds to drain to a stream.  The model classifies streams by thresholding flow accumulation.  But things break when a watershed is so small that there's not enough flow accumulation to classify a stream.  On the edge of your DEM I see a bunch of what look like truncated watersheds that look like this issue.  Since some of SWY's equations use a "distance to stream" value, we run into problems here. For reference, this also happens when the DEM doesn't route.  Flow accumulation is too short to classify streams in some watersheds.

    I made a zoomed picture of some of these watersheds on your data w/ the calculated stream layer underneath.  I circled two (of many) polygons that don't have streams in them: http://i.imgur.com/vogqEje.jpg

    This half a problem w/ InVEST, and another with the data.  We should have a better error here than "invalid value encountered in divide".  But even if we didn't, it's not clear what to do on these areas except discard them.  Likely I'll get to "fixing" this during the summer, you can follow the issue here: https://bitbucket.org/natcap/invest/issues/3642/fix-divide-by-invalid-error-on-swy).  Note I say "fix" because in the end, the best we can do is throw away those data.  In the meantime, you can either clip out those areas of the DEM that don't route streams. Or have a larger DEM that extends beyond those watersheds.  OR  if you're feeling punchy, you can reduce the threshold flow accumulation value to make more streams.  

    At any rate, I'm sorry for the hassle, and long delay on my end.  If I don't otherwise hear back from you, I'll post on this thread when I patch better error handling into the model.
  • ptelarolptelarol Member
    Hi Rich,

    Thanks for the support and information - it's been really helpful in trying to get this to work. 

    We've started to run the model on hydrologic regions or watersheds instead of counties to take care of the truncated watershed issue. However, we still seem to be getting the same exact errors in the same exact lines. I've run the model on one hydrologic region and then another four watersheds within that region separately. I've tried filling the DEM in two different ways: 1) Using Wang and Liu & 2) Using ArcGIS to find the max sink depth and then filling up to that depth. Neither of these methods seems to do the trick. I've attached a log file from one of the runs on a watershed. Happy to drop you the data if it helps. 

    On an unrelated note, when I run the model consecutively using the same inputs but a different DEM, I get another error "NoneType' object has no attribute 'SetProjection". Do you have any idea what is going on here? I've attached a log file with the run. 

    Thanks for any help and hope the summer is going well.

    -Pete
  • RichRich Administrator, NatCap Staff
    Hi Pete, there's a long weekend coming up, but can you dropbox me your new dataset?  I'll see if the same issue is occuring, and if so handle it in a way that doesn't make InVEST crash.

    Your second error is probably caused by this line in the "TIFFOpen:C:\Users\ptelarol\Dropbox\Output_Tests_Invest\June_28_2017\Si.tif: Permission denied"

    Sometimes an error like that can occur if a file is locked, or deleted, or running out of harddrive space or something similar.

    Anyway, feel free to dropbox that whole datastack along to richsharp@stanford.edu and I'll see what I can do.
  • ptelarolptelarol Member
    Hi Rich,

    Thanks for the quick reply.

    I finally figured the problem out. It looks like due to the resolution differences in the ETo and precip rasters compared to the DEM, that when the ETo and precip rasters were clipped that it created areas where the DEM overlapped with "No data" areas on the Eto and precip rasters. Once I corrected for the no data in these rasters, the model ran. The other problem has been fixed as well.

    I appreciate the help and patience from your end. If other issues pop-up, I'll post on here. 

    Have a good fourth!

    -Pete
  • RichRich Administrator, NatCap Staff
    Great thanks Pete, that makes sense.  I've sill got your original issue in my queue to patch when I get to it.  
  • ptelarolptelarol Member
    Great - will be on the lookout for the patch over the summer!
  • Hi all,

    I also have an error in the seasonal water yield model (see attached log-file). I would be happy if you could help me with it.

    KInd regards

    Markus


  • RichRich Administrator, NatCap Staff
    Hi Markus, unfortunately the exception isn't printed in the log. :\ but i see the last reasonable output was mapping the curve number raster.  If I were to guess, could there be a landcover value in your raster that's not in the biophysical/curve number table?  

    If that's not the issue and there was a big red exception in the user interface, could you screenshot that and post it here?

    And if there's no big red exception, can you dropbox me your datastack to richsharp@stanford.edu and I can take a look?
  • Hey Rich,

    Thank you very much for the hint. It worked out. 

    Best

    Markus 

Sign In or Register to comment.