Seasonal Water Yield error help

Hi - I am using the seasonal water yield model on a fairly large region. After about 2 hours running, I receive a Set Projection, get raster band error. It may be to me running out of disk space but I also see a lot of errors in the log file. I have attached 2 log files with this issue (should be approximately the same). Could you help me figure out why this is failing?

Thank you

Comments

  • swolnyswolny Member, NatCap Staff
    Hi @atutterow -

    Yes, it is due to running out of disk space. You can see a lot of "no space left on device" messages in the log file. These models, especially Seasonal Water Yield, can use a lot of disk space in the intermediate processing, especially over a large region or high-resolution dataset.

    ~ Stacie
  • Thanks, @swolny - I ran it again with > 500 GB of space. It failed again with the same message and I still have room on my drive. Could it possibly take up more space than this? I attached my log file again.
  • swolnyswolny Member, NatCap Staff
    @atutterow, I'm not sure what's going on. From the very beginning it is saying that there's not enough disk space on E:\Tulare2015, which shouldn't be if you have 500GB available. SWY can take up a lot of space, especially if your inputs are large. What is the size of your DEM? Whatever resolution that is, all of the intermediate and final layers will be as well. Still, I doubt that the DEM is that huge. 

    The only other thing I can think of offhand is if E:\Tulare2015 is some sort of network drive that might cause problems. Is this a local disk or something else?

    ~ Stacie
  • H @swolny - Thanks. My DEM has cell size 30 x 30 meters. It's "uncompressed size" is 6.20 GB. 

    E:\Tulare2015 is a 1 TB external hard drive (400 GB of it is taken up with other info).
  • swolnyswolny Member, NatCap Staff
    Wow, that's a really big DEM, and that means that each intermediate and final raster from the model will also be 6.2GB, which will definitely take up a lot of space. I usually try to keep mine under 1GB, both for disk space and memory. Are you sure you're not running out of memory instead of disk?

    If you're sure you're not running out of disk or memory, then I'll have to turn this over to our software team. They do a lot of work to make the models better able to handle large datasets, and they might have some ideas.

    ~ Stacie
  • @swolny - yes, it is big! I believe I shouldn't be running out of memory or disk space.
  • swolnyswolny Member, NatCap Staff
    Ok then, @Rich and @jdouglass, I'm punting to you!

    ~ Stacie

  • RichRich Administrator, NatCap Staff
    Hi @atutterow, those errors are coming directly from the underlying GDAL library and are consistent with an out-of-disk-space error (or a scary hardware error!).

    But assuming the disk isn't failing, that error is coming from the step that aligns the raster stack including the monthly precipitation rasters. The model resizes all rasters to be the same pixel size as the DEM and then clips everything to the raster stack's intersecting bounding box. The very first disk error seems to be happening on the very first precipitation reprojection. Is it possible that your precipitation rasters are already rather large but their pixel size is larger than the dem? If so, the model will be making the precipitation rasters EVEN LARGER by effectively increasing their pixel count. How big are your base precipitation rasters compared to your DEM?

    And even though it looks like nothing finished, could you see if you could inspect "F:\Tulare2016\prcp_a0.tif" or any of the other rasters generated there to see what their sizes are?
  • Hi @Rich - Thank you.

    The base precip rasters are 319 KB while the DEM is 6 GB. The precip rasters also encompass a larger area than the DEM does, but I have had success with even smaller DEMs with the same precip input.

    I am unable to check the size of the rasters generated - says it is invalid.

    Thanks!
  • RichRich Administrator, NatCap Staff
    Hi @atutterow, in that case it doesn't sound like the issue is a finer scaling... These are really large rasters, is there any way you could share them with me via dropbox or google docs? Something that would let me try it on my end to see what's going on?
  • Hi @Rich


    I am using the climate zone advanced settings as well - I included my raster and rain events for that.

    Thanks for your help!
  • Hello @Rich - any update?

    Thanks!
  • RichRich Administrator, NatCap Staff
    Hi @atutterow, I'm running it now and observing that it's making pretty big intermediate files when aligning the rasters. I'll watch it and let you know.

    p.s. I notice that your soil raster is mostly nodata except for some regions in southern California. If you were to clip your stack to the bounding box of the nodata region things might run a little faster because you'd have less data to process.

  • RichRich Administrator, NatCap Staff
    Hi @atutterow, I think I see what might have been happening on your end. This version of the seasonal water yield model uses an older version of our geoprocessing pipeline that makes temporary files via an operating system call. Those files are created in the system temp file folder which is likely *not* your workspace. It's likely you were running out of space on your system drive, not your workspace drive, then when the model crashed it cleaned up all those temporary files.

    This is unfortunate because we're aware of this issue and we're in the middle of upgrading seasonal water yield to the newer pipeline, but it is not trivial work, it may take me a 2-4 more weeks. In the meantime, if you want to continue with this, is there any chance you might be able to pare down your dataset a little if possible? Like I mentioned in the last post your soil dataset is mostly nodata and you might be able to clip everything down to a tighter bounding box, you'll otherwise get nodata value results everywhere you have a nodata soil pixel. If that's not possible, could you try browsing to your system's temporary file directory and seeing if you can remove anything there? (If you're on windows this is likely in C:\Windows\Temp) or just in general see if there's anything you can do to make a few gigabytes of space on your system drive?

    If none of that is possible, I'll post back here when we get the SWY model upgrade complete.
  • Hi @Rich - Thanks for the troubleshooting. I'll try to free up some more space but I believe I have already gone through and cleaned up my drive as much as possible. I'll also crop my inputs.

    Let me know when the upgrade is complete! That will help significantly. 

    Thanks again!
  • Hi @Rich!

    As you're working on the SWY upgrade, I wonder if it includes solving the bug on the baseflow index, which Perrine mentioned in 2017, in this forum.

    My output maps of baseflow and local recharge seems to be reversed: pixels of forested areas present low values of B (like 0 mm) while built areas show much higher values (e.g. 900 mm). I'm not sure if it's related to this bug or if it's my mistake. 

    Could you help me reading this results?

    Thanks!

    Mari

  • RichRich Administrator, NatCap Staff
    Hi @mari_abreu, I haven't completed the SWY upgrade yet, but I'm not aware of a bug that would cause this reversal. I'll see if I can get one of our hydrologists to help you.
  • swolnyswolny Member, NatCap Staff
    Hi Mari -

    I'm not a hydrologist, but have used the model. What kind of values are in your biophysical table for forest versus built areas? Could it be that there are high values for Kc for forest, such that the forest is using the water so there's little left for baseflow? And what do the quickflow values look like for forest vs built areas? Do those values make sense?

    As a side note, remember that the baseflow values should be used as relative values, or an index, not absolute values. Still, the relative ranking should make sense based on your inputs.

    ~ Stacie

  • Hi @Rich,

    thank you for you attention!

    Hello @swolny,

    I'm running SWY 3.5.0 and the watershed is in Southern Brazil. The main part of the forest is above soils B and C and built areas is basically on soil B. So, I have:

    Forest ------------- CN_B = 40 CN_C = 49 -------- Kc = 1,1
    Built areas ------- CN_B = 82 ------------------------ Kc = 0,2

    The Quickflow map is coherent, vegetaded areas show lower values of QF when compared to built areas and degraded pasture. I'm aware that quikcflow is the only quantitative result in SWY. 

    My problem is in defending the idea that urbanized areas are those that contribute most to baseflow, while areas with preserved native forest do not contribute at all. Maybe you're right, the forest is using so much water that there's nothing left for baseflow... but why built areas present such high values of B? Maybe its CN = 82 isn't high enough.

    Thank you very much, Stacie.




  • Hi @Rich

    Do you have a status update on when the SWY model upgrade will be complete?

    Thanks,
    Alyssa
  • RichRich Administrator, NatCap Staff
    Working on it!
  • swolnyswolny Member, NatCap Staff
    It's hard for me to comment on your CN and Kc values without knowing the land cover types and soil better. In the projects I've done, we've used higher CN values for forest, and just slightly higher CN for urban areas.

    As for how the model works, I defer to our hydrologists. @Perrine, @Rafael?

    ~ Stacie
  • Thank you @swolny!

    Maybe I should go over my biophysical table. I'm using adapted values of CN for Brazilian soils (Sartori, 2004).

    Nevertheless, it'd be very helpful to talk to a hydrologist. 


  • PerrinePerrine Moderator, NatCap Staff
    Hi Mari, 

    I'm just catching up with this thread. Could you remind me of the bug you are mentioning in an earlier post?

    Your CN and Kc values look reasonable, although Kc=0.2 and CN=82 for urban areas may both be too low for the model to behave as expected. This depends on many factors, climatic of course, but also what is considered "urban" in your LULC classification (i.e. how much pervious areas do urban pixels have?).

    I agree that baseflow from urban areas should be relatively low, probably lower than forests, but if you have a significant proportion of pervious areas on these urban pixels, they may still contribute to baseflow. This is essentially what a relatively low urban CN (82) reflects in the model. 
    If that's the case, Kc should be higher, reflecting that there's still a significant amount of retention and evapotranspiration occurring on urban pixels.

    The tradeoff between retention and evapotranspiration is a key determinant in the SWY model, and it may be useful to try a range of Kc values to see how it changes the results.
    Let me know about the potential bug too, just to make sure the issue isn't there!
  • Hi @Perrine!

    Thank you very much for you attention. You pointed out important things that I should analyze more carefully.

    About the bug, actually you mentioned it first on a comment in 2017 (can't find that message anymore), saying that there was a bug related to baseflow index. When I read that, I thought that it could be related to the problem I'm facing with my baseflow index values. I should have made you the question: Is there really a bug in the baseflow index? Sorry if I wasn't clear.

    Well, I changed my CN for urban areas to 95 and something weird happened: the QF_mean now is lower. It's 61,6 when we had 111,4 with CN = 82. Shouldn't it be higher, since now I have less pervious areas? When I finish all the tests using different values of kc and CN, I'll let you know.

    Now, a question about the streamdirection file that the model genarates. Is it ok if the stream segments are not connected? Because my drainage is very fragmented althought I had no problem when delineating the watershed with my DEM using TauDEM.

    My best,

    Mari





Sign In or Register to comment.