Seasonal Water Yield Memory problems

Hello people,

I'm having some problems trying to run the seasonal water yield model to a Brazilian city, and sub-watersheds. The program runs into a memory leak and simply stop working until th SO kills the process, can you help me?

I'm attaching the log file, my original files are a bit too big to send, but if really needed i can find a dropbox to put it.

Thanks,

Vitor Zanetti
ITA - Brazil

Comments

  • RichRich Administrator, NatCap Staff
    Hi Vitor, this internal GDAL error is one I haven't seen before: `GDALRasterBlock::Internalize : Out of memory allocating 262144 bytes.`

    Offhand I'd guess that's occurring because of an out of disk error (though the log does say you had 2+TB to start) or GDAL's cache is overloading somehow. Do you know how big your rasters are in terms of number of pixels wide/tall and how large your pixels are are in meters?

    In theory, there are no data input size limits to InVEST, so if your data size is triggering that, I'd consider it a bug. But depending on your use, would it make sense to downsample the data to a larger pixel size? I'd think 30-90m pixels would be just fine for the size of area you're describing and may help you skirt this issue if you need a result quickly.

  • vitorzvitorz Member
    edited February 17
    Hello Rich!

    i'll try donwscale the data to solve the problems, it's a very large database (12gb), and i'm working with 1m resolution, i'll try working with 10-20 m resolution and let you know if it solves the problem.

    Thanks for helping
    Post edited by vitorz on
  • Hello Rich, me again,

    I've downscale most of the maps, (only MDT i didn't change), and the error persists, in the generation of the same map (B), and same size, any other ideas?

    I'm Attaching the log file of the last attempt, another thing is that it's taking some 12-20 hours to run.
  • RichRich Administrator, NatCap Staff
    Hi Vitorz, you said you downscaled most of your maps... Did you do your DEM too?  This model will otherwise resample all the rasters the DEM cell size so even if you downscale a couple it won't save you from oversampling.

    In case you didn't, can you try upsampling your DEM too?  In general, we recommend 30m anyway since that takes out all the high frequency noise associated with very fine DEMs.
  • vitorzvitorz Member
    Rich, good morning!

    I'd not downscale the DEM (MDT in portuguese sorry for that), because i didn't want to loose information on small creeks on the region that are in danger for overuse of the land, but i'll try to downscale it to 10m and 20m and see how much information will be lost in it.

    Thanks again
  • vitorzvitorz Member
    Hello Guys,

    Last report, the problem was the DEM, i downscale it to 10m resolution (from 1 m), and the model is running just fine, i lost a little bit of information, but i think i can mantain my analysis.

    Thanks for all the help.

    Cheers!

Sign In or Register to comment.