natcap.invest (c)logging

Dave

Hey Team,

I'm reporting some strange -- but not necessarily unwanted -- behavior related to natcap.invest, logging, jupyter notebooks, and fiona.

I'm doing some pre/post-processing of invest results in a jupyter notebook where I've imported the following:

In my workflow, whenever I make a call to read/write a shapefile using geopandas (which then calls fiona), fiona spews out endless DEBUG messages to the console. It's especially problematic in Jupyter because all those printouts bog down the browser and eat up GB of memory. I noticed this only happens if I have natcap.invest imported.

I've got workarounds, so not looking for a fix, just thought this might reveal some unintended side-effects of how natcap.invest does it's logging. And I know how @James get's excited about logging!

Dave

Oops, I posted too soon and left out the part where I say which packages I'm using:

natcap.invest.coastal_vulnerability.coastal_vulnerability_core (3.3.3)

geopandas (0.2.1)

fiona (1.7.0)

Thanks!

Rich

Oh I bet that's somewhere where we set the logging level.  Offhand I bet it's because the logging level is something global.  You might try a `logging.basicConfig(level=logging.INFO)` or `WARN` at the top of your code but after the invest imports.  I'm sure James will have something more interesting to suggest too.

jdouglass

Hey guys, you're both right on the money ... there's an issue with the current version of natcap.invest where importing (almost) any of the models or the UI will set up a logging handler that sends ALL logging across all packages to stdout, b ut only if such a handler hasn't already been set.  This will be fixed with the next release of InVEST.

Until then, the best thing to do would be to do this in your script *before* importing natcap.invest:

import logging

logging.basicConfig(level=logging.ERROR)

This way, only log messages with an ERROR level or higher would print to stdout, and the buggy logging stuff in natcap.invest will be ignored.  The full list of logging levels is here: https://docs.python.org/2/library/logging.html#logging-levels, if you'd rather tweak how much logging is allowed through, and basicConfig has some more parameters as well if you'd like to format the logging messags that do make it through: 

Dave

Thanks guys, that did the trick!