This forum is shutting down! Please post new discussions at community.naturalcapitalproject.org

Help: PicklingError and Intersect error in Hydropower Water Yield.

Hello


I run the Hydropower Water Yield model on an area and get the following errors (see yellow for the errors I see in the log below).

I use InVEST 3.6.0 on a Mac. The model gives an output .tif of water yield but only for the borders of the sub watersheds.

I don't understand the errors very well... The subsub.shp file is the pink shapefile in the attached screenshot (LCraster on the background), and in does overlap with all the rasters in QGIS.

Furthermore I do not find what the PicklingError and certificate verify failed errors mean and I am not very familiar with Python language, so I was not able to solve this...


I am panicking a little bit since my deadline is approaching and I still need to run the model for a lot of watersheds.

Could anyone be so kind to help me out?


Thank you in advance!

 Looking forward to an answer!


Eva


Initializing...

Showing messages with level INFO and higher

04/10/2019 14:32:10 natcap.invest.utils INFO Writing log messages to /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/InVEST-Hydropower-Water-Yield-log-2019-04-10--14_32_10.txt

04/10/2019 14:32:10 natcap.invest.ui.model Level 100 Starting model with parameters:

Arguments for InVEST natcap.invest.hydropower.hydropower_water_yield 3.6.0:

biophysical_table_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/BiophystableCSV.csv

depth_to_root_rest_layer_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/root.tif

do_scarcity_and_valuation False

eto_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/ET.tif

lulc_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/LC.tif

n_workers -1

pawc_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/pawc.tif

precipitation_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/precCR2.tif

results_suffix 2

seasonality_constant 5

sub_watersheds_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/subsub.shp

watersheds_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/wsh14.shp

workspace_dir /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER


04/10/2019 14:32:10 natcap.invest.hydropower.hydropower_water_yield INFO Validating arguments

04/10/2019 14:32:10 natcap.invest.hydropower.hydropower_water_yield INFO Reclassifying temp_Kc raster

04/10/2019 14:32:10 natcap.invest.hydropower.hydropower_water_yield INFO Reclassifying tmp_root raster

04/10/2019 14:32:10 natcap.invest.hydropower.hydropower_water_yield INFO Reclassifying tmp_veg raster

04/10/2019 14:32:10 natcap.invest.hydropower.hydropower_water_yield INFO Calculate PET from Ref Evap times Kc

04/10/2019 14:32:10 natcap.invest.hydropower.hydropower_water_yield INFO Performing wyield operation

04/10/2019 14:32:10 pygeoprocessing.geoprocessing ERROR aggregate vector /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/subsub.shp does not intersect with the raster (u'/Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/intermediate/precip_2.tif', 1)

04/10/2019 14:32:10 taskgraph.Task ERROR Something went wrong when adding task subws_id_precip_mn_zonalstats (15), terminating taskgraph.

Traceback (most recent call last):

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 602, in add_task

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 1062, in _call

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 540, in zonal_stats_tofile

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1380, in dumps

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 331, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 401, in save_reduce

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 568, in save_tuple

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 754, in save_global

PicklingError: Can't pickle <function <lambda> at 0x11a4a8aa0>: it's not found as pygeoprocessing.geoprocessing.<lambda>

04/10/2019 14:32:10 natcap.invest.ui.model ERROR Exception while executing <function execute at 0x11f5cded8>

Traceback (most recent call last):

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/ui/model.py", line 1579, in _logged_target

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 413, in execute

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 602, in add_task

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 1062, in _call

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 540, in zonal_stats_tofile

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1380, in dumps

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 331, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 401, in save_reduce

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 568, in save_tuple

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 754, in save_global

PicklingError: Can't pickle <function <lambda> at 0x11a4a8aa0>: it's not found as pygeoprocessing.geoprocessing.<lambda>

04/10/2019 14:32:10 natcap.invest.ui.model INFO Execution finished

04/10/2019 14:32:10 natcap.invest.utils INFO Elapsed time: 0.19s

04/10/2019 14:32:10 natcap.invest.ui.execution ERROR Target <function _logged_target at 0x11a2d9938> failed with exception

Traceback (most recent call last):

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/ui/execution.py", line 68, in run

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/ui/model.py", line 1579, in _logged_target

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 413, in execute

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 602, in add_task

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 1062, in _call

File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 540, in zonal_stats_tofile

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1380, in dumps

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 331, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 401, in save_reduce

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 568, in save_tuple

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save

File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 754, in save_global

PicklingError: Can't pickle <function <lambda> at 0x11a4a8aa0>: it's not found as pygeoprocessing.geoprocessing.<lambda>

04/10/2019 14:32:10 natcap.invest.ui.execution INFO Execution finished

04/10/2019 14:32:11 natcap.invest.ui.usage._log_model WARNING an exception encountered when logging URLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),)

04/10/2019 14:32:11 natcap.invest.ui.usage._log_exit_status WARNING an exception encountered when _log_exit_status <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)>

Comments

  • swolnyswolny Member, NatCap Staff
    I'm not familar with the CERTIFICATE error either, but suspect that you can safely ignore that.

    However, the first error "subsub.shp does not intersect with the raster" suggests that the subwatershed shapefile might have a different coordinate system than your rasters. They need to be *exactly* the same projected coordinate system. Double-check this and if they don't match exactly, reproject/Warp to the coordinate system of your input rasters and see if that helps.

    ~ Stacie

  • DaveDave Member, Administrator, NatCap Staff
    Another thing to consider is the model will first clip all the input rasters by the "watersheds" extent. So if you have watersheds and subwatersheds, it assumes that subwatersheds are entirely within watersheds. If that's not true for your data, it might explain this first error message.

    Yes, safe to ignore the certificate error.
  • Hello Stacie and Dave

    Thank you for the quick answer and for your time!
    I was able to solve the intersect error now :) 
    The other 'picklinerror' however remains (log below)... Maybe this is related to the model stopping the computation when not finished yet, because the wateryield.tif only shows results for a part of the area (see screenshot).
    How can I solve this?

    Thank you in advance!
    Looking forward to the answer!

    Eva


    04/10/2019 15:35:13  natcap.invest.utils INFO     Writing log messages to /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/InVEST-Hydropower-Water-Yield-log-2019-04-10--15_35_13.txt
    04/10/2019 15:35:13  natcap.invest.ui.model Level 100 Starting model with parameters: 
    Arguments for InVEST natcap.invest.hydropower.hydropower_water_yield 3.6.0:
    biophysical_table_path        /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/BiophystableCSV.csv
    depth_to_root_rest_layer_path /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/root.tif
    do_scarcity_and_valuation     False
    eto_path                      /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/ET.tif
    lulc_path                     /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/LC.tif
    n_workers                     -1
    pawc_path                     /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/pawc.tif
    precipitation_path            /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/precCR2.tif
    results_suffix                4
    seasonality_constant          5
    sub_watersheds_path           /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/subsub.shp
    watersheds_path               /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/swsh14.shp
    workspace_dir                 /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER

    04/10/2019 15:35:13  natcap.invest.hydropower.hydropower_water_yield INFO     Validating arguments
    04/10/2019 15:35:13  natcap.invest.validation DEBUG    Validation warnings: []
    04/10/2019 15:35:13  pygeoprocessing.geoprocessing INFO     n_workers > 1 (2) so starting a processes pool.
    04/10/2019 15:35:14  pygeoprocessing.geoprocessing INFO     1 of 5 aligned: eto_4.tif
    04/10/2019 15:35:14  pygeoprocessing.geoprocessing INFO     2 of 5 aligned: precip_4.tif
    04/10/2019 15:35:15  pygeoprocessing.geoprocessing INFO     3 of 5 aligned: depth_to_root_rest_layer_4.tif
    04/10/2019 15:35:15  pygeoprocessing.geoprocessing INFO     4 of 5 aligned: pawc_4.tif
    04/10/2019 15:35:17  pygeoprocessing.geoprocessing INFO     5 of 5 aligned: clipped_lulc_4.tif
    04/10/2019 15:35:17  pygeoprocessing.geoprocessing INFO     aligned all 5 rasters.
    04/10/2019 15:35:17  natcap.invest.hydropower.hydropower_water_yield DEBUG    bio_lucodes set([0.0, 1, 2, 3, 4, 5, 6.0, 7, 8, 9, 10, 11, 12, 13, 14, 15.0, 16.0, 17.0, -9999.0])
    04/10/2019 15:35:17  natcap.invest.hydropower.hydropower_water_yield INFO     Checking that input tables have landcover codes for every value in the landcover map.
    04/10/2019 15:35:17  natcap.invest.hydropower.hydropower_water_yield INFO     Reclassifying temp_Kc raster
    04/10/2019 15:35:17  pygeoprocessing.geoprocessing INFO     starting stats_worker
    04/10/2019 15:35:17  pygeoprocessing.geoprocessing INFO     started stats_worker <Thread(Thread-6, started daemon 123145369608192)>
    04/10/2019 15:35:18  pygeoprocessing.geoprocessing INFO     100.0%% complete
    04/10/2019 15:35:18  pygeoprocessing.geoprocessing INFO     signaling stats worker to terminate
    04/10/2019 15:35:18  pygeoprocessing.geoprocessing_core DEBUG    payload is None, terminating
    04/10/2019 15:35:18  pygeoprocessing.geoprocessing INFO     Waiting for raster stats worker result.
    04/10/2019 15:35:20  natcap.invest.hydropower.hydropower_water_yield INFO     Reclassifying tmp_root raster
    04/10/2019 15:35:20  pygeoprocessing.geoprocessing INFO     starting stats_worker
    04/10/2019 15:35:20  pygeoprocessing.geoprocessing INFO     started stats_worker <Thread(Thread-7, started daemon 123145369608192)>
    04/10/2019 15:35:21  pygeoprocessing.geoprocessing INFO     100.0%% complete
    04/10/2019 15:35:21  pygeoprocessing.geoprocessing INFO     signaling stats worker to terminate
    04/10/2019 15:35:21  pygeoprocessing.geoprocessing_core DEBUG    payload is None, terminating
    04/10/2019 15:35:21  pygeoprocessing.geoprocessing INFO     Waiting for raster stats worker result.
    04/10/2019 15:35:23  natcap.invest.hydropower.hydropower_water_yield INFO     Reclassifying tmp_veg raster
    04/10/2019 15:35:23  pygeoprocessing.geoprocessing INFO     starting stats_worker
    04/10/2019 15:35:23  pygeoprocessing.geoprocessing INFO     started stats_worker <Thread(Thread-8, started daemon 123145369608192)>
    04/10/2019 15:35:24  pygeoprocessing.geoprocessing INFO     100.0%% complete
    04/10/2019 15:35:24  pygeoprocessing.geoprocessing INFO     signaling stats worker to terminate
    04/10/2019 15:35:24  pygeoprocessing.geoprocessing_core DEBUG    payload is None, terminating
    04/10/2019 15:35:24  pygeoprocessing.geoprocessing INFO     Waiting for raster stats worker result.
    04/10/2019 15:35:25  natcap.invest.hydropower.hydropower_water_yield INFO     Calculate PET from Ref Evap times Kc
    04/10/2019 15:35:25  pygeoprocessing.geoprocessing INFO     starting stats_worker
    04/10/2019 15:35:25  pygeoprocessing.geoprocessing INFO     started stats_worker <Thread(Thread-9, started daemon 123145369608192)>
    04/10/2019 15:35:26  pygeoprocessing.geoprocessing INFO     100.0%% complete
    04/10/2019 15:35:26  pygeoprocessing.geoprocessing INFO     signaling stats worker to terminate
    04/10/2019 15:35:26  pygeoprocessing.geoprocessing_core DEBUG    payload is None, terminating
    04/10/2019 15:35:26  pygeoprocessing.geoprocessing INFO     Waiting for raster stats worker result.
    04/10/2019 15:35:27  natcap.invest.hydropower.hydropower_water_yield DEBUG    Performing fractp operation
    04/10/2019 15:35:27  pygeoprocessing.geoprocessing INFO     starting stats_worker
    04/10/2019 15:35:27  pygeoprocessing.geoprocessing INFO     started stats_worker <Thread(Thread-10, started daemon 123145369608192)>
    04/10/2019 15:35:29  pygeoprocessing.geoprocessing INFO     100.0%% complete
    04/10/2019 15:35:29  pygeoprocessing.geoprocessing INFO     signaling stats worker to terminate
    04/10/2019 15:35:29  pygeoprocessing.geoprocessing_core DEBUG    payload is None, terminating
    04/10/2019 15:35:29  pygeoprocessing.geoprocessing INFO     Waiting for raster stats worker result.
    04/10/2019 15:35:29  natcap.invest.hydropower.hydropower_water_yield INFO     Performing wyield operation
    04/10/2019 15:35:29  pygeoprocessing.geoprocessing INFO     starting stats_worker
    04/10/2019 15:35:29  pygeoprocessing.geoprocessing INFO     started stats_worker <Thread(Thread-11, started daemon 123145369608192)>
    04/10/2019 15:35:30  pygeoprocessing.geoprocessing INFO     100.0%% complete
    04/10/2019 15:35:30  pygeoprocessing.geoprocessing INFO     signaling stats worker to terminate
    04/10/2019 15:35:30  pygeoprocessing.geoprocessing_core DEBUG    payload is None, terminating
    04/10/2019 15:35:30  pygeoprocessing.geoprocessing INFO     Waiting for raster stats worker result.
    04/10/2019 15:35:30  natcap.invest.hydropower.hydropower_water_yield DEBUG    Performing aet operation
    04/10/2019 15:35:30  pygeoprocessing.geoprocessing INFO     starting stats_worker
    04/10/2019 15:35:30  pygeoprocessing.geoprocessing INFO     started stats_worker <Thread(Thread-12, started daemon 123145369608192)>
    04/10/2019 15:35:31  pygeoprocessing.geoprocessing INFO     100.0%% complete
    04/10/2019 15:35:31  pygeoprocessing.geoprocessing INFO     signaling stats worker to terminate
    04/10/2019 15:35:31  pygeoprocessing.geoprocessing_core DEBUG    payload is None, terminating
    04/10/2019 15:35:31  pygeoprocessing.geoprocessing INFO     Waiting for raster stats worker result.
    04/10/2019 15:35:31  pygeoprocessing.geoprocessing DEBUG    <osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x116f4acf0> >
    04/10/2019 15:35:31  pygeoprocessing.geoprocessing DEBUG    n_workers == 1 so a threadpool is sufficient
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     1 of 1 aligned: clipped_rastereqsPM4.tif
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     aligned all 1 rasters.
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     build a lookup of aggregate field value to FID
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     creating disjoint polygon set
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     build shapely polygon list
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     build shapely rtree index
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     poly feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     build poly intersection lookup
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     poly intersection feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     maximal subset build 100.0% complete on swsh14.shp
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     processing 1 disjoint polygon sets
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     disjoint polygon set 1 of 1 100.0% processed on swsh14.shp
    04/10/2019 15:35:32  pygeoprocessing.geoprocessing INFO     rasterizing disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:33  pygeoprocessing.geoprocessing INFO     summarizing rasterized disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:33  pygeoprocessing.geoprocessing DEBUG    unset_fids: 0 of 1 
    04/10/2019 15:35:33  pygeoprocessing.geoprocessing DEBUG    gt [ 6.2911331e+05  8.6827106e+00  0.0000000e+00  5.8596815e+06
      0.0000000e+00 -8.6827106e+00] for (u'/Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/intermediate/precip_4.tif', 1)
    04/10/2019 15:35:33  pygeoprocessing.geoprocessing DEBUG    remaining unset_fids: 0 of 1 
    04/10/2019 15:35:33  pygeoprocessing.geoprocessing INFO     all done processing polygon sets for swsh14.shp
    04/10/2019 15:35:33  pygeoprocessing.geoprocessing DEBUG    <osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x116f4acf0> >
    04/10/2019 15:35:33  pygeoprocessing.geoprocessing DEBUG    n_workers == 1 so a threadpool is sufficient
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     1 of 1 aligned: clipped_raster4h4v2a.tif
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     aligned all 1 rasters.
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     build a lookup of aggregate field value to FID
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     creating disjoint polygon set
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     build shapely polygon list
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     build shapely rtree index
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     poly feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     build poly intersection lookup
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     poly intersection feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     maximal subset build 100.0% complete on swsh14.shp
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     processing 1 disjoint polygon sets
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     disjoint polygon set 1 of 1 100.0% processed on swsh14.shp
    04/10/2019 15:35:36  pygeoprocessing.geoprocessing INFO     rasterizing disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:37  pygeoprocessing.geoprocessing INFO     summarizing rasterized disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:37  pygeoprocessing.geoprocessing DEBUG    unset_fids: 0 of 1 
    04/10/2019 15:35:37  pygeoprocessing.geoprocessing DEBUG    gt [ 6.2911331e+05  8.6827106e+00  0.0000000e+00  5.8596815e+06
      0.0000000e+00 -8.6827106e+00] for (u'/Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/intermediate/pet_4.tif', 1)
    04/10/2019 15:35:37  pygeoprocessing.geoprocessing DEBUG    remaining unset_fids: 0 of 1 
    04/10/2019 15:35:37  pygeoprocessing.geoprocessing INFO     all done processing polygon sets for swsh14.shp
    04/10/2019 15:35:37  pygeoprocessing.geoprocessing DEBUG    <osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x1171251e0> >
    04/10/2019 15:35:37  pygeoprocessing.geoprocessing DEBUG    n_workers == 1 so a threadpool is sufficient
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     1 of 1 aligned: clipped_rasterxvsevk.tif
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     aligned all 1 rasters.
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     build a lookup of aggregate field value to FID
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     creating disjoint polygon set
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     build shapely polygon list
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     build shapely rtree index
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     poly feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     build poly intersection lookup
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     poly intersection feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     maximal subset build 100.0% complete on swsh14.shp
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     processing 1 disjoint polygon sets
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     disjoint polygon set 1 of 1 100.0% processed on swsh14.shp
    04/10/2019 15:35:38  pygeoprocessing.geoprocessing INFO     rasterizing disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:39  pygeoprocessing.geoprocessing INFO     summarizing rasterized disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:39  pygeoprocessing.geoprocessing DEBUG    unset_fids: 0 of 1 
    04/10/2019 15:35:39  pygeoprocessing.geoprocessing DEBUG    gt [ 6.2911331e+05  8.6827106e+00  0.0000000e+00  5.8596815e+06
      0.0000000e+00 -8.6827106e+00] for (u'/Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/output/per_pixel/aet_4.tif', 1)
    04/10/2019 15:35:39  pygeoprocessing.geoprocessing DEBUG    remaining unset_fids: 0 of 1 
    04/10/2019 15:35:39  pygeoprocessing.geoprocessing INFO     all done processing polygon sets for swsh14.shp
    04/10/2019 15:35:39  pygeoprocessing.geoprocessing DEBUG    <osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x117128150> >
    04/10/2019 15:35:39  pygeoprocessing.geoprocessing DEBUG    n_workers == 1 so a threadpool is sufficient
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     1 of 1 aligned: clipped_raster5Frzhx.tif
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     aligned all 1 rasters.
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     build a lookup of aggregate field value to FID
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     creating disjoint polygon set
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     build shapely polygon list
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     build shapely rtree index
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     poly feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     build poly intersection lookup
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     poly intersection feature lookup 100.0% complete on swsh14.shp
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     maximal subset build 100.0% complete on swsh14.shp
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     processing 1 disjoint polygon sets
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     disjoint polygon set 1 of 1 100.0% processed on swsh14.shp
    04/10/2019 15:35:40  pygeoprocessing.geoprocessing INFO     rasterizing disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:41  pygeoprocessing.geoprocessing INFO     summarizing rasterized disjoint polygon set 1 of 1 swsh14.shp
    04/10/2019 15:35:41  pygeoprocessing.geoprocessing DEBUG    unset_fids: 0 of 1 
    04/10/2019 15:35:41  pygeoprocessing.geoprocessing DEBUG    gt [ 6.2911331e+05  8.6827106e+00  0.0000000e+00  5.8596815e+06
      0.0000000e+00 -8.6827106e+00] for (u'/Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/output/per_pixel/wyield_4.tif', 1)
    04/10/2019 15:35:41  pygeoprocessing.geoprocessing DEBUG    remaining unset_fids: 0 of 1 
    04/10/2019 15:35:41  pygeoprocessing.geoprocessing INFO     all done processing polygon sets for swsh14.shp
    04/10/2019 15:35:41  pygeoprocessing.geoprocessing DEBUG    <osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x117128630> >
    04/10/2019 15:35:41  pygeoprocessing.geoprocessing ERROR    aggregate vector /Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/subsub.shp does not intersect with the raster (u'/Users/evatoebat/Desktop/INPUTSSWSH14PROBEER/intermediate/precip_4.tif', 1)
    04/10/2019 15:35:41  taskgraph.Task     ERROR    Something went wrong when adding task subws_id_precip_mn_zonalstats (15), terminating taskgraph.
    Traceback (most recent call last):
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 602, in add_task
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 1062, in _call
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 540, in zonal_stats_tofile
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1380, in dumps
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 331, in save
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 401, in save_reduce
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 568, in save_tuple
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 754, in save_global
    PicklingError: Can't pickle <function <lambda> at 0x11c69f848>: it's not found as pygeoprocessing.geoprocessing.<lambda>
    04/10/2019 15:35:41  natcap.invest.ui.model ERROR    Exception while executing <function execute at 0x11c484ed8>
    Traceback (most recent call last):
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/ui/model.py", line 1579, in _logged_target
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 413, in execute
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 602, in add_task
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/taskgraph/Task.py", line 1062, in _call
      File "/Users/jenkins/jenkins_build/workspace/natcap.invest/label/Stanford-osx-1/env/lib/python2.7/site-packages/natcap/invest/hydropower/hydropower_water_yield.py", line 540, in zonal_stats_tofile
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1380, in dumps
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 331, in save
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 401, in save_reduce
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 568, in save_tuple
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
      File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 754, in save_global
    PicklingError: Can't pickle <function <lambda> at 0x11c69f848>: it's not found as pygeoprocessing.geoprocessing.<lambda>
    04/10/2019 15:35:41  natcap.invest.ui.model INFO     Execution finished
    04/10/2019 15:35:41  natcap.invest.utils INFO     Elapsed time: 28.52s

  • DaveDave Member, Administrator, NatCap Staff
    Hmm I still see the same "does not intersect" error in your log as well. If you can't get to the bottom of it feel free to send your data to me and I can try to reproduce it. davefisher@stanford.edu
  • Hello Dave

    Oh, good you notice this. Thank you. 
    I just sent the data via email (my email is eva.toebat@student.kuleuven.be).

    Thank you for your time and help!

    Eva
  • DaveDave Member, Administrator, NatCap Staff
    This looks like it was as Stacie suggested,
    one of the watershed shapefiles had a different coordinate system than
    the rest of the data. The subwatersheds are UTM zone 19S, other datasets are UTM zone 18S. Projecting the subwatersheds should
    resolve these errors!
  • Hello Dave

    I must have looked wrong while rechecking the coordinate system.
    Now the model works!

    Thank you for the help!

    Eva
Sign In or Register to comment.