Error in SDR: biophysical table, watershed and workspace no recognised as a supported file format

Hello InVEST community!

I recently learned about the very promising opportunities of the different models. However I did not manage to execute the SDR succesfully. The log indicates that there seem to some errors with my biophysical table and the watershed-shape, but I couldn`t find the problem. The columns are named as required. Codes of land-use/land-cover clases of the biophysical table are consistant with the lulc-raster file.
All data is georeferenced in the same coordinate system (EPSG 23030, ED50 /UTM 30N).
Furthermore InVEST does also detect an error with the workspace, even thogh it is just an empty folder.

These are log-lines of erros:

02/25/2018 12:50:25  osgeo              ERROR    [errno 4] `C:/Users/tobia/Documents/Sync/CGRBIO/a_academico/Ecoinformatica/a_exercicio_final/SDR/datos/tabla_biofisica_srd_03_2.csv' not recognised as a supported file format.
02/25/2018 12:50:25  osgeo              ERROR    [errno 4] `C:/Users/tobia/Documents/Sync/CGRBIO/a_academico/Ecoinformatica/a_exercicio_final/SDR/datos/watershed/watershed_1.shp' not recognised as a supported file format.
02/25/2018 12:50:25  osgeo              ERROR    [errno 4] `C:\Users\tobia\Documents\Sync\CGRBIO\a_academico\Ecoinformatica\a_exercicio_final\SDR\datos\workspace' not recognised as a supported file format.

However InVEST continues working, creates some files and intermediate outputs in the workspace and ends up with the following error message:

ValueError: The following 1 raster values [ 0.] from "C:\Users\tobia\Documents\Sync\CGRBIO\a_academico\Ecoinformatica\a_exercicio_final\SDR\datos\workspace\aligned_lulc.tif" do not have corresponding entries in the `value_map`: {131.0: 0.0, 901.0: 0.27, 520.0: 0.00155, 650.0: 0.00155, 471.0: 0.21, 141.0: 0.0, 911.0: 0.05, 530.0: 0.00155, 915.0: 0.05, 510.0: 0.00155, 917.0: 0.05, 151.0: 0.0, 921.0: 0.07, 155.0: 0.0, 540.0: 0.00155, 925.0: 0.27, 415.0: 0.2, 419.0: 0.2, 932.0: 0.0, 550.0: 0.00155, 423.0: 0.0, 425.0: 0.08, 427.0: 0.08, 429.0: 0.08, 815.0: 0.07, 433.0: 0.2, 821.0: 0.08, 439.0: 0.00155, 825.0: 0.07, 315.0: 0.00155, 411.0: 0.21, 317.0: 0.00155, 830.0: 0.08, 191.0: 0.08, 193.0: 0.0, 451.0: 0.08, 455.0: 0.08, 725.0: 0.05, 715.0: 0.05, 461.0: 0.21, 721.0: 0.00155, 341.0: 0.004, 441.0: 0.08, 473.0: 0.12, 475.0: 0.13, 477.0: 0.12, 479.0: 0.12, 481.0: 0.2, 611.0: 0.00155, 934.0: 0.27, 465.0: 0.21, 489.0: 0.2, 615.0: 0.05, 621.0: 0.00155, 750.0: 0.00155, 111.0: 0.0, 625.0: 0.05, 115.0: 0.0, 117.0: 0.0, 121.0: 0.0, 345.0: 0.004, 895.0: 0.08}

The complete log-file, the biopysical table and the watershed-shape are attached.

Thanks in advance. I would be very glad about any help!



Comments

  • swolnyswolny Member, NatCap Staff
    Hi @tho033 -

    I'm not sure why InVEST is giving you those errors about not being recognized as a supported file format, I'll leave that to the software team to comment on. 

    However, looking at the biophysical table, I see two things that could be causing the final ValueError that stops the model from running. There's a blank line at the end of the .csv (which InVEST used to have a problem with, I'm not sure if that's still the case), and the lucodes are floating point values, when they need to be integer. If you look at the table in Excel these aren't obvious, but if you look at it in a text editor like Notepad++, you can see these two things. 

    So try changing the lucodes to integer, and remove the  blank line at the end of the biphysical table, and see if the ValueError goes away.

    By the way, looking at the watershed.shp attribute table, I also see that there are duplicate ws_id values (for example, several watersheds have value 22) - these need to be unique integers for each watershed, no duplicates.

    ~ Stacie




  • Hi swolny,
    thanks for the fast reply! I changed lucode to integer and dissolved the shape-file to have unique ws_id values. However I still got more or less the same error messages:

    02/25/2018 21:24:13  natcap.invest.ui.model ERROR    Exception while executing <function execute at 0x0E40D130>
    Traceback (most recent call last):
      File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\ui\model.py", line 1504, in _logged_target
      File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\sdr.py", line 117, in execute
      File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\utils.py", line 457, in build_lookup_from_csv
    ValueError: lucode expected in [u'lucode;"usle_c";"usle_p"'] for the CSV file at C:/Users/tobia/Documents/Sync/CGRBIO/a_academico/Ecoinformatica/a_exercicio_final/SDR/datos/watershed/tabla_biofisica_srd_03_3.csv
    02/25/2018 21:24:13  osgeo              ERROR    [errno 4] `C:/Users/tobia/Documents/Sync/CGRBIO/a_academico/Ecoinformatica/a_exercicio_final/SDR/datos/watershed/tabla_biofisica_srd_03_3.csv' not recognised as a supported file format.
    02/25/2018 21:24:13  natcap.invest.ui.model INFO     Execution finished
    02/25/2018 21:24:13  natcap.invest.utils INFO     Elapsed time: 0.08s
    02/25/2018 21:24:13  osgeo              ERROR    [errno 4] `C:/Users/tobia/Documents/Sync/CGRBIO/a_academico/Ecoinformatica/a_exercicio_final/SDR/datos/watershed/watershed_1_diss.shp' not recognised as a supported file format.
    02/25/2018 21:24:13  osgeo              ERROR    [errno 4] `C:\Users\tobia\Documents\Sync\CGRBIO\a_academico\Ecoinformatica\a_exercicio_final\SDR\datos\workspace2' not recognised as a supported file format.
    02/25/2018 21:24:13  natcap.invest.ui.execution ERROR    Target <function _logged_target at 0x0E40DA30> failed with exception
    Traceback (most recent call last):
      File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\ui\execution.py", line 68, in run
      File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\ui\model.py", line 1504, in _logged_target
      File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\sdr.py", line 117, in execute
      File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\utils.py", line 457, in build_lookup_from_csv
    ValueError: lucode expected in [u'lucode;"usle_c";"usle_p"'] for the CSV file at C:/Users/tobia/Documents/Sync/CGRBIO/a_academico/Ecoinformatica/a_exercicio_final/SDR/datos/watershed/tabla_biofisica_srd_03_3.csv

    again I attached the log-file and modified watershed - and biophysical table files
  • swolnyswolny Member, NatCap Staff
    Something definitely changed about the biophysical table - it now has values separated by semicolons (but InVEST requires commas, since it's a Comma-Separated-Value table, and your first version of the table used commas) and there are quote marks around the field names usle_c and usle_p, but not lucode.

    What software are you using to create the biophysical table? If Excel, you can try starting with a copy of the sample data biophysical table, keep the field names, and replace the lucodes and values with the ones you've created for your area. Then save as CSV (comma-delimited) and see what happens. You should be able to open the sample data and do a similar process in other software too.

    ~ Stacie


  • jdouglassjdouglass Administrator, NatCap Staff
    I can chime in on the OSGEO errors in the original post ... you can safely ignore them.  They're from the UI, confirming (albeit in a not-so-helpful way) that those files are not spatial datasets.  We're working on a fix to remove messages like these from the model's logging in a future version of InVEST.
  • Hi again, thanks for your help Stacie and jdouglass!
    In general I used Excel to create the biophysical table, but I also used Notepad++ to check the csv-file. So I changed delimiter to comma again and made shure that there are quote marks around lucode, usle_c and usle_p. However, again I get the initial errors. So while ignoring the OSGEO errors there still remains the ValueError:

    ValueError: The following 1 raster values [0] from "C:\Users\tobia\Documents\Sync\CGRBIO\a_academico\Ecoinformatica\a_exercicio_final\SDR\datos\workspace3\aligned_lulc.tif" do not have corresponding entries in the `value_map`: {131.0: 0.0, 901.0: 0.27, 520.0: 0.0016, 650.0: 0.0016, 471.0: 0.21, 141.0: 0.0, 911.0: 0.05, 530.0: 0.0016, 915.0: 0.05, 510.0: 0.0016, 917.0: 0.05, 151.0: 0.0, 921.0: 0.07, 155.0: 0.0, 540.0: 0.0016, 925.0: 0.27, 415.0: 0.2, 419.0: 0.2, 932.0: 0.0, 550.0: 0.0016, 423.0: 0.0, 425.0: 0.08, 427.0: 0.08, 429.0: 0.08, 815.0: 0.07, 433.0: 0.2, 821.0: 0.08, 439.0: 0.0016, 825.0: 0.07, 315.0: 0.0016, 411.0: 0.21, 317.0: 0.0016, 830.0: 0.08, 191.0: 0.08, 193.0: 0.0, 451.0: 0.08, 455.0: 0.08, 725.0: 0.05, 715.0: 0.05, 461.0: 0.21, 721.0: 0.0016, 341.0: 0.004, 441.0: 0.08, 473.0: 0.12, 475.0: 0.13, 477.0: 0.12, 479.0: 0.12, 481.0: 0.2, 611.0: 0.0016, 934.0: 0.27, 465.0: 0.21, 489.0: 0.2, 615.0: 0.05, 621.0: 0.0016, 750.0: 0.0016, 111.0: 0.0, 625.0: 0.05, 115.0: 0.0, 117.0: 0.0, 121.0: 0.0, 345.0: 0.004, 895.0: 0.08}

    As I thought this could indicate a problem in the lulc-raster layer, I made shure that the values of the raster are integer (and not floating values). I also again checked that lulc-classes of the biophysical table and the lulc-rater layer are exactly the same. However, the error still stays the same. Do you have any more ideas?

    PS: In the attached zip file you will find the lulc-raster layer and the biophysical table.
  • swolnyswolny Member, NatCap Staff
    I brought your land cover raster into ArcGIS and created a raster attribute table, and there are cells with a value of 0. So you will either need to add value 0 to the biophysical table, or remove it from the map (since it looks like it's in slivers along the map edges, which you're probably not analyzing.

    See if that helps.

    ~ Stacie
  • hi,

    have the same problem:

    04/20/2018 14:59:55 natcap.invest.hydropower.hydropower_water_yield INFO Starting Water Yield Core Calculations

    04/20/2018 14:59:55 osgeo ERROR [errno 4] `C:/Users/Fra_Senza_Profilo/Documents/TESI/PROVA/biophysical_table2.csv' not recognised as a supported file format.

    04/20/2018 14:59:55 osgeo ERROR [errno 4] `C:\Users\Fra_Senza_Profilo\Documents\hydropower_water_yield_workspace' not recognised as a supported file format.

    04/20/2018 14:59:55 osgeo ERROR [errno 4] `C:/Users/Fra_Senza_Profilo/Documents/TESI/PROVA/WSHEDE.shp' not recognised as a supported file format.

    04/20/2018 14:59:55 natcap.invest.ui.model ERROR Exception while executing <function execute at 0x0DAC22B0>

    Traceback (most recent call last):

    File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\ui\model.py", line 1504, in _logged_target

    File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\hydropower\hydropower_water_yield.py", line 139, in execute

    TypeError: float() argument must be a string or a number

    04/20/2018 14:59:55 natcap.invest.ui.model INFO Execution finished

    04/20/2018 14:59:55 natcap.invest.utils INFO Elapsed time: 0.19s

    04/20/2018 14:59:55 natcap.invest.ui.execution ERROR Target <function _logged_target at 0x0490B1B0> failed with exception

    Traceback (most recent call last):

    File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\ui\execution.py", line 68, in run

    File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\ui\model.py", line 1504, in _logged_target

    File "c:\users\natcap-servers\jenkins-home\workspace\natcap.invest\label\gce-windows-1\release_env\lib\site-packages\natcap\invest\hydropower\hydropower_water_yield.py", line 139, in execute

    TypeError: float() argument must be a string or a number

    04/20/2018 14:59:55 natcap.invest.ui.execution INFO Execution finished



    first of all thx a lot for your time,


    and how can i fix it??



  • swolnyswolny Member, NatCap Staff
    Hi @FRASELVINI -

    As James mentioned above, you don't need to worry about the error "...not recognised as a supported file format."

    The other error is a problem though, "TypeError: float() argument must be a string or a number". Is the .txt file that you attached your biophysical table? If so, it appears to be missing the actual coefficients that go along with the LULC codes and descriptions. You're running the water yield model, which requires root depth and Kc values, both of which need to be floating point. Please see the Water Yield Data Needs section of the User Guide for more information.

    ~ Stacie



  • thx a lot,

    i will try to solve the problem.


Sign In or Register to comment.