SDR in 3.4.0rc2 - cell alignment shift compared to input DEM

Hi, I've just used the pre-release version 3.4.0rc2 to run the SDR in East Kalimantan, Indonesia. 
Thanks for all of your work on this release.

Do you know why the outputs have the same projection and cell size as the input DEM, but are shifted (by less than 1 cell) relative to the DEM's cell alignment?
The size of this shift seems to be larger when larger watersheds are used. The shift can be in different directions if different watershed shapefiles are used (i.e. alternative, adjacent watersheds with identically aligned boundaries based on the same input DEM)

The DEM cell size is 30.84220252 m
Output TIFs all have the same cell size, but are shifted by up to 30% of one cell length. (e.g. shifted to the east by approx. 13 m and south by 3 m. 
The shift is largest (c. 13 m) when I run the model using watersheds covering all of East Kalimantan, and is less (c. 5 m) when run for a single large watershed (e.g. Berau 14,000 km2.)
This is true for the preliminary outputs, flow direction, stream TIF etc., so it may arise during one of the early reprojection steps?

This isn't a massive spatial shift, but I don't know why it happens, and regret that:
1. separate runs for adjacent watersheds do not align (their shifts are slightly different, when the only difference in the model setup is the use of an adjacent watershed with an identically aligned shapefile boundary)
2. the outputs have less accurate stream locations, compared with delineation from the original DEM. 

Perhaps this is not large enough to be considered a problem, but I thought I should ask in case there's anything going wrong with the initial alignment of all datasets to the DEM.

This also happens to some extent (each slightly different) in previous versions including 3.3.3 and 3.3.3.post147+nc374360e6f37. 

In case it's helpful to see the data, all of the inputs and example 'watersheds' files (SubWS) are on Dropbox here:
(Parameters file: sdr_kal_example.json)

Some background on the other inputs: (I don't think anything suggests a reason for the cell alignment shift)
The watersheds align exactly with the DEM (from which they were delineated).
All layers are in the same UTM projection (WGS 1984 UTM N50)
Cell size is similar or slightly larger for other inputs (e.g. LULC 33 m), but I thought resampling was performed for all layers to align with the DEM, so this should have no impact on the alignment of the outputs.

Thanks for any light you can shed on this, or even just reassurance that this spatial shift doesn't sound like a major concern relative to other uncertainties.
- Jessie


  • Update: 
    The shift in alignment occurs if the LULC layer has any slight difference in cell size or alignment compared to the DEM. (even a tiny size difference such as 30.2 m vs 30.0 m)

    It does NOT occur if the LULC layer has been previously resampled to align exactly with the DEM, before being provided as input.

    This suggests that the processing is somehow influenced by the size/alignment of the LULC layer. However, I thought it was only supposed to depend on the DEM.

    In previous versions of InVEST there were different rules about which layers (DEM or LULC) were used as the basis for resampling, is it possible this is some sort of carry-over from that?

    My resolution for the moment is to manually align the LULC to the DEM  before running, so that the stream layer and all other outputs will match exactly to the DEM (rather than a partial compromise between the LULC and the DEM).

    Is it worth recommending this to users as a pre-processing step?
    (to avoid shift/displacement, which may only be a significant concern for very large datasets, i.e. large extent and/or fine resolution)
  • RichRich Administrator, NatCap Staff
    Hi @JessieWells, good eye on that alignment. No need to recommend pre-alignment to users, I patched that issue and a few more in SDR tonight. If you like, you can try this pre-reviewed development version: 

    Note on the back end, I've refactored a lot of SDR to use the new version of PyGeoprocessing which is sliiiiightly faster, but is going to help us enormously for our next generation platform we're developing. If you're watching closely on your end (and you seem to be) you'll notice some slight numerical differences between this development version and 3.4.0rc2. Those differences, if you see them, will be on the order of less than a relative tolerance of 1e-6. That's on the order of what we'd expect with 32 bit floating point calculations we have on the back end. Just mentioning in case you run into that too.

    Thanks for the very detailed report, and if we don't otherwise hear from you, this fix will roll into InVEST 3.4.0 when we release it.

Sign In or Register to comment.