Speed of FIS vs Statistical API requests

[andre.stumpf]

Dears,

We’ve been happily using the FIS API for quite a while now to fetch S-2 B04, B08, SCL for single pixels and subsquently visualize the NDVI time-series.

The core function for this is fairly simple, the bounding box is typically 10x10m:

from sentinelhub import FisRequest

def download_l2a_ext(bbox, percent_cc, config, download_cache='D:/Temp', bands=['B04', 'B08', 'SCL']):
    
    time_interval = ('2017-01-01', '2021-12-30')
    eval_script = 'return[' + ','.join([s for s in bands]) + ']'

    fis_request = FisRequest(
        layer='BANDS-S2-L2A',
        geometry_list=[bbox],
        time=time_interval,
        maxcc=percent_cc/100,
        resolution='10m',
        data_folder=download_cache,
        config=config,
        custom_url_params={CustomUrlParam.EVALSCRIPT: eval_script},
    )
    fis_data = fis_request.get_data(save_data=True)
...

This is fairly fast and with a few further steps of local compute (i.e. mask clouds and shadows, compute NDVI, put it in a matplotlib plot) one typically gets the NDVI time series for 5 years within 3-4 seconds.

Since it has been announced that the FIS API will be deprecated we are having a look at the Statisitcal API where the equivalent seems to take one minute or more:

curl -X POST https://services.sentinel-hub.com/api/v1/statistics \
 -H 'Authorization: Bearer XXXX' \
 -H 'Accept: application/json' \
 -H 'Content-Type: application/json' \
 -d '{
  "input": {
    "bounds": {
      "bbox": [
        786460,
        4642960,
        786470,
        4642970
      ],
      "properties": {
        "crs": "http://www.opengis.net/def/crs/EPSG/0/32632"
      }
    },
    "data": [
      {
        "dataFilter": {},
        "type": "sentinel-2-l2a"
      }
    ]
  },
  "aggregation": {
    "timeRange": {
      "from": "2017-01-01T00:00:00Z",
      "to": "2021-12-31T23:59:59Z"
    },
    "aggregationInterval": {
      "of": "P5D",
      "lastIntervalBehavior": "SHORTEN"
    },
    "resx": "10",
    "resy": "10",
    "evalscript": "//VERSION=3\nfunction setup() {\n  return {\n    input: [{\n      bands: [\n        \"B04\",\n        \"B08\",\n        \"SCL\",\n        \"dataMask\"\n      ]\n    }],\n    output: [\n      {\n        id: \"data\",\n        bands: 3\n      },\n      {\n        id: \"scl\",\n        sampleType: \"INT8\",\n        bands: 1\n      },\n      {\n        id: \"dataMask\",\n        bands: 1\n      }]\n  };\n}\n\nfunction evaluatePixel(samples) {\n    let index = (samples.B08 - samples.B04) / (samples.B08+samples.B04);\n    return {\n        data: [index, samples.B08, samples.B04],\n        dataMask: [samples.dataMask],\n        scl: [samples.SCL]\n    };\n}\n"
  },
  "calculations": {
    "default": {}
  }
}'

Given that our use case if for live interaction with the user such a long wait time is quite a bottleneck.

Two questions:

1. Is there anything that seems odd in the Statistical API call above which we should change to improve response time? Could the usage of difference user accounts (payed account for FIS, free account for Statistical API) explain the performance difference.
2. If not is it save to continue to build on the FIS API (actually our preffered solution) for the foreseeable future (i.e. at least for one year)?

Thanks and best,
André

[william.ray]

Hi André,

Apologies for not responding faster. We are currently investigating this and will try and give you an answer and solution as soon as we can.

Best,

William

[chung.horng (Chung)]

Hi @andre.stumpf ,

Thank you for reporting the issue. Our developers are aware of the issue and working on improving the performance. We’ll keep you up to date regarding the improvement.

As for continuing using FIS, it is officially deprecated so switching to statistical api is still recommended.

A workaround to avoid long response time at the moment is to separate the multi-year request to several single-year requests. For example, making separate requests for 2017, 2018, 2019, 2020, and 2021 in your case could shorten the response time. Also, Statistical API can skip no data automatically so you can actually put P1D to get all available data in your time range as FIS does.

[andre.stumpf]

Hi Chung and William,
Thanks for the feedback and for looking into the issue. Indeed the requests for shorter time windows are aproximately linearly faster (i.e. 50% shorter time window > 50% shorter response time); we will check if we can do a workaround with mulitple parallel requests for shorter time spans.
Having this said, could you provide further details regarding when good-old FIS will be switched off for good and when you expect further performance boosts for the Statistical API? Would help us a lot to decide whether we should switch now or later…

Thanks and best,
André

[gmilcinski (Grega Milcinski)]

Hi Andre,

we don’t have the timing for turning off FIS. With such a large breaking change we would (and will) make an announcement at least 6 months ahead.
And until there are some significant performance issues for the “replacement service” (i.e. Statistical API), not limited to border issues, we will almost certainly keep it running as we don’t want to degrade the user experience. Therefore, your feedback here is noted and well appreciated.

In your case it might make sense to wait with the decision until you hera back from us on the mitigation plans.

Best Regards,
Grega

[andre.stumpf]

Thanks Grega,

That makes a lot of sense… I should mention that we already see several advantages of the Statistical API and really think it moves in the right direction; if it would not be for the NRT interaction we need for this specific use case we would switch already now.

Best regards,
André