Impacts of the migration to Evalscript V3

Why we’re doing it (reasons and benefits)

Currently we have four different types of scripts. Simple, V1, V2, V3. It is confusing, hard to maintain, and for older scripts limits available features. V3 scripts do everything the previous versions can and add numerous useful functionalities. These include dataMask, units, data fusion, mosaicking selection ability, precisely defining the output, etc. After the change only V3 and simple scripts will remain; we are keeping simple scripts for the reason that for basic functionality and beginner users they are most easy to use.
An additional change is the removal of the TRANSPARENCY and BGCOLOR parameters in OGC requests. This is to remove the ambiguity of using these parameters alongside evalscipts. In any case their usage can be entirely replicated (and improved) by slightly modifying your evalscripts.
Sentinel Hub has for quite some time now outgrown the OGC standards and the goal is to consolidate new features in V3 evalscripts, while still supporting OGC standards.

When and how it will happen

As briefly summarized in the April’s newsletter (https://www.sentinel-hub.com/newsletters/20200407), we have already:

  • added V3 support to OGC services
  • added simple script support to process API
  • converted the scripts in the Custom Script repository
  • provided an API to convert v1 and v2 evalscripts to v3

In the near future, that is, at least the next few months, we will still support all versions. We will inform users at least 6 weeks prior to the end of the support. We would however recommend you start porting your scripts today.

Why some things can’t be done automagically

Complex evalscripts and parameter combinations (especially through OGC) limit automatic conversion to V3. For most simple-enough scripts (which can be quite complex actually) this can be done. Unfortunately, we can’t cover all the use-cases so if you fall into that category a small amount of work will be required.
The OGC FORMAT parameter influences the returned output even if the evalscript is the same. Since we cannot know what parameter value(s) you use this makes autoconversion unreliable.

The most important consequences of transition and how it impacts you

We realize change can be inconvenient, however we believe the simplification it brings is worth it. During the migration some additional work on your evalscripts might be necessary, as well as testing to make sure it runs correctly. We are doing our best to keep migration issues at a minimum, however due to the complexity of everything we would be surprised if no further issues pop up. Contact us if you find anything that isn’t working as expected.

What will happen when migration to evalscript V3 is finalised

Evalscript

Evalscripts V1 and V2 will stop working.
DataMask will become available in simple scripts via OGC.

impacts on OGC services

  • The depth= suffix of the format parameter won’t be supported anymore. If used, an error will be returned. The bit depth and the value of the response is fully under control of the evalscript. If the output of the evalscript can’t be “fitted” to the format of the OGC request (e.g., the evalscript defines sampleType: "FLOAT32" and the requested format is an 8bit integer) an error will also be returned.
  • The transparent parameter is deprecated. If used, it will be ignored. If the evalscript produces an output that has a transparency channel in the sense of four channel RGBA and the format supports it (for example PNG) an image with transparency will be returned. If the evalscript returns more channels that the format supports, the output is clipped to the number of channels (e.g. the evalscript returns 4 channels, JPG is requested, then only the first 3 returned channels will be placed in the JPG). This allows “transparent” scripts to be used with non-transparent formats, so you don’t need a separate script for each.
  • The bgColor parameter is deprecated. If used, it will be ignored. If your require a specific background color, modify the evalscript to do so. For how to do this, see here
  • The maximum size limit of 2500x2500 px will now be enforced.

A while ago we announced here that we are migrating data processing at Sentinel hub (SH) to evalscripts V3. We have prepared a converter and added support for evalscripts V3 to the EO Browser and to the Playground. We are now taking further steps towards the migration:

  • For each of the data products which used evalscripts V1 or V2 we added their corresponding V3 version. When creating new layers you can now select only data products which use either evalscripts V3 or simple script.
  • We converted all V1 and V2 evalscripts in our configuration templates.
  • We converted all V1 and V2 evalscripts in the custom scripts repository and in our documentation.
  • In the Dashboard:
    • layers which use evalscript V1 or V2 (or data products V1 or V2) are now underlined with red color and marked with a small icon (up arrow)
    • you can convert V1 and V2 evalscripts (and data products) with a click of the button
    • you can convert and revert the conversion of the data products if needed
  • We prepared a detailed tutorial for you to help you migrate to V3

All these changes have had no effect on your existing SH settings and configurations. Your SH requests still work as they have so far. This will remain so until the November 1st. This transition period is meant to give you enough time to fully migrate your SH configurations to V3.

Once the transition period is over we will disable support for evalscripts V1 and V2. At that point we will automatically:

  • convert all non-converted evalscripts to V3 using the converter API and
  • swap old data products (V1 and V2) with the corresponding V3 ones.
    Depending on how you request the data from SH, this automatic conversion might result in obtaining different values from SH with existing SH requests as explained in impacts on OGC services chapter here. We thus encourage you to take full control over the conversion and convert your scripts and data products before the end of the transition period.

In case of any further questions, feel free to contact us on our forum.

1 Like

We would like to inform you that we disabled V1 and V2 processing pipelines as the last step of migration to evalscript V3. This has no impact on your system if you have already converted your evalscripts to V3 as explained in our V3 Transition guide. If you have not yet done so, you might notice that:

  • SH now returns an error if you request data using evalscript V1 or V2
  • All simple scripts are now processed with pipeline V3 which can result in different values obtained from SH in some cases. Check the guide (liked above) for more details.

Let us know if you experience any problems because of this change or if you need help with adjusting your evalscripts or requests.

Dear all,
thank your for your developments. Especially thanks for all the information and important notifications that warn the user in time!

I have a question on impact of the migration on OGC WMS and situation when you define equation directly in your request and not via script in configuration page.

In our processing we use OGC WMS service. In particular, we define layer=‘CUSTOM-SCRIPT’ and send the script with EVALSCRIPT=ourscript in the req. As script we use some super simple script (‘return [(B08 / B05) - 1];’) .

In the request itself we use following:
raw = get_image(lai_script, day, wkt, res=‘10m’, format=‘image/tiff;depth=32f’, crs=“EPSG:4326”).
resulting in…
http://services.sentinel-hub.com/ogc/wms/e67d8cd3-cdbf-MASKED?SERVICE=WMS&REQUEST=GetMap&VERSION=1.1.1&LAYERS=CUSTOM-SCRIPT&CRS=EPSG%3A4326&TIME=2020-02-27%2F2020-02-27&MAXCC=100&RESX=10m&RESY=10m&FORMAT=image%2Ftiff%3Bdepth%3D32f&ATMFILTER=NONE&TRANSPARENT=0&GEOMETRY=POLYGON+((17.93543815612793+47.67868046074383%2C+17.93380737304688+47.67157196089703%2C+17.93603897094727+47.67099399151448%2C+17.93698310852051+47.67035821780065%2C+17.93998718261719+47.66862425012441%2C+17.94419288635254+47.66995363049563%2C+17.94651031494141+47.67082059945155%2C+17.94891357421875+47.67891161521251%2C+17.94007301330566+47.67873824945702%2C+17.93543815612793+47.67868046074383))&EVALSCRIPT=cmV0dXJuIFsoQjA4IC8gQjA1KSAtIDFdOw==

Since 2 days we are getting 8bit for the above listed call.

What to do to the url to reflect changes and get again 16 or 32f?

thanks ahead for reply and thank you for your time,
Marcela

Dear @marcela.doubkova,
if you send EVALSCRIPT with a parameter, you simply need to change this parameter to include sampleType as described in this (and the following) use-cases:
https://docs.sentinel-hub.com/api/latest/user-guides/transition/#use-case-4-formatimagetiffdepth16-or-format-is-not-specified