400 Client Error for SentinelHubInputTask (Sentinel 1)

Apps / Services Python SDK
1

Hello.
Using SentinelHubInputTask, I want to retrieve S-1 data for a time period in multiple bboxes. I am requesting IW, descending, VV+VH, sigma0 backscatter coefficient, output in Float32.
I get the following Error:

sentinelhub.exceptions.DownloadFailedException: During execution of task SentinelHubInputTask: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
400 Client Error: Bad Request for url: https://services.sentinel-hub.com/api/v1/process
Server response: "{"error":{"status":400,"reason":"Bad Request","message":"Output custom requested but missing from function setup()","code":"COMMON_BAD_PAYLOAD"}}"

If create a request without the evalscript, it works (but I need dB conversion and I am also not sure that i get Sigma0 coef.).
I guess something is missing in my evalscript? The code I am using:

data_collection = DataCollection.SENTINEL1_IW_DES  # name of data source
res = 10  # resolution of EOPatches in meters

evalscript = """
//VERSION=3

function setup() {
  return {
    input: ["VV", "VH"],
    output: {bands: 2, 
    sampleType: "FLOAT32"} 
  }
}

// apply "toDb" function on input bands
function evaluatePixel(samples) {
  var VVdB = toDb(samples.VV)
  var VHdB = toDb(samples.VH)
  return [VVdB, VHdB]
}

// definition of "toDb" function
function toDb(linear) {
  var log = 10 * Math.log(linear) / Math.LN10   // VV and VH linear to decibels
  var val = Math.max(Math.min(log, 5), -30) 
  return val
}

"""

add_data = SentinelHubInputTask(evalscript=evalscript,
                                data_collection=data_collection,
                                bands_feature=(FeatureType.DATA, 'BANDS_S1'),
                                resolution=res,
                                time_difference=datetime.timedelta(minutes=120),
                                config=config,
                                aux_request_args={"processing":{"backCoeff": "SIGMA0_ELLIPSOID"}}
                                )

save = SaveTask(output_folder_eopatches_S1, overwrite_permission=OverwritePermission.OVERWRITE_PATCH)

workflow = LinearWorkflow(
        add_data,
        save)


execution_args = []
for bbox_id, bbox_coords in list_of_bboxes:
    execution_args.append({
        add_data: {'bbox': BBox(bbox_coords, crs=crs), 'time_interval': time_interval},
        save: {'eopatch_folder': 'eopatch_{}'.format(bbox_id)},
        #export_tiff: {'filename': '{}eopatch_{}/BANDS_S1_VV_VH.tiff'.format(output_tiff_location, bbox_id)},
    })

executor = EOExecutor(workflow, execution_args, save_logs=True, logs_folder=reports_folder)
executor.run(workers=5, multiprocess=False)
2

From the execution logs, it looks like the ogc/wfc request is used, I do not know, why… Isn’t SentinelHubInputTask a processing API task? Has this something to do with the 400 error?
Any help would be appreciated.

Log content:
2020-11-17 14:17:50,981 eolearn.core.eoworkflow DEBUG Computing SentinelHubInputTask(*(), **{‘bbox’: BBox(((236104.5, 2067085.5), (236344.5, 2067325.5)), crs=CRS(‘32616’)), ‘time_interval’: [‘2018-01-01’, ‘2019-12-31’]})

2020-11-17 14:17:50,981 sentinelhub.ogc DEBUG URL=https://services.sentinel-hub.com/ogc/wfs/INSTANCE_ID?SERVICE=wfs&WARNINGS=False&REQUEST=GetFeature&TYPENAMES=DSS3&BBOX=236104.5%2C2067085.5%2C236344.5%2C2067325.5&OUTPUTFORMAT=application%2Fjson&SRSNAME=EPSG%3A32616&TIME=2018-01-01T00%3A00%3A00%2F2019-12-31T23%3A59%3A59&MAXCC=100.0&MAXFEATURES=100&FEATURE_OFFSET=0

2020-11-17 14:17:51,513 sentinelhub.ogc DEBUG URL=https://services.sentinel-hub.com/ogc/wfs/INSTANCE_ID?SERVICE=wfs&WARNINGS=False&REQUEST=GetFeature&TYPENAMES=DSS3&BBOX=236104.5%2C2067085.5%2C236344.5%2C2067325.5&OUTPUTFORMAT=application%2Fjson&SRSNAME=EPSG%3A32616&TIME=2018-01-01T00%3A00%3A00%2F2019-12-31T23%3A59%3A59&MAXCC=100.0&MAXFEATURES=100&FEATURE_OFFSET=100

2020-11-17 14:17:51,993 eolearn.io.processing_api DEBUG Downloading 117 requests of type DataCollection.SENTINEL1_IW_DES
2020-11-17 14:17:59,742 root DEBUG EOWorkflow execution failed

3

Hi @adraksler,

try requesting S1 with this adjusted evalscript:

//VERSION=3

function setup() {
    return {
        input: [{
            bands: ["VV", "VH"]
        }],
        output: [
            { id:"custom", bands:2, sampleType: SampleType.FLOAT32 }
        ]
    }
}

function updateOutputMetadata(scenes, inputMetadata, outputMetadata) {
    outputMetadata.userData = { "norm_factor":  inputMetadata.normalizationFactor }
}


// apply "toDb" function on input bands
function evaluatePixel(samples) {
  var VVdB = toDb(samples.VV)
  var VHdB = toDb(samples.VH)
  return [VVdB, VHdB]
}

// definition of "toDb" function
function toDb(linear) {
  var log = 10 * Math.log(linear) / Math.LN10   // VV and VH linear to decibels
  var val = Math.max(Math.min(log, 5), -30) 
  return val
}

SentinelHubInputTask expects that also userdata output is defined in an evalscript. In the example above we achieve this by adding the updateOutputMetadata function. This is useful in case one requests S2 data in digital numbers and needs a normalization factor to convert to reflectance. This is irrelevant for S1 and we only added it here to satisfy requirments/limitations of SentinelHubInputTask. We will need to find a way to handle this in eo-learn better in the future. Thank you for reporting!

From the execution logs, it looks like the ogc/wfc request is used, I do not know, why… Isn’t SentinelHubInputTask a processing API task?

Yes, SentinelHubInputTask access the data using SH Process API. The requests you see in the logs are wfs requests which are used for searching (soon to be replaced by using SH Catalog API).

1 Like
4

Thanks for the adjusted evalscript and explanation, it works. The problem I have now is that I need 2000+ small EOPatches which means I would burn all available requests approximately after 700 EOPatches (145 requests per eopatch). Is there any workaround? I was thinking of getting the imagery for entire area first using WCS request and then create EOPatches using ImportFromTiff…
I tried this:

data_collection = DataCollection.SENTINEL1_IW_DES
data_layer = "BANDS-S1-IW-DES-SIG"
bbox = BBox(bbox=[(225000.0, 2059000.0), (241000.0, 2090000.0)], crs=CRS.UTM_16N)
time_interval = ('2017-01-01', '2017-03-30') #shorter for testing, should be 2017-2019

evalscript = """
//VERSION=3

function setup() {
    return {
        input: [{
            bands: ["VV", "VH"]
        }],
        output: [
            {bands:2, sampleType: SampleType.FLOAT32 }
        ]
    }
}

//function updateOutputMetadata(scenes, inputMetadata, outputMetadata) {
//    outputMetadata.userData = { "norm_factor":  inputMetadata.normalizationFactor }
//}

// apply "toDb" function on input bands
function evaluatePixel(samples) {
  var VVdB = toDb(samples.VV)
  var VHdB = toDb(samples.VH)
  return [VVdB, VHdB]
}

// definition of "toDb" function
function toDb(linear) {
  var log = 10 * Math.log(linear) / Math.LN10   // VV and VH linear to decibels
  var val = Math.max(Math.min(log, 5), -30)  // all values to interval from -30 to 5 dB
  var norm = (val+30)/35  // Normalization: (val-min)/(max-min)  max=5, min=-30
  return norm
}

"""

request = WcsRequest(data_folder=output_folder,
                     layer=data_layer,
                     data_collection=data_collection,
                     bbox=bbox,
                     time=time_interval,
                     time_difference=datetime.timedelta(minutes=120),
                     resx='10m',
                     resy='10m',
                     # image_format=MimeType.TIFF_d32f,
                     config=config,
                     custom_url_params={CustomUrlParam.EVALSCRIPT: evalscript}
                     )

request.get_data(save_data=True)

…and got 400 Client Error again:
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://services.sentinel-hub.com/ogc/wcs/INSTANCE_ID?SERVICE=wcs&WARNINGS=False&MAXCC=100.0&EvalScript=Ci8vVkVSU0lPTj0zCgpmdW5jdGlvbiBzZXR1cCgpIHsKICAgIHJldHVybiB7CiAgICAgICAgaW5wdXQ6IFt7CiAgICAgICAgICAgIGJhbmRzOiBbIlZWIiwgIlZIIl0KICAgICAgICB9XSwKICAgICAgICBvdXRwdXQ6IFsKICAgICAgICAgICAge2JhbmRzOjIsIHNhbXBsZVR5cGU6IFNhbXBsZVR5cGUuRkxPQVQzMiB9CiAgICAgICAgXQogICAgfQp9CgovL1NlbnRpbmVsSHViSW5wdXRUYXNrIGV4cGVjdHMgdGhhdCBhbHNvIHVzZXJkYXRhIG91dHB1dCBpcyBkZWZpbmVkIGluIGFuIGV2YWxzY3JpcHQsIGhlbmNlIHRoZQovL3VwZGF0ZU91dHB1dE1ldGFkYXRhIGZ1bmN0aW9uLiBPdGhlcndpc2UgaXJyZWxldmFudCBmb3IgUzEuIE1vcmUgaW5mbzoKLy9odHRwczovL2ZvcnVtLnNlbnRpbmVsLWh1Yi5jb20vdC80MDAtY2xpZW50LWVycm9yLWZvci1zZW50aW5lbGh1YmlucHV0dGFzay1zZW50aW5lbC0xLzI5MDYvMwoKLy9mdW5jdGlvbiB1cGRhdGVPdXRwdXRNZXRhZGF0YShzY2VuZXMsIGlucHV0TWV0YWRhdGEsIG91dHB1dE1ldGFkYXRhKSB7Ci8vICAgIG91dHB1dE1ldGFkYXRhLnVzZXJEYXRhID0geyAibm9ybV9mYWN0b3IiOiAgaW5wdXRNZXRhZGF0YS5ub3JtYWxpemF0aW9uRmFjdG9yIH0KLy99CgovLyBhcHBseSAidG9EYiIgZnVuY3Rpb24gb24gaW5wdXQgYmFuZHMKZnVuY3Rpb24gZXZhbHVhdGVQaXhlbChzYW1wbGVzKSB7CiAgdmFyIFZWZEIgPSB0b0RiKHNhbXBsZXMuVlYpCiAgdmFyIFZIZEIgPSB0b0RiKHNhbXBsZXMuVkgpCiAgcmV0dXJuIFtWVmRCLCBWSGRCXQp9CgovLyBkZWZpbml0aW9uIG9mICJ0b0RiIiBmdW5jdGlvbgpmdW5jdGlvbiB0b0RiKGxpbmVhcikgewogIHZhciBsb2cgPSAxMCAqIE1hdGgubG9nKGxpbmVhcikgLyBNYXRoLkxOMTAgICAvLyBWViBhbmQgVkggbGluZWFyIHRvIGRlY2liZWxzCiAgdmFyIHZhbCA9IE1hdGgubWF4KE1hdGgubWluKGxvZywgNSksIC0zMCkgIC8vIGFsbCB2YWx1ZXMgdG8gaW50ZXJ2YWwgZnJvbSAtMzAgdG8gNSBkQgogIHZhciBub3JtID0gKHZhbCszMCkvMzUgIC8vIE5vcm1hbGl6YXRpb246ICh2YWwtbWluKS8obWF4LW1pbikgIG1heD01LCBtaW49LTMwCiAgcmV0dXJuIG5vcm0KfQoK&BBOX=225000.0%2C2059000.0%2C241000.0%2C2090000.0&FORMAT=image%2Fpng&CRS=EPSG%3A32616&TIME=2017-02-17T09%3A52%3A53%2F2017-02-17T13%3A52%3A53&RESX=10m&RESY=10m&COVERAGE=BANDS-S1-IW-DES-SIG&REQUEST=GetCoverage&VERSION=1.1.2

The selected area is in Yucatan. I tested also in an area in Europe, and there was no problem.