Downloading rate hit exception

1

Hi,
I’m trying following process api request:

from sentinelhub import (SentinelHubRequest,DataCollection, 
                         MimeType, CRS, BBox, SHConfig,
                         MosaickingOrder,bbox_to_dimensions)

config = SHConfig()
bbox=BBox([451140,4931410,701140,5181410],crs="EPSG:32632")
resolution=100
size= bbox_to_dimensions(bbox, resolution=resolution) #2500x2500
time_interval=("2022-10-01","2022-10-15")

evalscript = """//VERSION=3
function setup() {
    return {
        input: [{bands: ["B02","B03","B04","B08","B8A","B11","CLM","dataMask"]}],
        output: [{id: "ndmi",bands: 1,sampleType: SampleType.UINT8,noDataValue: 0},
        {id: "ndsi",bands: 1,sampleType: SampleType.UINT8,noDataValue: 0},
        {id: "ndvi",bands: 1,sampleType: SampleType.UINT8,noDataValue: 0},
        {id: "rgb",bands: 3,sampleType: SampleType.UINT8,noDataValue: 0},
        {id: "date",bands:1,sampleType:SampleType.UINT8,noDataValue: 0 }
        ],
        mosaicking: Mosaicking.TILE
    };
}
function evaluatePixel(samples, scenes, inputMetadata, customData, outputMetadata) {
    // Your javascript code here
    let b02=0, b03=0,b04=0, b08=0,b8a=0, b11=0, day=0;
    for (let i=0;i<samples.length;i++){
        if(samples[i].CLM!=1 && samples[i].dataMask!=0){
            b02=samples[i].B02;
            b03=samples[i].B03;
            b04=samples[i].B04;
            b08=samples[i].B08;
            b8a=samples[i].B8A;
            b11=samples[i].B11;
            day=i+1
            break;
        }
    }
    outputMetadata.userData = { "metadata":  scenes.tiles }
    return {ndmi: [index(b8a,b11)*127+128],
    ndsi: [index(b03,b11)*127+128],
    ndvi: [index(b08,b04)*127+128],
    rgb: [2.5*b04*255,2.5*b03*255,2.5*b02*255],
    date:[day]
    };
}"""

request = SentinelHubRequest(evalscript=evalscript,
                                       input_data=[SentinelHubRequest.input_data(
                                           data_collection=DataCollection.SENTINEL2_L1C,
                                           time_interval=time_interval,
                                           mosaicking_order=MosaickingOrder.MOST_RECENT,
                                       )
                                                  ],
                                       responses=[SentinelHubRequest.output_response('ndmi', MimeType.TIFF),
                                                  SentinelHubRequest.output_response('ndsi', MimeType.TIFF),
                                                  SentinelHubRequest.output_response('ndvi', MimeType.TIFF),
                                                  SentinelHubRequest.output_response('rgb', MimeType.TIFF),
                                                  SentinelHubRequest.output_response('date', MimeType.TIFF),
                                                  SentinelHubRequest.output_response('userdata', MimeType.JSON)],
                                       bbox=bbox,
                                       size=size,
                                       config=config,
                                      )
    data=request.get_data()

The first try failed with “Download rate hit” exception (I couldn’t find any spec on download rate limit)

The following tries keep failing with the following:

DownloadFailedException: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
502 Server Error: Bad Gateway for url: https://services.sentinel-hub.com/api/v1/process
Server response: "502 Bad Gateway"

Could someone help me to understand why it keep failing?
Is it really a matter of download rate limit? If so what is this limit?
Any suggestion on preventing this throttling?
Thanks
Best regards

2

Hi Dario,
502 error would typically not be connected to related to Sentinel Hub’s rate limiting. Those would be either 429 or 403, depending which limit you hit.

502 is probably something internal, i.e. the error in data or error in our infrastructure.

In order to have a deeper look, we would kindly ask for:

  • enable logging and provide the logs (make sure to mask the second half of the client credentials)
  • let us know the first half of the client_id, so that we try to track the server logs on our side (even better, if you could create a new client id for this purpose, so that the request is solated)

As you have Enterprise subscription, you can also pass the above to our e-mail support (support@sentinel-hub.com), if you feel better about it. That said, if we can communicate here, it might help some other users as well.

Best,
Grega

3

Hi Grega,
thanks for your reply and useful references. From my side, as far as this could be useful to others too it’s ok to interact right here in the forum. Of course I will switch to direct support in case more specific/insight/urgent issues may arise. Thanks for your help!

Just retried the request with a new client_id (client_id=22b67972-157e-45b5-....), it worked smoothly!

Also retried with the client_id that failed this afternoon client_id=3e8c1f94-3abd-4eed-.....) and it succedded too.

Maybe was just a matter of temporary unavailablity/overload of the endpoint?

Best
Dario

4

Hi Dario,
happy to hear that things worked now.
It does indeed happen occassionally that some request fails. We work hard to reduce these issues, but with a system of such level of complexity and variability it is impossible to prevent all.
We recommend to introduce a retry (once or twice), which should work around just about all such problems.
I apologise for the inconvenience.
Best,
Grega

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.