Issue in downloading S2 L1C images

EO Products / Scripts
1

I am downloading S2 L1c data in several area worldwide. In most of them the data are obtained without issue, but in one area I found some errors.

Here is the code:

coordstat=[127.39,  70.68]

area1=[coordstat[0]-0.02,coordstat[1]-0.02,coordstat[0]+0.02,coordstat[1]+0.02]

roi_bbox = BBox(bbox=area1, crs=CRS.WGS84)

time_interval = (str(2021)+'-10-01T00:00:00.000000000', str(years[-1]+2)+'-09-30T23:59:59.000000000')
class AddValidDataMaskTask(EOTask):
    def execute(self, eopatch):
        #eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(eopatch.mask["CLM"].astype(bool) ) & ~(np.greater(eopatch.mask["CLP"],50))
        eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(np.greater(eopatch.mask["CLP"],cld_thr/100*255))
        return eopatch

download_task = SentinelHubInputTask(data_collection=DataCollection.SENTINEL2_L1C,
                                     bands=bands,
                                     bands_feature=(FeatureType.DATA, 'BANDS'),
                                     additional_data=[(FeatureType.MASK, 'dataMask'), (FeatureType.MASK, 'CLP')],
                                     resolution=10
                                    )
print('downloading data '+time_interval[0]+'-'+time_interval[1])
input_node = EONode(download_task)
filter_clouds = EONode(AddValidDataMaskTask(),inputs=[input_node])
output_node = EONode(OutputTask("final_eopatch"), inputs=[filter_clouds])
workflow = EOWorkflow([input_node, filter_clouds, output_node])
result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})
eopatch_clean = result.outputs['final_eopatch']

This is the error that I obtained:

DownloadFailedException: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process
Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to s3://sentinel-s2-l1c-index/tiles/51/W/XU/2023/3/10/0/B08.index. HTTP Status: '404'", "code": "RENDERER_EXCEPTION"}"

DownloadFailedException: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process
Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to s3://sentinel-s2-l1c-index/tiles/51/W/XU/2023/3/10/0/B08.index. HTTP Status: '404'", "code": "RENDERER_EXCEPTION"}"
2

I obtained something similar in another station:

coordstat=[-152.12, 69.36]

area1=[coordstat[0]-0.02,coordstat[1]-0.02,coordstat[0]+0.02,coordstat[1]+0.02]

roi_bbox = BBox(bbox=area1, crs=CRS.WGS84)

time_interval = (str(2016)+'-10-01T00:00:00.000000000', str(2023)+'-09-30T23:59:59.000000000')
class AddValidDataMaskTask(EOTask):
    def execute(self, eopatch):
        #eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(eopatch.mask["CLM"].astype(bool) ) & ~(np.greater(eopatch.mask["CLP"],50))
        eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(np.greater(eopatch.mask["CLP"],cld_thr/100*255))
        return eopatch

download_task = SentinelHubInputTask(data_collection=DataCollection.SENTINEL2_L1C,
                                     bands=bands,
                                     bands_feature=(FeatureType.DATA, 'BANDS'),
                                     additional_data=[(FeatureType.MASK, 'dataMask'), (FeatureType.MASK, 'CLP')],
                                     resolution=10
                                    )
print('downloading data '+time_interval[0]+'-'+time_interval[1])
input_node = EONode(download_task)
filter_clouds = EONode(AddValidDataMaskTask(),inputs=[input_node])
output_node = EONode(OutputTask("final_eopatch"), inputs=[filter_clouds])
workflow = EOWorkflow([input_node, filter_clouds, output_node])
result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})
eopatch_clean = result.outputs['final_eopatch']

And I got a similar error

DownloadFailedException: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process
Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to s3://sentinel-s2-l1c-index/tiles/5/W/NS/2023/4/7/0/B08.index. HTTP Status: '404'", "code": "RENDERER_EXCEPTION"}"
3

Hi @paolo.filippucci ,

Thank you for reporting the issue. I’ve forwarded the issue to our developers and will let you know when the issue is fixed.

4

Hi @paolo.filippucci ,

The issue should be fixed. Please try again and let us know if the issue persists.

5

Hi,
It has been solved, but I found another one with S3:

DownloadFailedException: Failed to download from:
https://creodias.sentinel-hub.com/api/v1/process
with HTTPError:
500 Server Error: Internal Server Error for url: https://creodias.sentinel-hub.com/api/v1/process
Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to creo://EODATA/Sentinel-3/OLCI/OL_1_EFR/2017/12/01/S3A_OL_1_EFR____20171201T040107_20171201T040407_20171202T082843_0179_025_104_2520_LN1_O_NT_002.SEN3/geo_coordinates.nc. HTTP Status: '404' On CreoDIAS, you get 'Illegal request 403 error' also when a file is missing. So first make sure that the file is present.", "code": "RENDERER_EXCEPTION"}"
6

Hi @paolo.filippucci ,

It seems this file EODATA/Sentinel-3/OLCI/OL_1_EFR/2017/12/01/S3A_OL_1_EFR____20171201T040107_20171201T040407_20171202T082843_0179_025_104_2520_LN1_O_NT_002.SEN3/geo_coordinates.nc is missing in the bucket.

I would recommend reporting the issue to CreoDIAS support.

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.