API 500 Server Error

Apps / Services Python SDK
1

I’m using the SentinelHubInputTask from eolearn to create a workflow to retrieve Sentinel-2 scenes (not a full scene, but a small 256x256 aoi).
I’ve used my code a while, but now I’m getting the following error:

Any suggestions on how to resolve this?
My guess is that this error is is specific to the tiles and datetimes I’m requesting data for.

sentinelhub.exceptions.DownloadFailedException: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process
Server response: "{"error":{"status":500,"reason":"Internal Server Error","message":"Illegal request to s3://sentinel-s2-l2a-index/tiles/37/T/DK/2018/1/21/0/qi/CLD_20m.index. HTTP Status: '404'","code":"RENDERER_EXCEPTION"}}"

This is my code:

resolution = 10
time_difference = datetime.timedelta(hours=1)
bands = ['B01','B02','B03','B04','B05','B06','B07','B08','B8A','B09','B11','B12']
input_task = SentinelHubInputTask(
			data_collection = DataCollection.SENTINEL2_L2A,
			bands= bands,
			bands_feature = (FeatureType.DATA, 'L2A_data'),
			bands_dtype = np.uint16,
			additional_data = [(FeatureType.MASK, 'SCL'), (FeatureType.MASK, 'CLD')],
			resolution = resolution,
			time_difference = time_difference,
			config = config(),
			max_threads = 4
			)

output_task = OutputTask("eopatch")
workflow_nodes = linearly_connect_tasks(input_task, output_task)
workflow = EOWorkflow(workflow_nodes)
result = workflow.execute(
    {
        workflow_nodes[0]: {"bbox": roi_bbox, "time_interval": time_interval}	    }
)
eopatch = result.outputs["eopatch"]
2

Hi Hamed,

Thanks for your question, happy to help look into this for you. Are you able to share any more of the code that you are using to help me debug the issue? I am not sure what you maybe doing before this part of the code.

In addition, have you tested your assumption that the error maybe specific to the tiles and date times in your original request?

UPDATE: It seems that the tile was missing, we have reprocessed this so your request should not return an error now.

1 Like
3

Hi @william.ray,

Thanks for checking on this, and adding the missing tile.
I believe there is at least one more tile that I’m running in to the same problem with.
Here is the error:

sentinelhub.exceptions.DownloadFailedException: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process
Server response: "{"error":{"status":500,"reason":"Internal Server Error","message":"Illegal request to s3://sentinel-s2-l2a-index/tiles/30/S/WH/2018/1/16/0/qi/CLD_20m.index. HTTP Status: '404'","code":"RENDERER_EXCEPTION"}}"

Best,
Hamed

4

Hi William,
Another update: I can confirm this is the only tile/scene that I have problem with, and my pipeline works well for others.
So probably this is another missing tile.
Let me know if you get the chance to reprocess it.

Thanks,
Hamed

5

Hi Hamed,

Thanks, we will reprocess the missing tile as soon as we can :slight_smile:

Happy that your pipeline is working well apart from that!

6

Thanks William, I appreciate it if you can update me here when it’s reprocessed.

Hamed

7

Hi @william.ray,
I have found more missing scenes.
The following two fail with similar errors as the previous ones:
sentinel-s2-l2a-index/tiles/36/S/XH/2018/1/29/0/qi/CLD_20m.index
and
sentinel-s2-l2a-index/tiles/37/T/EJ/2018/1/11/0/qi/CLD_20m.index

In addition, when trying to get data from the following scene:
sentinel-s2-l2a-index/tiles/43/S/CS/2018/3/7/

I get a 502 error with empty response (details at the bottom). This might be a similar issue in which one of the files for that scene is missing. Can you please look into these?
Also these have become a blocker on our training data generation. Would it be possible to prioritize their reprocessing?

Thanks,
Hamed

/usr/local/lib/python3.7/site-packages/eolearn/core/eoworkflow.py in execute(self, input_kwargs, raise_errors)
    152 
    153         output_results, stats_dict = self._execute_nodes(
--> 154             uid_input_kwargs=uid_input_kwargs, out_degrees=out_degrees, raise_errors=raise_errors
    155         )
    156 

/usr/local/lib/python3.7/site-packages/eolearn/core/eoworkflow.py in _execute_nodes(self, uid_input_kwargs, out_degrees, raise_errors)
    208                 node_input_values=[intermediate_results[input_node.uid] for input_node in node.inputs],
    209                 node_input_kwargs=uid_input_kwargs.get(node.uid, {}),
--> 210                 raise_errors=raise_errors,
    211             )
    212 

/usr/local/lib/python3.7/site-packages/eolearn/core/eoworkflow.py in _execute_node(self, node, node_input_values, node_input_kwargs, raise_errors)
    238         LOGGER.debug("Computing %s(*%s, **%s)", node.task.__class__.__name__, str(task_args), str(node_input_kwargs))
    239         start_time = dt.datetime.now()
--> 240         result, is_success = self._execute_task(node.task, task_args, node_input_kwargs, raise_errors=raise_errors)
    241         end_time = dt.datetime.now()
    242 

/usr/local/lib/python3.7/site-packages/eolearn/core/eoworkflow.py in _execute_task(task, task_args, task_kwargs, raise_errors)
    266         """Executes an EOTask and handles any potential exceptions."""
    267         if raise_errors:
--> 268             return task.execute(*task_args, **task_kwargs), True
    269 
    270         try:

/usr/local/lib/python3.7/site-packages/eolearn/io/sentinelhub_process.py in execute(self, eopatch, bbox, time_interval)
     94         LOGGER.debug("Downloading %d requests of type %s", len(requests), str(self.data_collection))
     95         client = SentinelHubDownloadClient(config=self.config)
---> 96         responses = client.download(requests, max_threads=self.max_threads)
     97         LOGGER.debug("Downloads complete")
     98 

/usr/local/lib/python3.7/site-packages/sentinelhub/download/sentinelhub_client.py in download(self, *args, **kwargs)
     49         self.lock = Lock()
     50         try:
---> 51             return super().download(*args, **kwargs)
     52         finally:
     53             self.lock = None

/usr/local/lib/python3.7/site-packages/sentinelhub/download/client.py in download(self, download_requests, max_threads, decode_data, show_progress)
     85         else:
     86             for future in as_completed(download_list):
---> 87                 data_list[future_order[future]] = self._process_download_future(future)
     88 
     89         if is_single_request:

/usr/local/lib/python3.7/site-packages/sentinelhub/download/client.py in _process_download_future(self, future)
     99             if self.raise_download_errors:
    100                 traceback = sys.exc_info()[2]
--> 101                 raise download_exception.with_traceback(traceback)
    102 
    103             warnings.warn(str(download_exception), category=SHRuntimeWarning)

/usr/local/lib/python3.7/site-packages/sentinelhub/download/client.py in _process_download_future(self, future)
     95         """
     96         try:
---> 97             return future.result()
     98         except DownloadFailedException as download_exception:
     99             if self.raise_download_errors:

/usr/local/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
    426                 raise CancelledError()
    427             elif self._state == FINISHED:
--> 428                 return self.__get_result()
    429 
    430             self._condition.wait(timeout)

/usr/local/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
    382     def __get_result(self):
    383         if self._exception:
--> 384             raise self._exception
    385         else:
    386             return self._result

/usr/local/lib/python3.7/concurrent/futures/thread.py in run(self)
     55 
     56         try:
---> 57             result = self.fn(*self.args, **self.kwargs)
     58         except BaseException as exc:
     59             self.future.set_exception(exc)

/usr/local/lib/python3.7/site-packages/sentinelhub/download/client.py in _single_download(self, request, decode_data)
    118             return None
    119 
--> 120         response_content = self._execute_download(request)
    121 
    122         if request_path and request.save_response and (self.redownload or not os.path.exists(request_path)):

/usr/local/lib/python3.7/site-packages/sentinelhub/download/handlers.py in new_download_func(self, request)
     52                 if attempt_num == download_attempts - 1:
     53                     message = _create_download_failed_message(exception, request.url)
---> 54                     raise DownloadFailedException(message) from exception
     55 
     56                 LOGGER.debug('Download attempt failed: %s\n%d attempts left, will retry in %ds', exception,

DownloadFailedException: Failed to download from:
https://services.sentinel-hub.com/api/v1/process
with HTTPError:
502 Server Error: Bad Gateway for url: https://services.sentinel-hub.com/api/v1/process
Server response: ""
8

Thanks Hamed, I will let the team know :slight_smile:

1 Like