"Out of retries" and other errors


#1

Hi,

We are currently working on the script contest, but we have some difficulties to test our scripts.

We use the technique describe here : https://www.sentinel-hub.com/faq/how-handle-long-scripts-eo-browser?_ga=2.230420703.251349194.1554207108-664072082.1537186943
Even the example scripte provided do not work on our side.
Our script neither (even using https://apps.sentinel-hub.com/sentinel-playground-temporal/?temporal=true).

So we tried the old way of injecting the script in the URL. This do not work, and we have some difficult to understand errors like “<![CDATA[ Out of retries ]]>” .

I also tried to generate image using script that worked in the past for sure. And they do not longer work.

And sometime, event Sentinel Playground temporal do not load at all, and I only have a webpage with the following message :
##
Error

<small>{"config":{"transformRequest":{},"transformResponse":{},"timeout":0,"xsrfCookieName":"XSRF-TOKEN","xsrfHeaderName":"X-XSRF-TOKEN","maxContentLength":-1,"headers":{"Accept":"application/json, text/plain, */*"},"method":"get","responseType":"text","url":"https://services.sentinel-hub.com/ogc/wms/<INSTANCE_ID>?SERVICE=WMS&amp;REQUEST=GetCapabilities&amp;time=1554367535065"},"request":{}}</small>

Are we doing something bad ?

Thank you,

(edited by GM to remove the public link to the script, in case you would want to use it for competition…)


#2

Hi @ibisa,

there are many issues mentioned in this ticket.

  1. The scripts itself
    The script, which you have linked to in GitHub (I have removed the link in case it contains some proprietary information of yours, please add it back if not), does not have appropriate return.
    For script to be visualized, it needs to either return 3-dimensional array (return [R,G,B]:wink: or 1-dimensional array (return [GRAY];).

  2. Long scripts
    If you are want to use the linked script in EO Browser, you have to provide direct link to it.
    E.g. this is not OK:
    https://github.com/sentinel-hub/custom-scripts/blob/master/sentinel-2/lai/script.js
    This is OK:
    https://raw.githubusercontent.com/sentinel-hub/custom-scripts/master/sentinel-2/lai/script.js
    (you need to click “Raw” in GitHub to get to source data)

  3. Out of retries error
    This error happened in the period, when Sentinel Hub system was under significant load and was (temporarily) only partially operational. You should not get these normally.

I hope this helps. Feel free to come up with follow-up questions.
Looking forward to see your submission.


#3

Hi @gmilcinski,

Thank you for your answer.

  1. Indeed :slight_smile: We have corrected that

  2. Yes this is what we did. But still it do not work. Even just applying this example do not work : https://www.sentinel-hub.com/faq/how-handle-long-scripts-eo-browser?_ga=2.25932670.251349194.1554207108-664072082.1537186943
    But this is not that much a problem, we use now directly the URL generator.

We have now another question.
We throw errors to debug our code.
One of the question we had was the number of scenes we filter.
Here is our filtering code :
const filterScenes = (scenes, metadataInput, nbPastYears) => {
const filteredScenes = scenes.filter((scene) => scene.date.getMonth() === metadataInput.to.getMonth()
&& scene.date.getFullYear() >= metadataInput.to.getFullYear() - nbPastYears)
throw new Error(filteredScenes.length + ’ / ’ + scenes.length)
}
global.filterScenes = (scenes, metadataInput) => {
return utils.filterScenes(scenes, metadataInput, config.default.nbPastYears)
}

The problem we see is that the total number of scenes increase if we change the date of the request. But the number of filtered scene is always 4.
Our objective is to filter scenes to extract only the scenes of the same month (eg January) over of the last 4 years. And we are surprise to have only 4 scenes.
So this imply that in 4 months, we only have 4 scenes ?
For information, MAXCC=100
Did we miss something ?

Thank you,


#4

There are too many bits of info missing to be able to provide valuable support.
Can you perhaps provide an example of the full request? With script encoded in the URL?
Do mask the INSTANCEID and do remove the bits of the custom script, which might have some IPR value for you.


#5

You can find the script in the link you edited in the first post.
The interesting line is the line 126.


#6

Huh, that script is so complicated that I cannot figure it out, what is happening.
I can only provide some general advice.

The basic skeleton for multi-temporal script is along the lines:

function setup (dss) {
  // get all bands for display and analysis
  setInputComponents([dss.B02,dss.B03,dss.B04,dss.B05,dss.B08,dss.B12]);
  // return as RGB
  setOutputComponentCount(3);
}

// you should reduce number of scenes you are processing as much as possible here to speed up the processing

function filterScenes (scenes, inputMetadata) {  
return scenes.filter(function (scene) {
	//filter scenes by date or order to only retrieve the ones needed in the evaluatePixel
}
function evaluatePixel(samples,scenes) {  
	//run through all scenes (for each pixel independently) and do the math with pixels values
}

If you change these three functions, they will probably not be called by our processing engine.

Therefore I suggest you replace

var filterScenes = function filterScenes(scenes, metadataInput, nbPastYears) {
  return scenes.filter(function (scene) {
    return scene.date.getMonth() === metadataInput.to.getMonth() && scene.date.getFullYear() >= metadataInput.to.getFullYear() - nbPastYears;
  });
};

with something along the lines:

function filterScenes (scenes, inputMetadata) {  
	var nbPastYears=5;
	var filteredScenes = [];
	for (i=0;var i < scenes.length ; i++)
	{
		if (scene[i].date.getMonth()===metadataInput.to.getMonth() && scene[i].date.getFullYear() >= metadataInput.to.getFullYear() - nbPastYears) 
		{
			filteredScenes.push(scene[i]);
		}
	}  
	return filteredScenes;
}

I feel that the nbPastYears variable in your function is undefined, which might explain why there are always only 4 scenes… only from the last year.
Perhaps your function would work simply if you put the variable in the function itself.