Not same outputs from Playground and EOBrowser

unap · May 10, 2019, 6:01pm

Recently, I created a configuration instance that I named max_NDVI (ea2b05a7-e51e-MASKED) by using max_ndvi script.

https://github.com/sentinel-hub/custom-scripts/blob/master/sentinel-2/max_ndvi/script.js

Instance consists of two layers:

max_NDVI. To calculate maximum NDVI value per pixel between today and las 31 days. setOutputComponentCount(3) to get three bands.
max_NDVI_value. Same than max_NDVI, but setOutputComponentCount(1) to get NDVI values.

I realized that at Playground it works accurately because I just see clouds over lakes (normally water giver lower NDVI values than clouds). Nevertheless, at EOBrowser, scene at same date is different; I see clouds over vegetation.

Here you can find both scripts:

max_NDVI:

//Basic initialization setup function
function    setup (dss) {
//List of all bands, that will be used in the script, either for visualization or for choosing best pixel
  setInputComponents([dss.B04,dss.B08]);
//This can always be the same if one is doing RGB images
  setOutputComponentCount(3);
}

/*
In this function we limit the scenes, which are used for processing. 
These are based also on input variables, coming from Playground. 
E.g. if one sets date "2017-03-01" ("TO date") and cloud coverage filter 30%, 
all scenes older than 2017-03-01 with cloud coverage 30% will be checked against
further conditions in this function.
The more scenes there are, longer it will take to process the data.
After 60 seconds of processing, there will be a timeout.
*/

function filterScenes (scenes, inputMetadata) {
    return scenes.filter(function (scene) {
//Here we limit data between "(TO date - 1 month) to (TO date)
	  return scene.date.getTime()>=(inputMetadata.to.getTime()-1*31*24*3600*1000) ;
    });
}

function calcNDVI(sample) {
  var denom = sample.B04+sample.B08;
  return ((denom!=0) ? (sample.B08-sample.B04) / denom : 0.0);
}
function evaluatePixel(samples) {  
  var max = 0;
  for (var i=0;i<samples.length;i++) {
      var ndvi = calcNDVI(samples[i]);
    max = ndvi > max ? ndvi:max;
  }
if (max<-1.1) return [0,0,0];
else if (max<-0.2) return [0.75,0.75,0.75];
else if (max<-0.1) return [0.86,0.86,0.86];
else if (max<0) return [1,1,0.88];
else if (max<0.025) return [1,0.98,0.8];
else if (max<0.05) return [0.93,0.91,0.71];
else if (max<0.075) return [0.87,0.85,0.61];
else if (max<0.1) return [0.8,0.78,0.51];
else if (max<0.125) return [0.74,0.72,0.42];
else if (max<0.15) return [0.69,0.76,0.38];
else if (max<0.175) return [0.64,0.8,0.35];
else if (max<0.2) return [0.57,0.75,0.32];
else if (max<0.25) return [0.5,0.7,0.28];
else if (max<0.3) return [0.44,0.64,0.25];
else if (max<0.35) return [0.38,0.59,0.21];
else if (max<0.4) return [0.31,0.54,0.18];
else if (max<0.45) return [0.25,0.49,0.14];
else if (max<0.5) return [0.19,0.43,0.11];
else if (max<0.55) return [0.13,0.38,0.07];
else if (max<0.6) return [0.06,0.33,0.04];
else return [0,0.27,0];

}

max_NDVI_values:

//Basic initialization setup function
function    setup (dss) {
//List of all bands, that will be used in the script, either for visualization or for choosing best pixel
  setInputComponents([dss.B04,dss.B08]);
//This can always be the same if one is doing RGB images
  setOutputComponentCount(1);
}

/*
In this function we limit the scenes, which are used for processing. 
These are based also on input variables, coming from Playground. 
E.g. if one sets date "2017-03-01" ("TO date") and cloud coverage filter 30%, 
all scenes older than 2017-03-01 with cloud coverage 30% will be checked against
further conditions in this function.
The more scenes there are, longer it will take to process the data.
After 60 seconds of processing, there will be a timeout.
*/

function filterScenes (scenes, inputMetadata) {
    return scenes.filter(function (scene) {
//Here we limit data between "(TO date - 1 month) to (TO date)
	  return scene.date.getTime()>=(inputMetadata.to.getTime()-1*31*24*3600*1000) ;
    });
}

function calcNDVI(sample) {
  var denom = sample.B04+sample.B08;
  return ((denom!=0) ? (sample.B08-sample.B04) / denom : 0.0);
}
function evaluatePixel(samples) {  
  var max = 0;
  for (var i=0;i<samples.length;i++) {
      var ndvi = calcNDVI(samples[i]);
    max = ndvi > max ? ndvi:max;
  }
if (max<-1.1) return [0,0,0];
else if (max<-0.2) return [0.75,0.75,0.75];
else if (max<-0.1) return [0.86,0.86,0.86];
else if (max<0) return [1,1,0.88];
else if (max<0.025) return [1,0.98,0.8];
else if (max<0.05) return [0.93,0.91,0.71];
else if (max<0.075) return [0.87,0.85,0.61];
else if (max<0.1) return [0.8,0.78,0.51];
else if (max<0.125) return [0.74,0.72,0.42];
else if (max<0.15) return [0.69,0.76,0.38];
else if (max<0.175) return [0.64,0.8,0.35];
else if (max<0.2) return [0.57,0.75,0.32];
else if (max<0.25) return [0.5,0.7,0.28];
else if (max<0.3) return [0.44,0.64,0.25];
else if (max<0.35) return [0.38,0.59,0.21];
else if (max<0.4) return [0.31,0.54,0.18];
else if (max<0.45) return [0.25,0.49,0.14];
else if (max<0.5) return [0.19,0.43,0.11];
else if (max<0.55) return [0.13,0.38,0.07];
else if (max<0.6) return [0.06,0.33,0.04];
else return [0,0.27,0];

}

Thank you very much in advance.

gmilcinski · May 10, 2019, 8:30pm

First observation I have is that in the second script, max_NDVI_values, your return is still with three bands:

 return [0,0,0];

It should be
return [ndvi];

Not sure though if this is an issue.
Can you paste screenshots, so that I get a better view?

unap · May 13, 2019, 9:08am

Thanks for your observation. I have changed scripts. You can find them below.

Screeshot for max_NDVI:

Screenshot for max_NDVI_value:

Please, note that clouds were only detected over lake (cloud NDVI is higher than water NDVI).
Nevertheless, when using max_NDVI_value detects clouds over ground. Both layes are configurated to calculate maximum NDVI for the last 2 years, so clouds should not be detected over ground.

Also, when I use them on EO-Browser, clouds are detected over ground when using max_NDVI:

Nevertheless, max_NDVI_value does not work when using it at EO-Browser.

max_NDVI script:

//Basic initialization setup function
function    setup (dss) {
//List of all bands, that will be used in the script, either for visualization or for choosing best pixel
  setInputComponents([dss.B04,dss.B08]);
//This can always be the same if one is doing RGB images
  setOutputComponentCount(3);
}

/*
In this function we limit the scenes, which are used for processing. 
These are based also on input variables, coming from Playground. 
E.g. if one sets date "2017-03-01" ("TO date") and cloud coverage filter 30%, 
all scenes older than 2017-03-01 with cloud coverage 30% will be checked against
further conditions in this function.
The more scenes there are, longer it will take to process the data.
After 60 seconds of processing, there will be a timeout.
*/

function filterScenes (scenes, inputMetadata) {
    return scenes.filter(function (scene) {
//Here we limit data between "(TO date - 1 month) to (TO date)
	  return scene.date.getTime()>=(inputMetadata.to.getTime()-2*365*24*3600*1000) ;
    });
}

function calcNDVI(sample) {
  var denom = sample.B04+sample.B08;
  return ((denom!=0) ? (sample.B08-sample.B04) / denom : 0.0);
}
function evaluatePixel(samples) {  
  var max = 0;
  for (var i=0;i<samples.length;i++) {
      var ndvi = calcNDVI(samples[i]);
    max = ndvi > max ? ndvi:max;
  }
if (max<-1.1) return [0,0,0];
else if (max<-0.2) return [0.75,0.75,0.75];
else if (max<-0.1) return [0.86,0.86,0.86];
else if (max<0) return [1,1,0.88];
else if (max<0.025) return [1,0.98,0.8];
else if (max<0.05) return [0.93,0.91,0.71];
else if (max<0.075) return [0.87,0.85,0.61];
else if (max<0.1) return [0.8,0.78,0.51];
else if (max<0.125) return [0.74,0.72,0.42];
else if (max<0.15) return [0.69,0.76,0.38];
else if (max<0.175) return [0.64,0.8,0.35];
else if (max<0.2) return [0.57,0.75,0.32];
else if (max<0.25) return [0.5,0.7,0.28];
else if (max<0.3) return [0.44,0.64,0.25];
else if (max<0.35) return [0.38,0.59,0.21];
else if (max<0.4) return [0.31,0.54,0.18];
else if (max<0.45) return [0.25,0.49,0.14];
else if (max<0.5) return [0.19,0.43,0.11];
else if (max<0.55) return [0.13,0.38,0.07];
else if (max<0.6) return [0.06,0.33,0.04];
else return [0,0.27,0];

}

max_NDVI_value script:

//Basic initialization setup function
function    setup (dss) {
//List of all bands, that will be used in the script, either for visualization or for choosing best pixel
  setInputComponents([dss.B04,dss.B08]);
//This can always be the same if one is doing RGB images
  setOutputComponentCount(1);
}

/*
In this function we limit the scenes, which are used for processing. 
These are based also on input variables, coming from Playground. 
E.g. if one sets date "2017-03-01" ("TO date") and cloud coverage filter 30%, 
all scenes older than 2017-03-01 with cloud coverage 30% will be checked against
further conditions in this function.
The more scenes there are, longer it will take to process the data.
After 60 seconds of processing, there will be a timeout.
*/

function filterScenes (scenes, inputMetadata) {
    return scenes.filter(function (scene) {
//Here we limit data between "(TO date - 1 month) to (TO date)
	  return scene.date.getTime()>=(inputMetadata.to.getTime()-2*365*24*3600*1000) ;
    });
}

function calcNDVI(sample) {
  var denom = sample.B04+sample.B08;
  return ((denom!=0) ? (sample.B08-sample.B04) / denom : 0.0);
}
function evaluatePixel(samples) {  
  var max = 0;
  for (var i=0;i<samples.length;i++) {
      var ndvi = calcNDVI(samples[i]);
    max = ndvi > max ? ndvi:max;
  }
return [ndvi];

}

The main goal is calculate next expression: ([Current NDVI] - [minimum NDVI]) / ([MAXIMUM NDVI] - [minimum NDVI])*100, in order to assess current vegetation compared with the maximum expected.

Notice also I changed Advanced Layer Editor parameter “temporal”: true, This is the Advanced Layer configuration for max_NDVI:

{
	"title": "max_NDVI",
	"orderHint": 0,
	"description": "",
	"datasetSource": {
		"@id": "https://services.sentinel-hub.com/configuration/v1/datasets/S2L2A/sources/2",
		"id": 2,
		"description": "Sentinel S2 - L2A",
		"settings": {
			"indexServiceUrl": "https://services.sentinel-hub.com/index"
		},
		"dataset": {
			"@id": "https://services.sentinel-hub.com/configuration/v1/datasets/S2L2A"
		}
	},
	"dataset": {
		"@id": "https://services.sentinel-hub.com/configuration/v1/datasets/S2L2A"
	},
	"styles": [
		{
			"name": "default",
			"description": "Default layer style",
			"evalScript": "//Basic initialization setup function\nfunction    setup (dss) {\n//List of all bands, that will be used in the script, either for visualization or for choosing best pixel\n  setInputComponents([dss.B04,dss.B08]);\n//This can always be the same if one is doing RGB images\n  setOutputComponentCount(3);\n}\n\n/*\nIn this function we limit the scenes, which are used for processing. \nThese are based also on input variables, coming from Playground. \nE.g. if one sets date \"2017-03-01\" (\"TO date\") and cloud coverage filter 30%, \nall scenes older than 2017-03-01 with cloud coverage 30% will be checked against\nfurther conditions in this function.\nThe more scenes there are, longer it will take to process the data.\nAfter 60 seconds of processing, there will be a timeout.\n*/\n\nfunction filterScenes (scenes, inputMetadata) {\n    return scenes.filter(function (scene) {\n//Here we limit data between \"(TO date - 1 month) to (TO date)\n\t  return scene.date.getTime()>=(inputMetadata.to.getTime()-2*365*24*3600*1000) ;\n    });\n}\n\nfunction calcNDVI(sample) {\n  var denom = sample.B04+sample.B08;\n  return ((denom!=0) ? (sample.B08-sample.B04) / denom : 0.0);\n}\nfunction evaluatePixel(samples) {  \n  var max = 0;\n  for (var i=0;i<samples.length;i++) {\n      var ndvi = calcNDVI(samples[i]);\n    max = ndvi > max ? ndvi:max;\n  }\nif (max<-1.1) return [0,0,0];\nelse if (max<-0.2) return [0.75,0.75,0.75];\nelse if (max<-0.1) return [0.86,0.86,0.86];\nelse if (max<0) return [1,1,0.88];\nelse if (max<0.025) return [1,0.98,0.8];\nelse if (max<0.05) return [0.93,0.91,0.71];\nelse if (max<0.075) return [0.87,0.85,0.61];\nelse if (max<0.1) return [0.8,0.78,0.51];\nelse if (max<0.125) return [0.74,0.72,0.42];\nelse if (max<0.15) return [0.69,0.76,0.38];\nelse if (max<0.175) return [0.64,0.8,0.35];\nelse if (max<0.2) return [0.57,0.75,0.32];\nelse if (max<0.25) return [0.5,0.7,0.28];\nelse if (max<0.3) return [0.44,0.64,0.25];\nelse if (max<0.35) return [0.38,0.59,0.21];\nelse if (max<0.4) return [0.31,0.54,0.18];\nelse if (max<0.45) return [0.25,0.49,0.14];\nelse if (max<0.5) return [0.19,0.43,0.11];\nelse if (max<0.55) return [0.13,0.38,0.07];\nelse if (max<0.6) return [0.06,0.33,0.04];\nelse return [0,0.27,0];\n\n}\n"
		}
	],
	"id": "MAX_NDVI",
	"instanceId": "ea2b05a7-e51e-499e-8f30-0dddea4783c4",
	"defaultStyleName": "default",
	"datasourceDefaults": {
		"type": "S2L2A",
		"temporal": true,
		"mosaickingOrder": "mostRecent",
		"maxCloudCoverage": 20
	},
	"userData": {}
}

Let me know if something is not clear enough.

gmilcinski · May 19, 2019, 6:49pm

Thank you for this input, it is getting a bit more clear.

In terms of difference of “max_NDVI” and “max_NDVI_value” visualized in Sentinel Playground. I believe this is due to this line in the “max_NDVI_value”:

return [ndvi];

It should be:

return [max];

You are returning the last ndvi value instead of the maximum one…

In terms of “EO Browser difference” - EO Browser client was not designed for multi-temporal use. There every WMS requests to the back-end is limited to one individual day (where Sentinel Playground, for simplicity purpose, shows the data from the last 6 months; the temporal instance of Sentinel Playground does it from beginning of time, I think). Best way to check this is to observe network console in Development tools of the browser. You will notice WMS requests having TIME parameter.
We do have in our mid-term plans to upgrade EO Browser to allow also for multi-temporal use, but for now it is not possible. You can obviously modify the code yourself (quick and dirty solution should not be too complicated as you only need to hard-code one parameter) as EO Browser is open-source on GitHub:

unap · May 20, 2019, 8:26am

Thanks for your answer. I changed it, but my playground is not working now. Anyway, I’ll check it when working.
Please, let me know wheter it is a problem related to my own account or with a general issue.
Kind regards.

gmilcinski · May 20, 2019, 8:41am

Today we have deployed a new version and there are some issues with use with user’s instances. We are working on it and should be fixed in the next couple of hours (or rolled-back). Please try latest tomorrow morning.