Statistical API showing NaN average even if any one datapoint in that time period is NaN

Apps / Services
1

Sentinel 5-p user trying to download CH4 concentration data:
Using the Statistical API, I am trying to download long-term values over a particular lat-lon (tiny area, so my custom bounding box is 0.1x0.1 degrees). I am interested in obtaining time series for that particular lat-lon. However, the Statistical API seems to return a NaN value for a date or a time period average if: 1. any one of the underlying pixels (of the many in a 0.1x0.1 grid) has a NaN value on that date, and 2. If any one of the dates in say a Month has a NaN value.

Any help is appreciated.

Thanks

2

Hi @srathod4 ,

You need to mask pixels having a value of NaN when using Statistical API. Below is an example script:

//VERSION=3
function setup() {
    return {
        input: [{
            bands: ["VV", "VH", "dataMask"]
        }],
        output: [
            {
                id: "vv",
                sampleType: "FLOAT32",
                bands: 1
            },
            {
                id: "vh",
                sampleType: "FLOAT32",
                bands: 1
            },
            {
                id: "dataMask",
                bands: ["vv", "vh"]
            }],
        mosaicking: "ORBIT"
    };
}

function evaluatePixel(samples, scenes) {
    let vv_valid = 0;
    let vh_valid = 0;
    if (samples[0].dataMask === 1 && Number.isFinite(samples[0].VV)) {
        vv_valid = 1;
    }
    if (samples[0].dataMask === 1 && Number.isFinite(samples[0].VH)) {
        vh_valid = 1;
    }
    return {
        vv: [samples[0].VV],
        vh: [samples[0].VH],
        dataMask: [
            samples[0].dataMask * vv_valid,
            samples[0].dataMask * vh_valid
        ]
    };
}
3

Thank you!
Could you please help me with the same on this code?

    bbox_param = 0.1
    
    # Credentials
    # Set up Sentinel Hub configuration
    config = SHConfig()
    config.sh_client_id = client_id
    config.sh_client_secret = client_secret
    
    evalscript = """
    //VERSION=3
    function setup() {
      return {
        input: [{
          bands: [
            "CH4",
            "dataMask",
          ]
        }],
        output: [
          {
            id: "data",
            noDataValue: -999,
            bands:1
          },
          {
            id: "dataMask",
            bands: 1
          }
        ]
      };
    }
    
    function evaluatePixel(sample) {
      return {
        data: [sample.CH4],
        dataMask: [sample.dataMask],
      };
    }
    """
    
    calculations = {
        "default": {
            "statistics": {
                "default": {
                    "percentiles": {
                        "k": [
                            5,
                            25,
                            50,
                            75,
                            95
                        ],
                        "interpolation": "higher"
                    }
                }
            }
        }
    }
    
    
    # Define the bounding box
    # The bounding box is defined by specifying the longitude and latitude coordinates of two opposite corners
    # lon1, lat1, lon2, lat2
    bbox = BBox(bbox=[loni - bbox_param, lati - bbox_param, loni + bbox_param, lati + bbox_param], crs=CRS.WGS84)
    
    # Create Sentinel Hub statistical request
    request = SentinelHubStatistical(
        aggregation=SentinelHubStatistical.aggregation(
            evalscript=evalscript,
            time_interval=(formatted_date0, formatted_date1),
            aggregation_interval='P1D'       
        ),
        input_data=[
            SentinelHubStatistical.input_data(
                DataCollection.SENTINEL5P,            
                other_args={"dataFilter": {"mosaickingOrder": "leastRecent","timeliness": "OFFL"},"processing": {"upsampling": "BICUBIC","downsampling": "BICUBIC","minQa": 30}},            
          ),
        ],
        bbox=bbox,
        calculations=calculations,
        config=config
    )
    
    # Send the request and retrieve the response data
    response = request.get_data()
    response
4

Hi @srathod4 ,

You could try to adapt your Evalscript based on the example I provided above. You just need to add a conditional statement in the evaluatePixel function to re-assign 0 to dataMask if sample.CH4 is NaN (Number.isFinite will be false if the value is NaN).

5

I tried this and it didnt make a lot of difference compared to my original script

    function evaluatePixel(sample) {
      if (isNaN(sample.CH4) || isNaN(sample.dataMask)) {
        return {
          data: [null],
          dataMask: [null],
        };
      } else {
        return {
          data: [sample.CH4],
          dataMask: [sample.dataMask],
        };
      }
    }
6

Hi @srathod4 ,

Please try the following:

//VERSION=3
function setup() {
  return {
    input: [{
      bands: [
        "CH4",
        "dataMask",
      ]
    }],
    output: [
      {
        id: "data",
        bands:1
      },
      {
        id: "dataMask",
        bands: 1
      }
    ]
  };
}

function evaluatePixel(sample) {
  let is_valid = 0;
  if (Number.isFinite(sample.CH4) && sample.dataMask === 1) {
    is_valid = 1;
  }
  return {
    data: [sample.CH4],
    dataMask: [is_valid],
  };
}

If you still get a lot of NaNs from the response, please check if there is data available at that time range.

7

Thank you! It certainly allowed more (2-5%) values to pass over a 10x10km2 box with SIMPLE mosaicking than in my original code. But it also lead to some values now being NaN compared to the original code. I will use your suggested code. Thank you!

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.