Multi spectral data for sentinel2-l2a

Apps / Services Python SDK
1

Hi,

I am trying to save a combined tiff image for the following bands for a specific bounding box. When outputting just the RGB bands I am able to see the image without any issues. However, as soon as I add the other bands the image is no longer visible (it is black with a white bar in the middle).

What am I doing wrong here? I tried playing with different scaling factor etc.
Should be using different unit or scaling factor?

function setup() {{
    return {{
        input: ["B02", "B03", "B04", "B08", "B11", "B12", "dataMask"],
        output: [{truecolor_str}],
        mosaicking: "TILE"
    }};
}}

function evaluatePixel(samples) {{
    var scaleFactor = 2.5;
    var b02_array = [];
    var b03_array = [];
    var b04_array = [];
    var b08_array = [];
    var datamask_array = [];

    samples.forEach((sample, index) => {{
        b02_array[index] = sample.B02 * scaleFactor;
        b03_array[index] = sample.B03 * scaleFactor;
        b04_array[index] = sample.B04 * scaleFactor;
        b08_array[index] = sample.B08 * scaleFactor;
        datamask_array[index] = sample.dataMask;
    }});

I am using sentinelHubRequest API

    response = SentinelHubRequest(
        data_folder="test_dir",
        evalscript=evalscript,
        input_data=[
            SentinelHubRequest.input_data(
                data_collection=collection,
                time_interval=time_interval,
                mosaicking_order=MosaickingOrder.LEAST_CC
            )
        ],
        responses=response_list,
        bbox=bbox,
        size=bbox_size,
        config=config,
    )
2

Hi @princeofpersia ,

Could you please provide one of your outputs (setup function) and returns (evaluatePixel function) in your Evalscript? Thank you.

3

This is what I have from the variable perspective.

timestamps = set(ts.replace("-", "").replace(":", "") for ts in ts_list)

truecolor_str = ", ".join([f'{{id: "M{scene}_trucolor", bands: 4}}' for scene in timestamps])

truecolor_list = [
    f'M{scene}_trucolor: [b02_array[{index}], b03_array[{index}], b04_array[{index}], b08_array[{index}]]'
    for index, scene in enumerate(timestamps)
]

return_str = ", ".join(truecolor_list)

Now, I am trying with just 4 bands and the following is the complete evalScript.

//VERSION=3

function setup() {
    return {
        input: ["B02", "B03", "B04", "B08", "B11", "B12", "dataMask"],
        output: [{id: "M20230813T210939Z_trucolor", bands: 4}, {id: "M20230808T210939Z_trucolor", bands: 4}],
        mosaicking: "TILE"
    };
}

function evaluatePixel(samples) {
    var scaleFactor = 2.5;
    var b02_array = [];
    var b03_array = [];
    var b04_array = [];
    var b08_array = [];
    var datamask_array = [];

    samples.forEach((sample, index) => {
        b02_array[index] = sample.B02 * scaleFactor;
        b03_array[index] = sample.B03 * scaleFactor;
        b04_array[index] = sample.B04 * scaleFactor;
        b08_array[index] = sample.B08 * scaleFactor;
    });

    return {
        M20230813T210939Z_trucolor: [b02_array[0], b03_array[0], b04_array[0], b08_array[0]], M20230808T210939Z_trucolor: [b02_array[1], b03_array[1], b04_array[1], b08_array[1]]
    };
}

function updateOutputMetadata(scenes, inputMetadata, outputMetadata) {
    var date;
    if (scenes.orbits && scenes.orbits.length > 0 && scenes.orbits[0].dateFrom) {
        date = scenes.orbits[0].dateFrom;
    } else if (scenes.tiles && scenes.tiles.length > 0 && scenes.tiles[0].date) {
        date = scenes.tiles[0].date;
    }
    outputMetadata.userData = {
        scenes:  scenes.tiles,
    };
}

The following is the generated value for response list.

[{‘identifier’: ‘M20230813T210939Z_trucolor’, ‘format’: {‘type’: ‘image/tiff’}}, {‘identifier’: ‘M20230808T210939Z_trucolor’, ‘format’: {‘type’: ‘image/tiff’}}, {‘identifier’: ‘userdata’, ‘format’: {‘type’: ‘application/json’}}]

Issue seems to be related to dataMask, but not 100% sure

Thank you!!

4

Hi @princeofpersia ,

I tested your complete script and it works totally fine. Could you provide your request which returns black with a white bar in the curl format as shown below (not the one with variable but the actual request sent to our API)?

curl -X POST https://services.sentinel-hub.com/api/v1/process \
 -H 'Content-Type: application/json' \
 -H 'Authorization: Bearer ' \
 -H 'Accept: application/tar' \
 -d '{
  "input": {
    "bounds": {
      "bbox": [
        12.44693,
        41.870072,
        12.541001,
        41.917096
      ]
    },
    "data": [
      {
        "dataFilter": {
          "timeRange": {
            "from": "2023-08-07T00:00:00Z",
            "to": "2023-08-14T23:59:59Z"
          }
        },
        "type": "sentinel-2-l2a"
      }
    ]
  },
  "output": {
    "width": 512,
    "height": 343.697,
    "responses": [
      {
        "identifier": "M20230813T210939Z_trucolor",
        "format": {
          "type": "image/tiff"
        }
      },
      {
        "identifier": "M20230808T210939Z_trucolor",
        "format": {
          "type": "image/tiff"
        }
      }
    ]
  },
  "evalscript": "//VERSION=3\n\nfunction setup() {\n    return {\n        input: [\"B02\", \"B03\", \"B04\", \"B08\", \"B11\", \"B12\", \"dataMask\"],\n        output: [{id: \"M20230813T210939Z_trucolor\", bands: 4}, {id: \"M20230808T210939Z_trucolor\", bands: 4}],\n        mosaicking: \"TILE\"\n    };\n}\n\nfunction evaluatePixel(samples) {\n    var scaleFactor = 2.5;\n    var b02_array = [];\n    var b03_array = [];\n    var b04_array = [];\n    var b08_array = [];\n    var datamask_array = [];\n\n    samples.forEach((sample, index) => {\n        b02_array[index] = sample.B02 * scaleFactor;\n        b03_array[index] = sample.B03 * scaleFactor;\n        b04_array[index] = sample.B04 * scaleFactor;\n        b08_array[index] = sample.B08 * scaleFactor;\n    });\n\n    return {\n        M20230813T210939Z_trucolor: [b02_array[0], b03_array[0], b04_array[0], b08_array[0]], M20230808T210939Z_trucolor: [b02_array[1], b03_array[1], b04_array[1], b08_array[1]]\n    };\n}\n"
}'
5

Please find the evalScript with all the bands.

//VERSION=3

function setup() {
    return {
        input: ["B02", "B03", "B04", "B08", "B11", "B12", "dataMask"],
        output: [{id: "M20230813T210939Z_trucolor", bands: 7}, {id: "M20230808T210939Z_trucolor", bands: 7}],
        mosaicking: "TILE"
    };
}

function evaluatePixel(samples) {
    var scaleFactor = 2.5;
    var b02_array = [];
    var b03_array = [];
    var b04_array = [];
    var b08_array = [];
    var b11_array = [];
    var b12_array = [];
    var datamask_array = [];

    samples.forEach((sample, index) => {
        b02_array[index] = sample.B02 * scaleFactor;
        b03_array[index] = sample.B03 * scaleFactor;
        b04_array[index] = sample.B04 * scaleFactor;
        b08_array[index] = sample.B08 * scaleFactor;
        b11_array[index] = sample.B11 * scaleFactor;
        b12_array[index] = sample.B12 * scaleFactor;
        datamask_array[index] = sample.dataMask; 
    });

    return {
        M20230813T210939Z_trucolor: [b02_array[0], b03_array[0], b04_array[0], b08_array[0], b11_array[0], b12_array[0], datamask_array[0]], M20230808T210939Z_trucolor: [b02_array[1], b03_array[1], b04_array[1], b08_array[1], b11_array[1], b12_array[1], datamask_array[1]]
    };
}

function updateOutputMetadata(scenes, inputMetadata, outputMetadata) {
    var date;
    if (scenes.orbits && scenes.orbits.length > 0 && scenes.orbits[0].dateFrom) {
        date = scenes.orbits[0].dateFrom;
    } else if (scenes.tiles && scenes.tiles.length > 0 && scenes.tiles[0].date) {
        date = scenes.tiles[0].date;
    }
    outputMetadata.userData = {
        scenes:  scenes.tiles,
    };
}

INFO:main:Response list:
[{‘identifier’: ‘M20230813T210939Z_trucolor’, ‘format’: {‘type’: ‘image/tiff’}}, {‘identifier’: ‘M20230808T210939Z_trucolor’, ‘format’: {‘type’: ‘image/tiff’}}, {‘identifier’: ‘userdata’, ‘format’: {‘type’: ‘application/json’}}]

This is the request API:

    response = SentinelHubRequest(
        data_folder="test_dir",
        evalscript=evalscript,
        input_data=[
            SentinelHubRequest.input_data(
                data_collection=collection,
                time_interval=time_interval,
                mosaicking_order=MosaickingOrder.LEAST_CC
            )
        ],
        responses=response_list,
        bbox=bbox,
        size=bbox_size,
        config=config,
    )

    return response

Response list variable has the following value passed:
[{‘identifier’: ‘M20230813T210939Z_trucolor’, ‘format’: {‘type’: ‘image/tiff’}}, {‘identifier’: ‘M20230808T210939Z_trucolor’, ‘format’: {‘type’: ‘image/tiff’}}, {‘identifier’: ‘userdata’, ‘format’: {‘type’: ‘application/json’}}]

Please find the .tif file uploaded. The compressed file is very small, but the system does not allow me to upload in compressed format.

lon-156_68098_lat20_89494_F20230809T121400_M20230813T210939Z_trucolor_cc_03_11.tiff (1.8 MB)

6

Dear Experts,

Any hint on what I am doing wrong? Is this expected ?

I am working on a project on wild-fire detection and trying to get the right set of data. Visible data/image is also needed for some cross validation

7

Hi @princeofpersia ,

I am investigating the issue and will update you as soon as I find something.

8

Hi @princeofpersia ,

The Evalscript is fine and it should return the correct image. I believe the issue is the size set in your SentinelHubRequest class. The dimension of your image is 7 x 512, and there are 512 pages (layers) when I open it in QGIS. It seems to me that the width of image and the number of bands are misplaced. Please double check your request using response.payload to inspect the actual request payload sent to the API.

9

Thank you for checking.

I typically set it to 512 x 512, 256 x 256 or system derive based on box size and resolution.

When I checked the image.shape, it shows (256, 256, 7) or (512 , 512, 7) depending on what size I set

The values in the image numpy array ranges from 0 to 255 with the last column (dataMask) always having 255.

Probably the way I am writing the image is incorrect or the data type (default uint8) is not the right choice.

I am saving the image using tifffile.imwrite(image-name, image)

The same code works without any changes (other than adjustment to output bands) when I just have the 3 RGB bands and datamask. No changes to the size parameter were done. QGIS only show the 3 RGB in this case. As soon as I add the additional bands, QGIS shows more than 6 layers (probably 256).

Other than the

10

Hi @princeofpersia ,

Have you tried to save your images with get_data(save_data=True)? It will help you save the response to your local machine and the TIFF filename will be the identifier set in the request.

11

I have not tried it. but I will try it. My save_image program add some additional details to the filename , which I am not sure I can do when I specify save_data=True.

12

It seems that the issue came from the process of writing TIFFs as I can get the correct image with save_data=True. I tried the tifffile.imwrite method a bit and got the same result as you did, but I’m not familiar with the library. If you need to have a pipeline to add additional details to the filename, I would recommend using rasterio which gives you more control on writing TIFFs (see example below).

import rasterio as rio
from rasterio.transform import Affine

transform = Affine(pixel_size_x, 0, upper_left_x, 0, -pixel_size_y, upper_left_y)
tif_meta = {
    "driver": "GTiff",
    "dtype": "uint8",
    "width": 512,
    "height": 512,
    "count": 1, # number of bands
    "crs": "EPSG:4326",
    "transform": transform,
    "nodata": 0,
    "compress": "lzw",
}

with rio.open("temp.tif", "w", **tif_meta) as dataset:
    dataset.write_band(1, raster)
13

Sorry for the delayed response. Thanks for testing it on your side.

Yes, it seems to be the tifffile or the way I am using it. rasterio works fine

thank you!!!

1 Like