BYOC with CREODIAS Bucket

ops_sentinelhub · November 18, 2022, 7:53pm

I am following the instructions on https://docs.sentinel-hub.com/api/latest/api/byoc/#byoc-deployment

From here How to generate ec2 credentials? - FAQ Networking - CREODIAS

I had to enable openstack_waw3_1 location
downloaded the openrc.sh file with the configurations
I got my ec2 credentials
In WAW-3:
I created a bucket (named OTvhr)
uploaded my cog files: tile_1/XXX.tiff and tile_2/YYY.tiff
From here https://docs.sentinel-hub.com/api/latest/af3077d767c7d1b7cfc39fd52e838270/creodias_set_bucket_access_policy.py:
I downloaded the script and completed access_key, secret_key and bucket_name.
I had to set host=‘https://s3.waw3-1.cloudferro.com/’ as the original value gave me “botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutBucketPolicy operation: Unknown” as an error.
I configured the Sentinel Hub collection:
using my project id from openrc.sh file and the bucket name
collection location is correctly identified as CREODIAS
tiles ingestion fails with: Failed to access the storage.

The only thing I can think of that I did that was not on the documentation is changing the using s3.waw3-1.cloudferro.com instead of s3.waw2-1.cloudferro.com. Is this the issue? How can I correctly configure the buckets?

Thanks

Regards

dorothyrono · November 21, 2022, 9:07am

Hi Javier,

Sentinel Hub doesn’t yet support s3.waw3-1.cloudferro.com endpoint. Any chance you could use a bucket in s3.waw2-1.cloudferro.com cloud meanwhile. No ETA yet when waw3-1 will be supported.

apologies for the inconveniences

ops_sentinelhub · November 21, 2022, 2:42pm

Thanks, I managed to create a bucket on waw2, but now I get:

“Found bands with invalid names that cannot be used in evalscripts”, should I open another thread?

dorothyrono · November 21, 2022, 3:21pm

That means the band name contains invalid characters. The documentation indicates that:

The band name should be a valid JavaScript identifier so it can be safely used in evalscripts; valid identifiers are case-sensitive, can contain Unicode letters, $, _, and digits (0-9), but may not start with a digit, and should not be one of the reserved JavaScript keywords.

There are several options to solve this

(1) You could rename you files to valid names based on the above constraints or

(2) You could manually configure your bands directly in the collection before ingesting the tiles. There is an option to configure bands. The manual band configuration can also be done using the API

If you still don’t solve it, please paste here an example of your tile path to help easily identify the error

ops_sentinelhub · November 25, 2022, 10:12am

Thanks, I got it working with your suggestions.

I am now faced with a render issue.

Images on the bucket are OK.

I can open the images and get decent renderization from QGIS without tweaking enhancements.

gdalinfo of one of the tiles:

gdalinfo byoc_tiles/PleiadesNeo_Balcarce_pansharp_1_1/B.tiff 
Driver: GTiff/GeoTIFF
Files: byoc_tiles/PleiadesNeo_Balcarce_pansharp_1_1/B.tiff
Size is 13832, 15360
Coordinate System is:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (-58.289053000000003,-37.815680000000000)
Pixel Size = (0.000002699902400,-0.000002699912435)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=DEFLATE
  INTERLEAVE=PIXEL
  LAYOUT=COG
  PREDICTOR=2
Corner Coordinates:
Upper Left  ( -58.2890530, -37.8156800) ( 58d17'20.59"W, 37d48'56.45"S)
Lower Left  ( -58.2890530, -37.8571507) ( 58d17'20.59"W, 37d51'25.74"S)
Upper Right ( -58.2517079, -37.8156800) ( 58d15' 6.15"W, 37d48'56.45"S)
Lower Right ( -58.2517079, -37.8571507) ( 58d15' 6.15"W, 37d51'25.74"S)
Center      ( -58.2703805, -37.8364153) ( 58d16'13.37"W, 37d50'11.10"S)
Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
  NoData Value=0
  Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 2 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
  NoData Value=0
  Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 3 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
  NoData Value=0
  Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 4 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
  NoData Value=0
  Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 5 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
  NoData Value=0
  Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 6 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
  NoData Value=0
  Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960

And the evalscript used:

//VERSION=3

let minVal = 0.1;
let maxVal = 0.11;

let viz = new HighlightCompressVisualizer(minVal, maxVal);

function setup() {
  return {
    input: [{
      bands: [
        "B1",
        "B2",
        "B3",
        "dataMask"
      ]
    }],
    output: {
      bands: 4,
      sampleType: "AUTO"
    }
  }
}

function evaluatePixel(sample) {
    let val = [sample.B3/65535, sample.B2/65535, sample.B1/65535];
    val = viz.processList(val);
    val.push(sample.dataMask);
    return val;
}

I really tried a lot of variants, but never got a decent render.

Thanks in advance!
Regards.

chung.horng · November 25, 2022, 2:10pm

Hi @ops_sentinelhub ,

Would you mind letting us know what bands you’re using, i.e., what are the B1 to B7 in your collection? Also, could you please share a screenshot of the expected visualisation?

Thank you!

ops_sentinelhub · November 25, 2022, 2:29pm

Hi @chung.horng,

The expected visualization is True Color using Bands 1,2,3 as R,G,B.

This screenshot is generated using QGIS and the pre-uploaded files.

Bands 4 and + are not used.

I just caught that the script invert the bands order, but using them in the right order gives the same kind of unwanted result.

Regards.

ops_sentinelhub · November 25, 2022, 3:17pm

I recreated the collection, but I did not rename the bands (the automatic nomenclature was B_1, B_2, etc. and I was renaming them to B1, B2, etc.), this time it worked.

Does anyone know why was this happening?

Regards

system · January 24, 2023, 3:18pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.