How to access S-2 meta-data in AWS after bucket goes to Requester pays?

Service is available over standard web-interface, you simply need to combine an URL as described in the first post in this thread:

Many thanks for your great work, Grega.
I cannot access to metadata.xml - e.g. https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/16/R/BV/2018/4/16/0/metadata.xml
The error is “503 ERROR The request could not be satisfied.”
Any solution?
Thaks a lot for your kind support.
Naoto

Hi Naoto,
I clicked the link you have provided and it works fine for me.
We have noticed that AWS Lambda sometimes simply does not work and we do not know, what is the reason for that. However, this happens rarely and few minutes later usually works fine.
Is the problem persisting?

Thanks a lot for your prompt reply!
Unfortunately, the problem remains and no access to “metadata.xml”
Because of this problem, the download command (e.g., sentinelhub.aws --tile T54HVH 2017-04-14 -e) doesn’t work.
I can access to “tileInfo.json” and “productInfo.json” but no access to “metadata.xml” and “preview.jpg”.
I will try alternative ways of downloading data.

Thanks a lot about the news.

But a have a doubt, maybe it is a problem because that changes. (This is a big question here)

Into my context, i get the access to the address in the first moment:
https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/29/S/PC/2018/8/23/0/

But in my process i need to do download file .jp2, so the complement here is:
https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/29/S/PC/2018/8/23/0/B8A.jp2

And download the file into my repository images is the end of my process, after… i will render it.

This scenario throw me a message error:

AccessDenied
Access Denied

Why this error code? I’m runnig my tests locally and i’m in in Europe region.

I appreciate any help.
Thanks,
Eder

Hi Eder,
as described in the first post in this thread, only meta-data are available over this interface. B8A.jp2 is not meta-data (see a list of available files on top).
For your case you need to use signed requests as described in this post, check options 1-4.

Hi,
I found that the last update of sentinelhub-py (vision = 2.4.1) on GitHub, and the config.json has changed from
"aws_base_url": "http://sentinel-s2-l1c.s3.amazonaws.com/"
to
"aws_metadata_base_url": "https://roda.sentinel-hub.com/"
and no “aws base url” anyway.

Is it means we cannot download S2 images as before by sentinelhub-py 2.4.1 because of these changes?

I have used the old version (2.0.1) to download image data (.jp2) successfully, which can replace the new version.

Regards

Hi @zheng.tang ,

you are correct - older versions of sentinelhub-py are now outdated and cannot download S2 images from AWS anymore. sentinelhub-py serves only as an interface for download and the data is not available at “http://sentinel-s2-l1c.s3.amazonaws.com/” anymore. Therefore I recommend upgrading the package.

By the way, sentinelhub-py 2.4.2 has just been released with a few improvements in handling jpeg 2000 images.

Hi @maleksandrov ,

That’s strange!

I am downloading images by using the old version, and the url is “http://sentinel-s2-l1c.s3.amazonaws.com/”

But I can’t download images by version 2.4.1, there’s some bugs with connecting to AWS S3 buckets, and the url is “https://roda.sentinel-hub.com”.

I assume the version 2.0.1 still works because you have set parameter "use_s3_l1c_bucket": true which then downloads everything from s3 bucket instead of “http://sentinel-s2-l1c.s3.amazonaws.com/”. In newer versions those metadata files which are available from “https://roda.sentinel-hub.com” are downloaded from there, hence slightly less requests to s3.

It is possible that newer versions are causing you problems because we changed library for reading Jpeg 2000 images from OpenCV to Pillow. Even though Pillow doesn’t raise errors during installation sometimes its capabilities to read Jpeg 2000 images don’t get installed properly (especially on Windows). Hence installation from wheel is recommended.

Anyway if you will still have some problems with the newest version of sentinelhub-py please create a new thread on this forum about it or raise an issue on Github. We will be happy to help you resolve the problem.

Hi @maleksandrov,

Yes, I set "use_s3_l1c_bucket" as true, and filled my aws id and key in "aws_access_key_id" and "aws_secret_access_key". Then I can download anything in S3.

Is that means I can download metadata only in “https://roda.sentinel-hub.com”? Then I need to write a new program to access S3 to download jp2 images.

Regards

Everything else, jp2 images included, will still be downloaded from s3 just like before. Only those metadata files which are available at “https://roda.sentinel-hub.com” will be downloaded from there. This is all handled internally by sentinelhub-py, hence your code would still work without having to make any changes.

Hi, when I try to download images, when sentinelhub.aws try to get the metadata I get the error:

503 ERROR
The request could not be satisfied.

Even if I access the example (https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/16/R/BV/2018/4/16/0/metadata.xml) directly from the browser I get the same error.

Hi @pablotcarreira,

I tried accessing the link you provided and it works fine. Is it possible that this service was temporarily unavailable when you tried to access the file?

This happens in some cases but only in limited AWS regions. We do not yet know the reason for this error and are looking into it.
Usually though it works soon after.

Ok, thank you for the quick response. I’m accessing from Brazil and I’m still getting the error, I’ll try again later.
Edit: I tried accessing the address from an Ec2 instance in Ohio, Zona A and it worked.

Hi @pablotcarreira and @geoinformatics2018,
can you please try, if the access to meta-data now works from your location?
e.g.
https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/1/K/AT/2018/7/20/0/tileInfo.json
or

Thanks,
Grega

Hi Grega,

Thanks for your effort!
I can access to “tileInfo.json”, “productInfo.json”, and “preview.jpg” but still no access to “metadata.xml”.
I’m from Japan.

Naoto

Hi, tested at 14:25 utc for two scenes. tileInfo.json works and metadata.xml return Error 503.

Tests:

https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/16/R/BV/2018/4/16/0/tileInfo.json OK
https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/16/R/BV/2018/4/16/0/metadata.xml ERROR
https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/1/K/AT/2018/7/20/0/tileInfo.json OK
https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/1/K/AT/2018/7/20/0/metadata.xml ERROR

The Error is:

503 ERROR
The request could not be satisfied.
The Lambda function associated with the CloudFront distribution is invalid or doesn’t have the required permissions.
If you received this error while trying to use an app or access a website, please contact the provider or website owner for assistance.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by following steps in the CloudFront documentation (http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/http-503-service-unavailable.html).

Generated by cloudfront (CloudFront)
Request ID: ba7JhQYbdPvUXe2osi0UDDhrwWiRLvibHlflNRAAZ_xv6kPZQg-k9g==

Can you try one more time? Thanks.