In your example link above to list available products, the results are capped at 1,000 items. Reviewing the S3 docs , it seems like the “NextContinuationToken” value should be included as the request parameter ‘continuation-token’ in order to fetch the subsequent pages. When I tested it however, I always seem to get back the first page only. Is there something else needed to page the results or is the parameter maybe not being forwarded to S3 properly?
I clicked the link you have provided and it works fine for me.
We have noticed that AWS Lambda sometimes simply does not work and we do not know, what is the reason for that. However, this happens rarely and few minutes later usually works fine.
Is the problem persisting?
Thanks a lot for your prompt reply!
Unfortunately, the problem remains and no access to “metadata.xml”
Because of this problem, the download command (e.g., sentinelhub.aws --tile T54HVH 2017-04-14 -e) doesn’t work.
I can access to “tileInfo.json” and “productInfo.json” but no access to “metadata.xml” and “preview.jpg”.
I will try alternative ways of downloading data.
as described in the first post in this thread, only meta-data are available over this interface. B8A.jp2 is not meta-data (see a list of available files on top).
For your case you need to use signed requests as described in this post, check options 1-4.
I found that the last update of sentinelhub-py (vision = 2.4.1) on GitHub, and the config.json has changed from "aws_base_url": "http://sentinel-s2-l1c.s3.amazonaws.com/"
to "aws_metadata_base_url": "https://roda.sentinel-hub.com/"
and no “aws base url” anyway.
Is it means we cannot download S2 images as before by sentinelhub-py 2.4.1 because of these changes?
I have used the old version (2.0.1) to download image data (.jp2) successfully, which can replace the new version.
you are correct - older versions of sentinelhub-py are now outdated and cannot download S2 images from AWS anymore. sentinelhub-py serves only as an interface for download and the data is not available at “http://sentinel-s2-l1c.s3.amazonaws.com/” anymore. Therefore I recommend upgrading the package.
By the way, sentinelhub-py 2.4.2 has just been released with a few improvements in handling jpeg 2000 images.
I assume the version 2.0.1 still works because you have set parameter "use_s3_l1c_bucket": true which then downloads everything from s3 bucket instead of “http://sentinel-s2-l1c.s3.amazonaws.com/”. In newer versions those metadata files which are available from “https://roda.sentinel-hub.com” are downloaded from there, hence slightly less requests to s3.
It is possible that newer versions are causing you problems because we changed library for reading Jpeg 2000 images from OpenCV to Pillow. Even though Pillow doesn’t raise errors during installation sometimes its capabilities to read Jpeg 2000 images don’t get installed properly (especially on Windows). Hence installation from wheel is recommended.
Anyway if you will still have some problems with the newest version of sentinelhub-py please create a new thread on this forum about it or raise an issue on Github. We will be happy to help you resolve the problem.
Everything else, jp2 images included, will still be downloaded from s3 just like before. Only those metadata files which are available at “https://roda.sentinel-hub.com” will be downloaded from there. This is all handled internally by sentinelhub-py, hence your code would still work without having to make any changes.