A few months ago Geoscience Australia, working on the Digital Earth Africa, asked us to perform atmospheric correction of the complete archive over Africa, back to 2017. It took us about a week and the data is now available in Sentinel Hub. In a follow-up discussion with AWS’ Open Data team we have agreed to expand the archive over the whole globe.
At the moment Sentinel-2 L2A data are available over Europe since September 2016, Africa since beginning of 2017 and since mid December 2018 elsewhere.
In a couple of weeks from now, early April 2020, as the processing finishes, all scenes, worldwide, since 2017, with cloud coverage less than 90%, will be available as atmospherically corrected (L2A, Bottom of the atmosphere) data.
We cannot wait to see what people will do with such a volume (global, 3 full years and counting) of Analysis Ready Data (ARD).
We are using official Sen2Cor v2.8 for the processing, with SRTM and CCI ancillary layer active.
An example of non-atmospherically corrected data (L1C)
Ruben from Vito has warned us about the Sen2Cor issue in areas without SRTM (below and above 60 degrees) and we have excluded these from further processing. We will deal with that later.
it seems that the processing consumed much of the available resources at AWS (eu-central-1 region) and the price of the spot instances was going up quite a bit (see below). This impacted our plans quite a bit as our budget is limited. We have now stopped the process a bit, waiting for price to go down, after which we will restart it with a bit lower capacity. So it might take a bit more, a week or two perhaps.
First of all thanks for making the Sentinel-2 Analysis Ready Data publicly available.
I would like to ask if it’s possible to add also the ‘tileInfo.json’ file to the ‘s3://sentinel-s2-l2a/products/’ archive, which includes information such as ‘dataCoveragePercentage’ or ‘coordinates’ of the product.
Moreover, I think there is a typo in the ‘https://roda.sentinel-hub.com/sentinel-s2-l2a/readme.html’ website and the ‘s3://sentinel-s2-l1c/tiles/10/S/DG/2018/12/31/0/’ s3-path should actually be ’ s3://sentinel-s2-l2a/tiles/10/S/DG/2018/12/31/0/’
Hi @lampros,
in the first year of Sentinel-2 distribution one product contained several tiles (usually 10, but also up to hundreds) so product-tile was 1-n connection. Therefore it did not make sense to add tileInfo.json there. In practice, if it would have been 1-1 from the beginning, we would not have split product and tile at all… But once we have had it, we wanted to keep the structure fixed.
Thanks for noticing the typo. It’s been fixed, will take a while before the cache is getting cleared.
Thank you for the response, I wasn’t aware of this. I am actually interested in the Sentinel-2 Analysis Ready Data since January 2019 and I find (personally) more efficient to search for products using the ‘aws-cli’ based on the ‘s3://sentinel-s2-l2a/products/’ archive rather than based on ‘s3://sentinel-s2-l1c/tiles/’ which includes all files.
A last question: Are both archives (‘products’ and ‘tiles’) updated simultaneously? For instance, as of 02-04-2020 there are 458 days of products available for years 2019 and 2020 based on the ‘s3://sentinel-s2-l2a/products/’ archive, so I expect that these products are available to download in the ‘s3://sentinel-s2-l1c/tiles/’ archive.
Short update about the status.
The data back to July 2017 are in the system.
We have however been alerted about, so it seems, missing SRTM model in some of the processed tiles and we have to analyze, what is the reason for that.
A short update - we have completed the procesing of all tiles with cloud coverage less than 90% and under/over lat 60 deg North/South.
Next steps:
-processing of tiles nearer to the poles
-identification and re-processing of tiles that have DEM missing.
