We needed a specific image (certain date and bounding box), and noticed that some of the layers (band combinations) came back fully white, while others (using some or all of the same bands) are not. Specifically some of the index layers (NDVI, NDWI) work, but their constituent bands (when configured manually in the EO Browser) do not.
As can be seen, in EO Browser the true color layer is plain white for the whole image. The imagery itself from that date is probably not the issue, the same image in the USGS EarthExplorer displays just fine.
Is there a standard procedure to report cases like this, apart from this forum?
It’s not that it is fully white, it is just that the band values fall out of the normal range. My guess is that it is Sen2Cor going wrong here, for this specific orbit (or rather, datatake), corrupting L2A data (if you check L1C data, you will see those are OK).
You can also manually configure the indices, just need to use the same stretch. E.g. B08 band.
With this volume of data you simply have to take into account that some have problems.
USGS EarthExplorer shows L1C data in this reason.
This forum is exactly the right place to report cases like this.
Investigaging a bit further - if you zoom out a bit, you will see that part of the orbit is OK, part not. The “not OK” part was processed by ESA in 2018, where the “OK” part was processed by us for the Digital Earth Africa exercise. We have used the newer version of Sen2Cor (2.8), where ESA used the one available in 2018… That might be buggy, or there was an issue in the ground segment at the time.
That is very interesting, thanks for the investigation. That also explains why the index layers work, since they are scale-invariant for a shared scaling factor in both bands.
Is there any chance of those images being re-processed? In our setup it would be fairly challenging to add a workaround for one specific image/date due to how everything works.
What are the chances of this being an isolated issue, vs. assuming that more images might be “corrupted”?
From what you have found out, would it be possible/feasible to detect the scale factor that would be needed to correct the image data based on the faulty image alone?
There is quite an overhead with reprocessing the data so we do typically not do this. We do occasionally exclude some items from the archive. At the moment we cannot commit on the timeline.
As you can imagine, changing our workflow has similar issues as with yours, but adding a work-around on a 15PB archive is an even larger challenge.
This kind of case has not been yet identified in the past so it seems to be pretty rare. There were other cases of corrupt data though, e.g. having strips over some bands, or some bands being missing, or… I believe the users of these datasets simply need to adjust to the fact that data are imperfect, as they are.
In terms of the “correcting the scale factor” - my recommendation would be to rather try to detect such cases and exclude them (automatically) from the workflow. It is not certain that a scale factor is the same for all bands.
If we decide to do something with these specific data, I will let you know.
As you can imagine, changing our workflow has similar issues as with yours, but adding a work-around on a 15PB archive is an even larger challenge.
I can very much imagine
If it happens rarely enough, maybe ignoring it is indeed the best course of action. We did actually ignore it at first, due to some basic sanity checks, and didn’t even notice. In a manual review process the question was raised why this specific image was ignored, since it was known to contain good data for the specified time frame.
we investigated this a bit further and actually found a systematic error in our system. These tiles were reprocessed at a later stage (3 months after observation) and in the mean time ESA has changed the processing baseline. Our processing flow however took the old baseline into account, thus resulting in different values.
Tiles in this specific orbit have been fixed. We are now looking if there are some more cases like this in the system.
Thank you for reporting this.
Best,
Grega
