Hi @tdrivas,
note that we are not working with Open Data Cube software.
I suggest you contact them directly:
https://www.opendatacube.org/contacts-check
Best,
Grega
Hey,
I can totally relate to your question, as I’ve been diving into the world of data cubes and file systems lately. It’s always a bit of a puzzle to figure out the most efficient setup. For me, when dealing with Open Data Cube software, I’ve found that a hybrid approach works wonders.
Here’s what I’ve been up to: I store my core mission data, like Sentinel-1, in their original formats, leveraging Cloud Optimized GeoTIff, JP2, zarr, and HDF5. If the data is already in the cloud, why duplicate, right?
For post-processing, I sometimes find it handy to temporarily park data in a distributed file system, like HDFS. But remember, that’s just my preference for my workflows. Your mileage may vary!
And a quick shoutout to tdrivas: Great question! Based on my experience, the choice of file system can significantly impact performance. Considering your use case and the type of analysis you’re running can guide your decision. Don’t hesitate to experiment a bit and find that sweet spot.