Import from Tiff issue

Hi,
I am trying to import data from Tiff images to EOPatch. Tiff images are located on a different drive (O:\), my working directory is on D:\. When I run the script for importing, I get the “No such file or directory” error (note, the path in the error message starts with the letter D:\, but it should with O:\).
I passed an absolute path of the tiff folder (O:\aitlas_slc_SI_sigma…), so I do not understand why the drive changes (I guess, somewhere in fs\osfs.py). I am working on Windows. I also tried to pass the path as raw string literals, double backslashes, and forward slashes, the result was the same.
Any idea how to solve this?

Example code:

import_from_tiff = ImportFromTiff(folder='o:/aitlas_slc_SI_sigma/yr17wk01_SLC_SIG_20170101_20170106_weekly/',
                                  feature=(FeatureType.DATA, 'weekly_data'),
                                  # timestamp_size=4,
                                  image_dtype=np.float32)
import_from_tiff.execute(eopatch, filename='20170101_20170106_weekly_SLC_SIG_ASC_VH_yr17wk01.tif')


Traceback (most recent call last):
  File "C:\Users\adraksler\.conda\envs\aitlas\lib\site-packages\fs\osfs.py", line 360, in openbin
    sys_path, mode=_mode.to_platform_bin(), buffering=buffering, **options
FileNotFoundError: [Errno 2] No such file or directory: b'D:\\aitlas_slc_SI_sigma\\yr17wk01_SLC_SIG_20170101_20170106_weekly\\20170101_20170106_weekly_SLC_SIG_ASC_VH_yr17wk01.tif'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\adraksler\.conda\envs\aitlas\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-13-4162f2179dee>", line 5, in <module>
    import_from_tiff.execute(eopatch, filename='20170101_20170106_weekly_SLC_SIG_ASC_VH_yr17wk01.tif')
  File "C:\Users\adraksler\.conda\envs\aitlas\lib\site-packages\eolearn\io\local_io.py", line 408, in execute
    with filesystem.openbin(path, 'r') as file_handle:
  File "C:\Users\adraksler\.conda\envs\aitlas\lib\site-packages\fs\osfs.py", line 360, in openbin
    sys_path, mode=_mode.to_platform_bin(), buffering=buffering, **options
  File "C:\Users\adraksler\.conda\envs\aitlas\lib\site-packages\fs\error_tools.py", line 90, in __exit__
    reraise(fserror, fserror(self._path, exc=exc_value), traceback)
  File "C:\Users\adraksler\.conda\envs\aitlas\lib\site-packages\six.py", line 702, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\adraksler\.conda\envs\aitlas\lib\site-packages\fs\osfs.py", line 360, in openbin
    sys_path, mode=_mode.to_platform_bin(), buffering=buffering, **options
fs.errors.ResourceNotFound: resource 'aitlas_slc_SI_sigma/yr17wk01_SLC_SIG_20170101_20170106_weekly/20170101_20170106_weekly_SLC_SIG_ASC_VH_yr17wk01.tif' not found
1 Like

Not a Windows user here, but in the example code, your drive letter is in lower-case. Could that be an issue?

Otherwise, try to list the file/folder first with

import os

os.path.exists('O:/...')

Once the response is True, try inputting this into the ImportFromTiff task.

PS: windows paths are notoriously quirky in python…

Hi @adraksler,

Please make sure that you are using eo-learn-io>=0.8.1. In earlier versions, we had a bug in this task where Windows paths didn’t work.

The correct way of specifying the path should be 'O:\\aitlas_slc_SI_sigma\\yr17wk01_SLC_SIG_20170101_20170106_weekly'. In the background of the task backslashes will then be converted into forward slashes because we use a filesystem abstraction package.

@batic, @maleksandrov thank you for a fast reply.
The os.path.exists always returns true (lower/upper-case drive letter, forward/backslashes) and I can easily list the filenames of all tiff files, using os.listdir or glob.glob.
I tried both versions, eo-learn-io 0.8.1 and 0.8.0, on different machines, no luck.

I did some searching and I found this: in module local_io.py there is _get_filesystem_and_paths which calls get_base_filesystem_and_path from fs_utils.py. The return of this function is also get_filesystem('/', **kwargs), which I believe will always return the filesystem the script is run from, am I right? Could this be the reason for drive letter change and therefore “No such file or directory” error?

I found a solution, probably not ideal, but it works for me. I am posting it here, in case it helps anyone.

In fs_utils.py, I added/modified two lines:

    def get_base_filesystem_and_path(*path_parts, **kwargs):
        """ Parses multiple strings that define a filesystem path and returns a filesystem object with a relative path
        on the filesystem

        :param path_parts: One or more strings defining a filesystem path
        :type path_parts: str
        :param kwargs: Parameters passed to get_filesystem function
        :return: A filesystem object and a relative path
        :rtype: (fs.FS, str)
        """
        path_parts = [str(part) for part in path_parts if part is not None]
        base_path = path_parts[0]
        if ':' in base_path:
            drive = base_path.split(':')[0]+':'  # added; get drive letter from path
        else: 
            drive = '/'

        if '://' in base_path:
            base_path_parts = base_path.split('/', 3)
            filesystem_path = '/'.join(base_path_parts[:-1])
            relative_path = '/'.join([base_path_parts[-1], *path_parts[1:]])

            return get_filesystem(filesystem_path, **kwargs), relative_path

        entire_path = os.path.abspath(os.path.join(*path_parts))
        pure_path = PurePath(entire_path)
        posix_path = pure_path.relative_to(pure_path.anchor).as_posix()

        return get_filesystem(drive, **kwargs), posix_path  # drive instead of '/', so different drives can be detected
2 Likes

Thanks @adraksler, we’ll make sure to include this fix in the next eo-learn release.

Hi @adraksler

May i know how do you arrange your data directories? I am currently searching how to properly use ImportFromTiff. I am just curious what is the proper arrangement of the data directories before using this fuction.