Cannot perform PointSamplingTask with multiple MASK_TIMELESS

Apps / Services Python SDK
1

Hi,

I do not know if this is intentional behavior, but performing point sampling on more than one MASK_TIMELESS seems to be impossible. In the example below, I sample an additional MASK_TIMELESS called SPLIT which is essentially a layer matching the extent of the LULC training data but predefines which polygons are to be used for training and which for test (SPLIT == 1 for training and SPLIT == 2 for test). This is a less biased way of assigning polygons within an eopatch, especially if your training is sparse and only available in few eopatches. Moreover, for independent testing and validation, it’s more rigorous to make full polygon assignments and only sample training from certain ones, and test from completely different ones (what @matic.lubej is doing in SI_LULC_pipeline.ipynb guarantees an independent test set, but with a spatial bias towards certain eopatches because an entire eopatch is chosen for test, rather than randomly sampled polygons across the AOI).

 spatial_sampling = PointSamplingTask(
    n_samples=n_samples,
    ref_mask_feature='LULC',
    ref_labels=training_val,
    sample_features=[  # tag fields to sample
        (FeatureType.DATA, 'FEATURES',),
        (FeatureType.MASK, 'IS_VALID',),
        (FeatureType.MASK_TIMELESS, 'LULC',),
       (FeatureType.MASK_TIMELESS, 'SPLIT',)
    ],
    even_sampling=True)

This does not work, but if assigning the new SPLIT layer to DATA_TIMELESS, it works:

spatial_sampling = PointSamplingTask(
    n_samples=n_samples,
    ref_mask_feature='LULC',
    ref_labels=training_val,
    sample_features=[  # tag fields to sample
        (FeatureType.DATA, 'FEATURES',),
        (FeatureType.MASK, 'IS_VALID',),
        (FeatureType.MASK_TIMELESS, 'LULC',),
       (FeatureType.DATA_TIMELESS, 'SPLIT',)
    ],
    even_sampling=True)

Therefore, it’s not a malfunction, but should be made explicit if this is intended behavior.

Thanks!

William

2

This is not intended behavior - user should be able to sample any feature in the patch. Can you describe in more details what happens in the first case (when using (FeatureType.MASK_TIMELESS, 'SPLIT',))? Is there an exception, feature is not sampled, …?

Thanks.

3

This class seems to be the problem:

class MoveFeature(EOTask):
    """
    Task to copy fields from one eopatch to another
    The fields are defined as a dictionary in the form {FeatureType: {feature_names_to_copy}}
    """

    def __init__(self, fields_to_copy):
        self.fields_to_copy = fields_to_copy

    def execute(self, *eopatches):

        dst_eopatch, src_eopatch = eopatches
        for key in self.fields_to_copy.keys():
            for name in self.fields_to_copy[key]:
                dst_eopatch.add_feature(key, name, src_eopatch[key][name])
        return dst_eopatch

It refuses to move more than one name of each key type.

I pass:

 move_features = MoveFeature({
    FeatureType.MASK_TIMELESS: {'LULC'},
    FeatureType.MASK_TIMELESS: {'SPLIT'},
    FeatureType.MASK: {'IS_VALID'}})

But the MoveFeature class only picks up FeatureType.MASK_TIMELESS: {'SPLIT'} and FeatureType.MASK: {'IS_VALID'}}).

I can’t figure out why though.
If I use add_feature to manually pass to the new eopatch all the FeatureType I want to erode and sample, it works, which means there is nothing wrong with the PointSamplingTask procedure.

Regards,

William

4

Hi William,

thanks for providing additional feedback. It looks like MoveFeature task needs some improvement as well.

Anyway, the conceptual issues about the PointSampling task still remain and need to be addressed.

Thanks again for all your feedback,

Anze

5

Hi William,

In this case the issue is that the dictionary that is passed has multiple fields with the same key. It should work if you pass the following

 move_features = MoveFeature({
    FeatureType.MASK_TIMELESS: {'LULC', 'SPLIT'},
    FeatureType.MASK: {'IS_VALID'}})

The alternative is to use the EOTask._parse_features() method to parse the features to move, instead of looping over keys and names.

Hope this helps.

1 Like