Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve object-oriented classification using PSE-TAE and LTAE methods #1261

Open
gilbertocamara opened this issue Jan 7, 2025 · 0 comments
Assignees
Milestone

Comments

@gilbertocamara
Copy link
Contributor

Describe the requested improvement

This rationale for this improvement has been pointed out by @charlotte-pel:

The foundational PSE-TAE algorithm paper by @VSainteuf (Vivien Garnot) and colleagues uses polygons as its assumed input data structure. The data are farm boundaries in France, and the algorithm aims to classify agricultural cultivation types. Garnot's algorithm has two steps. See section 3.1 and Figures 1 and 2 of the original paper (‘Satellite Image Time Series Classification with Pixel-Set Encoders and Temporal Self-Attention’). First, an MLP (multi-layer perceptron) combines the points within a polygon, and positional encoding organizes the result. Garnot calls this phase PSE (pixel-set encoder). In the second phase, the PSE result is input to an attention model called TAE (temporal attention encoder). See how Garnot describes the first part in section 3.1 of the original paper.

In the sits implementation of PSE-TAE and LTAE, we classify each time series individually. In most sits applications, training data is available as individual locations and not as polygons. For this reason, we did not implement the PSE part of Garnot's method. The temporal attention encoder receives individual points.

Segmentation with sits_segment() results in post-segmentation objects. In this case, we could have used the same strategy as Garnot, implementing a pixel-set encoder. However, currently we use the same pixel-based logic to train and classify the points within a polygon. Strictly speaking, our implementation of OBIA in sits does not correspond to the PSE-TAE algorithm and its LTAE derivative.

It would be very interesting to compare our implementation with Garnot's original algorithm. To do this, we would have to include a new algorithm and new training methods, namely:

(a) make a segmentation.
(b) classify the points inside polygons using a similar strategy as the one by Garnot.
(c) implement PSE-TAE in the form proposed by Garnot.

We will be able to make comparative statements between the pixel-based version of LTAE and the original object-based version of LTAE.

Associated sits API function
sits_tae() and sits_ltae() will have to be modified to include the polygon-based PSE part of Garnot's proposal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

4 participants