Remote File fetcher and 401/404 HTML body responses

[DiegoPino]

What?

Bug. If you fetch remotely Files via AMI and those URLs are e.g protected. And the you figure it out, you open the remotes, run AMI again, even with the re-download option during processing we end re-using the already downloaded files (HTML.. all wrong). Why? Bc the file persister actually fetches/downloads the Bodies BUT delivers an HTML... (the 401 controller). They end in the composter but until the composter run, we will keep re-using the already saved one.

Das, meine Damen und Herren, ist die definition von einem 🐞

@alliomeria will fix so we never have to deal with this again. Pull coming tomorrow AM.

[DiegoPino]

The problem happens here.

ami/src/AmiUtilityService.php

Line 498 in a9a167c

$response = $this->httpClient->get($uri, ['sink' => $path, 'timeout' => round($max_time,2)]);

the combined action of a sink (which means will download a file/body) with a 401/404 like the way Drupal provides leads to a File being saved.
The solution could be
1.- Download (so keep the sink, but delete after checking the status CODE)
2.- Better, do a HEAD request. If the HEAD has No Size/is 4XX don't even attempt to download.

[DiegoPino]

Resolved via 88899a7