borg2: sftp:// repo url throws "Error: Invalid Backend Storage URL"

[bjo81]

Issue from: #8548 (reply in thread)

borg 2.0.0b14

borg2 transfer --repo sftp://[email protected]:23/mail  --from-borg1 --other-repo mail
Enter passphrase for key /data/mail: 
Error: Invalid Backend Storage URL: sftp://[email protected]:23/mail

[ThomasWaldmann]

The URL looks syntactically correct.

Hmm, maybe the path isn't correct and the error msg needs to be improved:

• there was a change in how URLs are interpreted, they are now relative to the current directory on the sftp server, so your URL means $CWD/mail - is that the intended location you wanted to use?
• for absolute URLs, a double-slash in the URL is needed now. So if you wanted to use /mail as a toplevel directory, it would be sftp://[email protected]:23//mail.

[ThomasWaldmann]

Also: did you create the target repo as a related repository first?

[bjo81]

I created it with rclone:sb:mail, which points to the same location. Creating with sftp:// also throws the error:

borg2 repo-create -e repokey-chacha20-poly1305  --copy-crypt-key  --repo sftp://[email protected]:23/mail  --from-borg1 --other-repo mail
Error: Invalid Backend Storage URL: sftp://[email protected]:23/mail

[ThomasWaldmann]

For first try, I used a simplified setup and it worked for me:

ssh-add ~/.ssh/mykey  # a normal ssh priv key, my yubikey "sk" key didn't work.

sftp -P23 [email protected]  # should log in without asking for pw

export BORG_REPO=sftp://[email protected]:23/borg2b14-test  # repo in CWD!
borg repo-create -e none  # takes rather long (20mins?) for creating 64k nesting dirs
borg create --list docs-archive docs

I also did another try without the BORG_REPO env var and giving the same URL via --repo option. As expected, it did not make a difference.

[bjo81]

Tried it in the same way now and still getting the error:

root@backup:/data# export BORG_REPO=sftp://[email protected]:23/mail
root@backup:/data# borg2 repo-create -e repokey-chacha20-poly1305  --copy-crypt-key  --from-borg1 --other-repo mail
Error: Invalid Backend Storage URL: sftp://[email protected]:23/mail
root@backup:/data# borg2 repo-create -e none                                                                       
Error: Invalid Backend Storage URL: sftp://[email protected]:23/mail
root@backup:/data# export BORG_REPO=sftp://[email protected]:23/mail_test
root@backup:/data# borg2 repo-create -e none
Error: Invalid Backend Storage URL: sftp://[email protected]:23/mail_test

[ThomasWaldmann]

Does borg2 -V show it is 2.0.0b14?

[bjo81]

Yes:

borg2 2.0.0b14

BTW, it's https://github.com/borgbackup/borg/releases/download/2.0.0b14/borg-linux-glibc236 I'm using here.

[ThomasWaldmann]

Ah, guess I know what it is:

Due to build / requirements issues with paramiko (which is what borgstore uses to support sftp:// stores) on some platforms, I made the sftp support optional.

So, with pip, it needs pip install borgstore[sftp], but that is not the default how borg pulls in the borgstore dependency.

So, for building the binaries, I need to change the Vagrantfile so that for the platforms used for binary building, it uses that pip install command.

So, this is not a borg bug, but rather a bug in how the binaries are built. b14 just misses support for "sftp://".

BTW, rclone also supports sftp, you could try that.
If rclone is present and recent enough, that should work with the b14 binary.

[ThomasWaldmann]

I'll change the exception msg in borgstore to:

Invalid or unsupported Backend Storage URL: ...

It's non-trivial to do better there currently due to:

• the code being structurally able to support multiple implementations of a backend supporting "sftp://" urls (although we currently only have one)
• no logging yet, so it is just the exception msg