sftp:connect-program not working for passphrase protected keys

[rpgdev]

lftp is not asking me for my passphrase protected key. It used to work back in the day but right now it keeps asking me for the ftp password instead, if I try to bypass it it just never connects to the destination.

I set this line in the ~/.config/lftp/rc file (every alias there is loaded as well as the lftp prompt style).
set sftp:connect-program "ssh -a -x -i /home/user/.ssh/id_ed25519"

Am I doing something wrong? Is there anything else I'm missing?