If you are all linux, perhaps you’d like Borg. It’s pretty easy to set up, and perhaps it offers the de-duplication you crave. It’s also got some GUIs you can run, if you are into that kind of thing. If you don’t feel inclined to use any backup software, then using rsync over ssh (scp, yes) instead of SFTP is a solid way to go.
Which is somewhat BS, since we have a perfectly good Olympic Peninsula here in Washington State that it was certainly named after.