vfs/shell: handle remote errors on send

[tuffnatty]

Proposed changes

I've hit #3128 and tried to fix it.

The old workflow during moving a file to a write-protected remote location looked like:

1. copy_file_file() calls mc_open() on the destination vpath.
2. mc_open() calls shell_open()
3. shell_open() generates a directory entry in the local VFS cache and returns a handle to it.
4. copy_file_file() transfers the file content into the local temporary copy and sets dst_status to DEST_FULL.
5. copy_file_file() calls mc_close().
6. mc_close() calls shell_file_store().
7. shell_file_store() calls remote script, src/vfs/shell/helpers/send.
8. send answers with ### 001.
9. shell_file_store() receives ### 001 and starts to send file data.
10. send fails to truncate the destination file with > "${FILENAME}", ignores the error, and starts the loop to consume exactly the specified number of bytes, attempting to write them to the write-protected destination path and ignoring any errors.
11. After consuming the specified number of bytes, send answers with ### 200.
12. mc_close() returns successfully.
13. copy_file_file() tries to chmod and chown the file, fails, the user chooses to Ignore the errors.
14. copy_file_file() returns FILE_CONT, and move_file_file() removes the source file, leaving no copy neither in source nor in destination.

The simplest fix in the scenario with a write-protected destination is provided in the first patch, which adds an attempt to truncate the file or append 0 bytes to the file in shell_open() before creating the fake directory entry in the local VFS cache, and returns an error on failure. It also has a positive side effect of failing early, before creating the local temporary copy etc.

But the first patch alone does not solve the issue when the remote error occurs later during the copy, e.g. in an out of space situation. So the task was to make the send (and append) script detect and report the errors while also still working on the lowest common denominator of a shell environment. To do it, I had to implement some tests and some primitive test infrastructure for these scripts, which, as I've found out later, was already discussed in #3635. The test scripts are designed to run wherever the GitHub CI is running, they assume a fairly POSIX-compliant sh and are known to work in dash, bash, and FreeBSD Almquist shell. It's the subject of the second patch.

The third patch contains the updated send and append scripts. They are written in an archaic dialect, so that Heirloom Bourne shell can run them. They handle the write errors, consume the rest of the data, and respond with ### 500.

But even when the scripts return ### 500 and mc_close() returns an error, dst_status is still equal to DEST_FULL, which is obviously a lie. The fourth patch catches this condition and changes dst_status to DEST_SHORT_QUERY, asking the user to choose to keep or delete the partial file, and preventing the deletion of the source file in move_file_file().

• Resolves: dataloss: moving a file and skipping upon problem deletes original file #3128.
• Probably also makes savannah: Aborting a FISH file transfer is grossly inefficient #68, savannah: fish copies files to tmp before transfer -> fails with low disk space #77 more bearable in some situations.
• Probably is also a fix for Move over ssh shell will destory files when target FS full #3961, but it needs to be confirmed.

* Needs: #5155. Well it's not a hard dependency, so I have rebased this PR on the master branch to ease reviewing.

Checklist

• I have referenced the issue(s) resolved by this PR (if any)
• I have signed-off my contribution with git commit --amend -s
• Lint and unit tests pass locally with my changes (make indent && make check)
• I have added tests that prove my fix is effective or that my feature works
• I have added the necessary documentation (if appropriate)

[tuffnatty]

I've applied @ossilator 's suggestions and rebased the PR on master to ease reviewing.