SSH Frequently Asked Questions

I want to use sftp in a script, but it doesn't work — I can't use "here documents," and it keeps trying to prompt for stuff.

---

Authentication

First off, you must also use an authentication method which does not require user input, e.g. public-key with ssh-agent, or trusted-host. If the script is to be run unattended, for instance as a cron job, see this discussion.

shell-script "here documents"

A shell script using ftp with a "here document" looks like this:

  #!/bin/sh
  echo "OK, starting now..."
  ftp remotehost <<EOF
  cd pub
  binary
  get foo.tar.gz
  quit
  EOF
  # script goes on to do some other things...

If you try this with sftp, you'll get this:

Warning: tcgetattr failed in ssh_rl_set_tty_modes_for_fd: fd 0: Inappropriate ioctl for device

...and sftp will fail to work as you intend, because it's trying to treat its standard input as a terminal. sftp has a -b file switch, to have it read commands from a file instead of the terminal. So if you place the sftp commands in separate file, this will work.

However, it's nice to keep everything together in one file using "here documents." What you can do, is this:

  #!/bin/sh
  echo "OK, starting now..."
  sftp -b /dev/stdin remotehost <<EOF
  cd pub
  ...

Most modern Unix flavors provide a way to access the current process' standard streams via the filesystem. Linux has /dev/stdin; under Solaris, it would be /dev/fd/0 — yours may be different.

There is a problem using this technique under Linux with the csh and tcsh shells — the bash shell works properly, however. If you want to understand the problem, read on.

A particular problem with Linux and (t)csh

If you try this method under Linux with either csh or tcsh as the script interpreter, you get the following rather odd error from sftp (the tests were done with RedHat Linux, kernel 2.2.12-20):

Could not read the batchfile.

If you trace sftp, you find that its open of /dev/stdin fails with EPERM. It's hard to imagine why a process would not have permission to open its own standard input — you'd think that opening /dev/stdin should be equivalent to dup'ing fd 0. However, this doesn't appear to be the way it's implemented. Instead, /dev/stdin (actually a symlink to /proc/self/fd/0) behaves as a "phantom link" to the inode opened on standard input. This means that the permissions on that file are relevant, even though the process already has the file open and could get equivalent access just by reading or dup'ing fd 0, regardless of the file permissions.

The exact problem is this: the shell implements a "here document" for a program by placing the input lines in a temporary file, opening the file, immediately unlinking that file, and placing the open file descriptor on the command's standard input when the shell runs the command. (t)csh, however, takes the extra privacy precaution of specifying the file's modes as 000 when creating it, making it unreadable by anyone. This seems fine: since the file is immediately unlinked, no one can ever try to open it again anyway... oops, but they can: via the /dev/stdin mechanism! When sftp tries open /dev/stdin, it is prevented by the file permissions.

I think one can argue that the /proc/self/fd/* files should work differently: they should always appear to be owned by the caller's effective uid, and be mode 600. I intend to suggest this to the Linux kernel folks. In the meantime, fortunately sh and bash do not set the permissions like this, and so you can script sftp using those shells.