SSH: multiple authentication with private/public keys

As a complement to an older article on SSH key authentication and because you should never put all your eggs in one basket, here is a simple solution to use a specific key for a given identity on a given SSH server.

In our example we will use two keys: one with a passphrase for connecting to server.mydomain.tld by simply typing ssh serv and one without a passphrase mainly for running maintenance scripts on a small fleet of client machines cli01.parc01.mydomain.tld etc., whose SSH server listens on a specific port and the firewall only allows connections from server.mydomain.tld (let’s be crazy).

After generating the appropriate keys and deploying them to the respective machines:

$ ssh-keygen -f ~/.ssh/admin01.id
[...]
$ ssh-keygen -f ~/.ssh/parc01.id
[...]
$ ssh-copy-id -i ~/.ssh/admin01.id server.mydomain.tld
[...]
$ for i in {001..100};do
   ssh-copy-id -i ~/.ssh/parc01.id cli${i}.parc01.mydomain.tld;
  done
[...]

We simply edit our SSH config file ~/.ssh/config to define:

# Prevent "Too many authentication failure" when ssh agent try each key in .ssh directory
IdentitiesOnly yes

# We will connect by simply typing ssh serv
Host serv
  Hostname server.mydomain.tld
  User admin01
  IdentityFile ~/.ssh/admin01.id

# For each client machine
Host cli01.parc01.mydomain.tld
  Hostname cli001.parc01.mydomain.tld
  Port 2222
  User admin02
  IdentityFile ~/.ssh/parc01.id

The IdentitiesOnly yes is the line that does everything: without it, SSH tries every key in ~/.ssh/ in order, burns authentication attempts and ends up with “Too many authentication failures” before reaching the right key. With it, SSH only uses the key specified in IdentityFile for each host.

[edit: 2026] This mechanism became the standard pattern: one key per identity, IdentitiesOnly yes everywhere, host aliases in the config. The modern version (SSH in 2021 ) adds Include config.d/* for context splitting and myflec documents the full split: work, private, lab, every Host block in its own file. “Too many authentication failures” remains the most common error for those who accumulate keys without IdentitiesOnly.

[edit: 2026] The Ansible implementation of all this: the ssh role .