Most servers I connect to have the option
PasswordAuthentication set to
no, meaning I do more often than not see an error:
Permission denied (publickey).
The reasons for this are multiple, but in my scenario, this happens because there are no identities (keys) present in the SSH agent.
SSH agent holds OpenSSH keys used for public key authentication (public key authentication method is vastly superior to plain-text password authentication and should be preferred whenever possible), and hands the right one over to the server during connection. OpenSSH keys in itself is a rather broad term. For sake of this article, the term OpenSSH key here refers to the context of public key authentication. Public key authentication actually requires a key pair - obviously the public key and it's matched counterpart, a private key.
The default SSH agent shipped with OpenSSH is
ssh-agent. It has a rather
basic feature set, as it expected. There are software packages available
that can acts as an SSH agent and they feature sets differ. There can be
only one active SSH agent running on the system at a given time.
Keyrings and Password managers
Many distributions ship with Gnome Keyring, a password manager/keyring (branded also as User Credentials Manager, implying it is able to handle multiple types of credentials). These two terms are not the same, but for clarity, this article individually refers to Gnome Keyring as keyring and to KeePassXC as a password manager. Both software packages overlap greatly in the features primarily discussed here, thus the terms appear together.
Both KeePassXC and especially Gnome Keyring can handle multiple types of secrets or user credentials (some of them overlapping) such as passwords, security certificates and Freesdesktop.org secrets in addition to the keys such as GnuPG keys, and the core of this topic, OpenSSH keys. Keyring's/password manager's ability to handle OpenSSH keys means it also features an SSH agent implementation.
Private key passphrase
Private key can and (unless there is a valid reason not to) should be additionally protected with a passphrase. If protected, a valid passphrase is required before the key can be used for an actual authentication.
Extending the previous definition, Keyring's/password manager's ability to handle OpenSSH key, additionally means, it can store, and later recall, the stored private key's passphrase.
In the most basic sense,
ssh-agent prompts for a passphrase when the
passphrase protected key is added into it. This process is synchronous in a
sense that the terminal expects user to provide the passphrase before next
command can be issued. The agent is preventing subsequent prompts for the
same key already added into it, until the key is finally removed from it.
Reducing the amount of passphrase prompts is in fact the main
responsibility of the agent.
Keyring and Gnome Keyring work differently compared to the
respect to adding keys into their respective agent implementations. When
configured for this, user is not required to provide passphrases for
individual keys, as the passphrases are stored within keyring/password
manager's database. Instead, user is prompted for a single master password
when unlocking the keyring (with Gnome keyring there usually is no
visible prompt for the SSH passphrase at all. This is by design, as an
user account password is used as a keyring master password, unlocking the
keyring automatically after user logs in) or the database.
The problem arises in situations where the prompt is invoked asynchronously - when there is no terminal associated to it. This is generally the case with programs employing a graphical user interface (GUI). Since Gnome Keyring and KeePassXC have their own graphical interfaces, they are both affected.
SSH_ASKPASS and ssh-add
Manipulating keys in the agent is done by
ssh-add. While there are
currently multiple agents available, there's just a single widely used
ssh-add. All OpenSSH agents mentioned strive to be compatible with the
ssh-add command, otherwise they would not be very useful.
One particularly problematic scenario with the passphrase prompt and both
discussed GUI programs is the
ssh-add (while technically KeePassXC and Gnome Keyring would work well
enough as an agents without implementing this functionality, they did
implement it over time, because of the community requests for a full
compatibility with the
ssh-agent), flagging the key added to the agent
for confirmation before being used. The key flagged like this is still
added to the keyring/password manager's agent, but expects to prompt the
passphrase into a floating dialog the moment a key is actually used. This
functionality in turn requires at least a properly configured
environmental variable, pointing to a working graphical prompt
KeePassXC and user confirmation
There is an important and useful setting in a KeePassXC entry under SSH
tab, removing keys from agent when the database is locked (because of tight
coupling with user login session, Gnome Keyring doesn't offer an option to
the keys from the agent, which can theoretically increase the potential for
their misuse. Original
ssh-agent agent and specifically KeePassXC, both
operating separately from the user login session, have options to
automatically remove the keys from the agent). Enable it by checking:
- Remove key from agent when database is closed/locked
It is worth noting, that KeePassXC additionally to storing the passphrase
also offers to store the actual private key in the database as an
attachment. With above setting enabled, further enabling user
confirmation under the same SSH tab is a great hindrance when both, the key
and its passphrase are stored inside the same database. To increase
security, one should always strive to store different security factors on
different locations (it is a safer approach to store the different
factors of a security in different places. For instance, when using
password + 2FA/MFA in a form of TOTP, store password in the manager and the
other in the phone. This applies for SSH keys as well, for instance storing
the private key in the filesystem /something I own/ and its passphrase it
in a brain /something I know/. In practice, storing every available
factor in single one secure
.kdbx database with a strong master password
is still marginally better than omitting some available factors, such as
using just a password without TOTP or using a ssh key without a
Furthermore, when user confirmation is enabled, but
mis-configured, SSH will stop working, displaying an error:
sign_and_send_pubkey: signing failed for RSA "email@example.com" from agent: agent refused operation
With the above in mind, I would advise considering enabling user confirmation only when either:
- The actual private key and its passphrase are both stored in a different databases
- Keys do not get automatically removed from agent when the database is locked
Before making commands that require
ssh automatically prompt for a
KeePassXC database unlock dialog, there are some more steps required. Since
there can only be one SSH agent active at a time, other agents possibly
running have to be disabled or configured differently, before the KeePassXC
agent implementation can be used.
ssh-agentat login as described in SSH_keys#Start_ssh-agent_with_systemd_user
- When on distribution where Gnome Keyring is present, disable it:
chmod -x /usr/bin/gnome-keyring-daemon
Further, when on actual Gnome desktop environment, continue with GNOME/Keyring#Disable_keyring_daemon_components
Configure KeePassXC Settings accordingly:
General > Startup
- Start only a single instance of KeePassXC
- Minimize window after unlocking database
- Remember previously used databases
- Load previously open databases on startup
Security > Convenience
- Lock databases when session is locked or lid is closed
- Enable SSH Agent integration
- Configure an actual SSH entry accordingly:
Edit entry > Entry
Edit entry > SSH Agent
- Add key to agent when database is opened/unlocked
- Remove key from agent when database is closed/locked
Edit entry > SSH Agent > Private key
Insert either valid [Attachment] OR [External file]
Test the setup
Before continuing, test everything works properly. Start by unlocking the
KeePassXC database and running
ssh-add -l, the output should be:
4096 SHA256:9k/Nfk7fijei+JFj8F7YfyF7fhFHElSmpuFuew9+8f3 firstname.lastname@example.org (RSA)
Locking the database and running
ssh-add -l should yield precisely this:
The agent has no identities.
If this is not the case, consider consulting the Further reading section at the bottom. There are useful links from the other people using KeePassXC to their advantage.
Prompt KeePassXC unlock with ssh
When everything is prepared and tested, the actual thing is to implement
the script that gets called before any command employing
ssh is run. To
do so, create following two files:
- Paste the following line into
ProxyCommand $HOME/.ssh/keepassxc-prompt %h %p
- Last thing, create executable script
#!/bin/bash until ssh-add -l &> /dev/null do echo "Waiting for agent. Please unlock the database." keepassxc &> /dev/null sleep 1 done /usr/bin/nc "$1" "$2"
Done! Running any command that relies on
ssh while the database is still
locked, the KeePassXC unlocking dialog is invoked. After unlocking, the
keys are automatically added into the agent and the command succeeds. No
Permission denied (publickey). for the valid keys!
This is a 30th post of #100daystooffload.
Frequently Asked Questions (FAQ)
Why is the
keepassxc-prompt script so plain?
It serves as a proof of concept. Modify as needed. One good modification is to add a timeout.
ProxyCommand used for SSH forwarding?
Yes, but it appears to be working with this approach without problems.
Why not just alias
ssh to the wrapper script?
There are other commands that rely on
ssh. With an alias every other
command would not invoke the unlocking prompt, for instance
Why not replace
/usr/bin/ssh with a wrapper script?
Many reasons. It prevents proper upgrades, with something security delicate
as OpenSSH, it is advised to use updated software. Next, it requires on a
single user's file, so it would disable
ssh for other users on the
system. Asides, it is plainly ugly.
Can I use D-Bus to detect if the database is locked/unlocked?
Sure. If there are also others keys in the agent that are managed
separately of KeePassXC, simply use this instead of
ssh-add -l in the
qdbus org.keepassxc.KeePassXC.MainWindow /org/freedesktop/secrets/collection/Passwords org.freedesktop.Secret.Collection.Locked
/etc/sshrc or, when enabled,
~/.ssh/rc be used for this
No. These hook files run on the server when the connection is initiated.
For this to work, the hook must be run on the client. The latter also has
to be enabled via
PermitUserRC yes in
What are other situations where KeePassXC prompts for unlock automatically?
- When global Auto-type keyboard shortcut is used
- When KeePassXC Freesdesktop.org Secret Service is enabled and a program needs access
- On any browser plugin keyboard shortcut, assuming installed and configured properly
Settings > Browser Integration
- Enable browser integration
- Request to unlock database if it is locked