Register now for Tech Days 2023 to save on your Full Conference Pass    |    Get Your Pass

What is SSH and SSH Key Management?

Secure Shell (SSH) is a widely adopted protocol that allows users, typically system administrators, to securely authenticate and execute commands on remote systems. There are two ways to access systems using the SSH protocol – SSH key-based access and traditional passwords.

SSH keys are harder to crack and much easier to use than passwords, which makes them the default choice for the SSH protocol. However, to achieve these benefits, organizations must introduce proper SSH key management.

This guide will explore everything you need to know to get started with SSH key management, including:

  • What is SSH and how does it work?
  • What are the risks of unmanaged or mismanaged SSH keys?
  • Why is SSH key management so important?
  • 6 best practices for SSH key management

What is SSH?

Secure Shell (SSH) offers a secure way for a person or device to gain access to one system (or server) from another. It replaces the standard username and password with a trusted SSH key pair to provide simpler and more secure ongoing access to remote systems. This ongoing access can cover activities like system administration, automated processes, and managing mission-critical network infrastructure.

Gain Control of Your SSH Keys Today.

How SSH Works: Understanding SSH Keys

The SSH protocol uses key-based authentication to gain access to remote systems. Rather than requiring users to log in with a username and password, a pair of keys are used to verify identity and encrypt communications between client and server. Since SSH keys only require a one-time setup, providing ongoing access after the initial key pair generation, provides more secure and seamless access.

Public & Private SSH Keys

These SSH keys are based on asymmetric cryptography, which uses two different, but mathematically related keys (known as a key pair), to authorize and encrypt connections. In the case of SSH, private and public key pairs are generated to authenticate users for remote access. The public key sits on the remote system and provides access to any user or device that has the corresponding private key, which serves as a method for authentication.

How SSH Keys Are Used

In many organizations, SSH keys provide access to critical systems like databases, servers, and recovery systems. They are also used for more secure file transfers across systems (since the SSH protocol ensures that all information shared gets encrypted), issuing remote commands, and delivering support or updates to systems.

Why Is SSH Key Management So important?

SSH key management is essential to maintaining security since the entire SSH protocol, and therefore the security of it, is built around public and private key pairs. Proper SSH key management protects against critical risks that can lead to system failures or allow important information to fall into the wrong hands.

Risks of Untracked and Unmanaged SSH Keys

Failure to manage SSH keys, or simply mismanaging SSH keys, can open organizations to several risks and challenges, including:

Loss of Control

It’s Too Easy to Generate Keys

If there is no centralized control for SSH key management, anyone in the organization can easily generate or duplicate their own SSH keys for one-off use cases. When this happens, not only does the number of keys created get out of control, but it also becomes harder to take inventory and manage the keys that do exist. This situation can make it easier for malicious parties to access keys and lead to several other challenges noted below that open the organization to risk.

How an SSH Key is Generated (in 30 Seconds or Less)


Static Keys

There is No Expiration Date

Unlike SSL/TLS certificates, SSH keys don’t expire, which opens the door for increased risk over time. That’s because the longer SSH keys exist, the less likely they are to fall under any kind of key management or key security oversight. In turn, this creates situations where users leave the company, but still have access to their SSH keys. It can also make it difficult to rotate keys to maintain security, since the longer keys are in existence, the harder it becomes for security teams to identify their purpose. SSH key rotation and hygiene are critical but often avoided by IT and security admins out of fear that rotating keys will incidentally block access to critical systems.

Orphaned Keys

People Come and Go – Keys Stay

SSH keys become orphaned keys when they are no longer actively used and the location of their corresponding public or private key becomes unknown. When users change roles or leave the organization, they often leave behind once active SSH keys. This situation makes it difficult for security users to identify the purpose of the orphaned keys. When that happens, the security team is hesitant to retire the orphaned keys, as doing so could potentially cause a serious system failure.

Key Compromise

SSH-Based Attacks Are on the Rise

Widespread attacks such as Lemon_Duck and FrtizFrog highlight the importance of managing and protecting SSH keys. Even a single compromised private SSH key gives attackers the ability to impersonate trusted admins, hide in encrypted traffic, and move laterally across your network. Without proper protection and access control, malicious parties (i.e. hackers) or even non-malicious parties, such as users who have since left the organization but retain access to SSH keys, can access critical systems and confidential information.

Key Sprawl

SSH Keys Have Proliferated in DevOps & Cloud

Key sprawl occurs as the use of SSH key based access continues to multiply over time, often at exponential rates. At a certain point, the number of keys in circulation becomes too much to reign in and properly manage. Challenges like those listed above often contribute to growing key sprawl over time. Ultimately, the greater the key sprawl, the more opportunities exist for private SSH keys to become compromised since it is harder for security users to track and manage all of the SSH keys in existence.

What Happens if SSH Keys Are Compromised?

If SSH keys fall into the wrong hands, they can provide root or privileged key based access to systems and information. As a result, SSH keys are often viewed as a top target for hackers, and the impact of compromised SSH keys is high for organizations.

Compromised SSH keys can have a serious impact on everything from an organization’s reputation to the safety of its data (and that of its clients and partners) and the operational status of business-critical systems. Additionally, one compromised key can create a slippery slope by providing unintended users with access to additional SSH keys, which can in turn allow them to access more systems and data.

Specifically, compromised SSH keys can give malicious parties access to systems and allow them to:

  • Access, change or delete sensitive data
  • Introduce malware
  • Alter or destroy systems
  • Use other authentication credentials to gain access to additional systems
  • Disrupt data transfers between systems or organizations

SSH Key Management Best Practices

SSH key management is critical for any organization using the SSH protocol. However, given the number of keys typically in use (often in the hundreds of thousands), proper management can become challenging very easily.

The following best practices can help avoid these challenges to ensure full protection through proper SSH key management at all times.

01 Discover & Inventory SSH Keys

First and foremost, one of the primary pillars of proper SSH key management is maintaining an inventory of all the keys in use. This inventory should involve mapping SSH key pairs to both users and servers as they get created and running periodic checks on this list to ensure everything is still up to date.

This type of key inventory should help security users maintain a full understanding of what keys exist, which systems they grant access to, for which users, and the purpose of the keys. In turn, this understanding should help protect against orphaned keys and, in conjunction with some of the other SSH key management best practices outlined below, prevent key sprawl.

02 Implement Policy & Governance

Next, security users should set and enforce strict policies for how SSH keys get created and used. These policies should cover areas of concern including:

  • Who can create keys and for what purpose
  • How keys get created (e.g. enforcing a minimum key length adhering to accepted standards for asymmetric encryption like RSA, ECC and Diffie-Hellman)
  • How many users can be associated with a single key (in most cases, only one user should be associated with each key)
  • How long keys should last until they expire (i.e. how often keys should be rotated and who is responsible)
  • What should happen to keys when the user associated to them leaves the organization


Setting and enforcing these types of standards can help eliminate key sprawl caused by orphaned and expired keys. They should also help reduce the risk of keys getting compromised by eliminating key sprawl, tightening access to keys and strengthening the security of keys from the moment they get created.

03 Identify & Remediate Risks

Once all SSH keys are collected in a single inventory and mapped to associated users and machines, it becomes much easier to identify and remediate risks. Not only does this help organizations to identify the number of users with root access to privileged systems, it also allows them to locate SSH keys that are no longer in use.

By following SSH key management best practices, it’s possible to delete all SSH keys associated with various inactive users and machines in the network, and replace stale or unmanaged keys with freshly generated key pairs. Only defined users and/or groups should be able to create and deploy key pairs to systems to ensure compliance with security requirements.

04 Introduce SSH Key Rotation

One important best practice for SSH key management is to rotate keys at regularly set intervals as dictated by the organization’s policies. Key rotation requires users to generate new keys after a certain amount of time so that the same keys don’t get used for too long a period of time.

In most organizations, the number of SSH keys in use requires key rotation to occur via automation rather than manually. Overall, this type of key rotation helps strengthen security for SSH by generating net-new key pairs on a regular basis, which also helps protect against risks like key sprawl.

05 Enforce Granular Access Controls

Another critical part of SSH key management is introducing access controls, which should govern where any given key can be used from, how often, and the commands that key can be used to execute. It should also ensure that only authorized users can create SSH key pairs.

This type of access control creates more focused use cases for key based access. The combination of controlled usage location and executable commands make it more difficult for unintended users to access keys and for them to parlay access from one key to usage across multiple systems. As a result, strong access control can not only help reduce the likelihood of a private SSH key falling into the wrong hands, but it can also help lower the impact should any keys become compromised.

06 Continuously Audit & Monitor

Finally, auditing is a must for any program following SSH key management best practices. Specifically, security users should audit SSH key based authentication regularly to ensure that all keys in use adhere to policies and belong to users who are still active (versus those who have left the organization).

Once again, given the amount of SSH keys in most organizations, most of this auditing work should be automated so that users can accomplish it efficiently and effectively. Conducting regular audits of these factors should help with SSH key policy enforcement and allow security users to identify any areas of risk sooner rather than later.

Introducing SSH Key Management in Your Organization

If your organization uses SSH key based authentication to govern remote access to systems, then proper SSH key management is essential.

Most organizations that introduce SSH do so with the intention of improving security, but the only way to achieve that benefit is through proper key management. As outlined above, the risks of unmanaged or mismanaged SSH keys are plentiful and can lead to serious financial and reputational damage for your organization.

Fortunately, adopting these SSH key management best practices can help protect against these risks and allow your organization to use SSH in a highly secure way. Introducing these efforts will take time, but with the right processes and technology, they are well within reach for any security team that needs to reign in SSH key management.

Ready for more information?

Whether you’re first getting started with SSH or need help taking back control of SSH key sprawl in your organization, download our eBook to learn more.

Find out how the Keyfactor platform can modernize your PKI, prevent
certificate outages, accelerate DevOps security, and more.

The 2022 State of Machine Identity Management Report

Get actionable insights from 1,200+ IT and security professionals on the next frontier for IAM strategy — machine identities.

Read the Report →