How to Generate SSH Keys and Connect to Your VPS Securely (2025 Guide)

You’ve just spun up your first VPS and need to connect securely. While you could keep using passwords, there’s a better way. SSH keys eliminate the need to type passwords every time you connect, and they’re far more secure than any password you could create.

SSH keys work using public-key cryptography: you generate a pair of keys (one private, one public), keep the private key on your computer, and place the public key on your server. When you connect, the server checks if your private key matches the public key on file. If they match, you’re in.

By the end of this guide, you’ll know how to generate SSH keys, upload them to your VPS, and connect securely without passwords. We’ll also cover common mistakes and how to fix them when things go wrong.

TLDR

# Generate key
ssh-keygen -t ed25519 -C "[email protected]"

# Copy to server
ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

# Connect
ssh [email protected]

Why SSH Keys Are (Still) the Secure Choice

Unphishable: There’s no password to steal or reuse.
Strong cryptography: Ed25519 keys are small, fast, and secure.
Local protection: You can encrypt the private key with a passphrase.
Granular access: A key can be limited to specific users, hosts, or commands.
Auditable & revocable: Remove one public key without affecting others.

Popular Use Cases

VPS logins (e.g., Ubuntu/Debian servers you manage with Serversinc).
Git operations with GitHub/GitLab/Bitbucket.
Automations/CI that need server access without interactive passwords.
Tunneling/port-forwarding for secure access to internal services.

Generating Your SSH Key Pair

Always use Ed25519 unless you’re connecting to very old systems (pre-2014). It’s faster, more secure, and creates smaller keys than RSA.

ssh-keygen -t ed25519 -C "[email protected]"

When prompted:

File location: Press Enter for default (~/.ssh/id_ed25519) or specify a custom path
Passphrase: Always use one for VPS access** - it’s your last line of defense if someone gets your private key

You’ll see output like:

Your identification has been saved in /home/you/.ssh/id_ed25519
Your public key has been saved in /home/you/.ssh/id_ed25519.pub

Set Correct Permissions

This is crucial - SSH will refuse to work if permissions are too open:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

Uploading Your Public Key to Your VPS

You need to get your public key (the .pub file) onto your server. Never upload your private key.

Method 1: Using ssh-copy-id (Recommended)

ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

This automatically creates the right directories and sets correct permissions.

Method 2: Manual Upload

If ssh-copy-id isn’t available:

Display your public key:

cat ~/.ssh/id_ed25519.pub

Copy the entire output (it should be one long line starting with ssh-ed25519)
Connect to your server with password:

ssh [email protected]

Create the SSH directory and add your key:

mkdir -p ~/.ssh
echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... [email protected]" >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Testing Your SSH Key

Try connecting with your key:

ssh -i ~/.ssh/id_ed25519 [email protected]

If you used the default key name (id_ed25519), you can omit the -i flag:

ssh [email protected]

You should be prompted for your passphrase (not your server password). After entering it, you should be logged in.

Setting Up SSH Config (Optional but Recommended)

Create ~/.ssh/config to simplify connections:

Host myserver
    HostName your-server.com
    User yourusername
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Now you can connect with just:

ssh myserver

Security Best Practices

1. Disable Password Authentication (After Testing SSH Keys)

Once SSH keys work, disable password login on your server:

sudo nano /etc/ssh/sshd_config

Change or add these lines:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

Restart SSH:

sudo systemctl restart sshd

2. Key Management Rules

Never share your private key, treat it like a password
Use passphrases for any key that accesses servers
Separate keys for different servers/services
Store encrypted backups of your keys securely

3. Additional Server Hardening

Change SSH port from default 22: Port 2222 in sshd_config
Disable root login: PermitRootLogin no
Use fail2ban to block brute force attempts

Common Mistakes and How to Fix Them

”Permission denied (publickey)”

Most common causes:

Wrong permissions - Fix with:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 600 ~/.ssh/authorized_keys  # on server

Public key not properly added - Check the server’s ~/.ssh/authorized_keys:

cat ~/.ssh/authorized_keys

Should be one long line per key, no line breaks.

Wrong username or hostname - Double-check these match your server setup.

”Too many authentication failures”

SSH tries multiple keys and gives up after 6 attempts. Use IdentitiesOnly yes in your SSH config, or specify the exact key:

ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519 user@server

“Connection timed out”

Check if SSH service is running: sudo systemctl status sshd
Verify the port (default 22): ssh -p 2222 user@server if changed
Check firewall settings: sudo ufw status

Key Not Found Errors

SSH looks for keys in specific locations. If you used a custom name, either:

Specify it: ssh -i ~/.ssh/custom_key user@server
Or add to SSH config with IdentityFile directive

Accidentally Uploaded Private Key

If you accidentally put your private key in authorized_keys:

Remove it immediately: nano ~/.ssh/authorized_keys
Generate a new key pair
The old private key is now compromised

What to Do If You Get Locked Out

Prevention is better than cure, but if you lose access:

VPS providers usually offer console access through their web panel
Recovery mode or rescue system (most VPS providers support this)
Backup access method, always keep password authentication enabled until you’re 100% sure SSH keys work
Snapshot/backup your server before making SSH changes

Troubleshooting with Verbose Output

When things go wrong, add -vvv for detailed debugging:

ssh -vvv [email protected]

This shows exactly what SSH is trying and where it fails.

Platform-Specific Notes

Windows (PowerShell)

Windows 10/11 include OpenSSH:

ssh-keygen -t ed25519 -C "[email protected]"

Keys are stored in: C:\Users\YourName\.ssh\

macOS

Add key to keychain to avoid repeated passphrase prompts:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

Conclusion

SSH keys are essential for secure VPS management. They’re more secure than passwords, more convenient once set up, and industry standard for server access. Take time to set them up properly - the security benefits are worth the initial effort.

Remember: protect your private key like a password, use passphrases, and always test your setup before disabling password authentication.