SSH slow to login / ask for password

On your server, or computer that you are connecting to..
Edit /etc/ssh/sshd_config (NOT ssh_config)
Add to the end of the file:
UseDNS no

Reboot or restart sshd
/etc/init.d/sshd restart