Normally, Empathy only shows your contacts that are currently online. You can show all of your contacts instead, including those that are offline.
From the Contact List window, choose View Offline Contacts, or press Ctrl H.
To hide the offline contacts again, repeat the same procedure above.