It can happen when switching between clients. Iris has "Follow all" button that helps to get your following back up. Nostrgram.co has a history feature that may restore the original list.