It was usually Fournival pre-patch because his affection shot up through the quests you are given.