If yiu have a crush sleeve (spacer) yes spacer against the race. Otherwise use the washer.

You want the bearing to freely turn but not the race of the bearing