I'm black, the only time I've ever had an issue (twice). Both were when I accidentally ended up on sus tribal lands. They don't take kindly to outsiders.

One time in north Cali up by mt fanta area. There was a storm and I was looking for a hotel. Another time in Wyoming area a little west of Cody. Again weather conditions and had to get off the road in the middle of the night.