Many libertarians are closet conservatives. They pretend to be socially and fiscally open, but in practice they aggressively simp for social conservative dictators when given the chance. Their entire vibe changes and becomes more obviously conservative sometimes after lost elections.

That's a reason why they have limited support to begin with; a lot of people that might otherwise support their views kind of sense this vibe ahead of time and then it indeed plays out in practice.