It turns out they make special bits specifically for tile, and you should not use a regular drill bit attached to an impact driver...