If you are lean, muscular, with optimized fat and glucose metabolism, it's perfectly fine. Especially if it's your only form of swamping. Seek raw milk ice cream.
If you don't have great health, then ice cream is going to get in your way, metabolically speaking.