The empire can fall and society can go on.
I stole this quote from Wikipedia just because it defines societal collapse succinctly.
“Societal collapse is the fall of a complex human society characterized by the loss of cultural identity and of social complexity as an adaptive system, the downfall of government, and the rise of violence.”
The Roman Empire fell but the societies it encompassed continued and in many cases flourished.