There were/are attempts like Ubuntu Touch or Plasma Mobile. Still very basic support, but with lots of potential - at least for tech interested people.
The main problem is that most mobile SoCs are locked or don't release OpenSource drivers.
Maybe one day we will have free RISC-V SoCs which attempts to replace ARM.