I am not sure if i understand your question right, but of course mathematicians have explored the concept of geometry in non-Euclidean spaces, where the traditional rules of Euclidean geometry do not apply. Non-Euclidean geometries, such as hyperbolic and elliptic geometries, deal with planes that have different properties from those of Euclidean space.