I think you’re right. There is a whole class of theories in psychology and cognitive science called two-process theories. They basically propose pretty much what you described:

There is an energy-intensive, slow, deliberate mode of linear cognition that uses environmental cues to switch on. When the cues indicate less relevance, the brain switches back to the default mode, the other process, which is energy-saving, fast, intuitive, parallel. I haven't focused much on time perception in using these concepts, but I'd bet it applies well.

I haven't read it, but I assume Daniel Kahneman's book Thinking Fast and Slow refers to these concepts.