That's the only thing that makes sense to me. Otherwise the network would be at the mercy of whoever controlled the clock it references to determine the difficulty setting.