It's not a magnet that's moving. It's electric charge that's moving, which is how magnetic fields are generated.
It's fine to say we don't know for sure what's down there. It is based on inference from indirect sources, but the model of what's down there is fairly detailed and does account for the major phenomena we do observe directly.