no, the geographical pole does not shift slowly, the magnetic poles move around a lot.

https://www.ncei.noaa.gov/products/wandering-geomagnetic-poles

the geographical pole is confined by inertia and the coupling of the crust with the core, which is magnetically aligned to a strong degree with the pole of the sun.

the pole shift happens when the crust's binding with the core weakens (through increased magma temperature) and the external magnetic alignment conflicts with the current core field. it is a correction to bring the crust in alignment with the external and internal fields.

as for serbia... yeah, some of the best parts of my last 12 years were in that place. bulgaria was nice but serbia ... idk. it was just magical. i would have trouble explaining.