i wasnt paying attention to the part where you said turned off...

The device _shouldnt_ do anything when its turned off... but... there are reasons why smartphones generally dont have removable batteries and come with gps.

If you are concerned about privacy, keep it in a faraday cage, bag, or other signal blocking implement.