Your phone isn't listening to you without your consent; you're just too lazy to adjust your privacy settings.