What you can delete are the events signed by the pen, but not the pen itself.

The pen is just strung-together letters and numbers and you can't make a string stop existing.

All you can do is put the pen down and walk away.