No external signs under the new covenant? How about baptism and the Lord's supper, sir?

The new covenant is an expansion of the old. There are things that are abrogated in Christ as the once-for-all sacrifice for sins, such as the entirety of the sacrificial system, but other aspects are not abrogated, having instead been changed.

The initiatory sign of the covenant is of the latter sort. Instead of the outward sign of circumcision, applicable only to males, we have the outward sign of baptism, expanded to include both males and females. No longer a bloody right, because sufficient blood for all has already been shed in the Son.

Both circumcision and baptism are meant to represent an inward work of the Spirit, as well, but they are also both outward signs of initiation into the covenant people of God.

"And the Lord your God will circumcise your heart and the heart of your offspring, so that you will love the Lord your God with all your heart and with all your soul, that you may live." Deuteronomy 30:6

"...having been buried with him in baptism, in which you were also raised with him through faith in the powerful working of God, who raised him from the dead." Colossians 2:12