The size of the gravel is not uniform so you can not measure length with it. The same is true about fiat. Even though the unit is called 1 dollar the size of it is not fixed in time.
Discussion
You’re right that the size of gravel isn’t standardized, so the comparison I made is flawed.
It makes no sense to compare a unit of measurement (dollar) to a (somewhat) finite asset (gravel).
Just like the imperial measurement system makes no sense compared to the metric system. Yet people use both systems, but when exact data is required (I.e. landing a space ship on the moon) we use the metric system.