Measuring success for “onboarding” efforts … if not “active” users then what? (All ideas welcome)
nostr:note1jk2jv67apx7r2a2mem3n8pdxl3t3w5z5sgjz0x3sjhgx7l8zd2ashn355r
Discussion
Daily meme count
Every single product should try to optimize for the users' LTV (life time value).
Easy to say, hard to measure. But the closer you get with the measurement, the less noise there is, and the harder it is to game the metric.
- clicks: lot of noise, loose correlation with LTV
- zaps: less noise, higher correlation with LTV
So the question always is - how to define the LTV for the user of the product in a measurable way. Simple concept, hard execution.
I would probably go with some composite metric for engagement. Daily, weekly, and monthly unique engaged users.
Extra tricky thing here is that each user has different things in their LTV, and mostly they are unable to express it even if you ask them.
So … no answer?
On a similar note, how does one measure the LTV, or the value-to-growth ratio of other protocols?
Seems like applications would have better conceptual approaches to measuring and defining success.
Looking at the protocol level for metrics of success is a lot harder when the protocol doesn't aim to be entirely public.
All metrics are gameable.
With Bitcoin we have never had much of a perfect picture of value either. Value is perspective-based. Individually determined.
Definitely. Value can be much better defined and specified on the client level than on the protocol.
On the protocol - maybe active & returning npubs having repeatedly more than single type of engagement signals...
My point is more conceptual, you can always draw some sort of this schema for any tool or product:
*it's from this product development keynote "The alchemy and science of product metrics" - personal favorite recommendation for PMs; https://www.youtube.com/watch?v=xPT28IGDP_M
Lol, yea.
I mean, for Damus/Primal, I would go with the couple of the composite metrics as a bundle to keep track of for the longer progress. I don't think you can come up with a single north star bullet metric. But there is somewhat clear hierarchy in the quality of the metrics, and their proximity to LTV.
Ex.
- Unique users with a click (comment, zap) are better than total clicks (comments, zaps)
- zap is better than like
For individual iterations, I'd still lean more into UX feedback and product intuition.
The composite metric might be
Distinct(npub HAVING (note + comment + like + zap)>0
You can do >1; >10; >100, and over different time frames, to measure higher, and more repetitive and lasting levels of engagement.
...
At different stages of the product, you want to use diffently advancing metrics. The simple ones are good for the start, but can be skewing you towards the cheap engagement baits. At more advanced stages, you want to counterbalance them with even more quality, and that gets progressively harder to define and measure.
Daily notes(of any kind) count.
[Time period: daily, weekly etc] sats earned per npub (limited to those with LN address) would indicate the size of Nostr creator economy at given time