Subscribe   |   @sriramk   |   About

Building something no one else can measure

04/23/17


books

I wrote a tweet a few days ago that got a bunch of responses. What does that even mean? Is that just one of those tweets that is meant to look smart to tech insiders but doesn’t actually mean much? Well, yes but I decided take it upon myself to expand a ambiguous “strategy” tweet into a longer, more ambiguous blog post with 500% more ambiguous “strategy” in it.

Any large system picks a metric to goal itself on. Entire books and way-too-long Medium posts have been written on the importance of said metric - it influences everything from people’s incentives to how quickly you can optimize your business. In an organizational equivalent of Schrödinger’s cat, picking the metric itself can cause weird cultural distortion (see Goodhart’s Law).

Since it is near impossible to perfectly measure human behavior, most large teams/products pick a proxy metric to measure underlying behavior. For example - ‘clicks’ are a proxy for “did I read this?” and “will I buy this product sometime in the future?”, ‘time spent’ is a proxy for “did I enjoy this content?” and NPS is often a substitute for “do I love this company?”. You convert a nebulous human emotion/behavior to a quantifiable metric you can align execution on and stick on a graph and measure teams on. Engineers and data scientists can’t do anything with “this makes people feel warm and fuzzy”. They can do a lot with “this feature improves metric X by 5% week-over-week”. Figuring out the connection between the two is often the art and science of product management.

This is where opportunities arise for startups and insurgents.

These metrics never really capture the underlying human emotion or behavior they are trying to measure. To make things more interesting, they almost always create secondary behavior which makes the metric go up but in a way the system designers didn’t anticipate or want.

For example, in terms of what designers wanted, what they built/measured and what they unintentionally caused:

If you take this outside tech, you could vaguely apply this framework to broader issues. This and this are great posts which go into this in great detail where I stole some of these from. Some examples.

Companies know this of course, they’re not dummies . They just don’t have better options. At scale, you often have to settle for things you can measure/optimize or monetize. You also have to settle for metric that optimize for the short term - that’s what drives quarterly earnings or the competitive dynamics or product iteration loop at the moment. It’s just the best you can do and is often, good enough.

But you, as an outsider, you can do better. If you can figure out this this blind spot , you can build something that captures a space that the existing incumbent can’t go after easily. This is not easy and almost always needs some instinct or intuition on product/human beings/communities that is not broadly known and by its very nature, high beta.

Here are some crazy, impractical examples in this vein.

While on the surface these may sound just contrarian, what most of these do is to optimize for something long term/less measurable where the incumbents are constrained by time and what they can measure.

You might have noticed something else here - they’re almost all hard to monetize or build sustainable businesses as existing monetization/external systems all align with existing metrics too (a commerce company which tries to cap the amount of money a customer spends every year faces real challenges).

No one said this would be easy. But if you can thread a needle here and make it work, there is real value. In the frustrating words of all my childhood math textbooks, how to do that is left as an exercise to the reader.

Some fun further reading here, here, here, here , here and here.

Updated: Some more specific examples as opposed to theoretical ones.


Subscribe: I write a new post every week or two on tech/startups or whatever catches my fancy. You should sign up!