New Relic put together GetLeanSF on Monday evening, hosted at the fabulous Microsoft office above the Westfield Mall. Hundreds of people gathered to here Eric Ries talk about Lean Startup.

I’ve seen Eric speak a few times and read his book, yet the event still managed to inspire me — reinforcing previous ideas about Lean Startup, without feeling repetitive. Eric gave a short talk, followed by long Q&A with Chris Cook, President and COO of New Relic, and an unconference facilitated by David Nielson.

Eric Ries gave a brief background of how he started evangelizing Lean Startup. As a consultant, he found that people did not believe his advice would work, despite his actual experience from IMVU about continuous deployment and radical practices for early customer feedback. This led him to seek a foundation for his ideas and a framework to help others think about them.

The word “Lean” in Lean Startup comes from the lean manufacturing process which began in Japan with Toyota. He was surprised to learn that many of the practices on which modern software processes were modeled were considered obsolete. The core of lean manufacturing was to figure out:

How do we tell the difference between what activities add value to the customer, and which ones are wasteful?

The audience was engaged throughout the discussion of what Eric calls “the boring stuff,” the process of invention, of figuring out what parts of your new idea have value and tracking what makes a difference with customers.

He spoke of putting out your software early, when you have only features A & B, and you know that the product doesn’t make sense without C. If people complain that your product doesn’t have feature C that is the best, and least likely, scenario. That would be great news! The more likely thing to happen is that people don’t care about your product at all and never sign up to use it. Or maybe they sign up and ask for XYZ instead of C.

He cautioned to be wary of vanity metrics, “the numbers you put in the press release to make your competitors feel bad” — we sent 10M messages on our platform! …is that 10M people who tried it once and never came back? or one very active user? neither one shows traction. What you want to see (and should be tracking internally) are actionable metrics, using boring accounting techniques that let you see how those metrics change over time.

We’re so good at accounting now, that it is easy to forget about what it is for. Accounting was originally developed so that managers could be held accountable. He told of car companies with projections and forecasts for standard volumes, and sales targets that actually meant something because the industry was so stable. In our industry and our modern world, we need innovation accounting.

Eric advocates the use of cohort analysis and advises that you measure as few things as necessary to prove your hypothesis. In the early days of a startup, when you lack a large volume of data — don’t worry about it. Trade statistical measures for rich, in depth feedback
Unlike presidential elections, product market fit has a high signal to noise ratio. You are trying to figure out if anyone cares: is this the right product? are you showing your product to the right people?

A few more notes:

Ried Hoffman: “If you are not embarrassed, you waited too long”
Eric’s Corollary: “No matter how long you wait, you will be embarrassed by it.”

Hypothesize what they customers do when they use your product…”All you need is a Starbucks.”

MVP (minimally viable product) should be defined by what are you trying to learn. What are your risks… What are the biggest risks?

“Science is only as good as your hypothesis.”

“Vision is too precious a commodity to be wasted on faith-based initiatives.”

Overall, it was a delightful evening — many thanks to Eric, Chris, Dave and the folks at Microsoft and New Relic who made it happen.