How the engineering leader of Coinbase tracks productivity

Varun Srinivasan, the engineering leader of Coinbase explain how he tracks productivity at Coinbase.

*This interview is part from Stripe Atlas community forum.

How do you track your engineers’ productivity?

Measuring productivity in design fields like engineering is much harder than in operational fields like support.

For a support agent, you can track inbound tickets, time to resolve, and customer satisfaction and quantify productivity at every step of the funnel. For an engineer, the good output is hard to distinguish from the bad output with raw metrics. A really good engineer might take twice as long and ship less code than a bad engineer. But this may 10x the performance of your system or make it very easy for the next engineer to make changes, which is well worth the cost

So the first question I ask people trying to track engineering productivity is ‘Why?’

If you’re trying to increase productivity, approach it like a system debugging the problem. Instrument the system, find the biggest bottleneck, fix it, and move onto the next one. If you’re far away from day-to-day coding, sit down with an engineer and ship some code into production. Once you see everything laid out in steps, you can reason about the problem better.

If you’re trying to identify low performers, then you need a good performance framework. If this is a skillset where you’re not an expert (say, machine learning for example), then try to find and talk to 3 expert IC’s and ask them how they measure performance in their peers. At Coinbase, we measure an engineer’s performance based on five criteria:

• Scope – how big is the work you do?
• Quality – how bug-free and stable is the code you release?
• Execution – can you take big projects, break them down into chunks and deliver on time?
• Communication – how clearly can you communicate specs and requirements?
• Growth – how well do you mentor and support other engineers?

The specifics change at each level, but this gives us a frame of reference to examine the work someone does and compare it relative to others.

Do you measure productivity the same way across the company? Or do individual managers have their own methodologies?

Measuring productivity at the org level (several hundred people) is very different from measuring it at the team level (5-10), which is different from measuring it at the individual level.

As we grew to 50+ people, we created shared frameworks for individual and team performance. We have a leveling rubric for individual performance tracking (see the answer above) and OKR systems for team and organization performance tracking. It’s important to have a shared framework so that you’re not comparing apples to oranges.

Engineering productivity is qualitative so we measure the impact on business value (dollars generated, customers acquired) or the closest proxy that we can track (customer happiness, engagement, uptime). Engineering metrics like commit velocity and story points can be useful, but give little insight into value creation, which is what productivity should strive to measure.

If you want to find out more about how Waydev can help you, schedule a demo.