How To Set Data Science OKRs
Balancing the inherent uncertainty of data science with business reality
Your manager comes to you and asks for your OKRs for this half. Chances are, their manager has just asked for team OKRs, and now the task has been delegated to you. How do you write reasonable goals?
Wait, What Are OKRs?
Objectives and Key Results (OKRs) is a goaling methodology that originates from Andy Grove’s High Output Management. We’ve mentioned Grove’s book before, when discussing manager 1:1s. It has become the predominant management text within tech - most data science teams, knowingly or not, utilize the best practices espoused by Grove decades ago! In particular, many tech companies now use OKRs to create company and team specific goals.
OKRs are comprised of an Objective, a high-level business goal, along with the Key Results needed to accomplish it. Key results should be tangible markers required to realize the objective, and most companies will expect a metric or deadline to be associated with them. An example OKR could be:
Objective: Publish a book on data science
Key Result 1: Finish a 75,000 word draft within 12 months
Key Result 2: Share manuscript with 5+ potential publishers
Key Result 3: Negotiate and sign a book contract with a publisher
You get the idea. We want a big, important objective, along with the tangible key results needed to accomplish it.
Planning For Failure
OKRs ought to be aspirational (to a certain extent). It is not expected that all teams achieve 100% of their OKRs every half or quarter. If this happens, a company is setting OKRs incorrectly, and should look to be more ambitious.
A common success goal is for around 50%-60% of OKRs to succeed. Managers should also realize that this rate may be slightly lower for data science teams, as DS work tends to be closer to an R&D role than other functions within a company.
Setting Personal OKRs
As a data scientist, you should feel confident that you will achieve at least some portion of your OKRs. It may be tempting to write OKRs that you know can all be achieved, but be careful - good data science managers will understand when you aren’t stretching yourself, and may ask you to revise goals to shoot higher.
In general, within a given objective, your author finds it helpful to balance higher and lower risk key results. For instance, if we are looking to launch a new recommendations feature, an example OKR could be:
Objective: Generate X% Revenue Lift Through Improved Recommendations
Key Result 1: Prototype new collaborative filtering methodology by end Q3
Key Result 2: Launch and analyze experiment to Z% of customers by end Q4
Key Result 3: Partner with engineering to productionize feature by mid-Q1
These key results progressively increase in risk, from finishing a methodology (presumably achievable), to launching an experiment (some risk, will likely require collaboration), to productionizing the new feature (high risk, will require a successful experiment as well as significant engineering resources).
Call Out Dependencies + Risks
As a data scientist, you rarely work in isolation. Often, OKRs are shared across teams, or will be conditional on other factors beyond your control. When setting OKRs, make sure to identify these dependencies and bring them to your manager’s attention. It may be necessary to sync with other managers and ensure that a crucial input to your project is on the right team’s roadmap.
Don’t Forget Your “Usual” Work
When setting OKRs, it is enticing to focus on the new and novel - the projects that your team is excited about tackling this quarter. However, much of a team’s work is more routine - for instance, maintaining existing processes, or paying down technical debt. If this work accounts for a significant portion of your team’s time, then it should be present in your OKRs. This “business as usual” work can itself be quantified - for example, a key result might be to “Reach 100% documentation coverage across all data pipelines”.
Review And Reflect
At the end of each quarter or half, your team should be taking the time to look back on previous OKRs and analyze performance. Did you stay on track, or did unexpected distractions derail your goals?
Your author finds that most data science teams are too ambitious when setting OKRs, largely due to a failure to account for non-OKR time sinks - for instance, maintenance work, recruiting time, vacations, standard team attrition, ad-hoc requests, and more. If this is happening, it may be necessary to stack rank potential OKRs, and reduce them down to only the most valuable.