DECISIONS

Smarter. Better. Faster.

October 2, 2018 | San Francisco | presented by Split Software

Software teams around the world struggle making the decisions required to develop, release and scale risk-free software products that deliver value.

Are you spending your valuable development time on rollbacks and emergency fixes rather than on your next sprint’s work?

Are you unsure if your customers really require the features your team is building?

Do you lack visibility into how these features are impacting your company’s key metrics?

…if so, you’re not alone.

Speakers

Make an informed decision

Who is DECISIONS for?

This free one-day event brings together innovative engineers, product managers, and data scientists who are focused on building world-class software and delivering exceptional product experiences.

What will DECISIONS answer?

  • How you can release fast and often by unblocking dependencies that exist between feature releases.
  • Ways to mitigate risk in your release cycle.
  • How to minimize the blast radius of any feature release by gradually rolling out features.
  • Best practices for keeping up with the pace of product delivery without compromising quality.
  • Expert tips for building measurement into your product development cycle.
  • Real world testimonials about reducing cross-team friction when it comes to feature delivery.
  • The critical role data-driven decision making plays in cutting edge software development.

What’s the format of DECISIONS?

The format will feature a mix of thought-provoking keynote sessions, expert panels, and fireside chats with industry leaders, and hands-on practitioners alike. Between sessions, attendees will have the opportunity to exchange ideas with their peers and network with the industry’s most cutting-edge product development teams. And in true industry fashion, we’ll close the day with a great invite-only VIP party.

Why should you attend DECISIONS?

  • Interact with innovative engineers, product managers, and data scientists.
  • Take home practical advice your team can put to work immediately, as well as inspirational insights into the future of product development.
  • Network and exchange ideas with leading Engineers and Product ecosystem partners.

Agenda

Tuesday, October 2, 2018

7:30 AM

Registration & Coffee

Get your badge, swag, and coffee - on us.
9:00 AM

Kickoff Keynote

A warm welcome and an introduction to the key components of decision making in product development.

Trustworthy Online Controlled Experiments at Large Scale

Experimentation allows product teams at Microsoft to evaluate ideas objectively, and make data-driven DECISIONS, accelerating innovation. Learn how, and why Experimentation is vital for your organization. Read more
Every day at Microsoft, about 50 new controlled experiment treatments (e.g., A/B tests) start, across multiple products, using Microsoft’s ExP platform. These experiments have allowed product teams at Microsoft to evaluate ideas objectively, and make data-driven DECISIONS, accelerating innovation. After sharing some surprising real examples, some worth tens of millions to over $100M annually, we focus on valuable lessons learned over the years. Learn how, and why Experimentation is vital for your organization.

Fostering a Culture of Experimentation

Experimentation requires a cultural transformation. Learn how to apply what Ya learned during LinkedIn’s transformation at your organization. Read more
Those who experiment most, innovate best. But running large scale A/B tests is not just a matter of infrastructure and best practices. It requires instilling experimentation at the heart of the decision making process. With that comes the cultural challenges of issues like embracing failure, democratizing results, and more in-depth collaboration. These challenges must be addressed not just with individual teams or contributors, but with leadership as well. In this fireside, we’ll chat with Ya about how experimentation changed the culture at LinkedIn, why it was a necessary transformation, and how to apply what they’ve learned to transforming the culture of decision making in your organization.

Early Morning Sessions Q&A

Adil Aijaz, Ronny Kohavi, and Ya Xu will answer questions from the audience in a live Q&A session.
10:30 AM

Morning Break

Everyone needs a caffeine break mid-morning, us included. Bonus: Get your copy of Shoot Like a Girl signed by MJ Hegar!
11:00 AM

How to Optimize Your Product Using Analytics

One thing that separates great product teams from the rest is how well they use analytics to make decisions. Come learn best practices about how to use metrics to optimize your product. Read more

One thing that separates excellent product teams from others is how well they use analytics to make decisions. Product teams live in a data-driven world, and the bar is rising. Analytics increasingly offer greater transparency in understanding how your customers are using your product, what’s working, and what isn’t. Come to learn best practices in how to use metrics to optimize your product.

Product management expert Dan Olsen will share advice from his book The Lean Product Playbook on a repeatable process for how to improve your product with metrics, illustrated with real-world examples and case studies.

Finding the (Expensive) Needle in a Haystack: Anomaly detection, Root Cause Analysis, and Minimizing the Business Impact of Alerts at LinkedIn

ThirdEye, the internal alert & monitoring tool at LinkedIn, was built to perform anomaly detection and root-cause analysis at scale, in addition to determining the business cost of individual alerts. Read more

At LinkedIn, the real measure of operational excellence is defined by two key metrics - Mean time to Detect (MTTD) an issue, and Mean time to Recover (MTTR), which is time taken to triage and fix the issue. The reason why these two metrics are important is that the longer it takes to detect and recover from an issue, the higher the business impact. Getting an alert isn’t enough, because even if a team is made aware of an issue, the company will continue to lose revenue until the team can get to the cause of the alert and provide a fix. ThirdEye, the internal alert & monitoring tool at LinkedIn, was built to perform anomaly detection and root-cause analysis at scale, in addition to determining the business cost of individual alerts.

This session will cover how ThirdEye:

  • Employs a combination of data-driven algorithms along with business rules to improve MTTD without compromising on the quality of alerts.
  • Performs anomaly detection, metric de-aggregation, and analysis of related events in real-time as the user interacts with it.
  • Cross-references dimensional information of events and metrics to present likely explanations for the issue under investigation.
  • Helped identify swings in metrics at LinkedIn due to traffic spikes, conflicting deployments, ongoing experiments, and even simple holidays.

Be the Change: How You Can be a Catalyst for Large Scale Impact

In this talk, Major MJ Hegar reveals how to be a catalyst for change on a large scale, be it in your career, at home, or on a national or global level. Read more

In this talk, Major MJ Hegar reveals how to be a catalyst for change on a large scale, be it in your career, at home, or on a national or global level. “When someone tells you that something cannot be done,” she says, “the only way to prove them wrong is to do what they believe to be impossible.”

Her heroic actions in Afghanistan—when she was shot down on a Medevac mission and was pivotal in rescuing her crew and patients, and became the second woman in history to earn a Distinguished Flying Cross with Valor Device—changed the perception of women in the military. And, her historic lawsuit against the Secretary of Defense opened the door for women to participate in ground combat, a significant step forward for equal rights for women in the military. How did she do it? And what can we learn from her experiences? A magnetic speaker, she combines harrowing war stories and powerful lessons on social change with practical advice on leadership, diversity, teamwork, and motivation.

Late Morning Sessions Q&A

Dan Olsen, Kishore Gopalakrishna, and MJ Hegar will answer questions from the audience in a live Q&A session.
12:30 PM

Lunch

Refuel, mingle with Sponsors, and maybe even make a friend. Bonus: Get Jez Humble to sign your copy of Accelerate!
1:30 PM

Cloud Native and Ford vs. Tesla

There is a race between the Teslas of the world learning to become a car company compared to the Fords of the world learning to become a software company. Read more

The software industry is currently having a moment. Techniques and tools that have been proven out with technology-forward companies are now being adopted widely across more traditional industries. This adoption is driven, in large part, by these technology-driven companies targeting every industry.

There is a race between the Teslas of the world learning to become a car company compared to the Fords of the world learning to become a software company. This race is driving an enormous wave of innovation as we make techniques that were often reserved for big tech companies available much more widely. In this fireside chat, we’ll dig into Joe’s brain as he unpacks what these techniques are, and how you can apply them in this world where every business in every industry must digitally transform if they want to survive.

Atomic Use Case, Lifetime Value, and the ROI of Your Data Team

What should you expect from your data team? This talk will focus on how to use data to understand the key activation use case and how you can use that to quantify the Lifetime Value of your users. Read more
Many companies building in house data science teams are doing so because everyone else is doing it. However, not everyone understands how to measure the return on the investment they are making. What should you expect from your data team? In this session, I will focus on the kinds of impact you can expect to have from investing in building a data team. Specifically the talk will focus on how to use data to understand the key activation use case (Atomic Use Case) for onboarding of new users. Additionally, once you have understood the Atomic Use Case, how you can use that to quantify the Lifetime Value of your users.

Misadventures in Continuous Delivery

There are plenty of sources extolling the virtues of CD and how to implement it in your favorite tech stack. But what if you’ve already got a large code base that the company depends on? Read more
There are plenty of posts on Hacker News extolling the virtues of CD and how to implement it in your favorite tech stack. But what if you’re not starting from scratch? What if you’ve already got a large code base that the company depends on? What if the way your team worked for the better part of a decade requires a paradigm shift? We all know how open engineers are to change… The best way to grow is to learn from your mistakes, but it’s even better if you can learn from someone else’s blunders. So come with me on a cathartic journey through my team’s CD misadventures. From deployment rollback horror stories to Black Friday engineering nightmares, I’ll detail the good, the bad, and the ugly on our continuous delivery quest.

Afternoon Sessions Q&A

Joe Beda, Manu Sharma, and Dan DeMeyere will answer questions from the audience in a live Q&A session.
2:40 PM

Afternoon Break

Refreshments, and a sweet treat courtesy of Split! Bonus: Dan Olsen will be signing copies of Lean Product Playbook, so get in line early.
3:00 PM

Beyond Outcomes vs. Outputs — Russian Dolls & Broken Promises

In this talk, we’ll explore outcomes and outputs, and expand beyond overly simplistic views, while staying actionable and grounded in the reality of our day to day work. Read more

“Outcomes, not outputs!” is a common rallying call these days. To passionate problem solvers, this message likely resonates. Who wants to work in a Tayloristic feature factory as a glorified ticket taker? Who wants to ship prescriptive deliverables with disregard for the impact of our work (the “outcomes”)? No one!

Unfortunately, the “outcome vs. outputs” narrative and dichotomy is far too simplistic. At its worst, it marginalizes teams, encourages handoffs, and reinforces silos. The reality is rarely that binary. Teams eagerly attempt to realize the promise of outcome-driven approaches but struggle with the details. Tools like OKRs only seem to exacerbate the problem, becoming more of a nuisance and hoop-to-jump-through than a learning/alignment enabler. Even our work visualizations — kanban boards, for example — can obscure how our interventions and “bets” are genuinely related.

In this talk, we’ll explore a more nuanced view of outcomes and outputs through the lens of value creation networks, bets, and promises-in-progress. I will cover:

  • Impulse and response — why we tend to overlook the passing of time.
  • How to connect our day to day work to the “big picture.”
  • A framework to discuss “prescriptiveness.” Finding the sweet spot for your teams
  • Why outputs aren’t inherently evil — but separating them from outcomes is.
  • Assumption mapping, nested bets, and dueling with uncertainty
  • Promises-in-progress: how visualizing the work is not enough.

My goal for this talk is to expand beyond overly simplistic views while staying actionable and grounded in the reality of our day to day work. I’m excited to attend Decisions to learn, collaborate, and contribute!

The Evolving Product Manager Role — And Why Engineering Should Care

The evolution of the data-driven product manager means a new way of working, and a new set of solutions to enable that work. We’ll dig into the changes each of these companies has observed and what’s next. Read more

As engineering teams work to deliver faster, product teams must also work equally as quick to measure the value of what is being delivered. So who owns the responsibility of proving value, and how do they do it? This evolution of the data-driven product manager means a new way of working, and a new set of solutions to enable that work.

Earlier this year, the Product Cloud Alliance was announced, bringing together the most critical of those solutions with the goal of helping product leaders move from being tactical to strategic innovators within their organizations. In this panel, we’ll dig into why the Alliance exists, the changes each of these companies are seeing in the landscape of product development, and what shifts are coming next.

Experimentation 101: How to Get Started with Your Team

You and your team want to experiment, but now what? In this session, find out some of the tactics to take your team from newbies of statistics to experimentation stars. Read more
You and your team want to experiment, but now what? Knowing the correct tactics and processes, you need to get started can feel like a massive mountain to climb. In this session, find out some of the tactics Sophie has adopted to grow Skyscanner’s experimentation and testing activities. These best practices will help you get started with taking your team from newbies of statistics to experimentation stars.

21st Century Software Delivery

Learn how modern approaches developed over the last twenty years enable teams to much more rapidly deliver software that delivers value to users, and how to adopt these practices safely. Read more

For many years, software engineering meant delivering software the way we constructed buildings. Over time, this belief turned out to be a wildly inefficient way to create innovative products and services based on software.

In this talk, I’ll discuss how modern approaches developed over the last twenty years - lean, agile, DevOps, and user experience design - enable teams to much more rapidly deliver software that delivers value to users.

I’ll also talk about how you can adopt these practices in the safety of your own office, and explore the art of the possible and the patterns and methods used by high performing organizations.

Late Afternoon Sessions Q&A

John Cutler, Sophie Harpur, and Jez Humble will answer questions from the audience in a live Q&A session.
5:30 PM

Sponsor Highlights in Expo Hall

Learn how our Sponsor Partners can help you be Smarter, Better and Faster.
6:00 PM

Cocktail Party @ 620 Jones

Celebrate and Network with fellow attendees, and speakers during our evening cocktail reception at 620 Jones! Drinks, appetizers, and good times included.

Venue

Photo of the JW Marriott Union Square

JW Marriott Union Square

Join us at the newly renovated JW Marriott; a luxury hotel in the heart of downtown San Francisco.

515 Mason St, San Francisco, CA 94102

Sponsors

We’ve sold out!

We are overwhelmed by the amount of interest in this event. We may have sold out of tickets,
but you can still click the button below to get email updates from us.
If we end up finding space, we’ll reach out to you directly.