Design Meets Business

Sean: Product managers, designers,
and even engineers, we speak a foreign

language, and we forget it all the time.

And we use terms that we created
within our functions, and we

expect everyone else to understand
them in the exact same way we do.

We're working in an agile method.

We're going to do a discovery sprint.

We're going to do some contextual inquiry.

Um, We're going to do some validation.

And we use all these terms.

And sometimes the stakeholders
may even nod their head if

they think, okay, I understand.

I think what agile means, but
they don't know it the same way.

Christian: The discipline of design
is now key to building great products.

More and more companies are making
space for it at the higher levels.

More people than ever
want to become designer.

And most of us who already do the
job wants to find ways to have just a

little bit more impact in our teams.

Welcome to design meets business.

I'm Christian Vasile and on this
podcast, I bring you world class

product and design leaders who found
ways to shape products, companies,

and entire industries, and who are now
sharing what they know with you and me.

My hope is that we all get to learn
from the experiences, ideas, and stories

shared on this podcast and in the process.

Become better designers.

My guest today is Sean O'Neill.

Sean worked at Amazon since its early
days when he was only 2000 employees and

was previously CPO of GFK and a product
director at Tesco amongst other companies.

Sean very recently
became CPTO at Synchron.

Which only became official after
our recording of this podcast.

In our conversation, we discuss how
design can better work with product,

how to speak the language of our
senior stakeholders and what he's

learned about hiring from interviewing
over a thousand people at Amazon.

Get your notebooks out because this
episode is going to teach you a lot.

And with that, I bring you Sean O'Neill.

Sean, I'm so glad I get
to talk to you today.

Welcome to Design Meets Business.

I'm excited to have you on because
talking to design leaders is one

thing but talking to product leaders
can bring different perspective.

And I think we can learn different
things by talking to different people.

So thanks again for being on.

I've always seen product managers as
my best allies in my product teams.

So I hope I'll be able to get
into your mind a little bit today.

You've started in product all the way
back in 99, working on on Amazon's

bread and butter, really back then,
if I'm not mistaken, was books.

And since then you've taken on
some great roles and achieved some

fantastic results with your team.

So if that's okay with you,
if you can give us some of the

cliff notes of your career.

Sean: Sure.

And thank you so much for having me.

I've been really fortunate in my career
to work with some amazing people who have

taught me important lessons at companies
that were at really pivotal points.

of their evolution.

So last 25 years, I spent at
this intersection of commerce,

consumer behavior, data driven
feedback loops, and design.

And I've been managing product
and design teams for technical

products all of this time.

And I think I've got some
interesting points that I may

be able to share along the way.

So starting with Amazon in the early
days, When there were only 2000 employees

worldwide for Amazon, all the way up
through working for Tesco, a huge company

there, all the e commerce apps and
websites around the world, I have worked

for some of the smallest companies.

Two other founders in a garage
working on the new thing as a

startup for consumer loyalty rewards.

And most recently with GFK, a big data
and analytics information services company

that we just successfully exited from a
private equity sale to Advent from KKR.

And I'm delighted to chat with
you today on this podcast.

.
Christian: It's great to see that you've
been working with big companies and

small companies, and perhaps we can
start by diving a little bit into how

have you seen products act differently.

in different types of companies,
because I assume just like with

design and anything else, really, in
a startup, you behave differently.

You work differently.

The pace is different.

The appetite for risk is different.

How was that at Amazon in the
early days when he was a startup?

How was that at Tesco, which
is clearly not a startup?

And how was that in a garage
with two other funders?

Sean: You've got to bring the right energy
and priority to the challenge at hand.

So in a startup, there
is no three year plan.

There's Thursday's plan.

If you don't get the next investor funding
in, you're not going to make payroll.

And that really does laser focus you
on what is the most important consumer

validation evidence we can get to help
us raise the next round of funding.

And so there's value in the
startup side to really clarify.

You only have so many people, they only
have so much capacity, you better make

sure you're getting the right value out of
where you want them to spend their time.

And so you're not going to spend
time months getting lost in discovery

on some topic for a feature for the
startup where you got to move so fast.

That's going to be very different
even at the early days at Amazon.

The strength of Amazon has always
been a laser focus on the customer

and working backwards from
solving a need for the customer.

Do not get distracted by what
your competition is doing.

Pay attention to the competition, but
don't blindly copy them because so many

times a competitor may launch a feature
that actually is not working, is not

relevant for your users or for customers.

So be careful not to blindly copy them.

And a lot of the Amazon early
days was how fast can we recruit?

Funny enough, they were growing so fast.

We needed to constantly
recruit more people.

And, I've been fortunate to learn
skills that I've carried through to

this day on how to interview people.

I was trained as a bar raiser at Amazon.

Which is a responsibility to help
other teams fill their roles as well.

And I've interviewed over a thousand
people in my career from all job families.

And so it really helps you figure
out how to make sure you're bringing

the right talent in so that you've
got an A class team surrounding

you in all these functions.

Christian: Over a thousand people.

That's amazing.

If I'm not mistaken, this came
from Amazon who, or maybe it was

Apple, could have been Apple.

Now that I think about it, someone
said A people, or is it B people hire

C people hire D people, A people only
hire A people or something like that.

So that could have been Apple,
but it's also not relevant.

Tell us a little bit
about this bar raiser.

What was it exactly and . How
would you describe it?

Sean: Sure.

So the bar raiser concept is every
new hire should be better than

average of the current company.

So with every hire you are
trying to upgrade the average

capabilities of the whole company

.
So the idea of raising the bar is
every year the bar gets higher to be

qualified to work for the company.

so there's a lot of skills that go through
and Amazon was hiring so many people.

They could approach it as a
science and really use a lot of

rigor and controlled experiments.

For some engineers, they would have
them do online interviews and tasks

without talking to a single person and
make a hiring decision based off it.

And for other people, they
put them through full limps.

They were for others, they would
change what kind of questions they

asked and some mix of practical
and the personal questions.

And then for the people that got hired.

It's a running experiment.

They literally would track how were
those people rated at three months

in, at six months in, at 12 months in.

What was the retention like?

Were there any people that had
performance problems right off the start?

So they really were able to bring a
science and some of the key lessons that

I have used over the years, and I've
written my own training programs for

Tesco and for GFK and others to bring
interview excellence to these teams.

A big thing is beware of asking
theoretical questions only, because lots

of people are great at reading the theory.

They've read the books and they
can articulate, hypothetical,

Oh, this is what I would do.

I would motivate teams and I would
solve user problems and I would test it.

And you're like, Oh, that's great.

Instead look for evidence.

So the way I teach people and train them
for interviewing is I say, imagine You

are a investigative journalist and you
work for the New York Times or Wall Street

Journal and your editor gives you a story.

This candidate claims she's qualified
to work for your company in this role.

Go get the story.

And if you go and you interview
someone and you ask, are you

qualified to be this, lead designer
or head product manager or whatever?

And she says, yes, you can't
go back to your editor to say,

yes, she says she's qualified.

Here's my story.

They project it.

They said, go get the evidence.

So in looking for evidence,
you're trying to find examples.

Where people have actually
used the skills you want.

Give me an example of a time where
you had a difficult stakeholder

and how did you manage that?

Give me an example of a time where
you had a product that was failing.

What'd you do to turn it around?

Give me an example of a time you had
high dissatisfaction with one of your

products or services or features.

What'd you do to solve it?

And you hear how they work
through these problems.

Now, the use with this is they don't even
have to have succeeded in their example.

Maybe they couldn't turn around
a product that was failing.

But what you want to hear is
did they reflect on the problem?

Did they really?

It's backlit in my brain right now.

They look, they didn't actually know
what was going on here, but if they go

off, if they'll claim, Oh, I launched.

Microsoft, Azure, it was all me.

Wow that's pretty big business.

Let's dig into this.

And by digging in with these evidence
questions of what was your role, and

who else was involved, and what kind
of obstacles did you have to overcome,

you're going to pretty quickly figure
out if they have enough detail that they

were really there, or if they really
were just a passenger and had no real

involvement because they can't dig deep.

Looking for evidence.

Christian: This is great.

I remember quite a few interviews
that I've done, quite a few interviews

that I've been in myself where
this question often comes, which is

what was your part in this project?

And we work in product.

There's no such thing as I did everything.

Even if you were the only designer on the
team or the only PM on the team, you might

have asked another designer from another
team for feedback or something like that.

You, you're never the
only one who delivers it.

And I.

There's always been a struggle
to try to get to the bottom of

what did this person actually do.

You're saying looking for evidence,
asking questions, depending on how

they answer you, you can extract
some information from there.

How do you deal with that though, before
an interview, In a portfolio or when you

just apply for a job, cause it's much
easier to do when I have you face to face.

But when I'm just looking at your
portfolio and I know that part

of every single design project
there is stakeholder management.

How do you talk about stakeholder?

How do I talk about the fact that
I've sat in meetings with a lot

of senior stakeholders and I've run
them through pitch decks and whatever

it may be that I've done to convince
them to green light this project.

How do you talk about that?

Sean: There's a couple of different
ways in some, there's some

elements that you just will not
find out without a conversation.

You know, a Lot of these
conversations is also.

Gauging your communication skills.

It's gauging how well do
you simplify something.

Because so many businesses today,
it's all about written communication.

It's all email or it's slack,
and a lot can be lost between the

messages of confusion that leads
to teams going off the rails.

So to some degree, I am also testing for
how strong of a communicator are you.

Are you summarizing the right
headline points or are you diving

into the most granular, deep story?

And as a listener, I'm totally
lost on what mattered most in

this example you're telling me.

So you're testing for
some of those elements.

Some businesses like Amazon, where
they rely on the written word so

much, they will also give people
a small homework assignment.

And I'm trying to remember,
it's been a little.

nine years since I worked at Amazon,
but there was something like, pick

one of these three topics and write
a one pager summarizing them for us.

And one might be if you were going to
pitch a charity that you would like

to start, or another might be I can't
remember what the other ones were.

They might've been like a business
opportunity or something, but

something that you had the
context for and you understood.

And what they're really looking
for is how well do you articulate.

Why something is important, what kind
of evidence you bring, what are you

trying to get as a call to action?

Christian: I like this idea of Amazon
doing interviews as an experiment,

because obviously as part of product
organizations, experimentation is

one of the key tools that we're using
to Reach whatever goals we're trying

to reach and um, it sure sounds like
this was some sort of an A, B, C,

D, E test or something like that.

Sean: Oh many over the
years, many variations.

Christian: That's awesome.

That's great to hear how a good
organization with product at the core

of it implements experimentation not
only as part of the product itself,

but also as part of the company.

They're building the company just
like they're building a product

and that's really cool to see.

That's true.

There's, I read an article that you wrote.

Some time ago about this simple
question that you can ask in an

interview which was a a question
that I'll have to admit personally,

I always hate it and I never asked.

I was very negative towards
it, but I've read your article

and it swayed me a little bit.

Let's talk about this.

Where do you see yourself in
three to five years and why is

that such a key question to ask?

Sean: So this is a question
that I have asked hundreds and

hundreds of times over the years.

First, there is no right or wrong answer.

So as I'm asking somebody,
describe for me your ideal job

three to five years in the future.

And this is a valuable question because
it helps me understand their aspirations.

What is it that they hope to do?

When I ask this question, I always add a
couple factors or parameters for them to

consider because this is their response.

And I'll ask them, does the universe have
a title for this job you want to be doing

three years to five years in the future?

What kind of functional role is it, or
is it some brand new hybrid that you are

passionate about that doesn't yet exist?

Are there any industry sectors you're
deeply passionate about or not?

Are there societal trends?

It's all about circular
economy, perhaps, or not.

Are there any technology trends?

It's all about generative AI or not.

And what's helpful from this.

is it allows me to get a better
sense of the journey someone's on.

It allows me to have a sense
of their direction of travel.

How well does the role we're
talking about right now actually

match to that is useful.

buT it also helps me understand their
potential and where they might want to go.

Some people might want to stay an
individual contributor their whole

career, which is perfectly fine at a lot
of tech and design and product roles.

Others may want to lead larger teams.

That's useful as I think about
an organization over time and

how org designs are organic.

They should be evolving
and changing over time.

Helps me think about how this
person as they grow, develop in

her or his career may evolve.

Christian: So to play back, it's not
necessarily that you're asking because

you want to know where they'll be in
five years, because that's probably

not necessarily accurate anyway.

It's more to hear the sort of journey
that they see themselves being on.

Sean: Yes.

Especially for a lot of technical roles.

It's sometimes it's hard to find
people who want to be managers versus

want to be individual contributors.

So that helps me get a sense,
do they ultimately want to be?

Maybe they want to run their own
small design firm someday and they

want to be a general manager of it.

Okay.

They may need some exposure to profit
and loss and financial statements and

may need to manage more than one or
two people if they're going to have

that kind of larger responsibility.

So it helps me keep context.

Christian: And as someone who's being
asked this question, what's your advice?

Like, how would you,
how do you answer this?

Sean: Always answer honestly.

This is your And this is
your future and your focus.

Especially in larger firms, Amazon
many times, there are a hundred other

jobs that were open at the same time.

And so sometimes somebody was the right
candidate interviewing for the wrong role,

but fortunately the right role was open.

They just didn't know about
it or hadn't found that yet.

So

Christian: would you say that you gave
an example earlier about someone who

might want to open up their design
studio a few years down the line.

If I now apply for this job
and I get asked this question.

Considering that my goal is not
necessarily aligned with the

goal of the company that I'm
interviewing for, should I say that?

Or should I say, you know, well, perhaps
in the future, I'd like to lead a team

or should I align my goal with the goal
the company has at that point in time?

Or should I just straightforward
say, do you know what?

Five years down the line, I see
myself running my own design studio.

Sean: Five years is a long time.

That's why it's a useful question.

And for most people, it is one or two job
roles from today, And it also depends on,

frankly, the experience and maturity of
the person doing the interview, because if

they think, oh, I only want people who are
going to be here five years from now the

reality is your natural churn, probably
10 percent anyways, across the team And

so that's an unrealistic expectation.

Christian: Hey, that's good advice.

Read the room.

Learn to read the room.

Yes.

And the only way you can learn to read
the room is to do a lot of interviews.

Let's uh, change gears a little bit
and talk about the major subject

that I brought you here for.

And I thought I was so excited to
talk to you about this, which is the

relationship between design and product.

And this is like opening a can of worms.

There are a million questions I could ask
you here, but let's start from a couple

of examples, maybe one example that you've
seen in your career a good example of a

relationship between design and product.

Can you Can Talk us through
how that looks like.

Sean: So going back when I was still an
individual contributor, product manager.

So early on at Amazon, I had a number
of different roles across Amazon.

And at one point I was working on
all of the feedback and reputation

systems for the seller platform.

So the global product of.

ratings for the sellers, and we needed to
overhaul it because historically Amazon

had copied eBay's system of feedback
rating because the product ratings of

five stars already existed for Amazon.

Fortunately, Amazon kept the star
rating for rate your seller and your

shopping experience with the seller
and fulfillment, but the rest of

it was based on eBay system where
it was a lifetime feedback rating.

And buyer rates seller and seller
rates buyer because on eBay you

had to pay for your transaction.

So there are all these quirks to it that
made sense when Amazon first copied it.

Five years later, after Marketplace
had evolved into a very different

fixed price business for Amazon, the
same feedback system was still in

place and it wasn't fit for purpose.

So I had to redesign it and had a
great designer that worked with me

that I learned a lot from, but we
had to think about how do we help

people make better product decisions?

How do we help a purchase decision?

So how do we help a buyer?

Gravitate towards sellers.

At this time, it was a lot of booksellers
and music and DVD electronics.

And as the price of the electronics
items, we were moving into higher

categories the risk of a poor
choice is higher for a shopper.

So how do we help use the
feedback to steer people?

Because some of the sellers who
has started five years earlier with

Amazon and we're still selling.

Some of them were fantastic, but some
of them, the last year or two had been

horrible feedback for the purchases.

so if they had a lot of negative
feedback over the last year, but

four years of positive feedback, they
still could be a four star seller.

And so as a buyer, You had no idea
this four star seller was about to

give you a really bad experience.

So we had to change.

How do we show recency?

We also needed to eliminate the
whole buyer rate seller because some

sellers were badgering buyers and to
give them good feedback, I'll leave

negative feedback on you if you don't.

The reality is the feedback
on the buyer meant nothing.

It changed nothing.

But, there was this weird threat
that sellers would make the buyer.

So we had to really fix the experience.

And we also needed sellers to want to
improve their more recent experience.

And if they have defects to fix them.

So with a great designer working with
me on this, we were able to do lots of

research on buyers, research on sellers.

And ultimately came up with an
improved way of showing what was

a seller's performance over the
last 90 days of the last 180 days.

of the last year and over lifetime,
and we changed the default

feedback to be your last 365 days.

So that's a better representation,
including a peak Christmas period of what

kind of experience someone's going to get.

And so this really helped a long way.

The other challenge we had was most
feedback rate, or if you think about

a five star rating for movies, people
assume there's a normal distribution.

And that a four star movie
is still a pretty good movie

compared to a five star movie.

On feedback at the time, it was
actually a U distribution where 80

percent of people would leave fives.

And people who are unhappy, 20
percent would leave ones, and that's

a four star seller, which meant 20
percent of people were disappointed.

So most people don't expect that.

So they think a four star seller or 3.

8, they're great still.

Actually they aren't.

And so we needed to change that a bit.

And with the help of the designer,
we didn't want to change the

metaphor of the five stars that
had a lot of currency with people.

They understood how to rate
one to five, but we needed to

change how they interpreted it.

And so with our designer, we ended up
creating the one to five stars into

a four or five or a positive rating.

A three is a neutral and or one
or two is a negative rating.

And so then we could say, what
percentage positive did this seller have?

And that became one of the primary
elements that our designer put on

an offer listing page where you
would see all of these listings.

That number is more prominent.

And so this allowed a lot of
sellers to suddenly say, Oh, I've

been slacking on my performance.

I better clean this up or I'm
not going to get more sales.

And so it drove the right behaviors.

The good design drove the right behaviors.

Buyers wanting to prefer to go from
higher quality sellers, or if there's a

seller selling that product 10 cheaper,
and they are a three and a half star

seller, but you read their feedback.

Late shipping.

It arrived, but it was late.

You might actually be okay with that.

And then as a buyer , you're fully
informed to say, I expect it to show up.

I'm willing for it to take longer and
I'll take 10 less off the price to do it.

That could still be a good shopping
experience because you were informed.

Christian: And on a daily basis, how did
designing product for this project work?

Did you do the research together?

Did you, where did the goals come from?

Was there any negotiation there between,
what's possible and what's feasible

or what do we do in V1 versus V2?

General conversations that
we have nowadays quite a lot.

How did that look on a daily basis?

Sean: On a daily basis we
did the research together.

A very talented designer who
most importantly knew how

to design for testability.

And this is a trait that I sometimes see
lacking from some designers these days,

where they're given a brief and they
come up with their design, one design.

This is the answer.

As opposed to Amazon early on,
designers knew that it's a hypothesis.

We think we have one
approach to solve this.

Let's come up with two
or three alternates.

And let's have some alternate
design views that we can look at.

And in some cases, there was times we
were testing the buy box, for instance.

And we were testing different call to
actions , different primary colors of

the button, different background colors
of the background where the button,

because that buy box is the most high
value real estate on the entire page.

That is the primary call to action is when
it's the right product, push that button.

And so those kinds of tests, our designers
would automatically have two or three

different variables that you could do a
multivariate test on to come up with nine.

Christian: I like that idea of
coming up with more than one design

in your design phase and then
testing it against each other.

I think that there's a tendency today to,
even if you do take feedback from a lot

of stakeholders, then you go and work and
you come back with one because , I guess

that feels like we've solved the problem.

This is the way, this is our
way of solving the problem.

But I do like that idea of coming
up with a few alternates and then

testing them against each other.

And when it comes to testing, I think
this is really key, it's probably very

important to figure out before you test,
what are you testing for and how are

you going to compare these at the end?

Are they going to be compared on
what participants feel about them?

Is it conversion?

Is it, what is it going to be?

How do you come up with?

Those goals to make sure that however
you compare them, it's unfair and

accurate representation of how perhaps
that's going to behave in production.

Sean: So some of the tests are more
obvious what the goals would be.

So I worked on search quite a lot.

And so there you're going to have,
what is the search query conversion?

What's the basket per query?

What's the units per
session, revenue per session.

So there's a couple of obvious.

Goals that have very clear
commercial impact on it and

also customer friction goals.

So for instance, in search for people who
might be familiar, sometimes you have a

relevance ranking problem where the right
product is returned in search results,

but it's on page three, not page one.

And not everyone does paginate, but
those users who paginate to page three,

how many people are adding the item from
page three versus what's on page one.

So looking at signals like that, you
can test improved algorithms that

may actually improve the relevance so
that you will have a goal of more of

the products added are found on page
one versus page two or page three.

So there are times where you also can
test for user friction and usability.

In other cases, it
might not be so obvious.

What is the right this definitive outcome,
like on the reputation one, you could

choose to buy from someone with a slightly
worse reputation for a cheaper price.

It's not obvious to us what that behavior
should be, but what we were looking at

is how many people looked at a bunch
of sellers and then bought from zero.

So we're definitely trying to
look at conversion of people

getting to an offer listing page.

Cause that's the moment of truth where
you, you know, if you go to back in

the day when we sold lots of physical
books, you're looking at a whole list

of 25 people are selling Harry Potter.

Did you buy from any of them?

Which one did you buy from?

Christian: And I always encourage
designers whenever they do test different

alternatives against each other to
use their cross functional partners,

mostly product managers to figure out
what these metrics should be, because

as designers , we're not always.

aware of some of these metrics.

We're not always in tune with the
business as much as we should be.

And I always say just go and
talk to your PM partners.

They should know that's their job.

Would you say that's accurate?

Would you say that's the right advice?

Sean: It is absolutely the
product manager's job to know what

outcomes are we trying to affect?

Why is this item even prioritized?

It is the product manager's job.

Christian: Since we've gone there,
for designers who've not worked

with PMs before, And they now enter
perhaps a bigger company and then

suddenly they find themselves part of
a squad where there's PMs and analysts

and QA and whatever it may be, and
they have no idea what a PM does.

What does a PM do?

Sean: So a product manager, so I should
also caveat different companies use

different titles for the exact same job.

Yeah.

Product owner, product
manager, other companies.

Have totally different
jobs with the same title.

So it can be a little
bit confusing at times.

I'm coming from the Amazon approach and
I, we implemented this at Tesco and at

GfK, where there is no split between
product manager and product owner.

It's all one job family.

And the job of the product manager
is to be the bridge between

customer and business needs with
technological possibilities.

And bringing those two things together
and fundamentally the product owner

or product manager's job is to deliver
meaningful value for customers and

for the business through the design
process and the engineering process

and the data science process.

And how do we coordinate all of those
teams to be able to work together to

something that matters to the business.

Christian: I always define
it as a three legged stool,

technology, product and design.

It's you can launch a product
with just two of them.

But it's probably not going to be as
good because if you have just design and

technology, you might not always be aware
of where the best opportunities are.

You might not always be aware.

You might not always have
the pulse of the customer.

If he's just the technology and
product, then if design is not

there, it might not look good.

You might not.

Implement the actual correct or
the right or the best solution

for that specific problem.

So for me, the three, and I always
use this word partners, technology,

product and design are partners.

And you've described starting with
this situation at Amazon or this

project at Amazon, what a good PM.

and design relationship looks like.

and I hope that people who are listening
find themselves in one of these.

But the reality is that not everyone
does and the relationship between

PMs and design is not always so rosy.

What are some of the usual complaints
or challenges that product faces

when they deal with design?

Sean: The difficult part of the
role for both product managers and

designers is that everyone else in the
business thinks they can do those jobs.

That's sort of a funny situation,
but everyone thinks they

could design an interface.

I'll tell you exactly.

Wait.

Christian: Give me one second.

Sorry.

Give me one second.

Are you saying that not
everyone can design something?

Is that what you're saying?

Sean: Well, anyone, Anyone
could put pixels on a page.

It might be a train wreck, right?

And everyone in the business has an
idea of what the product should be too.

sometimes strongly held.

But for the engineering or data science
leg of the stool, most people in the

business may at least recognize, Hey,
I can't write code or I can't write

an algorithm and they'll show some
deference and respect for those jobs,

but they'll have no problem telling the
product manager, the designer exactly

what they think it should look like.

And that's a pain point in these roles.

Sounds very familiar.

Yeah.

Yes, it probably does sound familiar.

anD that also gets to what you and I,
when we were chatting before about having

this podcast conversation is what does it
take for design and for product to gain

more respect from the executive suite?

You know, the real challenge here
is product managers, designers, and

even engineers, we speak a foreign
language and we forget it all the time.

And we use terms that we created
within our functions and we expect

everyone else to understand them
in the exact same way we do.

We're working in an agile method.

We're going to do a discovery sprint.

We're going to do some contextual inquiry.

Um, We are going to do some validation
and we use all these terms and sometimes

the stakeholders may even nod their
head if they think, okay, I understand.

I think what agile means, but
they don't know it the same way.

And it creates these gaps
between expectation and

delivery, especially on time.

It can also create gaps where we
say, oh, we're going to be launching.

Some new feature that will
allow our clients to do planning

or pricing or promotions.

And we think we're delivering the 0.

1 version that's barely functional,
but is a proof of concept.

But on the stakeholder side, they
think it's all singing, all dancing.

And they're surprised and shocked.

when they see a version 0.

1 actually launch.

this is a big way that designers and
product managers can start building

more credibility is to understand
the language of their stakeholders.

And most stakeholders are not
talking about user friction.

They're not talking about of the site.

They're not talking about sometimes
how many sessions they have.

But they are thinking in
terms of active users.

They are thinking in terms of revenue.

They are thinking in terms of
cash flow, because that's what

they have to talk to the board
about and to the investors about.

So what's fundamental to build
credibility with these stakeholders

is to understand and talk about how
your work connects to outcomes the

rest of the business cares about.

And that will go a huge way
to building credibility.

Christian: we should spend more time
here because this is super valuable.

How do you then, as an individual
contributor, someone who perhaps

doesn't even have that much access to
the senior stakeholders, you're just

on the ground, boots on the ground,
doing the work, whatever, building

components in Figma or fixing bugs,
visual bugs, whatever it may be, right?

That, that work on a daily basis,
how do you then take that and

link it to these lofty words?

What's our revenue and what's our
MRR and just these things that our

senior stakeholders care about.

Sean: The first step is, do you know
what senior stakeholders care about?

Because in the all hands, they're
almost certainly talking about the top

objectives for the business and you
should take note of what are the most

important goals that the company is
tracking and how are we doing on it?

With GFK, we had revenue targets, we had
EBITDA cash flow targets but for some

of our key flagship products, we also
had an active user base target that we

wanted to grow and we had a specific
target around what percentage of active

users were senior decision makers.

We're a business to business company,
and we wanted to make sure that we also

were generating value as evidenced by
repeated use from more senior decision

makers, a chief marketing officer, a
managing director, a general manager of a

business unit, a vice president, examples.

So by tracking all these elements,
it helped our design team when

they were talking about product and
design, what should we prioritize?

Which of these goals is
this going to even move?

Our hypothesis on it.

you know, because you knew
you had this challenge of any

business to business application.

The more complicated it becomes, the
less likely a more senior decision maker

is going to use the tool on a regular
basis, because it becomes harder and

harder for them to dip in as a casual
user and get the answers they need.

The more, if it looks like a
Boeing, Airbus flight deck,

then they're not going to try.

They're not going to wade into it.

They'll just delegate.

Hey, tell me what that means.

And so that was an important
element for us to justify

keeping simplicity into design.

So one of our design principles
was enterprise grade insights

with consumer grade usability.

And by repeating that all the time
with our commercial teams and our

design teams and product and tech.

It allowed them to say we're doing
this piece of work to simplify or

refactor our navigation in order
to make sure some of these new

valuable capabilities have better
discoverability for the user in the tool.

It gave us the focus of why we're
doing the bit of work because it

laddered up to a goal we cared about.

Christian: I think that's a great example
of rehauling visually the navigation.

Okay.

I've worked on plenty of products where I
thought this looks crap, it doesn't work.

Let's rehaul it.

Which is quite a big piece of
work sometimes, not only for

design, but also engineering.

And unless you are able to link that
to something that someone higher

up cares about, it's going to be
a very hard sell for you to make.

So how do you push for some ideas
that perhaps, are going to make

a difference, but you don't have
any evidence that they will.

an example is this navigation, right?

I can look at navigation.

I can say, I've seen a lot of people in
usability testing being confused by it.

And one of our features is not
being very visible because of this.

And I know that if we change
the navigation, we'll be able to

highlight another feature much
more, but I have no proof of that.

How do I go about pushing for that
feature or that, that redesign?

Sean: So let me answer this in two ways.

The first one is when
you have no proof for it.

And the second answer is how
do you find proof for it?

So it's difficult when you have no
proof, but this is about also thinking

of what do stakeholders care about?

They understand that time is money.

how much effort you go into it is
a function of how confident are we

that this is worthy of working on.

So at GFK, we start a lot of
times with paper prototypes.

We're not doing high res, high
fidelity, mockups of elements

when we don't even know if it's a
real problem customers care about.

We'll do paper prototypes and interviews.

Hey, what about this?

Do you think this would improve it?

What if we did it this way?

And then we might go, because we're a
data and insights business, then we might

go to a functional data prototype and we
don't have a fancy, nice mockup of the

interface, we use Google Studio, and we'll
just load a data file in of real data,

and we'll just do a very basic outline.

It looks like Excel in Google Data
Studio, but it allows a real user to go

through a real use case with genuine data.

And would this help you make an
answer of what product you wanted

to put on promotion and by how
much to have the greatest value

or creativeness on your promotion?

So that's still making it
pretty cheap to be wrong.

One of my design principles
is make it cheap to be wrong.

And how do we make sure that we are
putting our appropriate amount of resource

investment into solving a problem based on
how confident we are we know the answer.

And over time as you get
more and more confident.

Then you get to, let's start developing
real code for this, or let's have a data

scientist build a real model, or let's
start building higher fidelity prototypes

of what it really might look like.

But it all is time is money, and you're
really trying to balance those things out.

Now let me answer the second half
of this, how can you get the

data to help you understand if the
features are being found or not?

Because this is a real important
problem for any B2B product,

especially even consumer subscription
products, where over time.

More and more features are
being added to the product.

And you really only have a very lagged
indicator of subscription renewals.

How many new subscriptions come in?

How many cancellations do you have?

And what's the renewal
rate on top of those?

And there might be 20, 30, 40 features or
more that are in this subscription bundle.

So how do you know if your
feature made a difference?

This is a real problem
we were trying to solve.

uh, JFK, we're using amplitude.

And they have a bunch of features that
have made it very easy for us to track

what's the awareness, what's the trial,
and what's the repeat of your feature.

And so for all new features rolling out
that were of a major release, I'd require

the product and design team to tell me,
give me your estimate, what percentage

of active subscribers eligible for this
product, or this feature, because they

already have the main subscription, what
percentage of active users will try your

product in the first four weeks and of
those users, what percentage of them

will repeat within four weeks after that?

And if you have 30 percent of
users try it and 2 percent of users

repeat we've got a value problem.

We've got a real issue because
they're not getting enough value.

And this is actually a super important
measurement tool for the commercial teams

because before at times the commercial
teams sometimes are beating the feature

drama, keep going to the next feature.

Our goal is not to launch features.

Our goal is to launch features,
clients value so much they will pay

for them and they will renew them.

And if you've got this example where
40, 30 percent tried it, 2 percent

renew or repeated, even commercial
is going to agree, Oh my God,

that's not generating enough value.

We're not done.

We need to iterate more.

And it helps justify time in the
sprint planning of, Hey, we're not

going to get to that next feature
that was queued up because we still

have more investment we need to make
in this one to get the value up.

Now, on the other side, if let's say
you have 2 percent of active users

try it, but 80 percent of them repeat.

That's an easier problem to solve.

That's an awareness problem.

So maybe you do have a discoverability
challenge in the navigation.

Maybe you do have some marketing needs,
or maybe you need some other way to

call attention and awareness to it.

In which case back to our
navigation overhaul, why are we

going to refactor navigation?

We've got some high value features that
are not getting enough discoverability and

they're not getting repeated usage enough.

And we're going to solve it this way.

Christian: I really like this
idea of these two targets

that you were talking about.

One of them shows you everything
around awareness and the other one

shows you anything around usefulness.

Are people aware of it and are they
using it often enough or repeatedly

enough that it proves that it's valuable?

I assume that these targets that you gave,
30%, whatever it may be are hypothetical.

But the question that I have is, do
you set any goals or any targets for

these features ahead of launching them?

Or is it more of a, let's just
launch it and see how it does.

Or do you say ahead of time, look,
if this doesn't have a 50 percent

repeat every four weeks, then it's not
worth investing more money into, or.

How do you deal with that part of putting
metrics on new features or new updates?

Sean: So that's, it's
an important question.

So we needed the estimates
before it launched.

And when we first started doing
this, I let the product managers

give me their estimates and I didn't
care if they were wildly wrong.

I wanted us to start with an assumption
and then let's learn and calibrate

of how it played out and over time we
would begin to see, depending on the

significance of the feature, depending
on how high it is in the navigation,

depending on how many users it might be
eligible for, you could start to see,

well I think 40 or 50 percent of users
should try it in the first four weeks.

And you need to get her a repeat
rate of at least 30 or 40 percent

in order to be demonstrating value.

Smaller features, I wouldn't expect
the same awareness, but I would

expect some repeat from that.

the second part of your question is
actually really interesting here,

because it gets to my overall point
on, are you delivering business value?

And too many teams, this
is not a design problem.

It's really more a product
management problem.

Are we over investing on the value
of what feature or proposition

we're trying to create here?

And if you go to two extremes, let's
say you've got some big subscription

proposition, and you've got a small
feature you want to add, to it.

And if that took an entire single
week to do across developers,

yeah, we should knock that out.

If that was going to take two full years
of three full squads to deliver some small

feature, You'd say, okay, that's wasteful.

That's not worth it.

The problem is we don't have
any real judgment in between.

But this is what your chief
financial officer cares about.

Your CEO, are you spending our precious
development design and product capacity

to work on the most important things?

And this is where speaking the language
of your stakeholders comes a long way.

If you can start to understand
where's the right balance, given

the potential impact or hypothesis
of how much value this might add.

If this is a major new proposition and
it's going to solve the biggest pain

point for your most important customer
segment, and that's going to change

your ability to sell into the whole
sector, then it might be worth six months

or 12 months of whole squad work on.

But if it's not, don't just blindly
ply on another quarter and another

quarter and another quarter.

We need to timebox.

some point, we need to say based
on how big the value is, this

is going to be good enough.

And I think that sometimes is a
challenge I've seen in some design

teams because they can be so passionate
about perfect design and the perfect

user experience that sometimes they
lack the judgment for what is the

development investment to get there.

And is that really the
most important thing?

Or are we better off saying?

This is good enough for us as a whole
squad to work on it for three weeks

or four weeks, and then get to the
next high value item in the backlog.

Christian: Something that I've learned is
that oftentimes you can use your engineers

help push back a little bit, because
you might design this perfect feature.

You might think it looks great.

The experience is amazing.

And then.

If you have an honest conversation
and a good relationship to your

engineers, they will have the courage
to say, we can make it like this.

If everything here is non negotiable,
fine, but we could spare, I

don't know, four days of work.

If you just let me change this copy,
instead of it being automatic to

something that's generic, and then
you can make that trade off of how

much does it really matter that is.

Personalized to you that specific piece of
copy versus that is something generalized.

And can we make that trade
off of, you know what, make it

generalized, spare four days of work.

But I think that sometimes as designers
and perhaps even as product people, we

don't really understand that trade off
before we talk to technology, before

they tell us what the trade off could be.

So that's why I said
the three legged stool.

And I find that relationship between
the three parts of a product team to be

very important because you can actually.

optimize and adjust each other's
expectations using The other

function skills in a way, right?

Because I, as a designer, don't know how
much it takes to code this necessarily.

Sean: Exactly.

And in fact, that exact example came
up in the early days when we were

building our GFK Neuron new platform.

We had lots of teams running in
parallel and the challenge was

some teams were designing the
interface and the engagement.

in a way that was not consistent
with other pages that a

user would be traversing.

And to make it worse, a lot of
this is data visualizations.

And early on, we were using
high charts as one of our

libraries for the visualization.

And this was also a product
manager's problem on my team.

Product manager and the designer came
up with a design that was beautiful

for the exact use case they wanted.

It was going to take six weeks of
development time versus it would have

taken a week had they accepted the high
chart default library out of the box.

So it's not just save four days.

It was say five weeks, right?

And they were insistent that they
wanted it their way, but they also

didn't have enough perspective of
what are all the other pages, the same

user is going to be interacting with.

And this is where.

This was early on for us to have
some maturity around a digital

design system that was more
consistent in the interaction modules

.

Christian: I think that negotiation
process comes in here as well.

Again, if you have a good relationship
with your cross functional partners,

because yeah, all details matter
and in an ideal world, we would

want the experience to be perfect.

But the fact of the matter is that.

There's only so much we have time for.

There's only so much investment
in terms of engineering hours.

We have this sprint or this
quarter, whatever it may be.

There's also space there.

And I always encourage people to
negotiate and say, do you know what,

if you can spare four days of work
doing this, just do it like this in V1.

Could we then next sprint if or perhaps
a couple of months from now when we've

launched this feature and we see that
there is some potential in it, could

we then keep these other things that
we haven't had time to invest in now?

Because perhaps the certainty,
the confidence for this feature

wasn't there necessarily.

Now that we have more confidence,
can we go back to some of those

details that we've left aside?

And then, so it's just because
you're negotiating the for an MVP or

for a version one, it doesn't mean
it's out of the window and you're

never going to be able to go back.

It is truly just a negotiation
process of what can we do now?

And what can we do later when we have
a bit more confidence that this feature

or whatever it is makes a difference.

Sean: I agree.

Christian: So you said something
earlier that I took a note here,

and it's certainly something that
I think it's worth repeating and

talking a little bit more about.

You said, make it cheap to be wrong.

I think that mentality and that approach
would save so much headaches for people.

let's.

Dive a little bit deeper into
make it cheap to be wrong.

What does that mean and
why is that important?

Sean: Oh, it is uh, so I've got on
my LinkedIn profile, which I think

we'll link to in the show notes here.

I've got 10 development principles
that I've come to value over the years.

And they're designed to help teams
make better decisions in place,

not wait for some escalation.

And make it cheap to
be wrong is important.

It is not fail fast because.

We are open to failure,
but we don't want to fail.

Especially if you're in a business
to business environment, your

clients do not want you to fail.

I assure you make it cheap to be wrong
means most decisions are reversible.

And it's not a one way door, it's a
two way door in a lot of these things.

And how do you test things
in a lightweight way?

And there's a couple product
development and design tools I've

used over the years that help.

One is working backwards.

The Amazon method, writing a press release
is a really cheap way to make it wrong.

Because all you do is write a Word doc.

It takes you, it's one page,
take you 20 minutes to write up.

What's this concept, this proposition?

Why is it so valuable that you're
solving such an important pain point?

for some of your key customers, And then
as you iterate on these Word docs, you

get lots of internal and external feedback
from these future press releases that

make the idea better and stronger because
people will come up with rebuttals.

And so you keep a frequently asked
questions list as part of this

Word doc, and you just keep adding,
even if you don't know the answer.

Someone asked what happens
uh, some edge case.

Don't know.

We'll figure it out.

And then over time, you'll come
back and you'll answer those facts.

And you've got a more robust
concept of this is a real problem.

If we solve this problem for our clients,
they will love us because we are 10 X

better than anything else in the market.

Then there's a second tool I use
a lot for, cheap to be wrong.

How do you communicate to other
stakeholders this future world

you're trying to talk about?

So the press release is good, it's
cheap because it is just a Word doc.

But sometimes you want to
give people more of a feel.

And so I use Storyboard.

So this is where I'm working
with design, graphic artists.

I want to tell a day in
the life of my customer.

And so I will have a four panel or six
panel day in the life of a customer in

the course of their life, some key pain
they've got, and how our proposition's

the hero in that moment of need for them
that solves all their problems in such an

effortless way that they love us for it.

And I've used these storyboards, it's
just like a movie panel, and I've used

them with Jeff Bezos to talk about Amazon
mobile payments and how Amazon can help

people through their daily life use their
Amazon wallet in the physical world.

I've used these with boards of directors
and other non technical stakeholders

.
Because you're not getting into any
of the feasibility or technical spec.

It's not how we're going to solve this.

It's for who and why, and that can
go a long way of getting the buy

in of how you are going to take the
friction out of somebody's world.

in such an important way that
they will pay you for it.

Then, two other tools I'll
just mention real quickly.

As you're making it cheap to be wrong,
you're trying to get as much validation

of your riskiest hypotheses before you get
to the most expensive part of building a

product, which is building the full code
base, that development, that deployment,

and all those things that are much
more expensive to fix after the fact.

You're trying to go through
this validation cycle.

And that's where paper prototypes can
go a long way with users trying to

figure out through our interviews.

Data prototypes also can be a
very cheap way to be wrong because

you're not writing real code.

You are not building high fidelity
interfaces and spending a lot of time

making it glossy and pixel perfect.

Instead, you're trying to figure
out, has this solved a pain point?

So those are a couple of different
options to make it cheap to be wrong.

Christian: That's awesome.

Thank you.

And we will definitely link to your.

10 principles and I'm sure there
are many more learnings there.

We're nearing the end of the episode,
but I do have a couple of more questions.

Since you are a product executive and you
have worked in a lot of companies, I'm

sure you have also worked in companies
where design is not a function of its own.

But design is under product.

And I know there are a lot of
opinionated people out there

who will say, no, this is a sin.

It shouldn't happen.

There are also a lot of people who
think, no, this makes a lot of sense.

I uh, would like to spend a couple
of minutes talking about design

reporting into product and why
sometimes that's a good thing.

And why sometimes perhaps design
should be its own division or

its own part of the company.

Sean: Another very important question.

And I know.

For tons of designers that I've
known and worked with over the years,

there's always this aspiration of
how do I get to the executive level?

And organizational design is a really,
it's a secret superpower to understand how

to create organizations, to bring people
together, to allow them to accomplish

important things for the business.

And everyone wants a
seat at the top table.

But you then have to ask, how many
seats are there at the top table?

And at what point does
there become so many seats?

That you've split your responsibility so
far, so you could have a chief marketing

officer, a chief design officer, a
chief customer officer, a chief product

officer, a chief product marketing
officer, a chief sales officer, a chief

revenue officer, a chief information
officer, a chief digital officer.

Who owns what with the proposition
and the customer experience,

and if a big initiative fails,
who's responsible for it?

So at some point there is a challenge
from an organization and an executive

perspective of how effective are
you as a decision-making body

solve these problems for your
customers and for your shareholder?

Most of my career I have managed
design as part of my organization

and I love design, but.

In different organizations, there
have been vice presidents of

design and others they've been
at a head of or director level.

And an organization is an answer
to a business need, just like a

product is there to serve a need.

Organizations are products as well.

In fact, as a sidebar, most people
think an organization is fixed

and they just got to make the best
use of the can instead of asking.

Like an archaeologist, what
was the business question

that led to this organization?

And oftentimes it'll tell you
some teams weren't talking.

So then they made one report into the
other and now all of a sudden they're

all coordinated in line because that does
go a long way to bring alignment up the

executive champion path for initiatives.

And product and design are
super close initiatives.

They're super close functions in terms
of how do we bring distinct skill

sets, but a similar goal to make a
brilliant proposition and solve needs

for users through some combination
of service and interfaces and

information to make a winning business.

I'll stop there to see if you
have other questions on that.

Christian: I think this is good
and I wanted to touch upon this

because I think this conversation
can oftentimes get heated.

And the reason that happens is because
people don't necessarily have an

understanding of why organizations
are the way they are sometimes.

It's not a matter of design not
mattering or design is not important or

product is more important than design.

So I thought your take on this, which
we've talked briefly about just before

we started the recording was interesting.

I haven't heard that before, so I thought
it was important to put it out there.

But Sean, let's bring this one home.

I have two more questions
we ask uh, at your eye.

I only say we, but I, it's just me.

So I asked the same questions at
the end of the episode each season,

there are two different questions.

The two for this time, and I'm going to
ask one at a time is What is one action

that you think led to your success
that separated you from your peers?

Sean: I don't think I've
separated from my peers.

I think I worked with some amazing peers,
but I will say what's one action that's

helped me be successful in my career.

And early on at Amazon, I was
fortunate that I got to work

on search and search results.

And because it opened my eyes to
the world of A B testing at scale.

And it really helped me think about most
digital problems over the last 20 years.

At their heart are a search and relevance
problems, personalization, targeting,

all these elements people are finding to
do at some point is about an information

retrieval, relevance, ranking problem.

So, That mindset has helped me a lot.

In fact, a funny thing, my, the first
A B tests I ran for Amazon, I was

running them on the book search page.

And at that point, book search
was 70 percent of Amazon's

revenue, just the search results.

in the books category.

So this is huge for the business,
publicly traded at that point in 1999.

And I was optimizing
the results of the page.

The page constraint I had, everything
on the page had to be 60 kilobytes

or less, not megabytes, 60 kilobytes.

And it was fascinating to do
variations to trade off bigger

books, images, smaller books.

Did I want the five star rating?

If I make it a GIF, that actually
is more page weight, and that

means I need fewer results.

Am I better off having more results for
more items to be on page one, or fewer

results with bigger images and the rating?

So, Fascinating problem.

Christian: Very interesting.

I can imagine working
with that constraint.

Sounds like it was very fun to make all
these different trade offs and figure out

which one works best for the business.

That's awesome.

Sean: At that time, the biggest
bandwidth, the biggest constraint

was bandwidth to people's computers.

It was dial up.

Of course.

I'm dating myself there, but.

Christian: Yeah.

Cool.

Awesome.

Thank you for that.

That little story there.

Second question is, what are we
not talking about when it comes

to design?

I

Sean: think the design industry is
not talking about enough, but you

heard it for probably 80 percent
of our call today, business impact.

We're not talking enough about is the
time spent matching the potential impact

and really just thinking about when
is good enough on a design and move on

to the next thing that has potentially
more impact customers for the business.

So that's probably my
strongest recommendation.

Understand as a designer, what
does your business care about most?

What do your stakeholders care about most?

And then ask, how do I
relate my work to it?

And you may have to talk to your boss,
or you may have to talk to some of

the product leads or commercial leads.

But the sooner you figure out how
your work links, the better decisions

you're going to make, and the more
stakeholder buy in you will get, which

leads to you being more credible.

Christian: It's a great place to
end on , thank you so much for

being part of the podcast today.

Any last words on where people can
find you, where they can follow

you, where they can read what
you're writing, anything like that?

Sean: You can find me on LinkedIn.

I post every now and then when I
find time to write up something

but yeah, reach out to me there.

Christian: Amazing.

Sean, again, thank you very much for being
part of the Design Meets Business journey.

This has been an absolute pleasure.

Sean: Thank you for having me.

Christian: If you've
listened this far, thank you.

I appreciate you and I hope you've
learned something that makes you just

a little bit better than yesterday.

You can check out the show
notes on designmeetsbusiness.

co If this has taught you anything,
please consider leaving a review

and sharing the episode with someone
else who could learn from it.

And I'll catch you in the next one.