The Data Daily
Less than 2 minutes to read each morning.
Not sure if you want in? Read the archives below.
5 days a week since May 1st, 2023.
Is my data architecture good?
Is my data architecture good?
Over the last few months I’ve had a number of clients ask me to review their data architecture.
When I say “sure” they share an architecture diagram and maybe some documentation that elaborates on their architecture. Then they ask something like - “Is our data architecture good? Do you notice any issues?”
It’s an impossible question to answer. So I usually reply with a question;
“It could be just fine. What challenges are you having? What can you not do that you want to do?”
A good data architecture doesn’t exist in isolation. There is no picture-perfect pattern to follow.
So what is a good data architecture? And how would you spot one?
There are only two kinds of data architectures. Those that work for your business. And those that don’t. Until I know if your architecture is “good” or not I have to know what challenges you are facing or what you wish you could do.
Only then can we talk about if your data architecture works. Or doesn’t.
Yes, we could nitpick about niche design patterns, hardware configurations, or development best practices. I have opinions of course. But none of that makes a good architecture. You could have an architecture that follows all the industry best practices perfectly and still fail if it doesn’t work for your business.
So instead of asking “do we have a good data architecture?”, try this.
Are we serving the needs of the business effectively? And how can we increase our capacity to empower the business?
I’m here,
Sawyer
How often matters more than What
You have to talk with your stakeholders. It’s part of the role of a data leader.
Interactions between business teams and data teams are fundamental to the success of any data team. When trust or collaboration breaks down between these groups, it’s hard for either side to confident when they have to work together.
Often, the advice to data teams is about WHAT to say or not say in a conversation. Yet, in spite of having great scripts for approaching our stakeholders, things still break down.
Instead, I’d argue HOW you have the conversation is more important than any of the WHAT you say. Here are key elements that go into the HOW of a conversation.
Context: This is the timing (morning/afternoon, busy season, etc), location (conference room, desk drive-by, break room) , and medium (in person, virtual, on-video/off-video)
Tone: Regardless of what you are saying, the tone of you words are a powerful communicator. Do you take on a tone of curiosity or boredom? Encouragement or frustration? Empathy or condescension? This is communicated in your facial expressions, body language, word choices, and volume.
Purpose: Who is centered in the conversation? Is the purpose for you to get what you want/need from the other person? Is it for them to share about their wants/needs? Is the purpose to encourage, inform, control, or challenge the other person?
Parties: What other people are in the conversation? Are managers or skip-managers present? How many peer team members are present? How much space in the conversation is there for each person to safely contribute?
There are many more factors you could probably list out as well, but these can serve as a starting point.
Context over content.
How over what.
They don’t care what you know or what you say if they don’t first know how much you care or know about them. Context is the solution.
I’m here,
Sawyer
When other people tell you who you are
Who tells you what your data team should be?
Because here’s the frustration I hear. Data leaders are exhausted by the whims and requests of business teams who think a data team should be a report factory.
It’s a perennial frustration for data leaders.
Like shoveling snow in a Michigan winter.
Or like trying to keep your house clean while raising kids.
Is there an end? How do you fix this?
There’s one fundamental disease under this symptom.
You let other people define what data means and what the data team should be.
Instead of defining what you believe your data team should be.
They think you are data cleaners. Chart builders. Query writers. Spreadsheet wizards. Database optimizers.
And yes you might do all those things, but that’s not who you are.
Your identity has been hijacked and there’s only one way to reclaim it.
Define your purpose and strategy for why data exists at your organization. And ruthlessly commit to that purpose and strategy.
Your purpose is why you exist.
Your strategy is how you show up.
Are you going to define that for yourself? Or let someone else pick it for you.
I’m glad you are here,
Sawyer
How do you win?
We all just want to know how to win.
The main thing your team is asking of you, their leader
And, if you are honest, the main thing you are asking of yourself
…how do we win?
All this striving, stressing, and strategy has to lead somewhere.
So where are we going? How do we know if we got there? Did we win?
They wonder if their code matters. If that report made a difference. If the detailed analysis was significant in any way.
…how do we win?
Winning doesn’t require someone else to lose.
It doesn’t require competition or cut-throat tactics.
(In fact, it probably requires the opposite. Collaboration, community, and connection.)
There’s no referee awarding points. No judge declaring a clear winner. No objective scoreboard crowning a victor. Data doesn’t have defined universal rules for winning.
Which is why your team is perpetually wondering…how do we win?
It’s up to you, their leader, to tell them how to win.
To define what success looks like.
Why does this matter?
If your team doesn’t know how to win, their courage to persist is limited.
If your team doesn’t know what the score is, the day is frustration instead of fulfillment.
If your team doesn’t know if they are successful, then they will find another job that does tell them.
Leaders of data teams, are looking at you to tell them how to win.
How confident do you feel in your definition of success? And does your team know what it is?
My Measuring Success Launchpad is a 2-week program that defines and designs success for your team that you can actually measure. It’s a great way to start 2025.
I’m here,
Sawyer
The two levers
If you missed the livestream yesterday about High Performing Data Teams: Agile Behaviors and Frameworks, you can still catch the replay!
————
For each decision you make you have two levers to pull.
The probability that something will happen.
The payoff that occurs if it happens.
In each decision, we are consciously or subconsciously evaluating these two things.
Skydiving has a very high chance of being successful. But a very significant loss if the parachute fails.
Buying an index fund has a very low chance of decreasing in value over a meaningful period of time.
Joining a community group might have a high chance of meeting new people and an average chance that one of those relationships will become significant in your life.
These are the two levers that are present in every decision. Probability and payoff.
Data won’t ever change the probability that something will occur. Neither will data change the size of the reward or loss for a specific decision.
But it will offer you a clearer view of reality. Maybe the High Risk, Low Reward option isn’t worth it. Maybe if the reward was 10% bigger it would be worth it. Only a clear view of reality allows you to ask different questions about the options before you.
Are there other options available?
How can I change this scenario to reduce the risk?
What opportunities are there to increase the probability of success?
Can I avoid this decision?
Ultimately, this requires two things:
Data to evaluate
Awareness of the outcome you want.
As data nerds, we have plenty of the first. But the second piece is a chronic challenge for leadership.
If you want tactical strategies for designing outcomes, success and decisions for your data team, jump into my Measuring Success Launchpad.
A previous workshop participant said this: “Yes, I would recommend this! The frameworks are really useful and it's always better to break down how it applies to your specific scenario with someone who knows it really well.”
Hit reply to get started.
I’m here,
Sawyer
You are invited
Most of the problems data leaders face aren’t new.
They are chronic and ubiquitous.
How do we effectively manage stakeholders and their expectations?
What are the best ways to manage work intake and prioritization?
How do we measure the effectiveness of our team performance?
What can we do to have a tighter feedback loop for our data work?
How do we stop being ticket-takers?
How do I know these are chronic and ubiquitous? Because they show up in my email inbox, DMs, and on phone calls with data leaders every week.
And I experienced them when I was in the trenches as a data engineer and data consultant at Microsoft.
This afternoon, I’m hosting a free Livestream where Business Performance Coach Kert Peterson will be joining me to talk about the issues that plague data teams.
Borrowing from the disciplines of agile, Kanban, and Scrum, we will discuss the challenges that prevent data teams from becoming High Performing Data Teams.
It’s free. This afternoon. 1pm ET.
The link is right here.
High Performing Data Teams: Agile Behaviors and Frameworks
Would love to see you there.
I’m here,
Sawyer
[Podcast] Increasing capacity at HBCUs with Data Analytics
Pre-s: Free LinkedIn live stream tomorrow at 1 pm ET.
High Performing Data Teams: Agile Frameworks and Behaviors
Join us!
————-
In this episode of Making Data Matter we sit down with Phillip Wallace - Director of Knowledge Management at UNCF (United Negro College Fund).
We shared a ton of laughs as Phillip offered his insights for you into
- How data is used to increase impact in Higher Ed
- The role of Qualitative and Quantitative Analytics
- The different functions or personas of a data professional
- The similarities between preaching and data analytics
- and more.
I’m here,
Sawyer
What are you optimizing for?
Pre-s: Free LinkedIn live stream this Wednesday at 1 pm ET.
High Performing Data Teams: Agile Frameworks and Behaviors
Join us!
—————
Decision-making is fundamental to business and organizational leadership.
The mantra in the data industry is “data-driven decisions”. But that statement gets simplified down to adages like “follow the data” and “make the best choice”.
The problem? You don’t know what you are optimizing for. And when you don’t know the end goal, following the data leads nowhere and the best choice is impossible to find.
Every decision requires multiple options. Each option has some likelihood of reaching an outcome. Quality decisions usually weigh how likely something is to happen with how desirable the result is.
Here’s what your decisions could look like:
Option 1: High likelihood of Result A.
Option 2: Low likelihood of Result B.
Option 3: Moderate likelihood of Result C.
So which one should you pick?
I have no idea. Because even when you have perfect knowledge of the likelihood of each result, you can’t decide without knowing which result you want to optimize for.
This is why decision-making always starts with outcomes.
Data can help you measure the likelihood of certain results. It can help you quantify the value of each result in neutral terms.
But data can’t tell you which result you should want. That’s the work of humans.
You can be as data-driven as you want, but you will always be dependent on the human factor.
Why does this matter to you?
Your frustrations with data, priorities, leadership, and progress are related to your clarity about outcomes. Your decision-making frameworks fall flat if you don’t know what you want.
I’m doing a 1-day workshop to help you build a High-Performing Data Team. It’s on November 12th for 4 hours. We begin our time focusing on defining the desired outcomes and end our time on decision-making frameworks.
Join me.
I’m here,
Sawyer
Negotiate with friction
Pre-s: I missed you all yesterday. It’s been a busy week and I was traveling to record lectures for a new course I’m teaching. Excited to be back.
——————-
Why you should care about friction
It starts with removing friction.
Surfers wax their boards.
Swimmers wax their legs.
Climbers and weightlifters coat their hands with chalk.
Sprinters wear spikes on their shoes.
Why?
They want optimal performance and so they negotiate heavily with friction.
In some cases, they want to increase friction and grip. Spikes. Chalk.
Other times they want smooth and as minimal friction as possible.
When millimeters and milliseconds matter the highest performing athletes become masters of friction.
Or for a more everyday example - Imagine driving your car on rough gravel vs a fresh interstate highway. The gravel road is slower, the bumps are jarring for the riders, and the rough ride puts excessive wear and tear on the vehicle.
“Highway miles” on an odometer are much different than “dirt road” miles.
Removing friction - even the smallest amount - improves speed, comfort, and longevity.
Friction, large and small, adds up. Compounds. And charges a heavy price.
Why should data teams care about friction?
Every time your business users and leadership interact with data they experience friction. How many clicks do they take. How confused they are by the chart. How much doubt enters their mind about the accuracy of the numbers?
Friction is the primary barrier between data teams and the business. A focus on removing friction requires collaboration between data and business teams and builds trust between them. It requires that you understand what their experience is like.
That you empathize, understand, and care.
The result? Trust. And trust is the fastest way to deliver value and get noticed by leadership.
I’m here,
Sawyer
My favorite data leaders
Good morning,
The data leaders I’ve talked with recently are planning for a strong close to end the year and a successful launch in 2025.
And that got me thinking about you - my favorite data people - here on this email list.
What are you focused on at the end of this year?
What challenges are you encountering on your team? What can I write about to help you? What resources could I provide that would have you feeling great about the end of 2024 and the beginning of 2025?
Hit reply.
I’m here,
Sawyer
Forget the vendors
Are you tired of the vendors?
Conferences, hype, product releases, vendor partners, and salespeople in the comments.
It’s hard to get through a day in the data world without getting pushed a vendor’s product.
That’s why this comment from a recent client stuck out to me:
“Many of the training and development opportunities I've participated in in recent years have focused more exclusively on vendor-driven technical discussions or a more surface-level view of one the topics that was unpacked in depth. I haven't encountered an opportunity that so seamlessly weaves together a data-informed decision-making framework, architecture, and the implementation and governance of teams and processes.”
This is true for my:
We rarely talk about vendors and specific products. Because what you, as a data leader most want and need, are first principles. Foundational ideas that you can build on and apply to your specific use case.
If you are frustrated by vendor-slanted conversations about data platforms, and you want a perspective that isn’t going to push a product, hit reply.
I’m here,
Sawyer
Your bridge has collapsed
Hey, data leader, Want to launch into 2025 with a strategy and purpose that you can actually measure?
Nov 10th I’m hosting a half-day workshop to help you define and design your success measures. More details and registration available here.
——————-
The infamous challenge for data professionals is interacting with the business stakeholders and leadership. Confusion about what they want, when they want it, how it should exist, and what importance it has is the primary complaint I hear in my inbox.
A key discipline that bridges this gap? Data modeling.
No, I’m not strictly talking about slow-changing dimensions or snapshot fact tables.
Rather, I’m thinking about data modeling at a higher level practice that maps raw data into business frameworks and languages.
It’s a practice where you, as a data leader work to represent the business concepts and workflow with data. You constrain, shape, expand, conform, and enrich data so that it business users and leadership see can their business in the data.
It’s not a discipline the business does. It’s the work of the data team to model data, but it does require connection and relationship with the business teams.
If you are feeling a breakdown between your data team and the rest organization.
If you are ignored by leadership for all the hard work you do.
If confusion abounds anytime you present data.
Take a hard look at your data modeling practice.
Your bridge might have collapsed (if it ever existed)
And only a chasm is left.
I’m here,
Sawyer
You are getting the results you designed for
Hey, data leader, launch into 2025 with a strategy and purpose that can actually measure.
Nov 10th I’m hosting a half-day workshop to help you define and design your success measures.
————
I think you should make mistakes.
Here’s why.
The impact of having safe and empowering leadership on your data team means your people get to be creative and courageous with their ideas. And that leads to mistakes.
What do you do with those mistakes?
Analyze the system.
Here are mistakes that could happen:
Pipeline fails because bad code was pushed.
A user reports bug that gets ignored for two weeks.
New easiest data requests get cherry-picked first by team members.
Documentation is 18 months old and unusable.
250 Job applicants, 17 interviews, and no candidate hired.
Errors aren’t the result of malice or vice. They are a result of the system. Humans don’t make mistakes. They behave exactly like the culture, system, and environment prepared them to act.
Mistakes aren’t an issue with the person. They give you unique insights into the failures of the processes, culture, and systems you have at hand.
Your team is getting the exact results (wins and failures alike) your team is designed for.
Want different results? Own the mistakes.
I’m here,
Sawyer
Outcomes don’t creep
“How do I get my business engaged in the work we are doing with data?”
Many ways to address this challenge. But here’s a simple place to start:
Replace Requirements Gathering with Outcome Gathering.
This is a fundamental shift in how data professionals interact with business teams. It changes nearly every question you ask.
It moves you from asking about what they want to asking about what they are trying to accomplish.
Instead of collecting detailed requirements about specific visualizations, data fields, or data latency. You get to ask about desired outcomes.
It shifts the conversation from being about technical details to being about business questions and opportunities. If you want the business team to be more engaged in data, then entering their domain with business-centric questions is a powerful posture.
A powerful side-effect of this approach? Scope creep disappears. No longer are you left fighting about whether a specific data point was included in the requirements gathering. Instead, both parties are focused on delivering an outcome.
And outcomes. Don’t. Creep.
I’m glad you are here,
Sawyer
A data platform always exists in this tension
You want a stable and robust data platform.
It should produce predictable results and remain resilient through change.
But
The organization, economic environment, leadership, customer needs, and team members are constantly changing.
Data has to respond, interact, and flex with the changes that are inevitable and constant.
You are always making trade-offs between these two extremes.
Excel is the ultimate form of flexibility with an effortless and dynamic ability to respond to changes.
Rigid pipelines that fail at a schema change are the other extreme.
All of your data products and platform elements exist somewhere between those two. That dashboard, those database tables, that graph database, those JSON APIs, and even those GoogleSheets that manages your whole warehouse inventory.
A data platform always exists in this tension.
Here is why you should care.
If you design for stability, predictability, and a structured platform, you will face constant frustrations by the ever-changing environment of the world around you.
If you rely on flexible and dynamic data designs, you will encounter headaches of scalability, reliability, and trust.
The point isn’t to pick one or the other. The goal is to acknowledge the tradeoffs when making choices about your data products.
Great mental health for data leaders is a commitment to reality at all costs. This is one of those realities you can’t ignore.
I’m here,
Sawyer
The virtuous cycle of a data driven org.
A flywheel is a momentum and leverage device. It stores energy to continue driving forward motion.
It’s weighted in a specific distribution so that once you get it turning, it collects energy and takes significantly less energy to keep it spinning.
When designed well, data is a flywheel for your business. Here’s how it works in steps.
We want business outcomes
We make decisions to get those outcomes
Data provides insights that reduce the uncertainty in a decision
Data tracks our progress toward the outcomes.
Our progress (or stalled progress) presents other decisions.
…and we use data to reduce uncertainty in the next decision
Make decisions. Track progress. Make more decisions.
Data is the weight that pushes this virtuous cycle forward.
And once you get the wheel spinning. It picks up momentum.
I’m here,
Sawyer
How do you gain business literacy?
Business literacy is empathy.
“Business literacy is so important for data people!”
This is what I read constantly (and write occasionally) on LinkedIn.
But what is business literacy…like actually?
Here’s my working definition:
“A curiosity, understanding and appreciation for business questions, challenges, and opportunities.”
This looks like:
Not assuming business challenges are easy
Asking more questions because the topic is more nuanced than you once believed
Acknowledging the complexity of the process, logic, and constraints they face.
Recognizing that things can be BOTH/AND.
Requirements gathering is about business outcomes, not punch lists.
Not assuming the business doesn’t know what they want.
Or, to put it in just a few word - Empathy for your business peers.
You feel, appreciate and are continually curious about their experience
and how you can remove the friction.
I’m here,
Sawyer
Should we cancel the race?
In the days leading up to the 2023 Twin Cities Marathon race organizers had decisions looming. The weather conditions were looking bad for the race. The forecast was far warmer than usual for an October 1st in Minnesota. They were faced with an important decision - should we cancel the race? This would be a major decision. Race organizers had been planning for months. Nearly 10K runners had spent months in rigorous training for this race. Canceling would be a huge disappointment to tens of thousands of people.
They had a number of ways they could have assessed the situation.
Send someone outside to see how “hot” it feels to them.
Send a group of people outside and ask them to rate how warm it feels to them, and aggregate the totals.
They open up their phone and google what the temperature will be that day.
Send someone out early to run the marathon and see if they made it to the end.
Consulted with weather experts, evaluated temperature, humidity, heat index, and a variety of other factors, and ranked the conditions on a rating scale to assess if the race should happen.
As you might expect, they went with the last option. Specifically, they relied on an established rating system known as EAS (Event Alert System). On the day of the race, at 5:19 am when most runners were already up preparing or traveling to the race site, they canceled the event.
They had hit a “Black Flag” Alert level on the EAS scale which means the weather is “Extreme and Dangerous”. But how do you know if the weather is extreme and dangerous? Well, there’s a more detailed measurement chart that. To hit “Black Flag” for hot weather the WBGT > 90 degrees. Ok, that’s more detailed, but what in the world is WBGT?
It’s the a detailed measurement called WetBulb Globe Temperature:
“(WBGT) is a measure of the heat stress in direct sunlight, which takes into account: temperature, humidity, wind speed, sun angle, and cloud cover (solar radiation). This differs from the heat index, which takes into consideration temperature and humidity and is calculated for shady areas. If you work or exercise in direct sunlight, this is a good element to monitor.”
For a highly nuanced situation with high stakes, they used a complex and specific measurement tool to make a decision about the race.
Why does this matter to you?
A data leader’s dream is to be a part of a “data-driven company.”
This is a practical example of a data-driven decision that leveraged a robust measurement tool.
Before those measurements, there was loads of uncertainty. How bad are the conditions? What are potential problems that could occur if we move forward? What is the risk to the runners? What is the cost of canceling the race?
They needed more certainty about a situation before they could make a decision. And because of those measurements, they had the clarity and certainty required to make a high-stakes decision and cancel the race a few hours before the start time.
(Thanks to my friend Matt who faithfully trained for the 2023 marathon only to have it cancelled. The 2024 marathon is this weekend - and Matt tells me its forecasted for “Green Flag” conditions”. He’s ready)
I’m here,
Sawyer
Are you teaching to the test?
Your metric needs an objective.
Or you will manage to the metric without knowing why.
This is a fundamental part of the work I do with data teams. We don’t just define success metrics - we define the purpose that metric holds.
What started as “Stakeholder Satisfaction” morphs into optimize the stakeholder survey to get higher numbers, which morphs into only sending the survey to people most likely to respond positively, which morphs into your team performing strictly to improve the survey results (a la “teach to the test”).
Without a metric objective, you can’t identify what negative gaming behaviors are.
Without a metric objective, you can’t tell if you’ve drifted from the original purpose.
Without a metric objective, your team will optimize a number rather than a result.
And you forget the point of the metric. At one moment in time, you believed your data team existed to serve the business teams and make them tremendously happy with data.
Your survey may or may not be serving that purpose anymore.
Jeff Bezos, when reflecting on an customer metric important customer metric at Amazon said this: “I don’t really care about this metric. I care about customer happiness. That metric is only useful to me so long as it reflects how happy customers are”
I’m here,
Sawyer
Giving up control
“Grant me the serenity to accept the things I cannot change, the courage to change the things I can, and the wisdom to know the difference.”
You control one side of the equation.
Things you can’t control
How the business thinks about you as a data team
What other people expect from the data people
If leadership just wants you to build reports
When stakeholders will reply to meeting requests
If a peer leader wants your collaboration or not
Whether leadership approves your budget request
How often you have to explain the same data report to the business.
They are out of your control. Wash your hands of all of them. Move on. Give up.
Many things are out of your control.
And when you do give up trying to control those things you can acknowledge the things you do control.
What do you control?
How you show up as a data team to the business.
The level of empathy you display when collaborating
What kind of requests you respond to from the business teams.
How you respond to requests for data that don’t make sense.
When and how to ask for budget and project approvals
If you want them to view you differently, you show up to the table differently.
When that happens, a huge ocean liner worth of freedom opens up when you step into the things you do control.
And all those things you don’t control? Well, after all Leadership isn’t about control at all.
It’s about influence.
You influence others, by your actions.
It was good to see you today,
Sawyer