Is Your Data Team Chopping, Cooking, and Serving All at Once?
This week we are focused on how to move faster as a data team. Yesterday we talked about source control and managing changes to your data solutions.
Today is about environment strategy.
But it will be more fun if we talk about a restaurant and food.
Every restaurant has a kitchen and a dining room. A place where food is prepped and a place where food is served. These are ‘environments’, often called Development (Dev) and Production (Prod).
Dev is the land where the staff works. Vegetables are chopped, steaks seared, and soup simmers in the “back of the house”. This is also where the dishes are cleaned. Unless specifically planned, guests do not see what happens back here.
Instead, guests stay in the dining room, where the experience is particularly designed for their enjoyment. No piles of messy dishes, frantic cooks, spilled oil, or burned asparagus.
In order for your data team to operate and deliver effectively you need separate environments. In the most minimal of data teams, this looks like just Dev and Prod. But in more advanced and complex organizations (or fancy restaurants) there will be more environments.
A Dev, where code is written (food is cooked). A Test, where changes are integrated with the existing code base (each element of the dish is plated). A Pre-Prod or Staging, where it goes through a final review before release (picture Gordon Ramsey meticulously reviewing a dish before it is served to guests).
Some companies have 7 environments. Most of 2-4.
But what happens if you skip this step? You are chopping, cooking, and plating all at your guest's table. Which works great except when it doesn’t. The restaurants that do this to create unique experiences are both exceptionally skilled and only choose the activities they are 100% confident they can get right every time. Everything else is still prepped in a backroom.
A basic environment strategy for your dev team looks like:
A place where the data team can make changes in safe ways that won’t affect end users.
A place where those changes and new features can be integrated with existing features to make sure nothing else breaks.
A place where required quality assurance and testing can take place (before away from end-user experience).
A place where users can consistently access high-quality data products without fear of bugs or data quality failure.
This is all about moving faster. Clean and well-thought-out environments make data teams more confident in development and deployment.
What do your data environments look like?
I’m glad you are here,
Sawyer
