A mistake we see companies make after they hire analytics engineers?
They try to shift all the analytics engineering work over from the data analysts.
While this is often presented as a move that helps data analysts, it’s detrimental to all parties in the long run. Let’s talk about why.
Commercial vs. full-stack data analysts
At the end of the day, data analysts should focus on work that has commercial impact.
That can include dashboards. But when stakeholders want a new metric data asset, or need something changed — and they always do — you want your analyst to be able to be able to implement that themselves.
Most of the time, this means going into dbt and updating models or created new ones.
If this process now requires looping in analytics or data engineering, you risk turning this into a coordination problem. You need to confirm the teammate has availability or go up the chain to reorder their priorities. Then you need to write up a ticket. It risks a back-and-forth.
For larger-scale analytics engineering problems, this is tolerable. For smaller stakeholder-facing changes, data analysts should want to control their workflow as much as they can.
This is why full-stack data analysts, who are strong both commercially and technically, can be of great value to companies.
The myth of dbt being too technical
There is a narrative going around that dbt is too technical for data analysts. That’s silly.
dbt was invented for data analysts. dbt is Python library that orchestrates SQL scripts. To use dbt in its most basic form, you only need to know SQL. And every analyst should know SQL.
dbt is a tool that enabled analysts to take more ownership over the data transformations and business logic that was otherwise gated to Python, Spark, Hadoop, etc. jobs. It’s one of the biggest drivers of increased analytics productivity in the last decade, analysts should not want it retconned away from them.
While it’s true that the architecture and design patterns of data modeling take some getting used to, that’s more used for architecting the data warehouse, not individual reporting models.
Division of labor between data analysts and analytics engineers
When both data analysts and analytics engineers are working in dbt, who should handle what?
This is going to vary for each company as well as the skillset of the team members. By default, I recommend:
- Analytics engineers transform raw data into reusable core data marts.
- Analysts take data from those data marts and write their business logic on top to create the metrics and reporting layers.
What ratio of data analysts vs. analytics engineers should I have?
Hiring full-time analytics engineers can be tricky. There’s a lot more work to do at the beginning of the analytics team’s lifecycle than there will be in a steady state. That’s why having a full-stack analyst on your team can be so powerful.
Broadly, teams fall into the trap of hiring too many analytics engineers. You really want to make sure that you have enough analysts actively partnering with the business.
A ratio of 4-5 analysts per analytics engineer was considered consensus before the AI age. We’re still calibrating what the right ratio is going forward.