Riverboat Consulting partnered with a client on a data warehousing project. The client was a Series A startup that worked with companies in the health & wellness space.
Due to their business model, they had data coming in from a variety of sources including their Postgres application database and Shopify. A business stakeholder had previously set up basic Metabase reporting directly querying Postgres, which was limited and siloed from other datasets. They felt like they grew out of this setup and wanted a more robust data platform as the company scaled.
Our goal was to set up their Snowflake data warehouse and set up data transformations with dbt for both the existing logic in Metabase and the foundational data marts they wanted.
What We Built
Snowflake Data Warehouse
Our counterpart at the client was a repeat client at a previous company of theirs, which simplified requirements gathering for the Snowflake warehouse architecture. In short, we wanted the same setup they were comfortable working in, and to coach them up on Snowflake administration.
Snowflake uses role-based access control (RBAC), the industry standard, but it can still be confusing to execute the correct access grants when you’re starting from scratch. We configured databases, schemas, warehouses, roles, and users for the client and provided the playbook for editing or adding new ones on their own.
Additionally, most of our work happened on a recorded Zoom meeting. Our client had 10 hours of recorded footage to rewatch if they had a question later (or have their AI agent search through).
dbt Project
We used dbt for data transformation, with work falling into one of two categories.
Metabase Lift-and-Shift
Like many other startups setting up a real data stack for the first time, there still was some tech debt. In this case, we had sets of reporting queries in Metabase that queried the Postgres application database. These queries were often unnecessarily complex and sometimes incorrect.
We needed to take the logic from Metabase and reference clean reporting tables in dbt (that would query the new Snowflake data warehouse), while ensuring the results matched between data sources. The client planned to migrate BI tools (they later chose Omni), so this was a prerequisite that reduced tool lock-in.
Foundational Data Modeling
Without the right data infrastructure in place, creating new data assets had previously been very tedious for the client. Now with Snowflake and dbt, it was the right time to create data models for key datasets and business logic. These assets would be easier/faster to query, reduce repeated filters/joins, and serve as building blocks for compounding reliance on data.
With the client, we mapped out the various data sources and the requirements to satisfy datasets needed for financial modeling. From there, we went from highest to lowest priority so we could show progress and impact.
Outcomes
The client got a ready-to-go data warehouse, armed with the training and documentation that gave them the confidence they could administrate it on their own. Then, we got them a head start on data modeling so that they could focus on stakeholder-facing work and demonstrate the benefits of a lean, effective data platform. Is your company just getting started implementing your data stack? Get in touch.