Conor O'Mara
As our CTO Chris explained in a previous post, modern data teams need software development lifecycle (SDLC) practices in BI in order to manage their data effectively.
As a solutions engineer, I’ve seen our customers leverage Omni’s SDLC workflows in tons of ways – from enterprise deployments that make changes safely across hundreds of reports, to scrappy startups ensuring their dashboards are always running so they can make quick decisions.
In this blog, I’ll show you how you can use Omni’s SDLC controls together to seamlessly enable your organization with data, including:
Keeping Omni in lockstep with your database
Applying upstream data changes to Omni workbooks and dashboards
Testing updates before pushing them to production
Manage changes to your data and content with Omni’s SDLC controls #
In the demo above, I walk through a common workflow for data analysts & engineers: making changes to your data, pulling those changes into your BI tool, and updating related content. Below, I’ll dive into the specifics about how these features help data teams adopt software development lifecycle practices to safely test and deploy at any scale.
Omni’s SDLC features #
You can think of these features as tools in your toolbox to manage your deployment. While each has its own purpose, utilizing them in tandem helps you build effective SDLC workflows.
Schema refreshes #
Database schemas change all the time – whether you’ve refactored an existing table, added new views, or renamed a column, you can easily keep Omni in sync. Schema refreshes can be triggered on-click in your shared model or on a branch, scheduled on a regular cadence, or triggered by an API (especially useful to call when merging changes to dbt).
Git integration #
Enabling the git integration syncs your Omni data model to a git repository. This ensures any changes to your data model are codified and creates backups of model files to reference later. Plus, it enforces pull requests while developing on the model in Omni, creating a collaborative and peer-reviewed editing process.
Built-in version control #
If you don’t need the robustness of git but still want governance of your data model, Omni’s built-in model history is a lightweight alternative that lets you view & restore previous versions of your model.
Content validator #
When you refresh your schema or change your shared data model, you may need to update any field or table references that have changed. Omni’s content validator lets you do just that. It’s a control tower from which all field, view, and topic references can be replaced in one step – like a global “find & replace” across all Omni content.
Branches #
Branches create a “sandbox” to test how changes to your data model may impact workbooks and dashboards. For example, to avoid “find & replace” on live dashboards, you can run the content validator in a branch, then push those changes live. They also enable you to swap between various dbt environments and database environments (more on that next).
Dynamic environments: dbt and database connections #
A common structure that data teams adopt in dbt and data warehouses is separate development and production environments. This gives you the ability to test your data transformations before shipping them.
Omni’s dynamic dbt environments and database environments let you test upstream data changes all the way in the visualization & presentation layer. From a branch, you can swap environments to see if your changes worked as expected or broke anything, and make fixes accordingly.
Content drafts #
When users edit a workbook and dashboard, they enter a draft – an isolated copy of that document that lets them make incremental edits safely, before pushing those changes live.
You can also attach drafts to branches. This opens up a powerful workflow across your database, Omni data model, and content. By entering a branch while on a draft, you can see how changes to your underlying data (via dynamic environments) and your data model (via the branch) ultimately impact the content you’re viewing. When you “merge” branch changes, you can simultaneously “publish” content drafts, keeping your entire data stack in sync without breaking the content that end users see.
Keeping it all running smoothly #
Getting all the parts of the data stack to operate seamlessly is challenging. Sometimes, the BI layer gets the short end of the stick – end users get subjected to broken dashboards & reports because it’s at the end of the data “assembly line.”
Omni is designed with SDLC controls to help you keep the whole data assembly line running smoothly. We’d love to show you how this works on your own data.