Looker tips, how to avoid breaking changes

Hi 👋

Today I want to jump on a simple trick in Looker in order to avoid some basic breaking changes on your instance.

The problem

You have some nightmares because you’re scared about schema evolution in your data warehouse? You’re scared because you hope business users will not find something broken in one of your explore before you…

Trust me, I’m feeling the same way 😇

The solution

As you may already know you can implement data tests directly in Looker. You implement tests in views, models or even directly in a dedicated space. One of my best practices is to add as soon as possible data tests.

With data tests you can perform unit tests for managing business logic of your data. But on my side I love to use data testes in order to check if nothing has changed in my data sources.

As you know, if one of the columns in your data source changes or is deleted, you will have an error in Looker. Those errors can be a nightmare because as a LookML developer you sometime do not know what is going on in the data warehouse.

So what you can do is implementing a simple tests like:

It’s a simple test and the SQL run behind will looks like:

With that little trick if the table schema change in production, you will be alerted because the test will fail.

And I alway activate the “require data tests to pass before deploying this project to production”. It’s basic but at each Pull Request you want to create you will launch data tests and you will be aware if something has changed in your schema.

Warning, if you’re dealing with a massive amount of data you should consider adding a filter on your test in order to hit the partition. If your etl is loading data everyday, you can add a filter on the date with “2 days’’ in order to just take the past 2 days of data.

And that’s it!

Conclusion

Data tests are often not really used in Looker Project but it’s a great way to start implementing CI/CD logic. It’s also a great way to define a strong process in your Pull Request logic meaning if in your LookML project you have Pull Request + Data Tests + Advanced Deploy Mode you start to have a better process to avoid a lot of pain points on your instance.

If you’re working on a single instance project, you should really consider working a lot on your deployment process. With those simple tricks, you already have some great best practices to implement.

Michaël