You Probably Don't Need a Data Dictionary

Great article by Michael Kaminsky on an important / hot topic:

While efforts to build a data dictionary are often undertaken out of a zeal for documentation that we would normally applaud, in practice data dictionaries and data catalogs end up being a large maintenance burden for little actual value, and tend to very quickly become out of date. Instead of investing in building out traditional data dictionaries, we recommend a few different approaches for achieving the same goals in ways that are less burdensome to maintain and better serve the original objectives as well.

I am 100% ideologically aligned with this post. The one thing I'd add is that projects like Marquez and Amundsen are redefining what is possible when it comes to metadata management / discovery / curation. I believe this is a big trend and has the potential to upend what is currently considered the state-of-the-art BI experience.


Want to receive more content like this in your inbox?