We've added a few search operators to help locate potentially stale assets quickly.
Find dataset assets that were last queried before a certain absolute date or a particular number of days.
Find dataset assets that were last refreshed (i.e. Last time the data in the table or view was changed due to a DML call (e.g. Insert, Update, etc.) ) before a certain absolute date or a particular number of days.
Find dataset assets that were created before a certain absolute date or a particular number of days.
These operators can be chained to narrow down the selection.
For example, to find all assets within the acme database that were created a year ago but haven't been queried in the last 30 days - use the query :
database:acme created_before:365d last_queried_before:30d
created_beforeis only supported for Snowflake, Bigquery and Databricks Unity Catalog.
Administrators can now subscribe to be notified by email if a crawler fails for any reason.
Metaphor now has new crawlers for :
Get Kafka metadata out of any Kafka compliant API
Get metadata from your Trino instance - onprem, using Startburst or AWS EMR
Import metadata from DataHub/Acryl
One click solution to migrate metadata out of legacy solutions
The Data Quality indicators in Metaphor have gotten a facelift. To make it easier for users to quickly see if they can trust the data in a table Metaphor now displays Green/Yellow/Red traffic indicators indicating Pass/Warn/Fail status of Quality tests.
Additionally if any table has failed tests, Metaphor automatically indicates a yellow warning light to any assets that are downstream of the failed table - cautioning the user to check and see if they should still trust the data.
Like the check engine light on a car dashboard, Metaphor provides a simple indicator to the end user, warning them about potential data problems detected by sensors within the data stack.
In addition to the Data Quality/Observability tools we already integrate with e.g. MonteCarlo, Soda, we've also added integration with dbt Tests.
Metaphor now supports displaying Snowflake Streams as a first-class asset.
We've made a few improvements to the PowerBI integrations to help data teams get better visibility into the PowerBI Ecosystem.
- PowerBI Tags now show up in Metaphor, users can search for all assets matching PowerBI tag for a quick inventory.
- PowerBI Usage Statistics such as View History, Most popular days of the week or time of the day a report has been viewed or number of email subscribers to that report. These statistics are enabled under Impact Analysis in the lineage graph.
Seeing the full picture of all assets within the data stack can be helpful for Data Engineering personas, but no one else in the organization typically care about non-production assets.
Metaphor lets you specify asset name patterns e.g. tmp, dev, test and the system automatically hides these assets from search listings, lineage etc.
Authorized users can choose to selectively enable viewing these assets if they'd like.
Need a handy way to endorse certain assets, create and apply an Endorsement tag making it highly visible to all users that this asset has been blessed by the data team?
Introducing Custom Governed Tags! Data admins and contributors can now personalize their governed tags in Metaphor by assigning colors and icons tailored to their organization. With the help of customized tags, all catalog users can easily spot important assets at a glance, and better understand their purpose.
Part 2 of our series on Coverage Metrics, this feature lets users see the percentage of assets within any given landing page that are documented, have governed tags or have contacts.
For example, you can quickly see what percentage of tables/views in a Database, or Schema have descriptions or have been assigned contacts.
Enable coverage metrics by selecting the ellipses next to any asset type in any asset landing page.
Are you tired of repeatedly writing long, complex search queries you’ve painstakingly crafted? The Saved Searches feature helps catalog users be more productive by allowing them to save search queries and filters, thus saving time and effort. It also makes teamwork a breeze by sharing searches with colleagues. That way, everyone can use the same search queries, making collaboration smoother and more consistent.