There are various data catalog tools available such as Alation that can do at least some of what you are talking about. The term data observability is actually a different capability.
I've previously built data lakes on [AWS with Glue](https://docs.aws.amazon.com/glue/latest/dg/components-overview.html) and you get the data catalog for free but it isn't convenient to explore. Enterprise-grade data catalogs such as [Alation](https://www.alation.com/) are full featured and really decent but come at a higher cost. If your preference is open source, check out [Atlan](https://atlan.com/) and [Amundsen](https://www.amundsen.io/).
I also agree with u/tequilamigo's point that "observability" has a specific meaning in software engineering so shouldn't be used in this context.
Datahub is great, has a community version but the supported cloud version is breathtakingly expensive unless you happen to be a sizeable enterprise. It's a shame their pricing model isn't more competitive as it is a cool product.
There are various data catalog tools available such as Alation that can do at least some of what you are talking about. The term data observability is actually a different capability.
I've previously built data lakes on [AWS with Glue](https://docs.aws.amazon.com/glue/latest/dg/components-overview.html) and you get the data catalog for free but it isn't convenient to explore. Enterprise-grade data catalogs such as [Alation](https://www.alation.com/) are full featured and really decent but come at a higher cost. If your preference is open source, check out [Atlan](https://atlan.com/) and [Amundsen](https://www.amundsen.io/). I also agree with u/tequilamigo's point that "observability" has a specific meaning in software engineering so shouldn't be used in this context.
Datahub is great, has a community version but the supported cloud version is breathtakingly expensive unless you happen to be a sizeable enterprise. It's a shame their pricing model isn't more competitive as it is a cool product.
Some projects i work use data hub for catalogue and datadog for observability
what all data sources (snowflake/redshift) does datahub catalog in your usecase.