Great Expectations (often abbreviated GX) is the open-source Python framework that defined the declarative approach to data quality. You write tests for your data the same way you write unit tests for your code — "expect this column to never be null," "expect these values to be in this set," "expect this row count to be between X and Y." If the expectations fail, the test suite fails, and your pipeline knows something is wrong.

If Monte Carlo and Bigeye are the automatic smoke alarms of data quality (they figure out what is normal and yell when it changes), Great Expectations is the explicit checklist (you write down exactly what you expect, and the framework verifies it). Both approaches are valid. They solve overlapping but distinct problems, and most mature data teams end up using both.

Origin Story

Great Expectations was created in 2017 by Abe Gong and James Campbell, two data scientists who had spent years dealing with the same problem at consulting clients: pipelines that "ran successfully" but produced silently wrong data. The first commit to the GX repo predates Monte Carlo's founding by about two years, which makes Great Expectations the direct ancestor of the entire data observability category. Barr Moses literally wrote her "What is Data Observability?" post in a world where Great Expectations already existed; the contribution of the observability vendors was to add automatic baselining and a polished SaaS experience on top.

The name is a Charles Dickens reference, but it also captures the core idea: you have expectations about your data, and you want to make those expectations explicit and testable rather than implicit assumptions buried in someone's head.

The commercial entity behind GX was originally called Superconductive, founded by Gong in 2020, and has since rebranded to just GX. The company raised a Series A from Index Ventures and a Series B from Tiger Global. The commercial product, GX Cloud, is a hosted version with a UI, scheduling, and team collaboration features, layered on top of the open-source core.

How It Actually Works

You install GX as a Python package (pip install great_expectations). You point it at a data source — a Postgres database, a Snowflake warehouse, a Pandas DataFrame, a Spark DataFrame, a Parquet file, whatever. GX profiles the data and lets you author expectations, which are roughly:

expect_column_values_to_not_be_null("user_id")
expect_column_values_to_be_in_set("status", ["active", "churned", "trial"])
expect_table_row_count_to_be_between(min_value=1000, max_value=1000000)
expect_column_mean_to_be_between("revenue", min_value=10, max_value=1000)

These expectations are bundled into an Expectation Suite, which is just a JSON file you check into git. You then run a Checkpoint, which validates a batch of data against the suite and produces a Validation Result. Failures can fail your pipeline, fire alerts, or produce a Data Doc (an auto-generated HTML report describing what passed and what failed).

The framework ships with about 50 built-in expectation types and an extensible system for writing your own. There is also a community-maintained "expectation gallery" with hundreds of contributed expectations.

Why Engineers Love It (And Why They Sometimes Hate It)

What engineers love about GX: it is explicit, version-controlled, and CI/CD-friendly. Your data quality rules live in git, get reviewed in pull requests, and run as part of your pipeline. There is no separate vendor SaaS to log into. There is no statistical magic you have to trust. If a test fails, you know exactly which expectation broke and why.

What engineers hate about GX: the API has historically been complicated and changed a lot. The framework went through several major redesigns (the v3 API, the "fluent" API, GX Core), and each migration broke existing setups. It is more verbose than dbt's built-in tests for simple cases. And configuration sprawl is real — a serious GX setup involves Datasources, BatchRequests, ExpectationSuites, Checkpoints, and Stores, which is more concepts than most teams want to learn.

These criticisms are part of why dbt's much simpler tests: block in YAML has eaten a lot of GX's market in transformation pipelines. For pure dbt shops, dbt tests are usually enough. GX shines when you need richer expectations than dbt offers, or when you are testing data outside of dbt (raw ingest, ML pipelines, streaming).

The Opinionated Take

Great Expectations is the technically correct answer to data quality for teams that care about engineering rigor. Declarative tests are inherently more trustworthy than statistical anomaly detection, because they fail or pass deterministically and the rules are visible in code review.

But correctness is not the same as adoption. The data observability category exists because most data teams will not write thousands of explicit tests, even if they should. Monte Carlo's pitch — "you do not have to write any tests, we figure it out" — is more attractive to busy teams than GX's pitch — "write tests for everything you care about." The result is that Great Expectations is the spiritual ancestor of the category but not the commercial winner.

GX Cloud is the company's attempt to fix this by adding auto-suggested expectations, a friendly UI, and managed scheduling. Whether it can compete commercially with Monte Carlo and Bigeye remains to be seen. The structural advantage GX has is that the open-source core is free, has a large installed base, and integrates everywhere. The structural disadvantage is that open-source companies in the data quality space have struggled to monetize because the easiest path is "self-host the OSS and never pay."

Where GX Fits in the Stack

Great Expectations runs inside your pipelines rather than alongside them. It is most commonly used:

• As a validation step inside Airflow, Dagster, or Prefect DAGs
• As a testing layer for dbt models (alongside dbt's native tests)
• As a quality gate for ML training data (where it has strong adoption among ML engineers)
• As a contract checker between data producers and consumers

A typical user is a data engineer or ML engineer who already writes Python and wants data tests to live in the same repo as the code that produces the data.