Live queries, extracts and caching
How Halo gets answers from your data: live by default, scheduled extracts, the result cache, freshness and the guardrails on every query.
When someone opens a chart, Halo Engine plans a query with its filters and row policies, then answers from the first of these that can: the result cache, the dataset's extract, or the source itself.
Live by default
A dataset with no extract sends each question to your database as a live query: parameterized, read-only SQL with a timeout. It suits a fast warehouse, or numbers that must be up to the minute, but your database does the work and reports wait on it.
Extracts
An extract is a copy of a dataset's tables, or of its custom SQL's result, that Halo keeps as Parquet files and refreshes on a schedule. Queries run in DuckDB over those files, so they're quick and your database isn't asked. Row policies and filters apply the same way.
Files are encrypted and count toward your plan's storage. At the limit, turning one on is refused, and a refresh fails and keeps the old files.
Turn one on
Open the dataset's Settings tab and find Extract.
Choose how often to Refresh, and when.
Optionally choose a Cursor column (below).
Choose Turn on extract. Its first refresh starts right away.
| Refresh | When |
|---|---|
| Every hour | On the hour |
| Every day | At an hour you pick, in your organization's time zone |
| Every week | On a weekday, at an hour |
| Only when refreshed by hand | When someone chooses Refresh now |
Refreshes run as jobs. New files replace old ones all at once, so queries see one refresh or the next, never a mix. If a scheduled refresh fails, whoever turned the extract on is told, and queries keep using the last good files.
Pause schedule stops scheduled refreshes, while the copy stays in use and Refresh now still works. Settings, Schedules lists every extract you can see. Turn off extract, in the panel's menu, deletes the files, and queries go back to the database.
Cursor columns and full copies
Without a cursor column, every refresh copies every row. With one, a date, timestamp or whole number that only grows, refreshes add rows from its last value on. That value is included, so rows loaded late for the last day still arrive.
Adding rows can't catch changed or deleted ones, so an extract with a cursor copies everything again regularly:
| Schedule | Full copy |
|---|---|
| Hourly or daily | The first scheduled refresh from the extract's hour on the first day of your organization's week. |
| Weekly | Every scheduled run |
| By hand | Refresh now, when the last full copy is over a week old |
It also copies everything after 48 incremental refreshes, after a column changes, and when you choose Copy every row now. Only full copies pick up rows with an empty cursor. The panel shows the last and next one.
When queries use the extract
Queries use the extract when it holds every table and column the dataset reads. Otherwise they run on the database, and the panel says why.
Joins across connections happen only inside an extract. A dataset with tables from two connections can be queried once its extract has refreshed.
Formulas compile for extracts automatically. Custom SQL fields run in DuckDB as written, and fail with DuckDB's message if it doesn't accept them.
A SQL dataset's own query runs at the source, during refreshes.
The result cache
Repeat questions are answered from saved results, for every dataset, live or extract. Under Result cache on the Settings tab, Keep results for is off, or from 1 minute to 7 days. The default is 15 minutes.
People never share rows. A saved answer depends on the query's SQL, values and row policies, so people who may see different rows never share one.
Changes count. A new dataset version or extract refresh means a new answer.
Results are encrypted. Failed queries and exports aren't saved.
Clear cached results throws away everything saved for the dataset.
Public links and embeds usually read a snapshot instead: a published report's answers, kept so they open without reaching your database.
Freshness
Reports, charts and the explorer say how old their data is, such as "Updated 4 minutes ago". The exact time, and whether it came from the cache, are in its tooltip. Refresh always asks again and replaces the saved result.
Data can be as old as the extract's last refresh, plus its time in the cache.
Guardrails
Every interactive query has limits, so one question can't hold up everyone else's. Your plan sets the first two: see Usage and plan limits.
| Guardrail | What happens |
|---|---|
| Timeout | Past it: "The source took too long to answer. Narrow the date range or filters and try again." |
| Row cap | A larger request is lowered: "Showing the first … rows, the most your plan returns from one query." |
| Concurrency | Each organization runs a limited number of queries at once. Others wait a moment, then: "Your organization is running as many queries as it can. Try again in a moment." |
Closing a tab cancels its query, and identical questions asked together share one run. Exports run in slots of their own.
Recent queries, on the dataset's History tab, lists what ran from the explorer, charts, reports and embeds, and how each ended. Slow queries are flagged, and answers from the cache never count as slow.