## What filtering is

A traced run sees **everything** your process and its children touch — not just your dataset and your model, but the Python interpreter reading its own standard library, pip reading `site-packages`, CUDA reading driver files, your logger writing to `/tmp`, and your experiment tracker scribbling into `wandb/`. Most of that is noise: it isn't part of your pipeline's real input→output story, and if `roar` recorded all of it every DAG would be buried under hundreds of incidental reads.

**Filtering is how `roar` keeps the recorded lineage to the files that actually matter.** It drops well-known noise before it becomes an artifact, so `roar dag`, `roar show`, and a registered lineage stay readable and reproducible.

> **Filtering shapes the record, not the run.** Filters only affect what `roar` *records* as inputs and outputs. Your command runs exactly as it always would — the same files are read and written on disk. A filtered file is one `roar` chose not to track, not one it hid from your program.

There are two layers, and they answer different questions:

| Layer | Answers | Configured in | Managed with |
|---|---|---|---|
| **Built-in noise filters** | "Is this a *category* of file nobody wants tracked?" (system files, installed packages, caches, `/tmp`) | `.roar/config.toml` `[filters]` | `roar config set filters.<key> <bool>` |
| **Project path filters** | "Is this a path *specific to my project* I want ignored?" (e.g. `wandb/`, `logs/`, a scratch dir) | `.roarconfig` `ignore_paths` | `roar filter add / list / remove` |

## Built-in noise filters

These are category filters `roar` applies out of the box. Each is a boolean in the `[filters]` table of `.roar/config.toml`, and **all default to `true`**.

| Key | Ignores |
|---|---|
| `filters.ignore_system_reads` | System file reads under `/sys`, `/etc`, `/sbin`. |
| `filters.ignore_package_reads` | Reads from installed packages (`site-packages` and the like) — already pinned in the captured dependency list, so tracking them as inputs is redundant. |
| `filters.ignore_torch_cache` | Torch/Triton compile caches (`/tmp/torchinductor_*`, etc.). |
| `filters.ignore_library_caches` | Well-known per-library user caches: `~/.cache/huggingface`, `~/.cache/pip`, `~/.cache/uv`, `~/.cache/poetry`, `~/.cache/wandb`, `~/.cache/mlflow`, and similar. A project-specific cache not on the curated list (e.g. `~/.cache/<your-project>/`) stays tracked. |
| `filters.ignore_tmp_files` | Everything under `/tmp`. |

Turn one off when you genuinely need that category in the record — for example, if an intermediate you care about is written under `/tmp`:

```bash
roar config set filters.ignore_tmp_files false
```

> **A read is kept even when its category is filtered, if nothing else explains it.** A `/tmp` file your run only *reads* (never wrote) is still surfaced — flagged as an *unsourced input*, because it won't exist when the pipeline is reproduced elsewhere. Filtering suppresses noise, not unexplained dependencies. The safe fix is to not stage real inputs in `/tmp`.

## Project path filters

Built-in filters cover categories that apply to everyone. Your project also has its *own* noise — an experiment tracker's `wandb/` directory, a `logs/` folder, a scratch workspace. That's what `roar filter` is for. It manages the `ignore_paths` list in a `.roarconfig` file at your project root:

```bash
roar filter add wandb/ logs/ .cache/wandb/   # add patterns
roar filter list                             # show current patterns
roar filter remove wandb/                     # drop a pattern
```

`roar filter list` prints the file it reads from and the patterns in it:

```
.roarconfig (/home/you/project/.roarconfig):
  wandb/
```

**Pattern matching** is deliberately simple, matched against the file paths the tracer observes:

- An **absolute** pattern (starting with `/`) matches as a **prefix**: `/data/scratch/` ignores anything whose path begins with it.
- A **relative** pattern matches as a **substring** anywhere in the path: `wandb/` ignores any path containing `wandb/`.
- A **trailing `/`** means "this directory and everything under it."

> **`.roarconfig` is meant to be committed.** It lives at the project root, separate from the local, git-ignored `.roar/` state directory. Checking it in means everyone tracing the project — and every CI run — shares the same project-specific filters, so lineage looks the same for the whole team.

## Seeing what was filtered

By default `roar run` reports filter activity as a one-line count so nothing is silently dropped:

```
hint: 10 files under /tmp ignored (filters.ignore_tmp_files).
hint:   to track them: roar config set filters.ignore_tmp_files false
```

To see the individual paths that were filtered, raise the output verbosity:

```bash
roar config set output.verbosity debug   # also lists every filtered file (can be large)
```

`output.verbosity` accepts `quiet`, `normal` (the default — status plus filter counts), `verbose` (also lists tracked reads/writes), and `debug` (also lists filtered files).

## Strict mode: cleaning up `/tmp` writes

By default `roar` *ignores* `/tmp` writes (`filters.ignore_tmp_files`) but leaves them on disk. **Strict mode** goes further: it deletes the `/tmp` files a run wrote once the run finishes, so scratch output doesn't linger between runs and can't be mistaken for a real input later.

```bash
roar config set cleanup.delete_tmp_writes true   # off by default
```

Strict mode (`cleanup.delete_tmp_writes = true`) overrides the passive `ignore_tmp_files` behavior — instead of merely not tracking `/tmp` writes, `roar` actively removes them. It only touches files the run itself wrote under `/tmp`; it never deletes anything your run only read. Leave it off unless you specifically want that scratch cleanup; it's a convenience for keeping a shared machine tidy, not a correctness requirement.

## Filtering vs. secret redaction

Path and category filtering decide **which files enter your lineage**. A separate mechanism, **omit filtering**, decides **which values are scrubbed from a lineage before it's published** to GLaaS — API keys in environment variables, credentials embedded in a git remote URL, and explicitly listed secret values. That runs at `roar register` / `roar put` time and is controlled by the `registration.omit.*` settings, not `[filters]`. See [Authentication](/docs/authentication) and [Scopes](/docs/scopes) for how registration handles secrets.

## What to reach for

- A whole *category* of files is cluttering every run → toggle a **built-in filter** (`filters.*`).
- One directory *in this project* is noise → add a **path filter** (`roar filter add`), and commit `.roarconfig` so the team shares it.
- An expected input or output is missing from the DAG → check the filter counts in the `roar run` output or run with `output.verbosity=debug`; a built-in filter (often `ignore_tmp_files` or `ignore_package_reads`) is the usual cause. See [Troubleshooting](/docs/troubleshooting).