Glaas minimal logo, light
Markdown

Experiment Tracking

On this page

We don't do experiment tracking

roar is not a metrics dashboard. It doesn't aggregate scalars across runs, render loss curves, or hold the place where you compare hyperparameter sweeps. Teams typically use a separate experiment tracker for that — Weights & Biases (W&B), MLflow, Neptune, and others — and roar is designed to coexist with that layer rather than replace it.

When you run a training script that already logs to one of those trackers, roar recognizes the convention and surfaces a direct link on the job's page on glaas.ai. The metrics dashboard stays where you put it; roar just makes the lineage-to-metrics jump one click instead of "now what was that run ID again?"

What roar does

During a roar run, an analyzer watches for the directory and file patterns each experiment tracker creates:

  • wandb/ and *.wandb for W&B.
  • mlruns/ and mlartifacts/ for MLflow.
  • .neptune/ for Neptune.

When detected, roar does two things:

  1. Filters those paths out of lineage. A W&B run produces thousands of tiny tracking files — recording each as an artifact would drown the DAG. Tracker output paths are excluded from artifact tracking automatically.
  2. Extracts the run reference (W&B run URL, MLflow run ID, etc.) and attaches it to the job record as metadata. The job page on glaas.ai then shows a "View in W&B" link (or equivalent) so you can jump straight to the metrics view.

The integration is one-way: roar links to the tracker, the tracker doesn't know about roar. No coordination needed.

Today: W&B surfaced

W&B is the primary surfaced tracker right now — the most common in ML teams, and the most reliable to detect by directory shape. MLflow and Neptune are detected and filtered from lineage, but their links may not yet appear on glaas.ai job pages — that's UI work in progress.

Roadmap. MLflow and Neptune link surfacing on glaas.ai is on the list. Other trackers (TensorBoard, ClearML, Comet) are candidates if there's demand. The detection layer is open to extension; the gating factor is the glaas.ai-side render.

Disable

If you don't want roar to detect or filter tracker output paths — say you have an unusual setup where you do want the wandb/ directory recorded as artifacts — disable the analyzer:

[analyzers]
experiment_trackers = false

You can also disable specific trackers individually:

[analyzers.experiment_trackers]
wandb = false
mlflow = true
neptune = true

When disabled, lineage no longer filters the tracker paths and no run links are extracted. The tracker itself is unaffected.

What's not collected

roar reads only the directory structure and the run reference (URL or run ID) — no metrics, no scalar values, no images, no config dumps, no model files. Metrics live in the tracker; roar just makes the link discoverable from your lineage.

Where to look next

  • Telemetry — roar's own anonymous usage telemetry, which is separate from experiment-tracker integration and follows different opt-out rules.
  • roar Guide — the analyzer config surface.