How to Build an AI Rank Tracker (AI Overviews + ChatGPT + Perplexity)

An AI rank tracker does for AI answers what a classic rank tracker does for blue links: it runs a fixed set of queries on a schedule, records who appears, and turns that into trend lines. The difference is that AI engines don't have ranks — they have mentions, citations and positions inside generated answers, and the same prompt can produce different answers on consecutive runs. This guide walks through the full architecture: prompt sets, the fetch layer, scheduling, a storage schema that survives non-determinism, scoring, and dashboarding. Code sketches included; everything here fits in a weekend prototype and scales to production.

What "rank" means when there's no rank

Classic tracking has one number per keyword: position. AI tracking needs at least four, per engine:

Metric	Definition	Analogue in classic SEO
Presence rate	% of runs where your brand is mentioned in the answer text	Impression share
Mention position	Where in the answer your brand first appears (1st entity, 2nd, ...)	Rank position
Citation rate	% of runs where your domain appears in the cited sources	Ranking at all
Citation share	Your citations ÷ all citations across runs for a prompt set	Share of voice

One structural point before any code: because answers vary run to run, a single sample is noise. Every metric above is a rate over N runs, not a point observation. Budget for 3–5 runs per prompt per engine per tracking period as a starting point, and treat day-over-day movement smaller than your run-to-run variance as no signal.

Architecture overview

Five components, none exotic:

prompt sets ──> scheduler ──> fetch layer (AI search API) ──> raw store
                                                                │
                                              scoring jobs <────┘
                                                    │
                                               dashboard / alerts

1. Prompt sets — the queries you track, versioned.
2. Scheduler — cron or a queue; fires prompt × engine × run jobs.
3. Fetch layer — calls an API that returns structured AI answers (we'll use llmdata's endpoints; the pattern works with any provider that returns parsed sources).
4. Storage — append-only raw responses + normalized mention/citation tables.
5. Scoring + dashboard — SQL views over the normalized tables, charted.

Step 1: Build the prompt set

Prompts are your keyword list, but they behave differently. Users ask AI engines full questions with intent baked in ("best CRM for a 5-person real estate team"), and engines internally fan a query out into multiple searches. Practical rules:

• Write buyer-phrased prompts, not keywords. "crm real estate" becomes "what's the best CRM for real estate agents?"
• Cover the fan-out. For each head query, add 3–5 variants (cheapest X, X for small teams, X alternatives, is X worth it).
• Version the set. When you add or reword prompts, stamp a prompt_set_version — otherwise your trend lines silently break.
• Start small. 25 prompts × 3 engines × 3 runs × daily = 225 calls/day. Prove the loop before scaling to thousands.

Step 2: The fetch layer

Use one function per engine that returns a normalized record. With llmdata, ChatGPT, Perplexity and AI Overview endpoints share the same response shape — answer text, ordered sources, extracted entities — which keeps the normalizer thin:

import httpx, os

API = "https://api.llmdata.dev/v1/monitor"
HEADERS = {"Authorization": f"Bearer {os.environ['LLMDATA_API_KEY']}"}

def fetch(engine: str, prompt: str, country: str = "US") -> dict:
    # engine: "chatgpt" | "perplexity" | "google-ai-overview"
    key = "query" if engine.startswith("google") else "prompt"
    r = httpx.post(
        f"{API}/{engine}",
        headers=HEADERS,
        json={key: prompt, "country": country, "include": {"sources": True}},
        timeout=120,
    )
    r.raise_for_status()
    return r.json()["result"]

Note the one real asymmetry: AI Overview is attached to a Google query, so the payload key is query, and the answer may simply not exist — the response carries aiOverview.present: false, which is itself a data point worth storing (it tells you Google is expanding or shrinking AI Overview coverage in your niche).

def normalize(engine: str, prompt_id: int, run: int, result: dict) -> dict:
    if engine == "google-ai-overview":
        aio = result.get("aiOverview", {})
        return {
            "prompt_id": prompt_id, "engine": engine, "run": run,
            "present": aio.get("present", False),
            "text": aio.get("text", ""),
            "sources": aio.get("sources", []),
            "entities": [],
        }
    return {
        "prompt_id": prompt_id, "engine": engine, "run": run,
        "present": True,
        "text": result.get("text", ""),
        "sources": result.get("sources", []),
        "entities": result.get("entities", []),
    }

Step 3: Scheduling

Cron is enough for v1. Three practices that save pain later:

• Jitter your runs. Spread the 3–5 runs per prompt across the day rather than firing them back-to-back; you're sampling a distribution, and consecutive-second samples are more correlated.
• Idempotent job keys. (prompt_id, engine, date, run_index) as a unique key means retries never double-count.
• Persist failures too. A timeout is different from present: false; record both distinctly or your presence rates drift.

# crontab: three staggered daily passes
0 6  * * * /usr/bin/python /opt/tracker/run.py --run-index 0
0 13 * * * /usr/bin/python /opt/tracker/run.py --run-index 1
0 20 * * * /usr/bin/python /opt/tracker/run.py --run-index 2

Step 4: Storage schema

Two layers: keep raw JSON forever (interfaces change; you'll want to re-parse), normalize what you score on. Postgres/Supabase sketch:

create table prompts (
  id bigserial primary key,
  text text not null,
  prompt_set_version int not null default 1,
  country char(2) not null default 'US'
);

create table runs (
  id bigserial primary key,
  prompt_id bigint references prompts(id),
  engine text not null,          -- 'chatgpt' | 'perplexity' | 'google-ai-overview'
  run_index int not null,
  ran_at timestamptz not null default now(),
  status text not null,          -- 'ok' | 'error'
  answer_present boolean,        -- false = AIO didn't trigger
  raw jsonb not null,
  unique (prompt_id, engine, ran_at::date, run_index)
);

create table mentions (
  run_id bigint references runs(id),
  brand text not null,           -- from entities[] or your own matcher
  first_position int             -- 1 = first entity mentioned
);

create table citations (
  run_id bigint references runs(id),
  url text not null,
  domain text not null,
  position int not null          -- order in sources[]
);

The extraction job walks entities and sources from each run into mentions and citations. For brands not in the extracted entities, add a matcher over the answer text — case-insensitive alias list per tracked brand (yours and competitors').

Step 5: Scoring the AI rank tracker

All four metrics fall out as SQL views. Presence rate and citation share, over a trailing 7 days:

-- Presence rate per brand, engine, day
create view presence_rate as
select r.engine, m.brand, r.ran_at::date as day,
       count(distinct m.run_id)::float
         / nullif(count(distinct r.id), 0) as presence
from runs r
left join mentions m on m.run_id = r.id and m.brand = 'YourBrand'
where r.status = 'ok'
group by 1, 2, 3;

-- Citation share: your domain's slice of all citations
create view citation_share as
select r.engine, r.ran_at::date as day,
       count(*) filter (where c.domain = 'yourdomain.com')::float
         / nullif(count(*), 0) as share
from citations c
join runs r on r.id = c.run_id
group by 1, 2;

Mention position averages the same way over mentions.first_position. One warning: never average positions across runs where the brand was absent — compute position conditional on presence, and show presence separately, or a single missed run masquerades as a rank crash.

Step 6: Dashboard and alerts

You need less than you think for v1:

• Per engine, per prompt group: presence rate, citation share, and mention position as 7-day rolling lines.
• A "movers" table: prompts where presence changed by more than your run-to-run variance week over week.
• AI Overview trigger rate: % of tracked queries where present = true — clients consistently find this chart alone worth the build.
• Alerting: a scheduled job that posts to Slack when a competitor's presence rate crosses yours on any prompt group.

Grafana, Metabase, or a Next.js page over the views all work. If you're an agency, the same views feed a white-label PDF.

Costs, non-determinism, and honest caveats

• Run count dominates cost. The multiplier is prompts × engines × runs × frequency. Most teams over-buy frequency and under-buy runs; daily with 3 runs beats hourly with 1.
• Variance differs by engine. In practice, generated-answer engines (ChatGPT, Perplexity) vary more run-to-run than AI Overviews for the same query set — measure your own variance in week one and set alert thresholds above it.
• Interfaces move. If you scrape engines yourself with headless browsers, the parser is the product — budget permanent maintenance. That maintenance burden is the main thing a structured API buys you.

Where llmdata fits

Everything above is provider-agnostic. We built llmdata to be the fetch layer: one API across ChatGPT, Perplexity, AI Overviews, AI Mode, Gemini, Grok, Copilot and classic Google SERP, returning the same sources-with-positions structure this schema expects. Plainly: llmdata is in early access — if you're building a tracker and want to test it against your prompt set, join the waitlist.

Related reading: what gets cited in AI answers, tracking brand mentions in ChatGPT step-by-step, and SerpApi alternatives for AI search data if you're still choosing a data provider.