Khoji documentation

Fetches one page through the SSRF guard, runs seventeen on-page checks (the original ten — title, description, h1, image alt, canonical, noindex, thin content — plus seven more: lang attribute, heading-order breaks, a canonical pointing elsewhere, mixed content on an https page, vague link text, incomplete Open Graph tags, and duplicate hreflang), stores the result, and — from the second run of the same URL onward — reports exactly what changed, flagging regressions separately from fixes. A bounded, robots.txt-respecting whole-site crawl now runs alongside the single-page audit — capped at 100 pages and depth 4, run inline so a caller can retry without wondering whether the first one is still going — and closes the four gaps the single-page audit could never see on its own: broken internal links, redirect chains, orphan pages that are in the sitemap but linked from nowhere, and site-wide duplicate titles.

Will not: Crawl past its own cap. A crawl stops at 100 pages and 4 levels deep; past that, result.truncated comes back true rather than the crawl quietly running forever against someone else's server. Fetch anything Core Web Vitals or load-time related as part of an audit or a crawl — that is a separate capability (site-speed-and-core-web-vitals-monitor) needing a free PageSpeed Insights key.

Fetches a sitemap (or an index of sitemaps) and the site's robots.txt, and reports every submitted URL that robots.txt then blocks — telling Google to index a page it is not allowed to read is the most common self-inflicted indexing mistake there is, and finding it needs no crawl. Khoji can also build a real sitemap.xml straight from one of its own crawls. Whether Google actually indexed a page is real arithmetic over pages someone has actually asked Khoji to inspect — POST .../pages/inspect and sitemap submission both go through the same Search Console OAuth connection google-search-console-insights uses.

Needs: a connected Google Search Console property (OAuth), for URL inspection and sitemap submission

Will not: Say whether Google indexed a page it was never asked to inspect — coverage is real arithmetic over pages this workspace has actually run through POST .../pages/inspect, not Google's whole-site index count. Fetch every URL in the sitemap to find which ones 404 from inside this route — that is what the bounded crawl (technical-site-audit) is for; firing hundreds of requests at a small business's shared hosting from inside one request was a deliberate design line not to cross.

Fifteen schema types — LocalBusiness, Product, FAQPage, Article, Service, Organization, WebSite, BreadcrumbList, Event, Recipe, JobPosting, HowTo, VideoObject, Review and AggregateRating — each generated only when its required properties are present and, if page text is supplied, only when name/headline/price/ratingValue/reviewCount actually appear on the page (digit-matching means ₹499 on the page satisfies 499 in the markup). The same rules run backwards over markup somebody else already put on a page.

Will not: Emit a value the supplied page text does not contain — this is a 400, not a warning. Guess which schema type a page needs — the type is always named in the request.

Needs no rank data at all: the owner's own declared page-to-keyword mapping is the input, and two pages declared for the same term is the conflict itself. Detection re-runs from those declarations on every read and recommends a primary page from the most recent crawl's word counts when one exists, never forcing a guess.

Will not: Merge pages or suppress the next scan automatically — resolving a flag is a person's decision recorded on that one detection; it does not silently stop re-flagging the same pages if they still conflict. Recommend a primary page without a recent crawl to base it on — with no crawl, recommended_primary_url comes back null rather than a guess.

Built entirely from a workspace's own crawl: a link graph (source, target, anchor text, in-degree per page) and a suggestion list matching under-linked pages to whichever other crawled page shares the most title words. A heuristic, not a model — no third party, no LLM — recomputed per request from one crawl's rows.

Will not: Work without a crawl to read — both routes take a crawl_id from a completed technical-site-audit crawl; there is no standalone link discovery outside of one. Call a language model to judge topical relevance — matching is by shared title words, a heuristic chosen for being fully explainable.

Scans any block of text — an agency proposal, a freelancer's plan, advice found online — for phrases that promise a ranking nobody can promise and for tactics named in Google's spam policies, and returns exactly which phrase triggered which finding and why.

Needs: a SERP data API, for benchmarking a draft against what is currently ranking

Will not: Score a draft against the pages currently ranking for a keyword — that needs a paid SERP data API this deployment does not have. Write or rewrite the content itself — it reviews text, it does not generate it.

Briefs are stored per keyword and feedback on them (useful / not useful, plus a comment) works fully. Generating the actual outline needs the pages currently ranking for the keyword and the real questions people ask around it — a SERP data API this deployment does not have. Writing an outline from a language model's own recollection instead, with no vendor connected, would be exactly the ungrounded, scaled content Google's spam policy targets, so Khoji refuses rather than fabricate one.

Needs: a SERP data API, to find what is currently ranking and what people are asking around a keyword, before an outline can be grounded in anything real

Will not: Generate an outline from a model's own recollection of a topic — every brief is created status: blocked and POST .../regenerate is a 409 naming the missing SERP data, never a plausible-looking outline with no source behind it. Hide that a brief is unusable — status and blocked_reason are on the brief itself, not buried in a log only an engineer would read.

A dedicated OAuth connection (separate from the drive.file connector any other agent might use) exchanges and refreshes a token through the platform's own generic connector framework. Once connected, Khoji pulls real Search Analytics rows — clicks, impressions, average position, by query and page — and writes rule-based findings from them: top query, a page with high impressions and near-zero clicks, a query sitting just outside page one (striking distance). No model is involved in any of it.

Needs: a Google Search Console connection for a property you have verified — start it from the Khoji dashboard, no technical setup beyond signing in with Google

Will not: Show anything before a workspace connects — GET /gsc/performance and GET /gsc/insights both refuse with a 409 naming the missing connection until POST /gsc/connect is called and OAuth consent is completed. Invent a finding — every insight in gsc/insights is a fixed template computed from rows actually synced from Search Console, never a model guessing at what the numbers might mean.

POST /vitals/check calls the real PageSpeed Insights v5 API and reads real-user field data (via CrUX) when Google has enough traffic on the URL to report it, falling back to a lab run otherwise — and always says plainly which one it is, rather than treating them as the same thing. Status is classified against Google's own 2024 Core Web Vitals thresholds, and a regression from the page's own previous reading writes an alert.

Needs: a PageSpeed Insights API key (free from Google Cloud Console) for a usable rate limit

Will not: Return anything reliably without a key — PageSpeed Insights works unauthenticated, but Google throttles that far below what a multi-tenant deployment can rely on, so an unconfigured deployment will see intermittent 502s rather than a stable check. Guess at Core Web Vitals from the page's own audit — this is a live call to Google's own measurement service, not a heuristic.

A canonical name/address/phone set once per workspace, citation entries checked against it, and an automatic match-status comparison (normalising whitespace, case and phone punctuation) that recomputes on every rescan. Discovery is manual, by design: you enter what a directory currently shows, Khoji does not scrape it — directory-by-directory scanning is high-maintenance and prone to breaking the moment a directory changes its markup. The comparison is what is automatic; the input is not.

Will not: Scan directories on its own — there is no crawler here reading Justdial, Google Business Profile or any other listing; you type in what the listing currently shows. Edit a listing on your behalf — mark-resolved records that you fixed it yourself; nothing here writes back to a directory.

A backlink the workspace already knows about is verified for real: Khoji fetches the claimed source page through the same SSRF-guarded client /audit uses and checks whether the target URL is still actually linked, raising new and lost alerts as that changes on every verification. This is verification, not discovery — finding backlinks nobody told Khoji about needs a backlink index API, since no search engine publishes a link graph and every commercial index is a vendor's own crawl.

Needs: a paid backlink-index provider, for discovering links nobody has entered manually

Will not: Discover a backlink on its own — GET /backlinks lists only what has been entered through POST /backlinks; there is no crawler here looking for who links to you. Mark a link active without actually fetching the source page — status changes only after a real fetch confirms or fails to confirm the link is still there.

Local keywords and manually-logged pack positions (bounded to Google's own pack size, 1–20) work locally. Profile-check is a real, first-party completeness score over fields the owner enters — name, address, phone, hours, category, description, photos — never framed as a ranking guarantee, since Google states none exists. Automated local-pack checking needs a SERP API with local-pack support; a live pull via the Google Business Profile API needs its own separate OAuth review and is not built.

Needs: a SERP API with local-pack support, for automated map-pack position checks; a live Google Business Profile pull is a separate, unbuilt OAuth integration

Will not: Pull your Google Business Profile automatically — GET /local/profile-check scores whatever fields you POST it; there is no live API read. Auto-check where you sit in the local pack — POST .../log accepts a position you actually observed; nothing here fetches one for you.

Projects, manually-entered candidate terms and shortlisting all work against Khoji's own database with no credential. Automatic expansion — turning one seed term into a list with real monthly search volume, a difficulty score and intent — needs a licensed keyword-data API (DataForSEO, SEMrush, Ahrefs and similar); Google itself does not publish search volume or difficulty for anyone to read for free.

Needs: a keyword-data API (e.g. DataForSEO, SEMrush, Ahrefs) for automatic expansion with real search volume and difficulty

Will not: Invent a search-volume or difficulty number — POST /keywords/expand refuses with a 409 naming the missing provider rather than returning a plausible-looking figure nobody measured. Stop you from tracking terms by hand — a project, its candidates and which ones are shortlisted all work today; only the automatic expansion step is blocked.

Competitor sets (a target domain plus up to 20 competitor domains, each validated) are stored and read against Khoji's own database with no credential. Generating an actual gap report needs a third-party SEO data API that indexes competitor ranking keywords — a business cannot pull Search Console data for a domain it does not own, so there is no first-party way to see what a competitor ranks for.

Needs: a paid SEO data API that indexes competitor ranking keywords, since Search Console only covers domains you own

Will not: Return an invented keyword list — every gap report on this deployment comes back status: blocked with entries: [] rather than a plausible list of terms nobody actually verified a competitor ranks for. Look at a domain's real traffic or backlinks to infer what it ranks for — there is no such shortcut; the data genuinely does not exist without the vendor.

Tracked keywords, deletion and history all work locally, and a person who checks Google themselves can log a real, dated position with POST .../log — genuine data, never a projection. Automated daily checking needs a paid or self-run SERP-fetch service; Google has no public rank-lookup API, and running an unlicensed scraper against Google is the tactic this product's own boundary and Google's terms both rule out.

Needs: a paid SERP-tracking provider, for automated daily position checks

Will not: Scrape Google itself to fill in a position — POST .../check refuses with a 409 naming the missing provider instead. Treat a manually-logged position as anything other than exactly what you saw — source: manual on every self-logged row, kept separate from an automated source that does not exist here.

Storage and the opportunity-extraction logic — which feature types (featured snippet, people-also-ask, image pack) are page-winnable and what format each one wants — are real and exercised directly against test data. Nothing populates the table on this deployment: it piggybacks on the same SERP data API rank-tracking needs, so both routes honestly return empty rather than an invented opportunity list.

Needs: the same SERP data API rank-position-tracker needs — this piggybacks on that fetch rather than requiring a second one

Will not: Fabricate a snippet opportunity — with no SERP snapshots stored, both GET /serp-features and GET /serp-features/opportunities return an empty items list and say why, not a guess. Guarantee winning a featured snippet even once connected — Google selects and can remove snippets algorithmically without notice; this can only ever be framed as an opportunity, never a promise.

Generating and reading a report is complete with zero setup: it aggregates whatever the other capabilities have already stored — crawl issues, tracked-keyword snapshots, verified backlinks, synced Search Console totals, citation mismatches — and every section carries its own data_as_of (or null) so a source that has never been synced reads as stale, not as a silent zero.

Needs: ZeptoMail (ZEPTOMAIL_API_KEY, a verified sending address) only for emailing a report — already configured on this deployment, so scheduling works today; generating and reading a report need nothing

Will not: Invent a number for a section nobody has populated — an unsynced source shows data_as_of: null instead of a zero that could be mistaken for genuinely no activity. Run on a schedule of its own — POST .../schedule sends the report once, right now; there is no recurring cron wired up for Khoji yet, so a repeat send means asking again.

Workspace member. A page that will not load is reported, not raised as an error: the one issue code http-error carries the status code, and it is stored and diffed like any other result.

{ "url": "https://example.in/pricing", "store": true }

{ "audit_id": "3fa8…", "url": "https://example.in/pricing", "status_code": 200,
  "page": { "title": "Pricing — Example", "meta_description": "…", "canonical": "…",
            "h1s": ["Pricing"], "lang": "en-IN", "word_count": 812,
            "images_total": 6, "images_without_alt": 2, "existing_schema_types": ["Product"] },
  "issues": [ { "code": "images-no-alt", "severity": "warning",
                "message": "2 of 6 images have no alt text.", "fix": "…" } ],
  "summary": { "error": 0, "warning": 1, "note": 2 },
  "compared_with": "2026-08-10T09:12:00+00:00",
  "changes": [ { "field": "robots", "message": "The robots meta tag changed.",
                 "before": "", "after": "noindex", "regression": true } ] }

Workspace member. Fewer than two stored audits of this URL returns comparable:false and a note to run it again — nothing is fetched.

{ "url": "…", "comparable": true, "before": "2026-08-10T09:12:00+00:00",
  "after": "2026-08-16T07:03:00+00:00",
  "changes": [ { "field": "robots", "message": "The robots meta tag changed.",
                 "before": "", "after": "noindex", "regression": true } ],
  "regressions": [ … only the entries above with regression: true … ] }

Workspace member. Fetches one page through the SSRF guard, runs seventeen on-page checks (the original ten — title, description, h1, image alt, canonical, noindex, thin content — plus seven more: lang attribute, heading-order breaks, a canonical pointing elsewhere, mixed content on an https page, vague link text, incomplete Open Graph tags, and duplicate hreflang), stores the result, and — from the second run of the same URL onward — reports exactly what changed, flagging regressions separately from fixes. A bounded, robots.txt-respecting whole-site crawl now runs alongside the single-page audit — capped at 100 pages and depth 4, run inline so a caller can retry without wondering whether the first one is still going — and closes the four gaps the single-page audit could never see on its own: broken internal links, redirect chains, orphan pages that are in the sitemap but linked from nowhere, and site-wide duplicate titles.

{ "url": "https://example.in" }

{ "origin": "https://example.in", "robots_txt_found": true, "status_code": 200,
  "crawlers": [ { "agent": "Googlebot", "allowed": true }, { "agent": "GPTBot", "allowed": false }, … 13 total ],
  "blocked_agents": ["GPTBot", "CCBot"], "sitemaps": ["https://example.in/sitemap.xml"],
  "note": "2 of the named crawlers are turned away by this file." }

Workspace member. max_pages and max_depth are both caller-set, capped at khoji_crawl.MAX_PAGES_CEILING and MAX_DEPTH_CEILING — a 400 names the ceiling if either is exceeded. Passing sitemap_url lets orphan-page detection compare the crawl against pages the sitemap claims but nothing on the site links to.

{ "root_url": "https://example.in", "max_pages": 100, "max_depth": 4 }

{ "id": "9b1c…", "root_url": "https://example.in", "status": "completed",
  "page_count": 87, "error_count": 2, "warning_count": 14, "note_count": 5,
  "truncated": false,
  "pages": [ { "url": "https://example.in/pricing", "depth": 1, "status_code": 200, … } ],
  "issues": [ { "code": "orphan-page", "severity": "warning",
                "message": "In the sitemap but not linked from any crawled page.", … },
              { "code": "duplicate-title", "severity": "warning", … },
              { "code": "redirect-chain", "severity": "note", … } ] }

Workspace member. Fetches one page through the SSRF guard, runs seventeen on-page checks (the original ten — title, description, h1, image alt, canonical, noindex, thin content — plus seven more: lang attribute, heading-order breaks, a canonical pointing elsewhere, mixed content on an https page, vague link text, incomplete Open Graph tags, and duplicate hreflang), stores the result, and — from the second run of the same URL onward — reports exactly what changed, flagging regressions separately from fixes. A bounded, robots.txt-respecting whole-site crawl now runs alongside the single-page audit — capped at 100 pages and depth 4, run inline so a caller can retry without wondering whether the first one is still going — and closes the four gaps the single-page audit could never see on its own: broken internal links, redirect chains, orphan pages that are in the sitemap but linked from nowhere, and site-wide duplicate titles.

{ "items": [ … one row per issue, same shape as a single-page audit issue … ],
  "total": 21, "summary": { "error": 2, "warning": 14, "note": 5 } }

Workspace member. Fetches a sitemap (or an index of sitemaps) and the site's robots.txt, and reports every submitted URL that robots.txt then blocks — telling Google to index a page it is not allowed to read is the most common self-inflicted indexing mistake there is, and finding it needs no crawl. Khoji can also build a real sitemap.xml straight from one of its own crawls. Whether Google actually indexed a page is real arithmetic over pages someone has actually asked Khoji to inspect — POST .../pages/inspect and sitemap submission both go through the same Search Console OAuth connection google-search-console-insights uses.

{ "url": "https://example.in/sitemap.xml" }

{ "origin": "https://example.in", "found": true, "sitemap_url": "…/sitemap.xml",
  "is_index": false, "child_sitemaps": [], "url_count": 214,
  "urls": [ …first 200… ], "blocked_by_robots": [ { "url": "…/admin/x", "path": "/admin/x" } ],
  "blocked_by_robots_count": 1,
  "note": "1 URL(s) are in the sitemap and blocked by robots.txt. That tells Google to index a page it is not allowed to read." }

Workspace member. Needs GOOGLE_SEARCH_CONSOLE_CLIENT_ID and GOOGLE_SEARCH_CONSOLE_CLIENT_SECRET set on the deployment, plus this workspace completing OAuth consent at POST /api/v1/oauth/google_search_console/start for a property it has verified in Search Console. Without that, this call is a 409 naming the missing connection.

{ "url": "https://example.in/pricing" }

{ "url": "https://example.in/pricing", "coverage_state": "indexed",
  "last_crawl_time": "2026-08-15T04:00:00Z", "inspected_at": "2026-08-19T09:00:00+00:00" }

Workspace member. Recomputed from whatever pages have actually been inspected — not Google's whole-site count.

{ "indexed": 41, "not_indexed": 3, "unknown": 5, "last_recomputed_at": "…" }

Workspace member. Omit a required property (LocalBusiness needs name and address) and this is a 400 naming it. Supply a value the page_text does not contain and it is a 400 naming that instead.

{ "type": "LocalBusiness",
  "values": { "name": "Example Salon", "address": "12 MG Road, Pune" },
  "page_text": "Example Salon — 12 MG Road, Pune. Open 10am–8pm." }

{ "type": "LocalBusiness",
  "json_ld": { "@context": "https://schema.org", "@type": "LocalBusiness",
               "name": "Example Salon", "address": "12 MG Road, Pune" },
  "note": "Paste this into a <script type=\"application/ld+json\"> tag on that page. …" }

Workspace member. Runs the generator's own refusal rules in reverse, against markup Khoji did not write. This is where a rating a page never displays is usually found.

{ "html": "<html>…page with a Product JSON-LD block already on it…</html>" }

{ "types_found": ["Product"], "blocks_found": 1,
  "issues": [ { "code": "schema-not-on-page", "severity": "error",
                "message": "The structured data claims things the page does not show: ratingValue=4.8.",
                "fix": "…" } ],
  "summary": { "error": 1, "warning": 0, "note": 0 },
  "note": "Structured data has to match what a visitor can see." }

Workspace member. Fifteen schema types — LocalBusiness, Product, FAQPage, Article, Service, Organization, WebSite, BreadcrumbList, Event, Recipe, JobPosting, HowTo, VideoObject, Review and AggregateRating — each generated only when its required properties are present and, if page text is supplied, only when name/headline/price/ratingValue/reviewCount actually appear on the page (digit-matching means ₹499 on the page satisfies 499 in the markup). The same rules run backwards over markup somebody else already put on a page.

{ "types": [ { "value": "LocalBusiness", "required": ["name","address"],
               "use_for": "a shop, salon, clinic or restaurant with a physical address" }, … 15 total ] }

Workspace member. The owner's own statement of what a page targets — repeat submissions of the same pair are idempotent.

{ "url": "https://example.in/services/hair-color", "keyword": "hair colour salon pune" }

{ "id": "…", "url": "https://example.in/services/hair-color", "keyword": "hair colour salon pune" }

Workspace member. Recomputed fresh from the workspace's own declared targets on every read, and upserted rather than duplicated on repeat calls.

{ "items": [ { "id": "…", "keyword": "hair colour salon pune",
               "page_urls": ["…/services/hair-color", "…/blog/best-hair-salon-pune"],
               "recommended_primary_url": "…/services/hair-color", "status": "open" } ], "total": 1 }

Workspace member. Built entirely from a workspace's own crawl: a link graph (source, target, anchor text, in-degree per page) and a suggestion list matching under-linked pages to whichever other crawled page shares the most title words. A heuristic, not a model — no third party, no LLM — recomputed per request from one crawl's rows.

{ "nodes": [ { "url": "…/pricing", "in_degree": 4 } ],
  "edges": [ { "source_url": "…/home", "target_url": "…/pricing", "anchor_text": "See pricing" } ] }

Workspace member. Only pages the crawl reached with a 2xx/3xx are considered as either a target or a source.

{ "items": [ { "target_url": "…/services/hair-color", "in_degree": 0,
               "suggested_source_url": "…/blog/best-hair-salon-pune",
               "shared_title_words": ["hair", "salon"] } ], "total": 3 }

Workspace member. Scans any block of text — an agency proposal, a freelancer's plan, advice found online — for phrases that promise a ranking nobody can promise and for tactics named in Google's spam policies, and returns exactly which phrase triggered which finding and why.

{ "text": "We guarantee a #1 ranking within 30 days and can arrange a few link exchanges." }

{ "held_for_review": true,
  "findings": [
    { "kind": "ranking_promise", "text": "guarantee a #1 ranking", "reason": "Nobody can promise a ranking or a traffic number — Google says placement cannot be bought or guaranteed. …" },
    { "kind": "spam_tactic", "text": "link exchange", "reason": "Link schemes is named in Google's spam policies. Following this can get the site penalised or removed entirely." }
  ],
  "summary": "2 things here would worry me." }

Workspace member. Briefs are stored per keyword and feedback on them (useful / not useful, plus a comment) works fully. Generating the actual outline needs the pages currently ranking for the keyword and the real questions people ask around it — a SERP data API this deployment does not have. Writing an outline from a language model's own recollection instead, with no vendor connected, would be exactly the ungrounded, scaled content Google's spam policy targets, so Khoji refuses rather than fabricate one.

{ "keyword": "hair colour salon pune" }

{ "id": "…", "keyword": "hair colour salon pune", "status": "blocked",
  "blocked_reason": "Generating an outline needs a SERP data API, which is not connected on this deployment." }

Workspace member. Always a 409 on this deployment — there is no partial or cached outline to fall back to.

{ "detail": { "message": "…needs a SERP data API…", "reason": "requires_configuration", "missing": "a SERP data API" } }

Workspace member. Briefs are stored per keyword and feedback on them (useful / not useful, plus a comment) works fully. Generating the actual outline needs the pages currently ranking for the keyword and the real questions people ask around it — a SERP data API this deployment does not have. Writing an outline from a language model's own recollection instead, with no vendor connected, would be exactly the ungrounded, scaled content Google's spam policy targets, so Khoji refuses rather than fabricate one.

{ "rating": "not_useful", "comment": "Still blocked, nothing to rate yet." }

{ "id": "…", "brief_id": "…", "rating": "not_useful", "comment": "Still blocked, nothing to rate yet." }

Workspace member. This route only records which verified property to read; the actual OAuth handshake is the platform's shared connector flow at /api/v1/oauth/google_search_console/start, not reimplemented here.

{ "site_url": "sc-domain:example.in" }

{ "id": "…", "site_url": "sc-domain:example.in", "oauth_connected": false,
  "note": "The property is saved, but Google account access is not connected yet. Complete OAuth at POST /api/v1/oauth/google_search_console/start, then retry (reason: not_connected)." }

Workspace member. Fetch-then-store, the same shape /audit uses for one page applied to Search Console: pulls fresh rows if the token is live, then always reports from what is actually stored, with a note explaining a failed sync rather than a silent gap.

{ "connection": { "site_url": "sc-domain:example.in", "last_synced_at": "…" },
  "oauth_connected": true, "window_days": 28,
  "totals": { "clicks": 412, "impressions": 9840 }, "rows_stored": 331 }

Workspace member. A dedicated OAuth connection (separate from the drive.file connector any other agent might use) exchanges and refreshes a token through the platform's own generic connector framework. Once connected, Khoji pulls real Search Analytics rows — clicks, impressions, average position, by query and page — and writes rule-based findings from them: top query, a page with high impressions and near-zero clicks, a query sitting just outside page one (striking distance). No model is involved in any of it.

{ "items": [ { "kind": "zero_click_high_impression",
               "message": "\"salon near me\" had 640 impressions and 2 clicks in the last 28 days.",
               "metric_ref": "salon near me" } ], "total": 4 }

Workspace member. Without PAGESPEED_API_KEY configured, Google's unauthenticated rate limit is the likely cause of an intermittent 502 on a busy deployment — the error response says so by name rather than presenting a bare failure.

{ "page_url": "https://example.in/pricing" }

{ "id": "…", "page_url": "https://example.in/pricing",
  "checked_at": "2026-08-19T09:00:00+00:00",
  "lcp_ms": 2450, "inp_ms": 180, "cls": 0.08, "status": "needs_improvement",
  "data_source": "field_crux" }

Workspace member. Only written when a later check's status is worse than the page's previous one.

{ "items": [ { "page_url": "…/pricing", "metric": "overall",
               "from_status": "good", "to_status": "needs_improvement",
               "message": "…/pricing got slower: good -> needs_improvement." } ], "total": 1 }

Workspace member. A canonical name/address/phone set once per workspace, citation entries checked against it, and an automatic match-status comparison (normalising whitespace, case and phone punctuation) that recomputes on every rescan. Discovery is manual, by design: you enter what a directory currently shows, Khoji does not scrape it — directory-by-directory scanning is high-maintenance and prone to breaking the moment a directory changes its markup. The comparison is what is automatic; the input is not.

{ "name": "Example Salon", "address": "12 MG Road, Pune", "phone": "+91 98765 43210" }

{ "name": "Example Salon", "address": "12 MG Road, Pune", "phone": "+91 98765 43210", "updated_at": "…" }

Workspace member. match_status is computed immediately from the canonical identity — mismatch, match or partial — never left pending.

{ "directory_name": "Justdial", "url": "https://justdial.com/…",
  "listed_name": "Example Salon", "listed_address": "12, MG Rd Pune", "listed_phone": "9876543210" }

{ "id": "…", "directory_name": "Justdial", "match_status": "mismatch", "resolved": false }

Workspace member. Recomputes match_status for every stored citation against whatever the canonical identity is right now — useful right after you correct the identity or update a listing by hand.

{ "checked": 6, "status_changed": 1 }

Workspace member. Verified synchronously on add — Khoji fetches source_url right away and checks it actually links to target_url.

{ "source_url": "https://blog.example.com/best-salons-pune", "target_url": "https://example.in/pricing" }

{ "id": "…", "source_url": "…", "target_url": "…", "status": "active", "last_checked_at": "…" }

Workspace member. Re-fetches the source page now; a link that used to be there and is not anymore flips status to lost and writes a BacklinkAlert.

{ "id": "…", "status": "lost", "last_checked_at": "…" }

Workspace member. A backlink the workspace already knows about is verified for real: Khoji fetches the claimed source page through the same SSRF-guarded client /audit uses and checks whether the target URL is still actually linked, raising new and lost alerts as that changes on every verification. This is verification, not discovery — finding backlinks nobody told Khoji about needs a backlink index API, since no search engine publishes a link graph and every commercial index is a vendor's own crawl.

{ "items": [ … ], "total": 4,
  "note": "Backlinks are verified on request, not discovered — Khoji checks a link the workspace already told it about, it does not crawl the web looking for new ones. Discovery needs a backlink index API." }

Workspace member. Local keywords and manually-logged pack positions (bounded to Google's own pack size, 1–20) work locally. Profile-check is a real, first-party completeness score over fields the owner enters — name, address, phone, hours, category, description, photos — never framed as a ranking guarantee, since Google states none exists. Automated local-pack checking needs a SERP API with local-pack support; a live pull via the Google Business Profile API needs its own separate OAuth review and is not built.

{ "term": "hair salon near me", "target_location": "Koregaon Park, Pune" }

{ "id": "…", "term": "hair salon near me", "target_location": "Koregaon Park, Pune" }

Workspace member. A real, deterministic completeness score over fields you supplied — never presented as a ranking prediction.

{ "fields": { "name": "Example Salon", "address": "12 MG Road, Pune", "phone": "+91…",
              "hours": "10am-8pm", "category": "Hair Salon", "description": "", "photos": 3 } }

{ "checked": true, "completeness_score": 71, "missing_fields": ["description"] }

Workspace member. Projects, manually-entered candidate terms and shortlisting all work against Khoji's own database with no credential. Automatic expansion — turning one seed term into a list with real monthly search volume, a difficulty score and intent — needs a licensed keyword-data API (DataForSEO, SEMrush, Ahrefs and similar); Google itself does not publish search volume or difficulty for anyone to read for free.

{ "name": "Salon services", "seed_term": "hair salon pune" }

{ "id": "…", "name": "Salon services", "seed_term": "hair salon pune" }

Workspace member. monthly_search_volume, difficulty_score and serp_features stay null on a manually-added term — nobody measured them.

{ "term": "hair colour salon pune", "intent_label": "commercial" }

{ "id": "…", "project_id": "…", "term": "hair colour salon pune", "source": "manual", "shortlisted": false }

Workspace member. 409 on every call on this deployment — nothing is invented in its place.

{ "project_id": "…", "seed_terms": ["hair salon pune"] }

{ "detail": { "message": "…needs a keyword-data API…", "reason": "requires_configuration", "missing": "a keyword-data API (e.g. DataForSEO, SEMrush, Ahrefs)" } }

Workspace member. At most 20 competitor domains per set, each validated as a real domain before it is stored.

{ "target_domain": "example.in", "competitor_domains": ["rival1.in", "rival2.in"] }

{ "id": "…", "target_domain": "example.in", "competitor_domains": ["rival1.in", "rival2.in"] }

Workspace member. Competitor sets (a target domain plus up to 20 competitor domains, each validated) are stored and read against Khoji's own database with no credential. Generating an actual gap report needs a third-party SEO data API that indexes competitor ranking keywords — a business cannot pull Search Console data for a domain it does not own, so there is no first-party way to see what a competitor ranks for.

{ "id": "…", "set_id": "…", "status": "blocked", "note": "…needs a third-party SEO data API…", "entries": [] }

Workspace member. Tracked keywords, deletion and history all work locally, and a person who checks Google themselves can log a real, dated position with POST .../log — genuine data, never a projection. Automated daily checking needs a paid or self-run SERP-fetch service; Google has no public rank-lookup API, and running an unlicensed scraper against Google is the tactic this product's own boundary and Google's terms both rule out.

{ "term": "hair salon pune", "target_location": "Pune", "device": "mobile" }

{ "id": "…", "term": "hair salon pune", "target_location": "Pune", "device": "mobile", "active": true }

Workspace member. A real, dated snapshot of what you personally saw searching Google — this is the honest way rank data gets into Khoji today.

{ "position": 7, "url": "https://example.in/pricing" }

{ "id": "…", "position": 7, "url": "https://example.in/pricing", "source": "manual" }

Workspace member. Tracked keywords, deletion and history all work locally, and a person who checks Google themselves can log a real, dated position with POST .../log — genuine data, never a projection. Automated daily checking needs a paid or self-run SERP-fetch service; Google has no public rank-lookup API, and running an unlicensed scraper against Google is the tactic this product's own boundary and Google's terms both rule out.

{ "detail": { "message": "…needs a SERP data API…", "reason": "requires_configuration", "missing": "a SERP data API" } }

Workspace member. Storage and the opportunity-extraction logic — which feature types (featured snippet, people-also-ask, image pack) are page-winnable and what format each one wants — are real and exercised directly against test data. Nothing populates the table on this deployment: it piggybacks on the same SERP data API rank-tracking needs, so both routes honestly return empty rather than an invented opportunity list.

{ "items": [], "total": 0,
  "note": "No SERP feature data yet. This piggybacks on the same SERP data API rank-tracking needs, which is not connected on this deployment." }

Workspace member. Would return, once populated, a keyword plus a recommended content format for the feature type it could realistically win.

{ "items": [], "total": 0 }

Workspace member. Generating and reading a report is complete with zero setup: it aggregates whatever the other capabilities have already stored — crawl issues, tracked-keyword snapshots, verified backlinks, synced Search Console totals, citation mismatches — and every section carries its own data_as_of (or null) so a source that has never been synced reads as stale, not as a silent zero.

{ "sections": ["crawl_issues", "rank_tracking", "backlinks", "search_console"], "range_days": 30 }

{ "id": "…", "sections": ["crawl_issues","rank_tracking","backlinks","search_console"],
  "range_days": 30,
  "snapshot": { "crawl_issues": { "data_as_of": "2026-08-18T…", "error": 2, "warning": 14 },
                "rank_tracking": { "data_as_of": null, "note": "No tracked keyword has a logged position yet." },
                "backlinks": { "data_as_of": "2026-08-10T…", "active": 12, "lost": 1 },
                "search_console": { "data_as_of": "2026-08-19T…", "clicks": 412 } },
  "generated_at": "2026-08-19T09:05:00+00:00" }

Workspace member. Sends now, through the platform's already-configured ZeptoMail sender — a genuinely recurring send is not wired up.

{ "to_email": "owner@example.in" }

{ "id": "…", "delivered_to": "owner@example.in", "delivery_status": "sent", "email_request_id": "…" }