Live test run — working principle & results

This page explains how Advisor works end-to-end, then records a real run against QueryCraft + OpenAI (restaurant shop id 11, 2026-07-22).

Related guides: Pipeline · Sessions · Postman · Redoc

---

Working principle

Advisor is not “one chat call.” It is a staged pipeline. Each stage has its own HTTP API so you can test it alone.

Owner problem (plain language)
        │
        ▼
┌───────────────────┐
│ 1. Auth           │  Who is the restaurant? → principalId / shop_id
└─────────┬─────────┘
          ▼
┌───────────────────┐
│ 2. Catalog        │  Load playbooks + recommendations (YAML, no LLM)
└─────────┬─────────┘
          ▼
┌───────────────────┐
│ 3. Classify       │  Problem text → 1–2 playbook_ids (OpenAI purpose=classify)
└─────────┬─────────┘
          ▼
┌───────────────────┐
│ 4. Query (optional)│  Single analytics ask — same engine the agent uses
│    /api/query/ask │  → QueryCraft POST /query/nlp (can take 15–120s)
└─────────┬─────────┘
          ▼
┌───────────────────┐
│ 5. Advisor session│  Agent loop (OpenAI purpose=investigate + tools)
│    start / answers│
└─────────┬─────────┘
          ▼
   concluded diagnosis
   (or awaiting_manager → resume with answers)

What the agent actually does

Inside POST /api/advisor/sessions the investigate model may call tools repeatedly:

Tool	Purpose
query_data	Ask QueryCraft (counts toward query_count, default max 8)
ask_manager	Pause session → status: awaiting_manager (max 3)
update_hypothesis	Move items on the hypothesis board
select_playbook	Switch / add playbook focus
conclude_diagnosis	Emit finding, evidence, actions, verification window

Prompt blocks:

Block	Role
A	Method, tone, hard rules (src/application/agent/block-a.ts)
B	This restaurant’s domains / channels (tenant + data-domains)
C	Selected playbook(s) + live hypothesis board

Important distinction

Call	What it proves
POST /api/query/ask	QueryCraft NLP + SQL + rows only
POST /api/advisor/sessions	Full Advisor: classify → tool loop → diagnosis

A successful query ask does not mean Advisor is configured. Advisor needs LLM_ADAPTER=openai (or mock for offline demos) and usually QUERY_ADAPTER=querycraft for live numbers.

---

Env used for this live run

AUTH_ADAPTER=querycraft
QUERY_ADAPTER=querycraft
LLM_ADAPTER=openai
OPENAI_ENABLED=true
CATALOG_ADAPTER=yaml
QUERYCRAFT_ENABLED=true
QUERYCRAFT_URL=https://querycraft.ncrts.pro
QUERYCRAFT_TIMEOUT_MS=120000

Notes:

• QueryCraft NLP often needs 1–2 minutes; keep QUERYCRAFT_TIMEOUT_MS ≥ 120000 or the Advisor aborts with This operation was aborted.
• With LLM_ADAPTER=mock, sessions return a scripted diagnosis and do not exercise a real investigate loop.

Confirm wiring:

GET /api/health

Expect adapters.llm = openai, adapters.query = querycraft, thirdParty.querycraft.timeoutMs = 120000.

---

Live run (shop 11) — step by step

Problem statement used throughout:

{ "problem_statement": "sales are down across the board" }

Step A — Auth

POST /api/auth/login → Bearer token
GET /api/auth/me

Field	Value
principalId / shop	11
Display name	Jessica Parker
Roles	restaurant, admin

All later QueryCraft SQL is scoped to this shop.

Step B — Classify (Advisor brain, cheap model)

POST /api/classify

Field	Result
Time	~3s
playbook_ids	["demand-decline"]
source	llm
adapters.llm	openai

Step C — Query alone (QueryCraft only — not full Advisor)

POST /api/query/ask

{ "question": "What were total sales yesterday?" }

Field	Result
HTTP	200 in ~25s
kind	data
Rows	sales_amount / formatted_sales ≈ $41.48
Scope	shop_id = 11, order_date = '2026-07-21'

This validates data access. It does not produce a diagnosis.

Step D — Start Advisor session (full loop)

POST /api/advisor/sessions with the same problem statement.

Field	Result
HTTP	201 in ~260s
session_id	0mrw7fpnsf70b794c05585ce7f
status / stop	awaiting_manager
playbook_ids	demand-decline
query_count	4 (four live QueryCraft tool calls)
ask_count	1
diagnosis	(none yet)

Hypothesis board after data probes

Hypothesis	Status	Note (short)
seasonality_not_decline	ruled_out	YoY/recent drop looks real, not seasonality
retention_leaking	investigating	Cohort repeat high but sample tiny
acquisition_dried_up	inconclusive	New-vs-returning query failed
channel_visibility_loss	ruled_out	Decline across channels

Pending manager questions

1. dq-01 — Has discovery changed (ads, signage, neighbor traffic)?
2. dq-03 — Location changes (construction, parking, competitor)?

Step E — Resume with answers

POST /api/advisor/sessions/{id}/answers

Sample answers used in the test:

{
  "answers": [
    {
      "probe_id": "dq-01",
      "answer": "We paused Facebook ads about 6 weeks ago and have not run any promos since."
    },
    {
      "probe_id": "dq-03",
      "answer": "Street construction started nearby 2 months ago; parking is harder. No new competitor that we know of."
    }
  ]
}

Field	Result
HTTP	200 in ~9s
status / stop	concluded
query_count	still 4
verify_after	2026-08-19

Updated board: acquisition_dried_up → supported (ads paused + no promos).

Diagnosis (summary)

Field	Value
Finding	Decline driven mainly by weaker acquisition after ads/promos stopped
Confidence	confirmed
Confirmed hypothesis	acquisition_dried_up
Action	local-visibility-audit (revive local visibility / marketing)
Verification	check_after_days: 28, probe dv-01

Step F — Snapshot

GET /api/advisor/sessions/0mrw7fpnsf70b794c05585ce7f
Returns the same concluded state (does not re-run the agent).

---

Postman order for this path

1. GET {{baseUrl}}/api/health — confirm openai + querycraft + timeout
2. POST {{baseUrl}}/api/auth/login — save token
3. POST {{baseUrl}}/api/classify — problem statement
4. (optional) POST {{baseUrl}}/api/query/ask — prove data
5. POST {{baseUrl}}/api/advisor/sessions — wait several minutes; save session_id
6. If awaiting_manager → POST .../sessions/{{sessionId}}/answers
7. GET .../sessions/{{sessionId}}

Set Postman request timeout ≥ QUERYCRAFT_TIMEOUT_MS (e.g. 120000+ ms). A session with several query_data calls can take 5–10+ minutes.

---

Failure modes we hit while building this run

Symptom	Cause	Fix
502 … operation was aborted ~15s	QUERYCRAFT_TIMEOUT_MS=15000	Raise to 120000+, restart
Session concludes in ~1s with “Mock diagnosis…”	LLM_ADAPTER=mock	Set openai + OPENAI_ENABLED=true
Query ask works, “Advisor does nothing real”	Only step 4 tested	Call /api/advisor/sessions with OpenAI

---

What is still stubbed

• Scheduled verification job (~14–28 days after verify_after)
• Notify webhook when verification is due
• SSE streaming of tool events (responses are sync JSON today)

See Tweaking to change prompts, playbooks, models, and caps.

Day chronicle: Work log — 2026-07-22.