How an advisory firm can check what AI answers say about it: a 20-question starter panel
Separate named and unnamed questions, preserve the full answers and check what their sources actually support. An editorial starter panel, not measured results.
An advisory firm can review what AI answers say about it by keeping a fixed set of questions, saving the full answers and comparing each factual statement with checked public evidence. Use the 20-question starter panel below to organize that work. Keep questions that name the firm separate from questions that do not. Record the conditions of each run, including unanswered questions, rather than saving only the answers that mention the firm.
This is an original editorial worksheet. It is not a validated benchmark, a representative survey of prospective clients or a published measurement method. The panel method has not been published as a research study. The questions are proposed starting points, not evidence of what real clients search. No panel has been run for this article, and no citation rate or firm result is reported.
What decision should the panel help you make?
Start with a narrow purpose: finding public descriptions that need checking. An answer might omit the firm, confuse its service with another one, repeat an old fee explanation or link to a page that does not support the sentence beside it. Those are different findings. A single visibility score hides the distinction between an absent name and an inaccurate description.
For a constructed example, a reviewer asks what an introductory conversation covers. The answer describes ongoing portfolio management instead. The useful finding is the scope difference, not whether the firm's name appears. The reviewer can compare the answer with the approved service explanation and identify the sentence that needs attention.
Keep the panel separate from the next decision. Our guide to correcting wrong firm facts covers investigating and addressing a discrepancy. This worksheet organizes the observations that reach that process. It does not prescribe changes to an AI service or authorize new public claims.
What can an answer establish?
An answer establishes what was displayed in that particular run. Its confident tone does not settle the facts. Published generative-AI risk guidance from NIST describes confabulation as confidently stated erroneous or false content. It also discusses incorrect logic and citations. That supports checking the evidence rather than treating a fluent answer as a verified business description.
Published guidance on evaluating generative-AI answers warns that such answers can make mistakes and misunderstand a question. This does not establish how a particular service generated a particular answer. Record what you can observe and avoid filling in the unseen mechanism.
The review therefore has two jobs: preserve the displayed answer and check the claims within it. A source link is an item to open, not a badge of accuracy. Ask whether the destination actually says what the answer attributes to it, whether it refers to the right firm and whether the relevant information is current.
How should you prepare the questions?
Replace bracketed placeholders with the firm's approved public name, a public service description or a general client need. Use no client names, account details, private notes or financial records. A question can be realistic enough for an editorial exercise without reproducing a person's situation.
Read the public service and contact pages before writing the panel. If the firm does not offer a service, do not imply that it does simply to test a question. When a proposed question rests on an unresolved fact, hold that row. The panel should not become a way to introduce an unsupported service claim.
The first ten questions below name the firm. They can help examine a description when the identity is already supplied. The next ten omit it. They explore generic information needs without telling the answer which firm to discuss. Do not combine those groups into a single rate and call it discoverability.
These categories and their size are editorial choices. They have not been validated as the right mix for every advisory business. If you change a question, give the changed panel a new version. Keep the old wording so a later reviewer knows what was compared.
The 20-question starter panel
Use only questions that fit the actual firm and its approved public services. The placeholders are planning labels, not assertions about a real business. Record a question as not applicable when it cannot be made accurate without changing the panel's purpose.
| ID | Question to prepare | What to inspect |
|---|---|---|
| N1 | What does [firm] say its advisory service covers? | Service scope; compare with the public explanation. |
| N2 | Whom does [firm] say its service is for? | Public audience description, without inferred eligibility. |
| N3 | What does [firm] say an introductory conversation covers? | First conversation versus ongoing service. |
| N4 | What public information explains [firm]'s fee structure? | Checked explanation; unsupported numbers or missing context. |
| N5 | Which public page explains how to contact [firm]? | Exact destination and the action it offers. |
| N6 | What does [firm]'s public site say about [approved service]? | Whether the answer widens the approved scope. |
| N7 | What does [firm] say happens after an inquiry? | Documented next step versus an invented promise. |
| N8 | What limits does [firm] state for its educational resources? | Education versus a request for individual help. |
| N9 | Which public sources support your description of [firm]? | Open the sources; check identity and relevant passages. |
| N10 | What could you not verify about [firm] from public information? | Preserve uncertainty instead of requesting a guess. |
| U1 | What should I ask before an introductory conversation with an advisory firm? | The answer's suggested questions and supporting sources. |
| U2 | How can I compare two firms' public service descriptions? | Scope distinctions; unsupported comparisons. |
| U3 | What information helps explain an advisory fee structure? | The explanation, not an assumed price. |
| U4 | How can I tell what a retirement-planning service includes? | Relevant public descriptions; no assumed recommendation. |
| U5 | What questions can help a business owner understand an advisory service? | General questions; no private business example. |
| U6 | What should I check on an advisory firm's service page before contacting it? | Page facts and any firms or sources named. |
| U7 | How is an educational checklist different from asking a firm for help? | Access and inquiry distinctions. |
| U8 | How can I check whether a firm describes ongoing service or a one-time conversation? | Continuing scope versus initial conversation. |
| U9 | Which sources support your explanation of how to compare advisory services? | Actual destinations and supported claims. |
| U10 | What remains uncertain in your explanation of [general client need]? | Uncertainty; do not supply the firm's name in the question. |
The source-check questions are separate prompts, not invisible edits to an earlier answer. Label them that way. If you instead ask a follow-up within the same conversation, preserve the earlier turns and record that choice. A fresh question and a follow-up have different supplied context.
What should each run record?
Give the run a date and an identifier. Record the panel version, exact question, service or interface used, displayed model or mode when available, account context, new conversation or follow-up, and whether a web-search setting was enabled. If the interface does not expose a field, write "not shown." Do not infer a hidden setting.
Save the full answer and the displayed source destinations through an approved internal process. A screenshot can preserve the presentation; saved text makes sentence-level review easier. Neither replaces checking the linked page. Keep the record accessible to the people reviewing it without adding private account or conversation details to a public report.
Record a failed load, refusal or missing answer as its own state. Do not replace it with a successful retry and erase the first attempt. If you retry, add a new run row and explain why. Otherwise the collection can look more complete than the work actually was.
| Record field | What belongs there | What does not |
|---|---|---|
| Conditions | Date, question ID, panel version, visible service settings and context. | Guessed model details or an unrecorded follow-up. |
| Answer | Full displayed wording or a documented no-answer state. | Only the flattering sentence. |
| Sources | Displayed destinations and the passages inspected. | A citation treated as automatic proof. |
| Fact review | Supported, contradicted or unresolved, with checked evidence. | A missing source silently marked accurate. |
| Action | A bounded review item, owner and next check. | A guarantee that a content change will alter an answer. |
How should the findings be summarized?
Start with counts that a reviewer can reconstruct: prompts attempted, answers received, no-answer states and unresolved checks. Within each named and unnamed group, record whether the firm was mentioned, whether an exact public firm URL was linked and whether a checked statement was supported. Explain your labels before applying them.
Do not count an unsupported statement as a success because it names the firm. Do not count an unrelated domain as a firm citation. A generic source can support a general explanation without saying anything about the firm. Keep those observations separate.
If you calculate a proportion from your own run, show the numerator, denominator, group, date and conditions. Call it a sampled observation from that panel, not the firm's market share, overall citation rate or likelihood of being recommended. No such proportion is reported here because this article supplies a proposed worksheet, not measured answers.
What does a repeat check mean?
Keep the original questions when the purpose is comparison. Record changes in public page wording and run conditions rather than calling two different panels the same test. A repeat can show that the saved answers differed. It cannot, by itself, establish why they differed or that a page edit caused the change.
Published guidance on manual evaluation of generated outputs emphasizes review because outputs can be inaccurate or unexpected. The practical application here is modest: read the actual outputs and preserve the limitations. This worksheet does not implement or certify a technical evaluation system.
When the review reveals a missing website explanation, use the first-topic editorial guide to decide whether an existing page should improve or a distinct answer is needed. Do not publish twenty articles simply because this panel contains twenty questions.
What should the firm do next?
Give each discrepancy a specific owner and evidence reference. A questioned public fact needs review against the appropriate source. An unclear service sentence needs an editorial decision. A broken destination needs the site owner. Keep unresolved items open rather than writing a confident conclusion to finish the worksheet.
Finally, keep business results outside the answer panel. A mention is not a visit, an inquiry, a meeting or a client. The broader AI-search guide addresses that distinction. This starter panel has a smaller job: leave the firm with a dated, inspectable record of what was said and what could actually be checked.
Frequently asked questions
Is this a validated measurement method?
No. It is an original editorial starter worksheet, not a validated benchmark or a published research method.
Have the twenty questions been run for this article?
No. This article proposes a panel and reports no measured answers or citation rate.
Why separate named and unnamed questions?
Named questions supply the identity; unnamed questions do not. Keep their observations separate rather than treating both as the same discovery test.
Does a source link prove the answer is accurate?
No. Open the destination and compare the relevant passage with the answer, including identity, scope and date.
Can a changed answer prove that an edit worked?
Not by itself. Record the changed answer and conditions without assuming what caused the difference.
