Automate observation and connect it to the next revision

Prevent observation tools from misclassifying results

A site’s text and external search results and answers change at different times. A single manual check cannot readily track whether new material was reflected after publication or old material was reused later. Observing fixed URLs and questions at set intervals lets you compare when changes happened and their results. If automated collection saves another answer or an earlier screen, you may fix the wrong cause, so preserve both the original and the object being assessed.

Automated indexing checks compare the URL entered with the URL assessed on the result screen. If the previous URL’s result remains or the response is unfinished, save it as unknown. Distinguish negatives, pending states, and unknowns first, so “indexing not completed” is not marked successful simply because it contains “indexing completed.” If the screen language or wording changes, preserve the original result and review the assessment rules.

Limit AI citation extraction to the completed answer to the current question and that answer’s sources area. Distinguish citation links from a user-entered URL, previous sidebar conversations, text the model is still writing, and mere name mentions. Store citations to the site as a whole and to the target detail page as separate values.

If the source list exceeds storage limits or part of a response is truncated, record the missing range. Do not conclude “no citations” when you have not checked the whole response. Treat access failures, login requirements, service restrictions, unfinished answers, and uninterpretable results as unknown values with their respective reasons. If historical records lacked accuracy or timeliness assessments, do not retroactively fill them as correct when adding those fields.

The Google URL Inspection API returns the state of the version stored in the index. Compare its last crawl date and selected canonical URL with the current page. The Google Indexing API notifies updates and deletions for JobPosting pages and BroadcastEvent pages within VideoObject. Give ordinary information pages crawl paths through sitemaps and internal links. URL Inspection · Indexing API scope

Manage scheduled observation volume and completion

Before automating observations, calculate request volume from the number of target URLs, search services, questions, AI services, and repetitions. With U URLs for indexing checks, E engines, Q AI questions, A services, and R repetitions, the basic requests per round are U×E + Q×A×R. Retries and requests used while waiting for responses add to that.

Separate cache refreshes at publication from external search and AI observations. Rerunning every question on each publication can repeat identical requests in a short period. Refresh target lists on a fixed schedule, identify changed URLs, and run the necessary observations. Operators set the interval; adding pages must not automatically increase frequency.

Give jobs keys identifying service, URL, question version, and observation round. Manage lease deadlines and completion records so multiple runners do not claim the same work. Define retry counts, intervals, and stop conditions. On a rate-limit response, delay that service’s next execution. When the waiting deadline expires, save the incomplete status and next retry time.

The administration screen separately shows registered jobs, actual attempts, normal responses, failures and unknowns, and last completion time. Update completion time only after both the original response and assessment are stored. Check the full sequence—execution, raw storage, assessment, and rereading the admin screen—and report failed services separately.

Use observation results to locate the next revision

For unindexed URLs, compare accessibility, robots rules, canonical URLs, duplicates, body quality, and the search tool’s reasons. If indexed but absent for relevant questions, inspect question-to-body alignment, internal links, and sufficiency of evidence. If cited but answered incorrectly, fix the dates, wording, stages, or links to historical material causing confusion.

If an answer is accurate but irrelevant, state the page’s audience and conditions clearly in the title, opening paragraph, and headings. If incomplete, add missing application procedures, exceptions, costs, or other question-required content using actual evidence. Link each revised sentence to the question ID and repeat the next observation under the same conditions.

Operational records retain the current observation, change from the previous one, possible causes, next revision, and next observation time. If an answer cites weekday hours for a Sunday question, record the cited URL. Add weekend conditions to that page, then check whether the next answer reflects Sunday hours and closure exceptions.

For example, if new HTML includes closure conditions and the crawl date has updated, but AI cites only 2025 records, investigate current and historical documents’ titles, periods, and links. If it cites the current document but omits an exception, compare that passage with the question’s conditions. If the document is already clear, check response variability too. A declining citation rate alone, without originals and cited URLs, cannot guide this choice.

If automated checks find a sudden increase in image bytes, save currentSrc, width, and format for the CMS route and investigate whether it reverted to the original URL. If only transformation counts rose, inspect storage keys and reuse paths. If a withdrawn item still returns 200, fix public-access decisions and CDN remnants before optimizing size. Separate result fields so different failures are not collapsed into one “SEO error.”

RSS checks compare public item IDs, links, and initial publication dates. A guid changed by a title edit can create duplicate reader entries; an item remaining after withdrawal requires fixing public-list refreshes. Video checks regularly verify segment URLs and media access, then review transcripts, summaries, and segment times when content changes. Observe search visibility and AI answers separately, with their own dates and questions, after these technical checks.

Follow the three initial diagnostic problems through to the end

Revisit the fictional library’s first three problems. If opening hours appear in the response HTML, the detail canonical points to that article, and old notices show their effective period and link to the current notice, you have confirmed the site changes. Also compare public, draft, and nonexistent document states against lists and sitemaps.

Next, record when the search tool crawled that URL and which canonical it selected. Ask AI the same Sunday question again under the same conditions. If it still uses an old timetable after the technical fixes, read the actual cited URL and answer, then reexamine historical links and time descriptions.

If citations appear but conditions are wrong, fix the correspondence between content and evidence. If access itself fails, fix the server, WAF, or cache. If technical results are normal but external crawl records are unchanged, observe recrawling status. Select the next action by cause and record the expected value it should change.

Connect raw storage to the next revision in one record

Include question ID, question version, service, and execution round in the job key. Preserve the completed original answer and its sources first. Keep assessments in separate records referencing that original. If criteria change, increment the assessment version without overwriting the original. Record retries as new attempts without erasing the first failed request.

Aggregation inputs must include response status, six judgments, evidence and reasons for each claim, and the raw-response path. The assessment code rejects completed rows with missing fields. The exercise below runs raw storage, assessment-file reading, conditional aggregation, and next-revision recording.

Run URL inspection requests and store originals

Save observe.mjs alongside metrics.mjs and assessment-fixture.mjs. Running node observe.mjs uses fixtures without external requests. It creates raw/index.json, original responses, assessments-v1.json, and report.json under observations/run-…. Rerunning the same job creates a new folder and never overwrites previous originals.

// Node.js 22. Default: fixture; --live sends a URL Inspection request. import { mkdir, mkdtemp, writeFile, readFile } from 'node:fs/promises'; import { resolve, join } from 'node:path'; import { pathToFileURL } from 'node:url'; import { grouped } from './metrics.mjs'; import { examples } from './assessment-fixture.mjs'; export async function inspectIndex({ inspectionUrl, siteUrl, token, request = fetch }) { if (!token) throw new Error('OAuth access token required'); const queriedAt = new Date().toISOString(); try { const response = await request('https://searchconsole.googleapis.com/v1/urlInspection/index:inspect', { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ inspectionUrl, siteUrl, languageCode: 'ko-KR' }), signal: AbortSignal.timeout(10000), }); const raw = await response.text(); const base = { inspectionUrl, siteUrl, queriedAt, httpStatus: response.status, raw }; if (!response.ok) return { ...base, status: 'failed', reason: 'HTTP ' + response.status, retryable: response.status === 429 || response.status >= 500, retryAfter: response.headers.get('retry-after') }; let data; try { data = JSON.parse(raw); } catch { return { ...base, status: 'unknown', reason: 'Invalid JSON' }; } const result = data.inspectionResult?.indexStatusResult; if (!result?.verdict) return { ...base, status: 'unknown', reason: 'Missing index verdict' }; return { ...base, status: 'complete', verdict: result.verdict, lastCrawlTime: result.lastCrawlTime ?? null, googleCanonical: result.googleCanonical ?? null, canonicalMatches: result.googleCanonical ? result.googleCanonical === inspectionUrl : null, coverageState: result.coverageState ?? null }; } catch (error) { return { inspectionUrl, siteUrl, queriedAt, status: 'failed', reason: error.name === 'TimeoutError' ? 'Timeout' : 'Network failure', retryable: true }; } } export async function runObservation(out, indexRecord) { await mkdir(out, { recursive: true }); const run = await mkdtemp(join(resolve(out), 'run-')); await mkdir(join(run, 'raw')); await writeFile(join(run, 'raw/index.json'), JSON.stringify(indexRecord, null, 2), { flag: 'wx' }); for (const row of examples) { await writeFile(join(run, row.rawPath), JSON.stringify({ fixture: true, question: row.question, answer: row.answer, reference: row.reference, status: row.status, failure: row.failure, model: row.model, sources: row.claims?.filter(c => c.sourceUrl).map(c => ({ url: c.sourceUrl, quote: c.quote })), }, null, 2), { flag: 'wx' }); } await writeFile(join(run, 'assessments-v1.json'), JSON.stringify(examples, null, 2), { flag: 'wx' }); // Read stored assessments and aggregate by condition. const rows = JSON.parse(await readFile(join(run, 'assessments-v1.json'), 'utf8')); const report = { index: indexRecord.status, groups: grouped(rows), next: rows.filter(r => r.status === 'complete' && (r.complete === false || r.timely === false)) .map(r => ({ questionId: r.questionId, rawPath: r.rawPath, action: r.timely === false ? 'Compare the historical notice’s effective period and link to the current notice.' : 'Compare the current notice’s first-Sunday closure wording with the answer’s omission.' })) }; await writeFile(join(run, 'report.json'), JSON.stringify(report, null, 2), { flag: 'wx' }); return { run, report }; } if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) { const record = process.argv.includes('--live') ? await inspectIndex({ inspectionUrl: process.env.INSPECTION_URL, siteUrl: process.env.SEARCH_CONSOLE_SITE, token: process.env.SEARCH_CONSOLE_TOKEN }) : { status: 'complete', verdict: 'PASS', googleCanonical: 'https://example.com/guides/opening-hours', fixture: true, queriedAt: '2026-10-24T00:00:00Z' }; console.log(JSON.stringify(await runObservation(process.env.OBSERVATION_DIR || './observations', record), null, 2)); }

inspectIndex puts inspectionUrl and siteUrl in the POST body and sends the OAuth access token in the Authorization header. It requires an account that can access the registered Search Console property and the webmasters.readonly or webmasters scope. URL-prefix properties include a trailing /; domain properties use sc-domain:example.com. Tokens are read from environment variables and excluded from stored observation records. Request and authentication contract

Provide INSPECTION_URL, SEARCH_CONSOLE_SITE, and SEARCH_CONSOLE_TOKEN as environment variables and run node observe.mjs --live to send a URL Inspection request. AI assessment input comes from assessment-fixture.mjs. To assess observed answers, replace that input with the service’s original responses, sources, and assessments.

For HTTP 401 or 403, repair authentication and property permissions before retrying. For 429 and 5xx, retain the failure response, retryAfter, and retryable, then end this execution. The scheduler records a new attempt after Retry-After or the chosen delay; it does not retry immediately forever. Even a 200 is unknown if verdict is missing or JSON malformed. Do not read PASS alone; compare lastCrawlTime and googleCanonical with the requested URL.

URL Inspection quotas are 2,000 per day and 600 per minute per property, and 10 million per day and 15,000 per minute per project. If multiple runners inspect the same property, total usage in shared storage. Storing the next permissible request time in the queue can reduce a burst of retries after a 429. Official quotas

runObservation stores originals first, then rereads the assessment file and aggregates with grouped. report.next retains original-response paths supporting omitted closure exceptions and use of historical material. The next worker compares those originals with the page, then records the revised sentence, document revision, and reobservation question. Failed requests and unassessable responses remain in the aggregation.

Storing originals first lets the judgments in this code be reexamined. Even if search-tool rules or AI evaluation criteria change, the response from that time remains readable. Follow the raw path in report.next to find the wrong period or omitted condition, and connect the revised sentence to the next observation. That turns automation into actual content revisions.