* utils: document binarySearch
* nes-datagen: generate training data from continuous recordings
Continuous enhanced telemetry now ships sliding-window recordings that, unlike per-request alternative-action recordings, carry no requestTime. The datagen pipeline needs a point to split each recording into edit history before/after, so this adds a pluggable pivot strategy (starting with Random, selectable via --pivot-strategy) and a new continuous/ pipeline module that replays a recording at the chosen pivot to produce a processed row.
Along the way this consolidates the pipeline's error and index handling: a shared WithRowIndex<T> replaces the ad-hoc { originalRowIndex, ... } pairs, per-record processing returns Result<IProcessedRow, Error> instead of field-presence unions, and failures surface as original Error objects (no string round-tripping). The telemetry sender's continuous payload is now the documented IContinuousRecording type.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* nes-datagen: label alt-action replay errors by originalRowIndex
Address PR review: the alternative-action path mislabeled diagnostics when
earlier records failed to parse.
- processAllRows: push replay errors with the row's true `originalRowIndex`
instead of its position in the filtered `rows` array (parse failures make
`rows` sparse, so the two diverge).
- loadAndProduceProcessedRows: resolve `languageForRow` via an
`originalRowIndex`-keyed Map rather than positional `rows[i]`, matching how
callers pass `e.originalRowIndex`.
- Clarify the `recordCount` doc: it counts successfully-parsed records (parse
failures are counted separately in `parseErrors`).
- Add a regression spec asserting replay errors carry the row index, not the
array position.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Agent Host changes for agents/adhoc-request-sender-mode-extension-55e2bb6f
* Remove unconfigured react-hooks/exhaustive-deps eslint directive
The eslint-disable directive referenced a rule that isn't registered in
this repo's ESLint config, which caused ESLint to error with
"Definition for rule 'react-hooks/exhaustive-deps' was not found" and
failed the Compile & Hygiene and Copilot - Test CI checks.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Coalesce adhoc tag-decoration rescans with requestAnimationFrame
Rescanning the whole editor text on every content change is wasteful for
bursty updates (e.g. a streamed response). Debounce the decoration update
to at most once per animation frame and cancel any pending frame during
cleanup so the callback can't run after the editor is disposed. The
initial scan stays synchronous so tags are highlighted immediately on mount.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR feedback: dispose token source; validate adhoc request JSON
- adhocRequestSender: always dispose the per-send CancellationTokenSource
in the finally block (separate from the current-send guard) so its
cancellation listeners don't leak across repeated Send/Stop cycles.
- simulationMain: validate and normalize the adhoc request JSON before use
so malformed input (missing/null/wrong-typed model/user/system) yields a
focused error message instead of a thrown stack trace.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: restore chat pipeline to windows-latest
* chore: remove node-gyp override
* chore: restore node-gyp override with comment
* refactor: rm dependency on key:sqlite
The module locks the node-gyp dependency to 8.x due to
its transitive sqlite3 native module dependency this in turn
blocks using newer windows CI, refs https://github.com/microsoft/vscode/issues/321267
The module can be replaced with built-in sqlite support
from Node.js without losing the on-disk cache format has
already been committed.
* chore: restore minimist
* chore: set sqlite busy timeout
* fix: decode json-buffer values for keyv cache compat
* nes-datagen: add cursor-jump (NCLP) task
Extend nes-datagen with a next-cursor-line prediction task alongside
the existing xtab path. Detects the user's next intentional cursor
move after the request bookmark and emits a training sample with the
production cursor-prediction prompt + the observed jump as the
expected response.
Three sub-modes via --sample-task:
- cursor-same-file: a jump farther than N lines from cursor at
request time
- cursor-cross-file: focus/selection on a different file
- cursor-both: either of the above
Reuses the production cursor-prediction prompt by capturing it via
the telemetry builder and a no-op fetcher; the cross-file target
line is resolved from a request-time content snapshot + post-request
replay so previously-opened targets get a correct line number
instead of being silently labelled :0.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: replace SAMPLE_TASK_VALUES tuple with a string enum
Convert the string-union + as-const tuple to a proper NesDatagenSampleTask
string enum. CLI surface is string-enum members keep theunchanged
kebab-case wire values ('xtab', 'cursor-same-file', ...). All consumers
(dispatch, fixtures, response metadata typing) updated to reference enum
members instead of string literals.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: lower default --same-file-jump-min-above to 2
Upward cursor jumps (back to a definition, an import, etc.) are
typically tighter than downward jumps after the user has been
writing. Lower the default threshold to match.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: rename NCLP to cursor-jump throughout
Drop the NCLP abbreviation in favor of the more descriptive
'cursor-jump' name already used in the production xtab provider.
cursorJumpPromptStep,
cursorJumpResponseStep), the capture request ids,
and all surrounding doc comments / test descriptions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: build documentIndexMapping from whole recording
path map from only the
pre-request slice and then re-walked the post-request slice to
backfill any documentEncountered entries that arrived later. Pass the
whole recording into documentIndexMapping instead so the helper sees
every document the user touched in a single pass; the backfill loop is
gone.
splitRecordingAtRequestTime now also returns the full entries array so
both callers can reuse it without re-deriving it from altAction.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: use shared Result type in cursor-jump detectors
Drop the bespoke { ok, value | reason } discriminated union in
detectJump.ts and reuse the existing Result<T, E> from
src/util/common/result. JumpDetectionResult<T> is now just an alias
for Result<T, string>.
.isOk(),
.err) and the spec file accordingly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: strip raw cursor-jump prompt from emitted telemetry
cursorJumpRawMessages and cursorJumpKeptRange were added to
IStatelessNextEditTelemetry so in-process debug / datagen tooling
could read them back via getStatelessNextEditTelemetry(). However
LlmNESTelemetryBuilder.build() spreads ...this._statelessNextEditTelemetry
into the emitted payload, so those two fields would leak to telemetry
cursorJumpRawMessages can contain full prompt content (sourcesinks
code), which must never leave the process.
Destructure them out before spreading into the build() payload. They
remain readable via getStatelessNextEditTelemetry() for tooling.
Documented the privacy contract on the IStatelessNextEditTelemetry
field declarations so future edits don't forget.
Addresses copilot-pull-request-reviewer feedback on PR #320113.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: fail cross-file detection when no selection lands on target
detectCrossFileJump previously returned Result.ok with toLine
undefined when only a focused event was seen for the target doc (no
selectionChanged). That left generateCrossFileResponse to drop the
sample later while the detector still reported a successful jump.
Treat focused-without-selectionChanged as a failed detection
('crossFileTargetNoSelection') so callers can skip early, and tighten
ICrossFileJump.toLine to non-undefined now that ok results always
have a usable line number. Removes the dead error path in
generateCrossFileResponse.
Adds a regression test that focused-only triggers the new error.
Addresses copilot-pull-request-reviewer feedback on PR #320113.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: capture cursor-jump prompt via logContext, not telemetry
The datagen pipeline previously stashed the raw cursor-jump prompt and
keptRange on IStatelessNextEditTelemetry so cursorJumpPromptStep.ts could
read them back via LlmNESTelemetryBuilder.getStatelessNextEditTelemetry().
That leaked raw prompts into the telemetry payload (worked around by a
destructure-strip hack in LlmNESTelemetryBuilder.build()) and was
asymmetric with the xtab path, which captures via
InlineEditRequestLogContext.rawMessages.
Move the cursor-jump capture vehicle onto InlineEditRequestLogContext to
match xtab:
- Add cursorJumpRawMessages / cursorJumpKeptRange fields and
setCursorJumpPrompt(messages, keptRange) to InlineEditRequestLogContext.
- XtabNextCursorPredictor.predictNextCursorPosition now takes a logContext
parameter and writes to it directly. The xtabProvider callsite passes
the same logContext it already had in scope.
- cursorJumpPromptStep reads from logContext instead of the telemetry
builder.
- Remove cursorJumpRawMessages / cursorJumpKeptRange from
IStatelessNextEditTelemetry, plus the corresponding setter/getter on
StatelessNextEditTelemetryBuilder and the getter on
LlmNESTelemetryBuilder.
- Revert the destructure-strip hack in LlmNESTelemetryBuilder.build().
The pre-existing cursorJumpPrompt telemetry field (JSON-stringified, fed
by setCursorJumpPrompt(messages)) is intentional and unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: cursor-jump ground truth is first user EDIT, not cursor landing
Selection-based detection treated peek, navigation, IDE auto-scroll, and
recursive cursor settling as if they were the user's next intended edit
location. The model's job is to predict where the user will EDIT next, so
key off the first 'changed' event after the request bookmark instead.
Same-file detector:
- Walks for the first 'changed' on the active doc; uses the first edit's
start offset to compute toLine; applies the linesAbove/linesBelow
threshold. Bails with editsAnotherFileFirst when a non-active doc is
edited first (lets the cross-file detector claim the sample in
cursor-both mode). 'selectionChanged' is no longer consulted, so the
settle-after-edit filter is gone it was a workaround for thetoo
selection-based approach.
Cross-file detector:
- Walks for the first 'changed' on a non-active doc; uses the first
edit's start offset, resolved against the target doc's snapshot
just-before applying the event. Drops focused / selectionChanged
heuristics and the crossFileTargetNoSelection error path (a focused
event without an edit no longer counts; background peek can't
pollute the dataset).
buildLineResolver: tightened i <= entryIndex to i < entryIndex so the
resolver returns the pre-edit line when entryIndex is itself a 'changed'
event. The bound is equivalent for the old selectionChanged caller.
Spec: switched ground-truth events from selChanged to changed; added
coverage for first-edit-of-multi-edit, editsAnotherFileFirst, and
active-doc-then-other-doc ordering.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* xtab: predictNextCursorPosition takes RequestTracingContext
Every other helper in xtabProvider takes RequestTracingContext (the
{ tracer, logContext, telemetry } bundle). The cursor predictor was the
odd one out, taking the three pieces as separate positional params with
the latter two that asymmetry made the new logContext-captureoptional
plumbing look more invasive than it is and forced an awkward
?.setCursorJumpPrompt chain at the use site.
Switch the predictor to take RequestTracingContext directly:
- Export RequestTracingContext from xtabProvider so the predictor can
type-import it (TS-erased to avoid the runtime circular import).
- predictNextCursorPosition signature collapses from 5 params to 3.
- Drop the optional chains; tracing.telemetry / tracing.logContext are
always present in production and the spec constructs a real bundle.
- Spec adds a createTestTracingContext helper using the cheap
InlineEditRequestLogContext / StatelessNextEditTelemetryBuilder
constructors already used by other inlineEdits specs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: rename splitRecording 'entries' field to 'wholeRecording'
Review feedback: the field on the splitRecordingAtRequestTime return
shape was named 'entries' but in context it carries the whole unsplit
recording (i.e. before slicing into prior/after parts). 'wholeRecording'
matches the comment at the consumer (documentIndexMapping callsite).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: inline JumpDetectionResult<T> as Result<T, string>
Review feedback: the one-line alias was used in exactly two places in
the same file and gave nothing over the underlying Result type.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: discriminated union for sample task + jump metadata
Review feedback: ISampleMetadata had 'task' + an optional 'jump' field
with toFilePath also optional. That let xtab samples accidentally carry
a jump and let cursor-cross-file samples omit toFilePath. Replace with
a discriminated union on task:
- xtab: no jump
- cursorSameFile: jump with fromLine/toLine/distance
- cursorCrossFile: jump with required toFilePath
assembleSample now takes a single SampleClassification arg, removing
the parallel task/jump parameters that callers had to keep in sync.
cursorJumpResponseStep is split into ISameFileGeneratedResponse and
ICrossFileGeneratedResponse so the generator return types map cleanly
to the union variants without a non-null assertion at the assembly
site.
DetectedJump no longer needs an assistantTask hint: the pipeline
constructs the classification directly from the response shape.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix dup import
* nes-datagen: address round-4 review feedback on pipeline.ts
Five review threads on pipeline. all addressed in-place.ts
- modelResponse: cursor samples were emitting an empty string for the
expected response. Populate it with the assistant content (which IS
the expected output) so downstream tooling has the gold label.
- Promise.all unbounded throws: wrap the limiter callback body in
try/catch so an unexpected exception from generateCursorPromptFromRecording
becomes a recorded per-row error instead of aborting the whole batch
via Promise.all's first-rejection semantics.
- Inline import for OffsetRange: replace the inline import('...').OffsetRange
type expression with a regular top-of-file import.
- Duplicated config-override block: both pipelines applied the same
applyConfigFile + four setConfig debounce/cache disables. Extract
into applyBatchModeConfig(configService, configs) and call from both.
- runInputPipeline parallelism + memory: add a doc comment clarifying
that this is the single-process entry point, that cursor-jump tasks
also benefit from runInputPipelineParallel (--sample-task is
propagated to workers), and that loadAndParseInput is in-memory by
design (sized per worker; use --parallelism > 1 for large inputs).
Full architectural unification of the parallel and non-parallel
paths is intentionally left as a follow- the surface area isup
large and out of scope for this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: add e2e tests for cursor-jump pipeline
Mirrors the existing xtab pipeline.e2e.spec.ts: drives two fixture rows
(a same-file jump and a cross-file jump) through the full
`runInputPipeline` for each `sampleTask` mode (cursor-same-file,
cursor-cross-file, cursor-both) and asserts on the JSONL output.
Coverage:
- only the matching row is emitted per mode; both rows are emitted in
cursor-both
- emitted samples carry strategy=next-cursor-line-prediction and the
correct discriminated `task` field (cursor-same-file / cursor-cross-file)
- assistant message targets the jumped-to line / file
- metadata.modelResponse mirrors the assistant content (the round-4 fix)
- --row-offset is reflected in metadata.rowIndex
Test fixtures are constructed in
`fixtures/cursorJumpFixtureData.ts` with synthesized recordings: an
explicit no-op edit + selectionChanged before the bookmark so the
cursor-prediction path's recent-edit gating is satisfied, then a
single post-request `changed` event the detector picks up.
The cursor pipeline needs a prompting strategy whose response handler
tolerates an empty stream — use `xtabUnifiedModel` in a dedicated
`cursorJumpConfig.json` (the existing patchBased02 config crashes on
empty output, which is acceptable in production but breaks the
prompt-only capture path).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Strengthen cursor-jump e2e assertions
Replace fuzzy matchers (toMatch(/25/), arrayContaining for tasks) with
exact assertions on assistant content, metadata.task, and metadata.jump.
In cursor-both, locate samples by filePath so a row→classification swap
would now be caught instead of passing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Make cursor-jump e2e helper accept partial nesDatagen overrides
Helper previously took Partial<RunPipelineOptions>; if a caller passed
`nesDatagen`, the spread fully replaced the default block and the
configured path. Now the helper accepts a partial nesDatagen overlay
and merges field-by-field, so the row-offset test only specifies the
two fields it actually changes and there are no non-null assertions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add within-threshold cursor-jump negative fixture
Scenario C: cursor on line 10, post-request edit on line 12 (only 2
lines below). Default threshold is ±5 lines, so neither the same-file
nor the cross-file generator should emit a sample for this row.
Asserted in cursor-both via a dedicated 'does not emit a sample for
the within-threshold row' test, and implicitly in cursor-same-file /
cursor-cross-file (their existing count==1 assertions would fail if
the threshold guard regressed).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Ulugbek Abdullaev <ulugbekna@github.com>
* nes-datagen: stream large JSON array inputs to avoid 2 GiB readFile limit
Reading the whole input via fs.readFile fails for files larger than 2 GiB
(and exceeds V8's max string length). Add a streaming JSON-array parser and
use it in both the sequential and parallel pipeline paths so multi-GB
recordings can be processed with bounded memory.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: also accept JSON Lines (NDJSON) input
Auto-detect the input format from the first non-whitespace character: a
leading '[' is parsed as a single JSON array, otherwise the file is parsed
as JSON Lines (one JSON object per line). Both formats are streamed so
multi-GB inputs work regardless of shape. Rename streamJsonArray ->
streamJsonRecords to reflect the broader purpose.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: infer JSON vs JSON Lines input format from file extension
Use the file extension (.jsonl/.ndjson -> JSON Lines, otherwise JSON array)
to select the streaming parser instead of sniffing the content.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: validate streamed JSON arrays for truncation and malformed input
The new streaming parser previously accepted any prefix of a JSON array
silently: a truncated file (no closing ']'), a missing element between
commas, a trailing comma or trailing data after the array all produced
zero or fewer records rather than an error. That is especially dangerous
for the multi-GB inputs this parser was introduced for, because the
underlying file is much more likely to be incomplete.
Tighten the state machine to surface these as errors, matching what the
old whole-file JSON.parse would have done, and add tests for each case.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: stream worker-result merging and final output write
For multi-GB inputs the parent process was still hitting V8's ~512 MiB
max-string-length limit in two places after the input-side fix:
1. Merging worker result files used fs.promises.readFile + JSON.parse on
each result, but with a 5+ GB input split N ways each per-worker file
is hundreds of MB of similar-shaped data and easily exceeds the
string limit.
2. writeSamples serialized the entire validSamples array via a single
JSON.stringify(arr, null, 2) before writing, which has the same
problem on output.
Switch both to stream over individual records:
- A new shared openWriteStream(filePath) helper wraps fs.createWriteStream,
attaches an 'error' listener immediately (so async write failures don't
surface as uncaughtException and skip cleanup), awaits backpressure via
the per-write callback, and exposes an idempotent close().
- writeChunkFiles uses the helper inside a try/finally so any mid-stream
ENOSPC/EIO bubbles up cleanly and the tmp dir is still removed.
- The merge step now uses streamJsonRecords<ISample>(resultPath), so the
parent never materializes a single worker's output as one string.
- writeSamples emits the output JSON array incrementally: per-sample
JSON.stringify(..., null, 2) (indented two spaces to match the previous
layout) joined with ',\n'. Byte size is accumulated for the existing
IWriteResult.fileSize.
Also documents that single-process loadAndParseInput still buffers the
full row set in memory and that --parallelism is required for very large
inputs (workers each only load their slice).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: switch output to JSON Lines
Both the final user-facing output and the per-worker intermediate result
files now use JSON Lines (one record per line) instead of a
pretty-printed JSON array. JSONL is dramatically simpler to write and
read incrementally: no surrounding brackets/commas to track, no
multi-line per-element indentation, just JSON.stringify + '\n' per
record on the write side and split-on-newline + JSON.parse per non-empty
line on the read side (this is what streamJsonRecords already does when
it detects the .jsonl extension).
Changes:
- writeSamples emits one JSON.stringify(sample) + '\n' per validated
sample via no array wrapper, no pretty-printing.openWriteStream
- resolveOutputPath defaults the implicit output to <input>_output.jsonl
(was <input>_output.json).
- Per-worker result files in runInputPipelineParallel are now
result_${w}.jsonl, so the merge step's streamJsonRecords auto-picks
the JSONL parser from the extension.
- E2e tests updated to read JSONL (split on newline, JSON.parse per
line) and to use .jsonl output paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: surface worker-result parse errors and update --out help text
Two review follow-ups:
1. The merge step that streams each worker's result file used to wrap
the iteration in a try/catch that downgraded any parse error to a
console.error warning. With the new streaming reader that is unsafe:
streamJsonRecords yields N valid records first and then throws on a
malformed/truncated tail, leaving those N partial records already in
allSamples. The pipeline would then quietly emit a truncated
training-data output. Drop the swallowing try/catch so a corrupt
worker result aborts the run non-zero.
2. The --out help text in simulationOptions.ts still advertised the old
JSON-array default (<input>_output.json). Update it to reflect the
JSONL output, and also note in --input that the format is inferred
from the .jsonl/.ndjson extension.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove leftover code from /tests test-generation codelens
The codelens feature for /tests test generation was removed in
c0eaa94d6a, but related infrastructure remained as dead code:
- ITestGenInfoStorage was never populated (the deleted testGenAction.ts
was the only writer), so TestFromTestInvocation and Test2Impl always
took the 'undefined' branch.
- IParserService.getTreeSitterAST().getTestableNode/getTestableNodes
and the underlying testGenParsing helpers + testableNodeQueries had
no remaining callers.
Delete the storage service, the parser API, the tree-sitter queries,
and the related tests. The /tests slash command itself is preserved;
`_findLastTest` stays since testFromSrcInvocation.tsx still uses it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Strip Copilot Memory (CAPI) feature entirely
Removes the CAPI-backed Copilot Memory that synced repository-scoped facts
to GitHub. The local file-based MemoryTool with user/session/repo scopes
remains as the sole memory mechanism.
- Delete AgentMemoryService and its test.
- Remove the github.copilot.chat.copilotMemory.enabled setting and its NLS string.
- Remove ConfigKey.CopilotMemoryEnabled.
- Strip all CAPI gating in memoryTool.tsx, memoryContextPrompt.tsx, tools.ts.
- Drop _dispatchRepoCAPI / _repoCreate / _sendRepoTelemetry.
- /memories/repo/ now always routes to local storage.
- Update memoryTool.spec.tsx: remove mock CAPI services and CAPI-only tests.
- Update simulationExtHostToolsService.ts for the new ToolsContribution arity.
* Make xtab modelConfiguration an advanced setting
Move InlineEditsXtabProviderModelConfiguration from TeamInternal to Advanced. Renames the key from chat.advanced.inlineEdits.xtabProvider.modelConfiguration to chat.inlineEdits.xtabProvider.modelConfiguration with a migration for backward compatibility, and exposes it via package.json. Experiments continue using the separate modelConfigurationString setting.
* Fix localModelConfig type to allow null
* Use fully-qualified key in simulation help text
Fixes#312210
This is using the old search endpoint. We'll start switching over the new one once it's ready
Co-authored-by: Copilot <copilot@github.com>
Starts working through some basic repo wide eslint rules for the copilot extension. Stuff like missing semicolons and missing readonly modifiers for disposables
* Allow invoking simulationMain with alternative action input
* Address review comments: rename CLI opts, extract pipeline, fix correctness issues
- Rename CLI options with --train- prefix (--train-input, --train-strategy,
--train-out, --train-row-offset, --train-worker) and document all options
- Extract runInputPipeline/runInputPipelineParallel to test/pipeline/trainPipeline.ts
- Preserve original row index through parse/replay/prompt pipeline to fix
sample numbering drift when rows are filtered out
- Fix parseSuggestedEdit: use JSON.parse for escaped text, handle missing delimiter
- Fix line number regex to accept optional space after | (WithoutSpace format)
- Clamp concurrency to >= 1, type samples as ISample[], wrap dispose in try/finally
- Gate verbose logging in loadAndParseInput behind verbose flag
- Use splitLines from existing utility instead of local duplicate
* move nes-datagen to a subcommand
* more code reuse around setting promptStrategy and model config
* Address review: use ResponseFormat, Limiter, assertNever, and raw messages
* minor refactor runPipeline
* finalize
* use POT instead of custom code
* move files from script/ to test/pipeline/
---------
Co-authored-by: ulugbekna <ulugbekna@gmail.com>
* feat: add OTel GenAI instrumentation foundation
Phase 0 complete:
- spec.md: Full spec with decisions, GenAI semconv, dual-write, eval signals,
lessons from Gemini CLI + Claude Code
- plan.md: E2E demo plan (chat ext + eval repo + Azure backend)
- src/platform/otel/: IOTelService, config, attributes, metrics, events,
message formatters, NodeOTelService, file exporters
- package.json: Added @opentelemetry/* dependencies
OTel opt-in behind OTEL_EXPORTER_OTLP_ENDPOINT env var.
* refactor: reorder OTel type imports for consistency
* refactor: reorder OTel type imports for consistency
* feat(otel): wire OTel spans into chat extension — Phase 1 core
- Register IOTelService in DI (NodeOTelService when enabled, NoopOTelService when disabled)
- Add OTelContrib lifecycle contribution for OTel init/shutdown
- Add `chat {model}` inference span in ChatMLFetcherImpl._doFetchAndStreamChat()
- Add `execute_tool {name}` span in ToolsService.invokeTool()
- Add `invoke_agent {participant}` parent span in ToolCallingLoop.run()
- Record gen_ai.client.operation.duration, tool call count/duration, agent metrics
- Thread IOTelService through all ToolCallingLoop subclasses
- Update test files with NoopOTelService
- Zero overhead when OTel is disabled (noop providers, no dynamic imports)
* feat(otel): add embeddings span, config UI settings, and unit tests
- Add `embeddings {model}` span in RemoteEmbeddingsComputer.computeEmbeddings()
- Add VS Code settings under github.copilot.chat.otel.* in package.json
(enabled, exporterType, otlpEndpoint, captureContent, outfile)
- Wire VS Code settings into resolveOTelConfig in services.ts
- Add unit tests for:
- resolveOTelConfig: env precedence, kill switch, all config paths (16 tests)
- NoopOTelService: zero-overhead noop behavior (8 tests)
- GenAiMetrics: metric recording with correct attributes (7 tests)
* test(otel): add unit tests for messageFormatters, genAiEvents, fileExporters
- messageFormatters: 18 tests covering toInputMessages, toOutputMessages,
toSystemInstructions, toToolDefinitions (edge cases, empty inputs, invalid JSON)
- genAiEvents: 9 tests covering all 4 event emitters, content capture on/off
- fileExporters: 5 tests covering write/read round-trip for span, log, metric
exporters plus aggregation temporality
Total OTel test suite: 63 tests across 6 files
* feat(otel): record token usage and time-to-first-token metrics
Add gen_ai.client.token.usage (input/output) and copilot_chat.time_to_first_token
histogram metrics at the fetchMany success path where token counts and TTFT
are available from the processSuccessfulResponse result.
* docs: finalize sprint plan with completion status
* style: apply formatter changes to OTel files
* feat(otel): emit gen_ai.client.inference.operation.details event with token usage
Wire emitInferenceDetailsEvent into fetchMany success path where full
token usage (prompt_tokens, completion_tokens), resolved model, request ID,
and finish reasons are available from processSuccessfulResponse.
This follows the OTel GenAI spec pattern:
- Spans: timing + hierarchy + error tracking
- Events: full request/response details including token counts
The data mirrors what RequestLogger captures for chat-export-logs.json.
* feat(otel): add aggregated token usage to invoke_agent span
Per the OTel GenAI agent spans spec, add gen_ai.usage.input_tokens and
gen_ai.usage.output_tokens as Recommended attributes on the invoke_agent span.
Tokens are accumulated across all LLM turns by listening to onDidReceiveResponse
events during the agent loop, then set on the span before it ends.
Ref: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-agent-spans/
* feat(otel): add token usage attributes to chat inference span
Defer the `chat {model}` span completion from _doFetchAndStreamChat to
fetchMany where processSuccessfulResponse has extracted token counts.
The chat span now carries:
- gen_ai.usage.input_tokens (prompt_tokens)
- gen_ai.usage.output_tokens (completion_tokens)
- gen_ai.response.model (resolved model)
The span handle is returned from _doFetchAndStreamChat via the result
object so fetchMany can set attributes and end it after tokens are known.
This matches the chat-export-logs.json pattern where each request entry
carries full usage data alongside the response.
* style: apply formatter changes
* fix: correct import paths in otelContrib and add IOTelService to test
* feat: add diagnostic span exporter to log first successful export and failures
* feat: add content capture to OTel spans (messages, responses, tool args/results)
- Chat spans: add copilot.debug_name attribute for identifying orphan spans
- Chat spans: capture gen_ai.input.messages and gen_ai.output.messages when captureContent enabled
- Tool spans: capture gen_ai.tool.call.arguments and gen_ai.tool.call.result when captureContent enabled
- Extension chat endpoint: capture input/output messages when captureContent enabled
- Add CopilotAttr.DEBUG_NAME constant
* fix: register IOTelService in chatLib setupServices for NES test
* fix: register OTel ConfigKey settings in Advanced namespace for configurations test
* fix: register IOTelService in shared test services (createExtensionUnitTestingServices)
* fix: register IOTelService in platform test services
* feat(otel): enhance GenAI span attributes per OTel semantic conventions
- Change gen_ai.provider.name from 'openai' to 'github' for CAPI models
- Rename CopilotAttr to CopilotChatAttr, prefix values with copilot_chat.*
- Add GITHUB to GenAiProviderName enum
- Replace copilot.debug_name with gen_ai.agent.name on chat spans
- Add gen_ai.request.temperature, gen_ai.request.top_p to chat spans
- Add gen_ai.response.id, gen_ai.response.finish_reasons on success
- Add gen_ai.usage.cache_read.input_tokens from cached_tokens
- Add copilot_chat.request.max_prompt_tokens and copilot_chat.time_to_first_token
- Add gen_ai.tool.description to execute_tool spans
- Fix gen_ai.tool.call.id to read chatStreamToolCallId (was reading nonexistent prop)
- Fix tool result capture to handle PromptTsxPart and DataPart (not just TextPart)
- Add gen_ai.input.messages and gen_ai.output.messages to invoke_agent span (opt-in)
- Move gen_ai.tool.definitions from chat spans to invoke_agent span (opt-in)
- Add gen_ai.system_instructions to chat spans (opt-in)
- Fix error.type raw strings to use StdAttr.ERROR_TYPE constant
- Centralize hardcoded copilot.turn_count and copilot.endpoint_type into CopilotChatAttr
- Add COPILOT_OTEL_CAPTURE_CONTENT=true to launch.json for testing
- Document span hierarchy fixes needed in plan.md
* feat(otel): connect subagent spans to parent trace via context propagation
- Add TraceContext type and getActiveTraceContext() to IOTelService
- Add storeTraceContext/getStoredTraceContext for cross-boundary propagation
- Add parentTraceContext option to SpanOptions for explicit parent linking
- Implement in NodeOTelService using OTel remote span context
- Capture trace context when execute_tool runSubagent fires (keyed by toolCallId)
- Restore parent context in subagent invoke_agent span (via subAgentInvocationId)
- Auto-cleanup stored contexts after 5 minutes to prevent memory leaks
- Update test mocks with new IOTelService methods
- Update plan.md with investigation findings
* fix(otel): fix subagent trace context key to use parentRequestId
The previous implementation stored trace context keyed by chatStreamToolCallId
(model-assigned tool call ID), but looked it up by subAgentInvocationId
(VS Code internal invocation.callId UUID). These are different IDs that don't
match across the IPC boundary.
Fix: key by chatRequestId on store side (available on invocation options),
and look up by parentRequestId on subagent side (same value, available on
ChatRequest). Both reference the parent agent's request ID.
Verified: 21-span trace with subagent correctly nested under parent agent.
* fix(otel): add model attrs to invoke_agent and max_prompt_tokens to BYOK chat
- Set gen_ai.request.model on invoke_agent span from endpoint
- Track gen_ai.response.model from last LLM response resolvedModel
- Add copilot_chat.request.max_prompt_tokens to BYOK chat spans
- Document upstream gaps in plan.md (BYOK token usage, programmatic tool IDs)
* test(otel): add trace context propagation tests for subagent linkage
Tests verify:
- storeTraceContext/getStoredTraceContext round-trip and single-use semantics
- getActiveTraceContext returns context inside startActiveSpan
- parentTraceContext makes child span inherit traceId from parent
- Independent spans get different traceIds without parentTraceContext
- Full subagent flow: store context in tool call, retrieve in subagent
* fix(otel): add finish_reasons and ttft to BYOK chat spans, document orphan spans
- Set gen_ai.response.finish_reasons on BYOK chat success
- Set copilot_chat.time_to_first_token on BYOK chat success
- Document Gap 4: duplicate orphan spans from CopilotLanguageModelWrapper
- Identify all orphan span categories (title, progressMessages, promptCategorization, wrapper)
* docs(otel): update Gap 4 analysis — wrapper spans have actual token usage data
The copilotLanguageModelWrapper orphan spans are the actual CAPI HTTP
handlers, not duplicates. They contain real token usage, cache read tokens,
resolved model names, and temperature — all missing from the consumer-side
extChatEndpoint spans due to VS Code LM API limitations.
Updated plan.md with:
- Side-by-side attribute comparison table
- Three fix approaches (context propagation, span suppression, enrichment)
- Recommendation: Option 1 (propagate trace context through IPC)
* feat(otel): propagate trace context through BYOK IPC to link wrapper spans
- Pass _otelTraceContext through modelOptions alongside _capturingTokenCorrelationId
- Inject IOTelService into CopilotLanguageModelWrapper
- Wrap makeRequest in startActiveSpan with parentTraceContext when available
- This creates a byok-provider bridge span that makes chatMLFetcher's chat span
a child of the original invoke_agent trace, bringing real token usage data
into the agent trace hierarchy
* debug(otel): add debug attribute to verify trace context capture in BYOK path
* fix(otel): remove debug attribute, BYOK trace context propagation verified working
Verified: 63-span trace with Azure BYOK (gpt-5) correctly shows:
- byok-provider bridge spans linking wrapper chat spans into agent trace
- Real token usage (in:21458 out:1730 cache:19072) visible on wrapper chat spans
- hasCtx:true on all extChatEndpoint spans confirming context capture
- Two subagent invoke_agent spans correctly nested under main agent
- Zero orphan copilotLanguageModelWrapper spans
* refactor(otel): replace byok-provider bridge span with invisible context propagation
Add runWithTraceContext() to IOTelService — sets parent trace context
without creating a visible span. The wrapper's chat spans now appear
directly as children of invoke_agent, eliminating the noisy
byok-provider intermediary span.
Before: invoke_agent → byok-provider → chat (wrapper)
After: invoke_agent → chat (wrapper)
* refactor(otel): remove duplicate BYOK consumer-side chat span
The extChatEndpoint no longer creates its own chat span. The wrapper's
chatMLFetcher span (via CopilotLanguageModelWrapper) is the single source
of truth with full token usage, cache data, and resolved model.
Before: invoke_agent → chat (empty, extChatEndpoint) + chat (rich, wrapper)
After: invoke_agent → chat (rich, wrapper only)
* fix(otel): restore chat span for non-wrapper BYOK providers (Anthropic, Gemini)
The previous commit removed the extChatEndpoint chat span, which was correct
for Azure/OpenAI BYOK (served by CopilotLanguageModelWrapper via chatMLFetcher).
But Anthropic and Gemini BYOK providers call their native SDKs directly,
bypassing CopilotLanguageModelWrapper — so they need the consumer-side span.
Now: always create a chat span in extChatEndpoint with basic metadata
(model, provider, response.id, finish_reasons). For wrapper-based providers,
the chatMLFetcher also creates a richer sibling span with token usage.
* fix(otel): skip consumer chat span for wrapper-based BYOK providers
Only create the extChatEndpoint chat span for non-wrapper providers
(Anthropic, Gemini) that need it as their only span. Wrapper-based
providers (Azure, OpenAI, OpenRouter, Ollama, xAI) get a single rich
span from chatMLFetcher via CopilotLanguageModelWrapper.
Result: 1 chat span per LLM call for all provider types.
* fix: remove unnecessary 'google' from non-wrapper vendor set
* feat(otel): add rich chat span with usage data for Anthropic BYOK provider
Move chat span creation into AnthropicLMProvider where actual API response
data (token usage, cache reads) is available. The span is linked to the
agent trace via runWithTraceContext and enriched with:
- gen_ai.usage.input_tokens / output_tokens
- gen_ai.usage.cache_read.input_tokens
- gen_ai.response.model / response.id / finish_reasons
Remove consumer-side extChatEndpoint span for all vendors (nonWrapperVendors
now empty) since both wrapper-based and Anthropic providers create their
own spans with full data.
Next: apply same pattern to Gemini provider.
* feat(otel): add rich chat span for Gemini BYOK, clean up extChatEndpoint
- Add OTel chat span with full usage data to GeminiNativeBYOKLMProvider
- Remove all consumer-side span code from extChatEndpoint (dead code)
- Each provider now owns its chat span with real API response data:
* CAPI: chatMLFetcher
* OpenAI-compat BYOK: CopilotLanguageModelWrapper → chatMLFetcher
* Anthropic: AnthropicLMProvider
* Gemini: GeminiNativeBYOKLMProvider
- Fix Gemini test to pass IOTelService
* feat(otel): enrich Anthropic/Gemini chat spans with full metadata
Add to both providers:
- copilot_chat.request.max_prompt_tokens (model.maxInputTokens)
- server.address (api.anthropic.com / generativelanguage.googleapis.com)
- gen_ai.conversation.id (requestId)
- copilot_chat.time_to_first_token (result.ttft)
Now matches CAPI chat span attribute parity.
* feat(otel): add server.address to CAPI/Azure BYOK chat spans
Extract hostname from urlOrRequestMetadata when it's a URL string
and set as server.address on the chat span. Works for both CAPI
and CopilotLanguageModelWrapper (Azure BYOK) paths.
* feat(otel): add max_tokens and output_messages to Anthropic/Gemini chat spans
- gen_ai.request.max_tokens from model.maxOutputTokens
- gen_ai.output.messages (opt-in) from response text
- Closes remaining attribute gaps vs CAPI/Azure BYOK spans
* fix(otel): capture tool calls in output_messages for chat spans
When model responds with tool calls instead of text, the output_messages
attribute was empty. Now captures both text parts and tool call parts
in the output_messages, matching the OTel GenAI output messages schema.
Also: Azure BYOK invoke_agent zero tokens is a known upstream gap —
extChatEndpoint returns hardcoded usage:0 since VS Code LM API doesn't
expose actual usage from the provider side.
* fix(otel): capture tool calls in output_messages for Anthropic/Gemini BYOK spans
Same fix as CAPI — when model responds with tool calls, include them
in gen_ai.output.messages alongside text parts. All three provider
paths (CAPI, Anthropic, Gemini) now consistently capture both text
and tool call parts in output messages.
* fix(otel): add input_messages and agent_name to Anthropic/Gemini chat spans
- gen_ai.input.messages (opt-in) captured from provider messages parameter
- gen_ai.agent.name set to AnthropicBYOK / GeminiBYOK for identification
Closes the last attribute gaps vs CAPI/Azure BYOK chat spans.
* fix(otel): fix input_messages serialization for Anthropic/Gemini BYOK
- Map enum role values to names (1→user, 2→assistant, 3→system)
- Extract text from LanguageModelTextPart content arrays instead of
showing '[complex]' for all messages
- Use OTel GenAI input messages schema with role + parts format
* docs(otel): add remaining metrics/events work to plan.md
Coverage matrix showing:
- Anthropic/Gemini BYOK missing: operation.duration, token.usage,
time_to_first_token metrics, and inference.details event
- CAPI and Azure BYOK (via wrapper) fully covered
- Tool/agent/session metrics covered across all providers
- 4 tasks (M1-M4) to close the gap
* feat(otel): add metrics and inference events to Anthropic/Gemini BYOK providers
Both providers now record:
- gen_ai.client.operation.duration histogram
- gen_ai.client.token.usage histograms (input + output)
- copilot_chat.time_to_first_token histogram
- gen_ai.client.inference.operation.details log event
All metrics/events now have full parity across CAPI, Azure BYOK,
Anthropic BYOK, and Gemini BYOK.
* fix(otel): fix LoggerProvider constructor — use 'processors' key (SDK v2)
The OTel SDK v2 changed the LoggerProvider constructor option from
'logRecordProcessors' to 'processors'. The old key was silently
ignored, causing all log records to be dropped.
This is why logs never appeared in Loki despite traces working fine.
* docs: add agent monitoring guide with OTel usage and Claude/Gemini comparison
* docs: remove Claude/Gemini comparison from monitoring guide
* docs: add OTel comparison with Claude Code and Gemini CLI
* docs: reorganize monitoring docs — user guide + dev architecture
- agent_monitoring.md: polished user-facing guide (for VS Code website)
- agent_monitoring_arch.md: developer-facing architecture & instrumentation guide
- Removed internal plan/spec/comparison files from repo (moved to ~/Documents)
* fix(otel): restore _doFetchViaHttp body and _fetchWithInstrumentation after rebase
* fix(otel): propagate otelSpan through WebSocket/HTTP routing paths
The otelSpan was created in _doFetchAndStreamChat but not included
in returns from _doFetchViaWebSocket and _doFetchViaHttp, causing
the caller (fetchMany) to always receive undefined for otelSpan.
Fix: await both routing paths and spread otelSpan into the result.
* docs(otel): improve monitoring docs, add collector setup, fix trace context
- Expand agent_monitoring.md with detailed span/metric/event attribute tables
- Add BYOK provider coverage, subagent trace propagation docs
- Add Backend Considerations: Azure App Insights (via collector), Langfuse, Grafana
- Add End-to-End Setup & Verification section with KQL examples
- Add OTel Collector config + docker-compose for Azure App Insights
- Fix: emit inference details event before span.end() in chatMLFetcher
(fixes 'No trace ID' log records in App Insights)
- Fix: pass active context in emitLogRecord for trace correlation
- Update launch.json to point at OTel Collector (localhost:4328)
* docs(otel): merge Backend Considerations and E2E sections to remove redundancy
* docs(otel): remove internal dev debug reference from user-facing guide
* docs(otel): remove Grafana section and Jaeger refs from App Insights section
* docs(otel): trim Backend section to factual setup guides, remove claims
* docs(otel): final accuracy audit — fix false claims against code
- Mark copilot_chat.session.start event as 'not yet emitted' (defined but no call site)
- Mark copilot_chat.agent.turn event as 'not yet emitted' (defined but no call site)
- Mark copilot_chat.session.count metric as 'not yet wired up'
- Fix OTEL_EXPORTER_OTLP_PROTOCOL desc: only 'grpc' changes behavior
- Fix telemetry kill switch claim: vscodeTelemetryLevel not wired in services.ts
- Remove false toolCalling.tsx instrumentation point from arch doc
- Fix docker-compose comments: wrong port numbers (16686→16687, 4318→4328)
- Add reference to full collector config file from inline snippet
* docs(otel): remove telemetry.telemetryLevel references — OTel is independent
* feat(otel): wire up session.start event, agent.turn event, and session.count metric
- emitSessionStartEvent + incrementSessionCount at invoke_agent start (top-level only)
- emitAgentTurnEvent per LLM response in onDidReceiveResponse listener
- Remove 'not yet wired' markers from docs
* chore: untrack .playwright-mcp/ and add to .gitignore
* chore: remove otel spec reference files
* chore(otel): remove OpenTelemetry environment variables from launch configurations
* fix(otel): add 64KB truncation limit for content capture attributes
Prevents OTLP batch export failures when large prompts/responses are
captured. Aligned with gemini-cli's limitTotalLength pattern.
Applied truncateForOTel() to all JSON.stringify calls feeding span
attributes across chatMLFetcher, toolCallingLoop, toolsService,
anthropicProvider, geminiNativeProvider, and genAiEvents.
* refactor(otel): make GenAiMetrics methods static to avoid per-call allocations
Aligned with gemini-cli pattern of module-level metric functions.
Eliminates 17+ throwaway GenAiMetrics instances per agent run.
* fix(otel): fix timer leak, cap buffered ops, rate-limit export logs
- storeTraceContext: track timers for clearTimeout on retrieval/shutdown,
add 100-entry max with LRU eviction
- BufferedSpanHandle: cap _ops at 200 to prevent unbounded growth
- DiagnosticSpanExporter: rate-limit failure logs to once per 60s
* docs(otel): fix Jaeger UI port to match docker-compose (16687)
* chore(otel): update sprint plan — mark P0/P1 tasks done
* fix(otel): remove as any casts in BYOK provider content capture
Use proper Array.isArray + instanceof checks instead of as any[]
casts for LanguageModelChatMessage.content iteration.
* refactor(otel): extract OTelModelOptions shared interface
Replaces 3 duplicated inline type assertions for _otelTraceContext
and _capturingTokenCorrelationId with a single shared interface.
* refactor(otel): route OTel logs through ILogService output channel
Replace console.info/error/warn in NodeOTelService with a log callback.
OTelContrib logs essential status to the Copilot Chat output channel
for user troubleshooting (enabled/disabled, exporter config, shutdown).
* fix(otel): remove orphaned OTel ConfigKey definitions
OTel config is read via workspace.getConfiguration in services.ts,
not through IConfigurationService.get(ConfigKey). These constants
were unused dead code.
* test(otel): add comprehensive OTel instrumentation tests
- Agent trace hierarchy (invoke_agent → chat → execute_tool, subagent
propagation, error states, metrics, events)
- BYOK provider span emission (CLIENT kind, token usage, error.type,
content capture gating, parentTraceContext linking)
- chatMLFetcher two-phase span lifecycle (create → enrich → end,
error path, operation duration metric)
- Service robustness (runWithTraceContext, startActiveSpan error
lifecycle, storeTraceContext overwrite)
- CapturingOTelService reusable test mock for all OTel assertions
* chore: apply formatter import sorting
* chore: remove outdated sprint plan document
* feat(otel): add OTel configuration settings for tracing and logging
* fix(otel): ensure metric reader is flushed and shutdown properly
* Add support for local repository memory and update telemetry events
* Update memory command labels for clarity in the UI
* update test
* Add repository memory section to snapshot tests for clarity
* ghost: use CompletionsFetchService
also comes with lots of fixes in CompletionsFetchService
* ghost: stream: refactor: extract CopilotAnnotations types to platform/
* completionFetchService: remove hard-coding n=1
* support for copilot_annotations
* fix incorrect import
* fix incorrect destroy of http connection
* fix: update response accumulation logic in CompletionAccumulator
* refresh token more correctly
* Fixes for external ingest
- Make sure we use relative paths
- Add command to clear
- Avoid requests if the index is already up to date
* Small cleanup
* fetcherService: change `FetchOptions#json` to be `unknown` instead of `any` and document it
* completionsFetchService: don't use `any`
* completionsFetchService: less logic within service
try reducing logic such that the service's easier to evolve
* completionsFetchService: refactor: cleanup
* address comments
* fix ci
Enables the same `no-unexternalized-strings` with have in `vscode` in this repo. This make sure we have a more consistent style across repos and when generating edits
* init
* Re-work code search implementation
Prepping for future work by making it easier to have individual repos in a workspace handle code search their own way. The previous implementation had gotten kind of unwieldy after we bolted on the ADO support. Extending it further would have made this even worse
* tool can give alternative definition by model
* Pass IChatEndpoint in getEnabledTools for calling tools' alternativeDefinition
* Introduce a ICopilotToolExtension to support override existing built-in tools and provide alternative definition.
* Fix errors
* add gpt-5-codex
* remove gpt-5
* remove references to old setting `github.copilot.chat.advanced.inlineChat2`
* play with `InlineChatIntent`
* wip
* move things, better/simpler prompt
* cleanup, renames, stuff
* more wip
* done after tool call
* edit and generate stest for new InlineChatIntent
* use codebook for diagnostics
* inline chat fixing stests
* stest run
* remove old Inline2 tests
* remove slash commands for v2, remove the editCodeIntent path for v2
* 💄
* 💄
* Don't use `diagnosticsTimeout` when with inline chat because the new diagnostics will never be read but slow down the result
* fix compile error
* stest run
* update baseline
* prevent some JSON errors from empty output
* unfresh baseline.json
* use `MockGithubAvailableEmbeddingTypesService` in stests
* back to hamfisted skipping of stests
* send telemetry from inline chat intent
* tweak some stests
