* utils: document binarySearch
* nes-datagen: generate training data from continuous recordings
Continuous enhanced telemetry now ships sliding-window recordings that, unlike per-request alternative-action recordings, carry no requestTime. The datagen pipeline needs a point to split each recording into edit history before/after, so this adds a pluggable pivot strategy (starting with Random, selectable via --pivot-strategy) and a new continuous/ pipeline module that replays a recording at the chosen pivot to produce a processed row.
Along the way this consolidates the pipeline's error and index handling: a shared WithRowIndex<T> replaces the ad-hoc { originalRowIndex, ... } pairs, per-record processing returns Result<IProcessedRow, Error> instead of field-presence unions, and failures surface as original Error objects (no string round-tripping). The telemetry sender's continuous payload is now the documented IContinuousRecording type.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* nes-datagen: label alt-action replay errors by originalRowIndex
Address PR review: the alternative-action path mislabeled diagnostics when
earlier records failed to parse.
- processAllRows: push replay errors with the row's true `originalRowIndex`
instead of its position in the filtered `rows` array (parse failures make
`rows` sparse, so the two diverge).
- loadAndProduceProcessedRows: resolve `languageForRow` via an
`originalRowIndex`-keyed Map rather than positional `rows[i]`, matching how
callers pass `e.originalRowIndex`.
- Clarify the `recordCount` doc: it counts successfully-parsed records (parse
failures are counted separately in `parseErrors`).
- Add a regression spec asserting replay errors carry the row index, not the
array position.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Agent Host changes for agents/adhoc-request-sender-mode-extension-55e2bb6f
* Remove unconfigured react-hooks/exhaustive-deps eslint directive
The eslint-disable directive referenced a rule that isn't registered in
this repo's ESLint config, which caused ESLint to error with
"Definition for rule 'react-hooks/exhaustive-deps' was not found" and
failed the Compile & Hygiene and Copilot - Test CI checks.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Coalesce adhoc tag-decoration rescans with requestAnimationFrame
Rescanning the whole editor text on every content change is wasteful for
bursty updates (e.g. a streamed response). Debounce the decoration update
to at most once per animation frame and cancel any pending frame during
cleanup so the callback can't run after the editor is disposed. The
initial scan stays synchronous so tags are highlighted immediately on mount.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR feedback: dispose token source; validate adhoc request JSON
- adhocRequestSender: always dispose the per-send CancellationTokenSource
in the finally block (separate from the current-send guard) so its
cancellation listeners don't leak across repeated Send/Stop cycles.
- simulationMain: validate and normalize the adhoc request JSON before use
so malformed input (missing/null/wrong-typed model/user/system) yields a
focused error message instead of a thrown stack trace.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: restore chat pipeline to windows-latest
* chore: remove node-gyp override
* chore: restore node-gyp override with comment
* refactor: rm dependency on key:sqlite
The module locks the node-gyp dependency to 8.x due to
its transitive sqlite3 native module dependency this in turn
blocks using newer windows CI, refs https://github.com/microsoft/vscode/issues/321267
The module can be replaced with built-in sqlite support
from Node.js without losing the on-disk cache format has
already been committed.
* chore: restore minimist
* chore: set sqlite busy timeout
* fix: decode json-buffer values for keyv cache compat
* nes-datagen: add cursor-jump (NCLP) task
Extend nes-datagen with a next-cursor-line prediction task alongside
the existing xtab path. Detects the user's next intentional cursor
move after the request bookmark and emits a training sample with the
production cursor-prediction prompt + the observed jump as the
expected response.
Three sub-modes via --sample-task:
- cursor-same-file: a jump farther than N lines from cursor at
request time
- cursor-cross-file: focus/selection on a different file
- cursor-both: either of the above
Reuses the production cursor-prediction prompt by capturing it via
the telemetry builder and a no-op fetcher; the cross-file target
line is resolved from a request-time content snapshot + post-request
replay so previously-opened targets get a correct line number
instead of being silently labelled :0.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: replace SAMPLE_TASK_VALUES tuple with a string enum
Convert the string-union + as-const tuple to a proper NesDatagenSampleTask
string enum. CLI surface is string-enum members keep theunchanged
kebab-case wire values ('xtab', 'cursor-same-file', ...). All consumers
(dispatch, fixtures, response metadata typing) updated to reference enum
members instead of string literals.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: lower default --same-file-jump-min-above to 2
Upward cursor jumps (back to a definition, an import, etc.) are
typically tighter than downward jumps after the user has been
writing. Lower the default threshold to match.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: rename NCLP to cursor-jump throughout
Drop the NCLP abbreviation in favor of the more descriptive
'cursor-jump' name already used in the production xtab provider.
cursorJumpPromptStep,
cursorJumpResponseStep), the capture request ids,
and all surrounding doc comments / test descriptions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: build documentIndexMapping from whole recording
path map from only the
pre-request slice and then re-walked the post-request slice to
backfill any documentEncountered entries that arrived later. Pass the
whole recording into documentIndexMapping instead so the helper sees
every document the user touched in a single pass; the backfill loop is
gone.
splitRecordingAtRequestTime now also returns the full entries array so
both callers can reuse it without re-deriving it from altAction.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: use shared Result type in cursor-jump detectors
Drop the bespoke { ok, value | reason } discriminated union in
detectJump.ts and reuse the existing Result<T, E> from
src/util/common/result. JumpDetectionResult<T> is now just an alias
for Result<T, string>.
.isOk(),
.err) and the spec file accordingly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: strip raw cursor-jump prompt from emitted telemetry
cursorJumpRawMessages and cursorJumpKeptRange were added to
IStatelessNextEditTelemetry so in-process debug / datagen tooling
could read them back via getStatelessNextEditTelemetry(). However
LlmNESTelemetryBuilder.build() spreads ...this._statelessNextEditTelemetry
into the emitted payload, so those two fields would leak to telemetry
cursorJumpRawMessages can contain full prompt content (sourcesinks
code), which must never leave the process.
Destructure them out before spreading into the build() payload. They
remain readable via getStatelessNextEditTelemetry() for tooling.
Documented the privacy contract on the IStatelessNextEditTelemetry
field declarations so future edits don't forget.
Addresses copilot-pull-request-reviewer feedback on PR #320113.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: fail cross-file detection when no selection lands on target
detectCrossFileJump previously returned Result.ok with toLine
undefined when only a focused event was seen for the target doc (no
selectionChanged). That left generateCrossFileResponse to drop the
sample later while the detector still reported a successful jump.
Treat focused-without-selectionChanged as a failed detection
('crossFileTargetNoSelection') so callers can skip early, and tighten
ICrossFileJump.toLine to non-undefined now that ok results always
have a usable line number. Removes the dead error path in
generateCrossFileResponse.
Adds a regression test that focused-only triggers the new error.
Addresses copilot-pull-request-reviewer feedback on PR #320113.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: capture cursor-jump prompt via logContext, not telemetry
The datagen pipeline previously stashed the raw cursor-jump prompt and
keptRange on IStatelessNextEditTelemetry so cursorJumpPromptStep.ts could
read them back via LlmNESTelemetryBuilder.getStatelessNextEditTelemetry().
That leaked raw prompts into the telemetry payload (worked around by a
destructure-strip hack in LlmNESTelemetryBuilder.build()) and was
asymmetric with the xtab path, which captures via
InlineEditRequestLogContext.rawMessages.
Move the cursor-jump capture vehicle onto InlineEditRequestLogContext to
match xtab:
- Add cursorJumpRawMessages / cursorJumpKeptRange fields and
setCursorJumpPrompt(messages, keptRange) to InlineEditRequestLogContext.
- XtabNextCursorPredictor.predictNextCursorPosition now takes a logContext
parameter and writes to it directly. The xtabProvider callsite passes
the same logContext it already had in scope.
- cursorJumpPromptStep reads from logContext instead of the telemetry
builder.
- Remove cursorJumpRawMessages / cursorJumpKeptRange from
IStatelessNextEditTelemetry, plus the corresponding setter/getter on
StatelessNextEditTelemetryBuilder and the getter on
LlmNESTelemetryBuilder.
- Revert the destructure-strip hack in LlmNESTelemetryBuilder.build().
The pre-existing cursorJumpPrompt telemetry field (JSON-stringified, fed
by setCursorJumpPrompt(messages)) is intentional and unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: cursor-jump ground truth is first user EDIT, not cursor landing
Selection-based detection treated peek, navigation, IDE auto-scroll, and
recursive cursor settling as if they were the user's next intended edit
location. The model's job is to predict where the user will EDIT next, so
key off the first 'changed' event after the request bookmark instead.
Same-file detector:
- Walks for the first 'changed' on the active doc; uses the first edit's
start offset to compute toLine; applies the linesAbove/linesBelow
threshold. Bails with editsAnotherFileFirst when a non-active doc is
edited first (lets the cross-file detector claim the sample in
cursor-both mode). 'selectionChanged' is no longer consulted, so the
settle-after-edit filter is gone it was a workaround for thetoo
selection-based approach.
Cross-file detector:
- Walks for the first 'changed' on a non-active doc; uses the first
edit's start offset, resolved against the target doc's snapshot
just-before applying the event. Drops focused / selectionChanged
heuristics and the crossFileTargetNoSelection error path (a focused
event without an edit no longer counts; background peek can't
pollute the dataset).
buildLineResolver: tightened i <= entryIndex to i < entryIndex so the
resolver returns the pre-edit line when entryIndex is itself a 'changed'
event. The bound is equivalent for the old selectionChanged caller.
Spec: switched ground-truth events from selChanged to changed; added
coverage for first-edit-of-multi-edit, editsAnotherFileFirst, and
active-doc-then-other-doc ordering.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* xtab: predictNextCursorPosition takes RequestTracingContext
Every other helper in xtabProvider takes RequestTracingContext (the
{ tracer, logContext, telemetry } bundle). The cursor predictor was the
odd one out, taking the three pieces as separate positional params with
the latter two that asymmetry made the new logContext-captureoptional
plumbing look more invasive than it is and forced an awkward
?.setCursorJumpPrompt chain at the use site.
Switch the predictor to take RequestTracingContext directly:
- Export RequestTracingContext from xtabProvider so the predictor can
type-import it (TS-erased to avoid the runtime circular import).
- predictNextCursorPosition signature collapses from 5 params to 3.
- Drop the optional chains; tracing.telemetry / tracing.logContext are
always present in production and the spec constructs a real bundle.
- Spec adds a createTestTracingContext helper using the cheap
InlineEditRequestLogContext / StatelessNextEditTelemetryBuilder
constructors already used by other inlineEdits specs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: rename splitRecording 'entries' field to 'wholeRecording'
Review feedback: the field on the splitRecordingAtRequestTime return
shape was named 'entries' but in context it carries the whole unsplit
recording (i.e. before slicing into prior/after parts). 'wholeRecording'
matches the comment at the consumer (documentIndexMapping callsite).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: inline JumpDetectionResult<T> as Result<T, string>
Review feedback: the one-line alias was used in exactly two places in
the same file and gave nothing over the underlying Result type.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: discriminated union for sample task + jump metadata
Review feedback: ISampleMetadata had 'task' + an optional 'jump' field
with toFilePath also optional. That let xtab samples accidentally carry
a jump and let cursor-cross-file samples omit toFilePath. Replace with
a discriminated union on task:
- xtab: no jump
- cursorSameFile: jump with fromLine/toLine/distance
- cursorCrossFile: jump with required toFilePath
assembleSample now takes a single SampleClassification arg, removing
the parallel task/jump parameters that callers had to keep in sync.
cursorJumpResponseStep is split into ISameFileGeneratedResponse and
ICrossFileGeneratedResponse so the generator return types map cleanly
to the union variants without a non-null assertion at the assembly
site.
DetectedJump no longer needs an assistantTask hint: the pipeline
constructs the classification directly from the response shape.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix dup import
* nes-datagen: address round-4 review feedback on pipeline.ts
Five review threads on pipeline. all addressed in-place.ts
- modelResponse: cursor samples were emitting an empty string for the
expected response. Populate it with the assistant content (which IS
the expected output) so downstream tooling has the gold label.
- Promise.all unbounded throws: wrap the limiter callback body in
try/catch so an unexpected exception from generateCursorPromptFromRecording
becomes a recorded per-row error instead of aborting the whole batch
via Promise.all's first-rejection semantics.
- Inline import for OffsetRange: replace the inline import('...').OffsetRange
type expression with a regular top-of-file import.
- Duplicated config-override block: both pipelines applied the same
applyConfigFile + four setConfig debounce/cache disables. Extract
into applyBatchModeConfig(configService, configs) and call from both.
- runInputPipeline parallelism + memory: add a doc comment clarifying
that this is the single-process entry point, that cursor-jump tasks
also benefit from runInputPipelineParallel (--sample-task is
propagated to workers), and that loadAndParseInput is in-memory by
design (sized per worker; use --parallelism > 1 for large inputs).
Full architectural unification of the parallel and non-parallel
paths is intentionally left as a follow- the surface area isup
large and out of scope for this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: add e2e tests for cursor-jump pipeline
Mirrors the existing xtab pipeline.e2e.spec.ts: drives two fixture rows
(a same-file jump and a cross-file jump) through the full
`runInputPipeline` for each `sampleTask` mode (cursor-same-file,
cursor-cross-file, cursor-both) and asserts on the JSONL output.
Coverage:
- only the matching row is emitted per mode; both rows are emitted in
cursor-both
- emitted samples carry strategy=next-cursor-line-prediction and the
correct discriminated `task` field (cursor-same-file / cursor-cross-file)
- assistant message targets the jumped-to line / file
- metadata.modelResponse mirrors the assistant content (the round-4 fix)
- --row-offset is reflected in metadata.rowIndex
Test fixtures are constructed in
`fixtures/cursorJumpFixtureData.ts` with synthesized recordings: an
explicit no-op edit + selectionChanged before the bookmark so the
cursor-prediction path's recent-edit gating is satisfied, then a
single post-request `changed` event the detector picks up.
The cursor pipeline needs a prompting strategy whose response handler
tolerates an empty stream — use `xtabUnifiedModel` in a dedicated
`cursorJumpConfig.json` (the existing patchBased02 config crashes on
empty output, which is acceptable in production but breaks the
prompt-only capture path).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Strengthen cursor-jump e2e assertions
Replace fuzzy matchers (toMatch(/25/), arrayContaining for tasks) with
exact assertions on assistant content, metadata.task, and metadata.jump.
In cursor-both, locate samples by filePath so a row→classification swap
would now be caught instead of passing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Make cursor-jump e2e helper accept partial nesDatagen overrides
Helper previously took Partial<RunPipelineOptions>; if a caller passed
`nesDatagen`, the spread fully replaced the default block and the
configured path. Now the helper accepts a partial nesDatagen overlay
and merges field-by-field, so the row-offset test only specifies the
two fields it actually changes and there are no non-null assertions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add within-threshold cursor-jump negative fixture
Scenario C: cursor on line 10, post-request edit on line 12 (only 2
lines below). Default threshold is ±5 lines, so neither the same-file
nor the cross-file generator should emit a sample for this row.
Asserted in cursor-both via a dedicated 'does not emit a sample for
the within-threshold row' test, and implicitly in cursor-same-file /
cursor-cross-file (their existing count==1 assertions would fail if
the threshold guard regressed).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Ulugbek Abdullaev <ulugbekna@github.com>
* chore: update @github/copilot and @github/copilot-sdk to version 1.0.61 and 1.0.1 respectively
- Bump @github/copilot from 1.0.60 to 1.0.61 in package.json and package-lock.json
- Update @github/copilot-sdk from 1.0.0 to 1.0.1 in package.json and package-lock.json
- Modify postinstall script to copy tgrep files instead of sharp files
- Update tests to include tgrep binaries
- Change model in e2e tests from 'claude-opus-4.7' to 'gpt-5-mini'
* fix: add libm.so.6(GLIBC_2.27)(64bit) to referenceGeneratedDepsByArch
* nes-datagen: stream large JSON array inputs to avoid 2 GiB readFile limit
Reading the whole input via fs.readFile fails for files larger than 2 GiB
(and exceeds V8's max string length). Add a streaming JSON-array parser and
use it in both the sequential and parallel pipeline paths so multi-GB
recordings can be processed with bounded memory.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: also accept JSON Lines (NDJSON) input
Auto-detect the input format from the first non-whitespace character: a
leading '[' is parsed as a single JSON array, otherwise the file is parsed
as JSON Lines (one JSON object per line). Both formats are streamed so
multi-GB inputs work regardless of shape. Rename streamJsonArray ->
streamJsonRecords to reflect the broader purpose.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: infer JSON vs JSON Lines input format from file extension
Use the file extension (.jsonl/.ndjson -> JSON Lines, otherwise JSON array)
to select the streaming parser instead of sniffing the content.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: validate streamed JSON arrays for truncation and malformed input
The new streaming parser previously accepted any prefix of a JSON array
silently: a truncated file (no closing ']'), a missing element between
commas, a trailing comma or trailing data after the array all produced
zero or fewer records rather than an error. That is especially dangerous
for the multi-GB inputs this parser was introduced for, because the
underlying file is much more likely to be incomplete.
Tighten the state machine to surface these as errors, matching what the
old whole-file JSON.parse would have done, and add tests for each case.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: stream worker-result merging and final output write
For multi-GB inputs the parent process was still hitting V8's ~512 MiB
max-string-length limit in two places after the input-side fix:
1. Merging worker result files used fs.promises.readFile + JSON.parse on
each result, but with a 5+ GB input split N ways each per-worker file
is hundreds of MB of similar-shaped data and easily exceeds the
string limit.
2. writeSamples serialized the entire validSamples array via a single
JSON.stringify(arr, null, 2) before writing, which has the same
problem on output.
Switch both to stream over individual records:
- A new shared openWriteStream(filePath) helper wraps fs.createWriteStream,
attaches an 'error' listener immediately (so async write failures don't
surface as uncaughtException and skip cleanup), awaits backpressure via
the per-write callback, and exposes an idempotent close().
- writeChunkFiles uses the helper inside a try/finally so any mid-stream
ENOSPC/EIO bubbles up cleanly and the tmp dir is still removed.
- The merge step now uses streamJsonRecords<ISample>(resultPath), so the
parent never materializes a single worker's output as one string.
- writeSamples emits the output JSON array incrementally: per-sample
JSON.stringify(..., null, 2) (indented two spaces to match the previous
layout) joined with ',\n'. Byte size is accumulated for the existing
IWriteResult.fileSize.
Also documents that single-process loadAndParseInput still buffers the
full row set in memory and that --parallelism is required for very large
inputs (workers each only load their slice).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: switch output to JSON Lines
Both the final user-facing output and the per-worker intermediate result
files now use JSON Lines (one record per line) instead of a
pretty-printed JSON array. JSONL is dramatically simpler to write and
read incrementally: no surrounding brackets/commas to track, no
multi-line per-element indentation, just JSON.stringify + '\n' per
record on the write side and split-on-newline + JSON.parse per non-empty
line on the read side (this is what streamJsonRecords already does when
it detects the .jsonl extension).
Changes:
- writeSamples emits one JSON.stringify(sample) + '\n' per validated
sample via no array wrapper, no pretty-printing.openWriteStream
- resolveOutputPath defaults the implicit output to <input>_output.jsonl
(was <input>_output.json).
- Per-worker result files in runInputPipelineParallel are now
result_${w}.jsonl, so the merge step's streamJsonRecords auto-picks
the JSONL parser from the extension.
- E2e tests updated to read JSONL (split on newline, JSON.parse per
line) and to use .jsonl output paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* nes-datagen: surface worker-result parse errors and update --out help text
Two review follow-ups:
1. The merge step that streams each worker's result file used to wrap
the iteration in a try/catch that downgraded any parse error to a
console.error warning. With the new streaming reader that is unsafe:
streamJsonRecords yields N valid records first and then throws on a
malformed/truncated tail, leaving those N partial records already in
allSamples. The pipeline would then quietly emit a truncated
training-data output. Drop the swallowing try/catch so a corrupt
worker result aborts the run non-zero.
2. The --out help text in simulationOptions.ts still advertised the old
JSON-array default (<input>_output.json). Update it to reflect the
JSONL output, and also note in --input that the format is inferred
from the .jsonl/.ndjson extension.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove leftover code from /tests test-generation codelens
The codelens feature for /tests test generation was removed in
c0eaa94d6a, but related infrastructure remained as dead code:
- ITestGenInfoStorage was never populated (the deleted testGenAction.ts
was the only writer), so TestFromTestInvocation and Test2Impl always
took the 'undefined' branch.
- IParserService.getTreeSitterAST().getTestableNode/getTestableNodes
and the underlying testGenParsing helpers + testableNodeQueries had
no remaining callers.
Delete the storage service, the parser API, the tree-sitter queries,
and the related tests. The /tests slash command itself is preserved;
`_findLastTest` stays since testFromSrcInvocation.tsx still uses it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add archived state management and worktree sharing for chat sessions
- Introduced `archived` property in `ChatSessionMetadataFile` to track archived sessions.
- Updated `IChatSessionMetadataStore` interface to include methods for setting and getting archived state.
- Implemented logic in `ChatSessionMetadataStore` and `MockChatSessionMetadataStore` to handle archived sessions.
- Added `getBlockingSiblingSessionsForFolder` utility to identify sessions that share worktrees.
- Modified CLI chat session commands to respect archived state during session deletion and worktree cleanup.
- Updated tests to cover new functionality and ensure proper behavior of session management.
* Add worktree sharing edge-case unit tests
Co-authored-by: DonJayamanne <1948812+DonJayamanne@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Introduces dedicated endpoint resolver classes (CopilotUtilitySmallChatEndpoint,
CopilotUtilityChatEndpoint) in place of the ModelAliasRegistry indirection, and
republishes the resolved utility endpoints under their new family ids
('copilot-utility-small', 'copilot-utility') as LanguageModelChatInformation
entries so that workbench callers using selectLanguageModels({ vendor: 'copilot',
id: 'copilot-utility-small' }) keep working.
- Renames the internal family identifiers everywhere they're consumed:
callers, tests, and workbench code in src/vs/workbench/contrib/chat/.
- Drops src/platform/endpoint/common/modelAliasRegistry.ts.
- CopilotUtilitySmallChatEndpoint.resolve tries a small primary model and
falls back to a second small model if the primary is unavailable.
- CopilotUtilityChatEndpoint.resolve returns the API-marked default base
model (is_chat_fallback === true), preserving existing behavior.
- Updates _copilotBaseModel field and 'copilot-base' literal in
ModelMetadataFetcher to _copilotUtilityModel / 'copilot-utility'.
No user-facing behavior change. The setting-driven 'disabled'/'default'/BYOK
selector support will be layered on top in a follow-up.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
remove comment
remove code smell
remove more code smell
* feat: make stream a caller-controlled passthrough in Messages API
Allow callers to set stream: false via requestOptions instead of
hardcoding stream: true. Add non-streaming response handler for the
Anthropic Messages API that parses single JSON responses.
- createMessagesRequestBody: stream: true → options.requestOptions?.stream ?? true
- preparePostOptions: stream: true as default before spread (callers can override)
- processResponseFromMessagesEndpoint: auto-detect via Content-Type header
- processNonStreamingResponseFromMessagesEndpoint: new handler for JSON responses
with tool call support in finishedCb delta, defensive parsing, cache-token
consistency warning, unknown block type logging
- Remove stale 'stream not respected' comment from fetch.ts
- Remove stream: false from agentIntent.ts inline summarization
- 10 new tests for non-streaming handler
* fix: add telemetry parity for non-streaming path and bump cache salt
* regenerate simulation cache for review-inline tests
* Regenerate simulation cache after rebase
* Temporarily disable multifile-edit-claude variant (#315940)
claude-3.5-sonnet returns model_not_supported from the endpoint, breaking
simulation cache regen. Re-enable when the test is updated to use a
currently-supported Claude model.
* Fix terminal strict-mode crash on empty suggestions + update baseline
- terminal.stest.ts: guard strict-mode `ok()` predicate so when the model
returns no code block, the test fails cleanly with the existing message
instead of crashing with 'Cannot read properties of undefined (reading match)'.
Also drop the stale commented-out debug block.
- baseline.json: refresh scores (68.01 -> 68.69) and drop the 14 entries for
the disabled multifile-edit-claude variant (see #315940).
- Remove now-orphaned multifile-edit-claude-panel.json outcome file.
* Apply CI-observed score improvements for cpp inline scenarios
CI on Linux scores 4 cpp InlineChatIntent scenarios higher than my local
macOS run does (likely platform-specific line-ending/whitespace normalization
in the cpp grader). Update baseline.json to match the Linux scores:
- edit-InlineChatIntent [inline] [cpp] - edit for cpp: 5 -> 9
- edit-InlineChatIntent [inline] [cpp] - edit for macro: 0 -> 2
- generate-InlineChatIntent [inline] [cpp] - cpp code generation: 3 -> 10
- generate-InlineChatIntent [inline] [cpp] - templated code gen: 0 -> 10
Overall score: 68.69 -> 68.86.
* Populate cpp diagnostic cache via Docker for cross-platform parity
The earlier rebase cache regen produced new LLM responses for the cpp
inline tests but failed to populate the clang diagnostic provider cache
for those new inputs, because clang detection on macOS is broken (Apple
clang prints '-v' output to stderr, but findIfInstalled only checks
stdout) and Docker wasn't running. As a result the cpp diagnostic cache
was missing entries for the new LLM responses, and CI re-ran clang live
on each platform with diverging results:
- Linux CI: clang available, scored highest (9, 2, 10, 10)
- Windows CI: no clang, errored out (5, 0, 10, 10 with worsening)
- macOS: Apple clang misdetected as missing, Docker off, errored
This commit:
1. Bumps CLANG_DIAGNOSTICS_PROVIDER_CACHE_SALT 5 -> 6 to invalidate
any contaminated entries.
2. Adds two new cache layers populated by running cpp tests via Docker
(using the mcr.microsoft.com/devcontainers/cpp image, same Linux
clang as CI). All 14 cpp scenarios now produce deterministic,
platform-independent diagnostic results when read from cache.
Verified with --require-cache: all cpp scenarios pass without invoking
clang/docker at runtime.
Strip Copilot Memory (CAPI) feature entirely
Removes the CAPI-backed Copilot Memory that synced repository-scoped facts
to GitHub. The local file-based MemoryTool with user/session/repo scopes
remains as the sole memory mechanism.
- Delete AgentMemoryService and its test.
- Remove the github.copilot.chat.copilotMemory.enabled setting and its NLS string.
- Remove ConfigKey.CopilotMemoryEnabled.
- Strip all CAPI gating in memoryTool.tsx, memoryContextPrompt.tsx, tools.ts.
- Drop _dispatchRepoCAPI / _repoCreate / _sendRepoTelemetry.
- /memories/repo/ now always routes to local storage.
- Update memoryTool.spec.tsx: remove mock CAPI services and CAPI-only tests.
- Update simulationExtHostToolsService.ts for the new ToolsContribution arity.
* fix(nes-datagen): discard xtab-275 oracle edits outside edit window
The NES model can only edit lines inside the prompt's <|code_to_edit|>
window [K, N). formatAsEditWindowOnly was expanding the window to cover
stray oracle edits, so the assistant text spilled out of the window and
duplicated surrounding context when applied.
Instead, keep edits in order and drop the first edit that isn't fully
contained in [K, N) along with every later edit (later offsets assume
earlier edits were applied). Add unit tests covering both the spillover
repro and the all-in-window control case.
* refactor(nes-datagen): extract filterEditsInsideEditWindow helper
Pull the line-containment filter out of formatAsEditWindowOnly into a
reusable helper, plus a small getEditLineRange utility to dedupe the
offset->line conversion. No behavior change.
* refactor(nes-datagen): route xtab-275 dropped-edit warnings through pipeline logger
Replace the console.warn in formatAsEditWindowOnly with a structured
return shape ({ assistant, droppedCount }) and a ResponseLogger threaded
through generateResponse / generateAllResponses. The pipeline now logs
dropped edits via its existing log callback (so warnings are visible to
dataset curators and captured by the e2e test logs), and surfaces the
count on IGeneratedResponse.droppedEditCount.
* fix(nes-datagen): correct netLineChange math for full-line deletions
splitLines("L6\n") returns ['L6', ''] (length 2), so deleting a single
terminated line was counted as -2 lines instead of -1. Compute the delta
by counting newlines in the old segment vs. the new text, which gives
the correct line-count delta for any combination of insertions,
deletions, or replacements without the trailing-empty quirk.
* test(nes-datagen): cover line-changing and boundary-straddling oracle edits
Add cases for in-window inserts that grow the slice, in-window deletes
that shrink it, edits that straddle the window boundary (must be
discarded), and the all-edits-outside fallback (assistant equals the
original window slice with droppedCount equal to all edits).
* fix(nes-datagen): filter oracle edits independently against edit window
Oracle edits come from StringEdit.compose().replacements upstream
(processor.ts) and applyEditsToContent applies them all to the original
doc by sorting offset-descending — so each edit's offsets are
independent. The earlier drop-and-truncate rule (which discarded every
edit after the first out-of-window one) was over-conservative and threw
away in-window edits that were perfectly applicable.
Filter each edit on its own merits and update the test that relied on
the truncate behavior to assert independent filtering instead.
* feat(nes-datagen): surface dropped-edit count in pipeline summary
Sum droppedEditCount across all generated responses and append it to
the [4/5] log line when nonzero, so dataset curators running the
pipeline can see at a glance how many oracle edits were discarded as
out-of-window. Silent omission was the original bug; making it visible
in the summary closes the feedback loop.
* Make xtab modelConfiguration an advanced setting
Move InlineEditsXtabProviderModelConfiguration from TeamInternal to Advanced. Renames the key from chat.advanced.inlineEdits.xtabProvider.modelConfiguration to chat.inlineEdits.xtabProvider.modelConfiguration with a migration for backward compatibility, and exposes it via package.json. Experiments continue using the separate modelConfigurationString setting.
* Fix localModelConfig type to allow null
* Use fully-qualified key in simulation help text
This prevents this annoying warning from getting printed all the time
```
[watch-copilot ] [watch:esbuild ] (node:12319) [MODULE_TYPELESS_PACKAGE_JSON] Warning: Module type of file:///Users/matb/projects/vscode/extensions/copilot/.esbuild.ts is not specified and it doesn't parse as CommonJS.
[watch-copilot ] [watch:esbuild ] Reparsing as ES module because module syntax was detected. This incurs a performance overhead.
[watch-copilot ] [watch:esbuild ] To eliminate this warning, add "type": "module" to /Users/matb/projects/vscode/extensions/copilot/package.json.
[watch-copilot ] [watch:esbuild ] (Use `node --trace-warnings ...` to show where the warning was created)
```
Co-authored-by: Copilot <copilot@github.com>
Fixes#312210
This is using the old search endpoint. We'll start switching over the new one once it's ready
Co-authored-by: Copilot <copilot@github.com>
Starts working through some basic repo wide eslint rules for the copilot extension. Stuff like missing semicolons and missing readonly modifiers for disposables
* refactor: remove getSessionIdForWorkspaceFolder method and update clearWorkspaceChanges to handle both session IDs and folder URIs
* Address comments
