* Allow invoking simulationMain with alternative action input
* Address review comments: rename CLI opts, extract pipeline, fix correctness issues
- Rename CLI options with --train- prefix (--train-input, --train-strategy,
--train-out, --train-row-offset, --train-worker) and document all options
- Extract runInputPipeline/runInputPipelineParallel to test/pipeline/trainPipeline.ts
- Preserve original row index through parse/replay/prompt pipeline to fix
sample numbering drift when rows are filtered out
- Fix parseSuggestedEdit: use JSON.parse for escaped text, handle missing delimiter
- Fix line number regex to accept optional space after | (WithoutSpace format)
- Clamp concurrency to >= 1, type samples as ISample[], wrap dispose in try/finally
- Gate verbose logging in loadAndParseInput behind verbose flag
- Use splitLines from existing utility instead of local duplicate
* move nes-datagen to a subcommand
* more code reuse around setting promptStrategy and model config
* Address review: use ResponseFormat, Limiter, assertNever, and raw messages
* minor refactor runPipeline
* finalize
* use POT instead of custom code
* move files from script/ to test/pipeline/
---------
Co-authored-by: ulugbekna <ulugbekna@gmail.com>
Enables the same `no-unexternalized-strings` with have in `vscode` in this repo. This make sure we have a more consistent style across repos and when generating edits
* Unify embeddings paths
Our code has two embeddings paths for legacy reasons:
- The capi api based one that only supports text3small. This is used everywhere except `#codebase`
- The new github embeddings endpoint based. This one supports additional embedding types and options. However it is currently only used in `#codebase`
This change switches everything to use the new github embeddings endpoint instead
* Fix static deps in tests
* add ability to specify modelConfig from file
* use custom model in test endpoint provider
* fix model check
* change model config to be independant from specifying model to run
* support yaml
* use readFileSync
* remove yaml parsing
* improve comment
* fix api key
* fix comment
* make opt parsing alig to comment
