-
Notifications
You must be signed in to change notification settings - Fork 4
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
compareRuns accepts negative/NaN tolerance
good first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.#30 In AgentPostmortem/Evalgate;Scorer weight: Infinity passes validation and poisons aggregation
good first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.#29 In AgentPostmortem/Evalgate;contains scorer with empty-string needle always passes
good first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.#28 In AgentPostmortem/Evalgate;Suite validation accepts empty prompts and bogus message roles
good first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.#27 In AgentPostmortem/Evalgate;Invalid numeric CLI flags silently fall back instead of erroring
good first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.#26 In AgentPostmortem/Evalgate;Cost is one hardcoded rate per provider, applied to every model and both token directions
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#16 In AgentPostmortem/Evalgate;Provider calls have no timeout and no retry, so a 429 fails the build
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#15 In AgentPostmortem/Evalgate;One provider or scorer error discards the entire run
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#14 In AgentPostmortem/Evalgate;Deleting a failing case makes the gate green: removed cases are never regressions
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#13 In AgentPostmortem/Evalgate;cost scorer passes with a perfect score when the provider reports no cost
bugSomething isn't workingSomething isn't workingStatus: Open.#7 In AgentPostmortem/Evalgate;Bug: a run with zero cases reports passed and exits 0, so the gate silently disables itself
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is neededStatus: Open.#6 In AgentPostmortem/Evalgate;--tags and --concurrency are ignored by baseline and compare
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is neededStatus: Open.#5 In AgentPostmortem/Evalgate;