Cases
A case is a finished task kept as an example. Run a skill against its cases to see whether a change made it better or worse before you publish.
On this page9 sections
What a case is#
A case records three things about a task that went well:
- what the task was asked to do;
- the files it read;
- the files that stand as the right answer.
Running a case gives the same task to the agent again, with the skill as it is now, and compares what it produces with the answer. A set of cases tells you whether a change to a skill fixed one thing and broke another.
Add a case#
There are two ways, and both start from real work.
From a finished task. In a conversation's Task List, a task that is done and used a skill has Add to cases.
- Click Add to cases.
- Check What the task was asked to do, the files it read, and the files that stand as the answer.
- Choose Judged by: A script or A person.
- Optionally set a Review by date, then click Add case.
If the task's review was corrected by a person, the corrected files become the expected answer.
From the track record. In the skill's tab, Promote to case on a recent use does the same with the task's own details.
Cases are listed in the Cases section of the skill's tab, in its home. Each shows what it asks, the task it came from, who added it and its last result.
How a case is judged#
| Judged by | How it works |
|---|---|
| A script | A check script in the skill's folder compares the output with the expected answer and reports pass or fail. |
| A person | Someone reads what was produced and says whether it is right. |
If a skill has no check script, a person judges each case.
Run the cases#
- In the skill's tab, open Cases.
- Under Against, choose The draft or a published version.
- Click Run cases, then Run to confirm.
Each case runs in its own conversation, listed under Case runs in the workspace. Your working files and other conversations are not touched, and the agent is not shown the expected answer.
A run carries on after you close the tab. Progress is shown as it goes. Click Cancel run to stop; cases that had not finished say so.
Every case ends as one of:
| Result | Meaning |
|---|---|
| Passed | The check passed, or a person said the output is right. |
| Failed | The check failed, or a person said it is wrong. |
| Needs a person | There is no check for this case. Someone has to read the output. |
| Could not run | The case did not complete. A reason is given. |
Compare results#
Compare results shows each case's latest result side by side for the versions and drafts you have run, up to four columns. Differences are marked Now passes, Now fails or Changed. Show output opens what a run produced.
There is no overall score. The comparison is there to be read.
Give a verdict#
On a result marked Needs a person, read what was produced, add a note if useful, and click Pass or Fail. Your name and the time are recorded. You can change a verdict later.
When a run leaves cases waiting for a verdict, the skill's owners get one skill review for the run. It closes when the last verdict is given.
Verdicts and results inform the decision to publish. They do not block it.
Edit and delete cases#
A case is a folder in the skill's home, so you can open and edit its files. An edited case is marked as having unsaved edits until you click Save edits or Discard edits.
To delete a case, open the menu on its row, choose Delete case and type delete. This removes the case and its files, and cannot be undone.
Keeping cases current#
A Review by date flags a case for another look once the date has passed. Nothing is deleted automatically. Leave the date blank to keep the case with no flag.
Limits#
- A case can hold up to 24 files, each up to 16 MB.
- One run per skill at a time, two cases at a time.
- A case is given up to 30 minutes.