Skip to content

admin: update AGENTS.md - #2179

Merged
lgritz merged 2 commits into
AcademySoftwareFoundation:mainfrom
jinhgkim:agents-docs
Oct 7, 2026
Merged

lgritz merged 2 commits into
AcademySoftwareFoundation:mainfrom
jinhgkim:agents-docs

Conversation

@jinhgkim

@jinhgkim jinhgkim commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

I came across some stale information in AGENTS.md while working on another PR. The first commit contains the necessary fixes: outdated minimum version info, and redundant or unclear instructions. Every wrong instruction misleads AI tools and wastes tokens, so this brings the file up to date. The second commit is a nice-to-have. It makes some of the wording more specific and adds an instruction about keeping AGENTS.md up to date.

Assisted-by: Claude Code / Opus-5.5

I initially noticed a couple of outdated details, then used AI to scan the file for more stale information and any wording that could confuse AI agents.

Checklist:

  • I have read the guidelines on contributions and code review procedures.
  • I have read the Policy on AI Coding Assistants
    and if I used AI coding assistants, I have an Assisted-by: TOOL / MODEL
    line in the pull request description above.
  • I have updated the documentation if my PR adds features or changes
    behavior.
  • I am sure that this PR's changes are tested in the testsuite.
  • I have run and passed the testsuite in CI before submitting the
    PR, by pushing the changes to my fork and seeing that the automated CI
    passed there. (Exceptions: If most tests pass and you can't figure out why
    the remaining ones fail, it's ok to submit the PR and ask for help. Or if
    any failures seem entirely unrelated to your change; sometimes things break
    on the GitHub runners.)
  • My code follows the prevailing code style of this project and I
    fixed any problems reported by the clang-format CI test.

Assisted-by: Claude Code / Opus-5.5
Signed-off-by: Jinnie Kim <jinhgkim@gmail.com>
Assisted-by: Claude Code / Opus-5.5
Signed-off-by: Jinnie Kim <jinhgkim@gmail.com>
Comment thread AGENTS.md
Comment on lines -72 to -73
- Read `testsuite/TESTSUITE-README.md` before updating references or
diagnosing failures

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can confirm this readme exists in OpenImageIO, but it's missing in this repo. Either drop it or add one?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's drop until we add a proper one that's right.

Comment thread AGENTS.md
Comment on lines +191 to +192
`Assisted-by: <TOOL> / <MODEL>`. The human author is responsible for
understanding, testing, and defending all changes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The human author is responsible for understanding, testing, and defending all changes.

This line doesn't do much from an AI's point of view. If you want to be strict about this policy, you could rephrase it as "Do not submit PRs, post comments, or approve or merge on the user's behalf. Leave every change for the human author to review, test, and submit themselves."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that this does help -- I can tell because when I ask it to commit what it's worked on, it frequently (maybe always?) tells me something like "Following the project's instructions, I have added 'Assisted-by' rather than 'Co-authored by' that my system instructions suggested." And often it says "I can commit, but I won't push, per the project's rules."

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha, I see. I often get "Do you want me to push?", so I've added an extra safeguard on my end in my ~/.claude, worded like my suggestion above. Now it says, "Here's the draft. I won't push, commit, or post comments." It's certainly a subtle difference, both leave the human author responsible, but one asks for your opinion and leaves a small possibility open (I've never said "Yes, push!", so I don't know what it would have done next), whereas the other is a hard no. This is just a small suggestion coming from my personal experience, and something to keep in mind.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're strictly talking about the sentence "The human author is...", then you might be right that it doesn't do anything useful. I don't know how to measure it.

@lgritz lgritz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, most of these are great, and I appreciate your taking a look at this. I made just a few comments where I thought things should be different.

Comment thread AGENTS.md
Comment on lines -72 to -73
- Read `testsuite/TESTSUITE-README.md` before updating references or
diagnosing failures

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's drop until we add a proper one that's right.

Comment thread AGENTS.md
than overwriting
- Be conservative loosening image diff thresholds — use the minimum needed
than overwriting; a test passes if its output matches any file in `ref/`
with the same extension

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
with the same extension
with the same extension.

Comment thread AGENTS.md
Comment on lines +75 to +76
- Be conservative loosening image diff thresholds (`failthresh`, `hardfail`,
`failpercent`, set per test in its `run.py`) — use the minimum needed

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know I talked about image diff thresholds in the original, but now I'm wondering if we should even say this at all. I think we definitely want to be extremely clear that agents should never loosen the thresholds or add new ref images unless a human explicitly asks it to do so after actually viewing the images and confirming that they are a visual match and differ for expected reasons.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's good to keep it. It's just one more rule for LLMs to follow. What makes you wonder whether we should say this at all?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I was worried that if we mention loosening thresholds, it might do that to make tests pass? But I think we probably never want it to do that. I think I'm saying: What behavior do we expect to elicit with this bullet item that it would do differently if it were absent?

Comment thread AGENTS.md
## Key Dependencies

- **LLVM 14+** (JIT compilation), **OpenImageIO 2.5+** (textures, image I/O, utilities), **Imath 3.1+** (math types), **Flex/Bison** (parser generation), **pybind11** (Python bindings, optional)
- **LLVM 14+** (JIT compilation), **OpenImageIO 3.0+** (textures, image I/O, utilities), **Imath 3.1+** (math types), **Flex/Bison** (parser generation), **pybind11 2.7+** or **nanobind 2.8+** (Python bindings, optional)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should take out the version numbers here, since they change from time to time?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I very much agree. I also wonder if we even need this "Key Dependencies" section at all.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's always hard to tell! Like I said, perhaps there is merit to stripping this whole file down to the bone and then building it up as we see the need. But my intuition is that maybe by mentioning the core tech stack (by name, not by version), it might be crucial context for it to formulate things in the right direction? But I'm not sure. But as an example, what I was thinking at the time is that just by briefly mentioning Imath or OIIO, it would be much more inclined to use things in those libraries rather than others, or replicating things on its own?

Comment thread AGENTS.md
Comment on lines +191 to +192
`Assisted-by: <TOOL> / <MODEL>`. The human author is responsible for
understanding, testing, and defending all changes.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that this does help -- I can tell because when I ask it to commit what it's worked on, it frequently (maybe always?) tells me something like "Following the project's instructions, I have added 'Assisted-by' rather than 'Co-authored by' that my system instructions suggested." And often it says "I can commit, but I won't push, per the project's rules."

@lgritz

lgritz commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Pending your adjusting a few things based on my comments (if you agree), I'm happy to merge this.

By the way, for a week or two now, the latest versions of Claude Code do recognize AGENTS.md as the rest of the harnesses do these days. So I think that also will let us simplify the setup-agent script a bit, I just haven't gotten to it yet.

Your doing this reminds me, though, that I've been wondering if we shouldn't tear down AGENTS.md and rewrite it. As it stands, it's probably 8-9 months old -- older even than the git history implies (May) because I was using it for a while before I got the confidence to make a PR to add it to the repo. The models have gone through a few major revisions since then and are much more capable, probably know a lot of the things in this file without being told.

I've been reading about this lately, and I believe that the growing consensus is that for the latest models, the AGENTS.md instructions can be stripped way down, and probably should be organized to say very little but have references. "If you need X, look in file Y" for lots of topics. That way, the startup minimum context stays small, and it works more like the kind of "selective disclosure" that the SKILL files use, where it only takes context for the brief description of when each part is applicable, and reads the rest only when the situation seems to call for that topic.

I remember seeing a talk recently from Boris Cherney (he leads the Claude Code team) who says that he thinks that ever new model generation, you should throw out all your AGENTS and skills and start over... first seeing what the new models do "bare" and minimally adding things back only as needed when you see what it consistently needs help with.

Thoughts? We can do this PR for now, but then also think about really tearing it down and give it a big rethink, if you also are thinking along those lines.

I never was, and still am not, especially sure how to tell if an AGENTS file is making things better or not, or is doing so enough to justify it token cost. Is everybody just guessing blind, or is there a methodology that we should be using?

Of course, this PR is in the OSL project, but let's assume this discussion applies equally to the OIIO side of the house.

@jinhgkim

jinhgkim commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

By the way, for a week or two now, the latest versions of Claude Code do recognize AGENTS.md as the rest of the harnesses do these days. So I think that also will let us simplify the setup-agent script a bit, I just haven't gotten to it yet.

It's definitely more convenient that Claude Code now supports AGENTS.md, but we might want to be careful with that. Last time I read about this, Claude only reads CLAUDE.md if we have both AGENTS.md and CLAUDE.md. I think you can tweak the settings so Claude reads both (I wonder if this is configurable by project setting or should be configured by per user - I don't know, I need to read more). But I think simplifying the setup-agent script doesn't really buy us anything, and keeping the agent-specific md file seems safer. It behaves as expected, without things slipping through unnoticed.

the AGENTS.md instructions can be stripped way down, and probably should be organized to say very little but have references. "If you need X, look in file Y" for lots of topics. That way, the startup minimum context stays small, and it works more like the kind of "selective disclosure" that the SKILL files use, where it only takes context for the brief description of when each part is applicable, and reads the rest only when the situation seems to call for that topic.

Yes! "If you need X, look in file Y" is the way to go. We give pointers, not the full context, unless something is very important and we don't want agents to miss it or skip it based on their own judgment call.

I remember seeing a talk recently from Boris Cherney (he leads the Claude Code team) who says that he thinks that ever new model generation, you should throw out all your AGENTS and skills and start over... first seeing what the new models do "bare" and minimally adding things back only as needed when you see what it consistently needs help with.

Yeah, I think I saw something similar a while ago where Boris said his own setup is surprisingly vanilla and he doesn't customize it much. Too much context degrades performance, so they keep it lean and only add small details gradually.

I never was, and still am not, especially sure how to tell if an AGENTS file is making things better or not, or is doing so enough to justify it token cost. Is everybody just guessing blind, or is there a methodology that we should be using?

I do think it helps the LLM navigate a massive project. One improvement I've wanted to make, looking at this repo, is to keep a small AGENTS.md in the key subdirectories instead of having everything in a single AGENTS.md at the project root (the AGENTS.md at the project root would contain only the really important things, because it's loaded in every new session). Each one would be fairly small and focused on that directory only, and in Claude Code, it's loaded only when the agent reads a file there. It acts like a guidebook.

I think of it like having a new hire on the team who doesn't have much context yet. When you introduce the codebase, you don't give every detail; you just give pointers on where to look. Without the pointers, the new hire will eventually find what they need, but it takes longer. And unlike a new hire, the agent starts from scratch every session, so it pays that exploration cost every time.

@jinhgkim

jinhgkim commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

https://code.claude.com/docs/en/memory#agents-md

This doc is what I was referring to when I talked about agents.md vs claude.md in an earlier comment. The other sections are also a good read to understand how this AI tool works in general.

@lgritz

lgritz commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

It's definitely more convenient that Claude Code now supports AGENTS.md, but we might want to be careful with that. Last time I read about this, Claude only reads CLAUDE.md if we have both AGENTS.md and CLAUDE.md.

Yeah, so I think you are saying: If we only have AGENTS.md, a developer may inadvertently create what they think is a local CLAUDE.md that adds, but in reality masks the material in AGENTS? Currently, the setup-agent script sets up a .claude/CLAUDE.md that contains @AGENTS.md, which causes it to be included. Maybe you're right and we should leave that. (Too bad, it's a bit of complexity I wish we could drop. It's a shame that Claude doesn't just read both instead of using the presence of one to ignore the other.)

Yeah, I think I saw something similar a while ago where Boris said his own setup is surprisingly vanilla and he doesn't customize it much. Too much context degrades performance, so they keep it lean and only add small details gradually.

Grain of salt: I bet he is operating without any token limits. :-)
So maybe he cares less than we do if everything he asks it to do involves extra rummaging through the project to figure out basic things that we could establish with some things in AGENTS.md.

keep a small AGENTS.md in the key subdirectories

Claude now reads a top level AGENTS.md (if there is no CLAUDE.md or .claude/CLAUDE.md, but it's not clear to me if AGENTS anywhere else would be read.


If this is an interest of yours, feel free to make a specific proposal as a PR. (I think we should merge this one first, with whatever minor revisions we agree on, because it does make a number of helpful fixes.)

I'm sure there are some parts of what we have that can be dropped because it's totally general advice that the model doesn't need.

And some parts are helpful but can be converted to "if you need X, look at Y" and put those instructions in other files (perhaps in docs/dev) that it will only read when on certain tasks.

@lgritz

lgritz commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

I do think it helps the LLM navigate a massive project. One improvement I've wanted to make, looking at this repo, is to keep a small AGENTS.md in the key subdirectories instead of having everything in a single AGENTS.md at the project root (the AGENTS.md at the project root would contain only the really important things, because it's loaded in every new session). Each one would be fairly small and focused on that directory only, and in Claude Code, it's loaded only when the agent reads a file there. It acts like a guidebook.

Turns out this is fine! I read the docs, and it does honor AGENTS.md at various subdirectories, as long as there isn't a CLAUDE.md at that particular subdirectory.

@jinhgkim

jinhgkim commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

It's a shame that Claude doesn't just read both instead of using the presence of one to ignore the other.

Well, you can tweak the settings so that Claude Code reads both. What I am worried about is, what happens we have everything written in AGENTS.md in the project, but some user has CLAUDE.md somewhere in their local directory e.g., ~/.claude/CLAUDE.md and so on. And another question that follows is, if we set the settings so that Claude reads both, does it follow the project settings or the user's settings? I haven't fully read their docs about AGENTS vs CLAUDE md rules, so I don't know. But it seems like just supporting their native markdown files at all times makes things simpler and no need to worry about edge cases.

Grain of salt: I bet he is operating without any token limits. :-)

Haha! Fair enough.

@jinhgkim

jinhgkim commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

I believe I have addressed all of your comments and questions that can be addressed at this point in the discussion. Feel free to merge unless I have missed something.

@lgritz

lgritz commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Well, you can tweak the settings so that Claude Code reads both. What I am worried about is, what happens we have everything written in AGENTS.md in the project, but some user has CLAUDE.md somewhere in their local directory e.g., ~/.claude/CLAUDE.md and so on.

Wonder no more, because I do that!

As long as the .claude/CLAUDE.md contains @AGENTS, it will read it. You can also add anything else you want.

OIIO and OSL have a .agents/setup-agent shell script, have you looked at that? (It's heavily commented with explanation. But, grain of salt: as I said, it needs slight updating in its explanations now that Claude does read AGENTS.md under certain circumstances.)

The project has AGENTS.md and some skills and other things in .agents. As long as you use one of the agent harnesses that use those, you are fine as-is. If you use Claude or some others that expect things in different places, we supply that setup-agent script that, depending on the agent you say you want to use, sets up a couple links and whatnot so it should all work.

If you do setup-agent claude, it will create .claude/CLAUDE.md that contains @AGENTS. That's enough to get it to read AGENTS.md. This is a one-time (per checked out tree) setup, after which you can put any other customizations you want in your CLAUDE.md (don't check it in), or append to .claude/CLAUDE.md (don't get rid of the @AGENTS.md).

The intent is for the out-of-the-box checkout to work with as many harnesses as possible, and the script can set you up to use one of the "nonstandard" ones.

@lgritz

lgritz commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

If there's a different way to set up what is checked in, what is linked, and what contains references, that is better, we can change.

Checking a CLAUDE.md into the repo seemed too much like endorsing a particular product and I wanted to avoid that.

It seemed to me like AGENTS.md and .agents, being generic and non-branded, were the best for what's checked in, and any adaptation for tools that expect something different can either handle it themselves or try our setup-agent script.

@lgritz

lgritz commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

I'll merge what you have now. But I do think there is merit to a much larger restructure of this file, as we have discussed.

@lgritz
lgritz merged commit 0dc86fc into AcademySoftwareFoundation:main Oct 7, 2026
3 checks passed
@lgritz lgritz changed the title docs: update AGENTS.md admin: update AGENTS.md Oct 7, 2026
@jinhgkim

jinhgkim commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Ah, I think I explained it poorly. Yes, I think the current .agents/setup-agent setup is good because it basically points each agent-specific file back to our single AGENTS.md. For example, Claude references the root AGENTS.md through the @AGENTS.md line in the .claude/CLAUDE.md that the script generates.

What I meant when I kept mentioning CLAUDE.md is that we should keep that mechanism and keep referencing AGENTS.md, instead of generating .claude/AGENTS.md. I assumed (my bad) that by simplifying .agents/setup-agent, you meant dropping CLAUDE.md and having Claude read AGENTS.md natively, as Codex does.

@lgritz

lgritz commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Oh, right, I think we're on the same page.

I never thought we should generate .claude/AGENTS.md. I did hope (but you corrected me) that Claude would read the top level AGENTS.md unconditionally and that we could eliminate the part of setup-agents that writes @AGENTS.md into .claude/CLAUDE.md. But that doesn't seem like it works yet, so let's stick to what we have (except for correcting the comments in setup-agents to make it correctly explain the current Claude behavior).

I do hope that all the products coalesce on uniformly using AGENTS.md and .agents/skills, and only use other names for additional customizations specific to those tools. I find the "read AGENTS.md unless CLAUDE.md exists" non-intuitive and kind of hostile to projects that want a single setup that can work for developers using a number of agents of their choice.

@jinhgkim

jinhgkim commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

I do hope that all the products coalesce on uniformly using AGENTS.md and .agents/skills, and only use other names for additional customizations specific to those tools. I find the "read AGENTS.md unless CLAUDE.md exists" non-intuitive and kind of hostile to projects that want a single setup that can work for developers using a number of agents of their choice.

That would be the ideal scenario, but unfortunately we are not living in a perfect world! I think the setup-agent script works great for the time being.

@jinhgkim

jinhgkim commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Perhaps there's already an agent shim somebody implemented as open source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants