Repository navigation
admin: update AGENTS.md - #2179
Conversation
Assisted-by: Claude Code / Opus-5.5 Signed-off-by: Jinnie Kim <jinhgkim@gmail.com>
| - Read `testsuite/TESTSUITE-README.md` before updating references or | ||
| diagnosing failures |
There was a problem hiding this comment.
I can confirm this readme exists in OpenImageIO, but it's missing in this repo. Either drop it or add one?
There was a problem hiding this comment.
Let's drop until we add a proper one that's right.
| `Assisted-by: <TOOL> / <MODEL>`. The human author is responsible for | ||
| understanding, testing, and defending all changes. |
There was a problem hiding this comment.
The human author is responsible for understanding, testing, and defending all changes.
This line doesn't do much from an AI's point of view. If you want to be strict about this policy, you could rephrase it as "Do not submit PRs, post comments, or approve or merge on the user's behalf. Leave every change for the human author to review, test, and submit themselves."
There was a problem hiding this comment.
I believe that this does help -- I can tell because when I ask it to commit what it's worked on, it frequently (maybe always?) tells me something like "Following the project's instructions, I have added 'Assisted-by' rather than 'Co-authored by' that my system instructions suggested." And often it says "I can commit, but I won't push, per the project's rules."
There was a problem hiding this comment.
Aha, I see. I often get "Do you want me to push?", so I've added an extra safeguard on my end in my ~/.claude, worded like my suggestion above. Now it says, "Here's the draft. I won't push, commit, or post comments." It's certainly a subtle difference, both leave the human author responsible, but one asks for your opinion and leaves a small possibility open (I've never said "Yes, push!", so I don't know what it would have done next), whereas the other is a hard no. This is just a small suggestion coming from my personal experience, and something to keep in mind.
There was a problem hiding this comment.
If you're strictly talking about the sentence "The human author is...", then you might be right that it doesn't do anything useful. I don't know how to measure it.
lgritz
left a comment
There was a problem hiding this comment.
Thanks, most of these are great, and I appreciate your taking a look at this. I made just a few comments where I thought things should be different.
| - Read `testsuite/TESTSUITE-README.md` before updating references or | ||
| diagnosing failures |
There was a problem hiding this comment.
Let's drop until we add a proper one that's right.
| than overwriting | ||
| - Be conservative loosening image diff thresholds — use the minimum needed | ||
| than overwriting; a test passes if its output matches any file in `ref/` | ||
| with the same extension |
There was a problem hiding this comment.
| with the same extension | |
| with the same extension. |
| - Be conservative loosening image diff thresholds (`failthresh`, `hardfail`, | ||
| `failpercent`, set per test in its `run.py`) — use the minimum needed |
There was a problem hiding this comment.
I know I talked about image diff thresholds in the original, but now I'm wondering if we should even say this at all. I think we definitely want to be extremely clear that agents should never loosen the thresholds or add new ref images unless a human explicitly asks it to do so after actually viewing the images and confirming that they are a visual match and differ for expected reasons.
There was a problem hiding this comment.
I think it's good to keep it. It's just one more rule for LLMs to follow. What makes you wonder whether we should say this at all?
There was a problem hiding this comment.
Maybe I was worried that if we mention loosening thresholds, it might do that to make tests pass? But I think we probably never want it to do that. I think I'm saying: What behavior do we expect to elicit with this bullet item that it would do differently if it were absent?
| ## Key Dependencies | ||
|
|
||
| - **LLVM 14+** (JIT compilation), **OpenImageIO 2.5+** (textures, image I/O, utilities), **Imath 3.1+** (math types), **Flex/Bison** (parser generation), **pybind11** (Python bindings, optional) | ||
| - **LLVM 14+** (JIT compilation), **OpenImageIO 3.0+** (textures, image I/O, utilities), **Imath 3.1+** (math types), **Flex/Bison** (parser generation), **pybind11 2.7+** or **nanobind 2.8+** (Python bindings, optional) |
There was a problem hiding this comment.
I wonder if we should take out the version numbers here, since they change from time to time?
There was a problem hiding this comment.
I very much agree. I also wonder if we even need this "Key Dependencies" section at all.
There was a problem hiding this comment.
It's always hard to tell! Like I said, perhaps there is merit to stripping this whole file down to the bone and then building it up as we see the need. But my intuition is that maybe by mentioning the core tech stack (by name, not by version), it might be crucial context for it to formulate things in the right direction? But I'm not sure. But as an example, what I was thinking at the time is that just by briefly mentioning Imath or OIIO, it would be much more inclined to use things in those libraries rather than others, or replicating things on its own?
| `Assisted-by: <TOOL> / <MODEL>`. The human author is responsible for | ||
| understanding, testing, and defending all changes. |
There was a problem hiding this comment.
I believe that this does help -- I can tell because when I ask it to commit what it's worked on, it frequently (maybe always?) tells me something like "Following the project's instructions, I have added 'Assisted-by' rather than 'Co-authored by' that my system instructions suggested." And often it says "I can commit, but I won't push, per the project's rules."
|
Pending your adjusting a few things based on my comments (if you agree), I'm happy to merge this. By the way, for a week or two now, the latest versions of Claude Code do recognize AGENTS.md as the rest of the harnesses do these days. So I think that also will let us simplify the Your doing this reminds me, though, that I've been wondering if we shouldn't tear down AGENTS.md and rewrite it. As it stands, it's probably 8-9 months old -- older even than the git history implies (May) because I was using it for a while before I got the confidence to make a PR to add it to the repo. The models have gone through a few major revisions since then and are much more capable, probably know a lot of the things in this file without being told. I've been reading about this lately, and I believe that the growing consensus is that for the latest models, the AGENTS.md instructions can be stripped way down, and probably should be organized to say very little but have references. "If you need X, look in file Y" for lots of topics. That way, the startup minimum context stays small, and it works more like the kind of "selective disclosure" that the SKILL files use, where it only takes context for the brief description of when each part is applicable, and reads the rest only when the situation seems to call for that topic. I remember seeing a talk recently from Boris Cherney (he leads the Claude Code team) who says that he thinks that ever new model generation, you should throw out all your AGENTS and skills and start over... first seeing what the new models do "bare" and minimally adding things back only as needed when you see what it consistently needs help with. Thoughts? We can do this PR for now, but then also think about really tearing it down and give it a big rethink, if you also are thinking along those lines. I never was, and still am not, especially sure how to tell if an AGENTS file is making things better or not, or is doing so enough to justify it token cost. Is everybody just guessing blind, or is there a methodology that we should be using? Of course, this PR is in the OSL project, but let's assume this discussion applies equally to the OIIO side of the house. |
It's definitely more convenient that Claude Code now supports AGENTS.md, but we might want to be careful with that. Last time I read about this, Claude only reads CLAUDE.md if we have both AGENTS.md and CLAUDE.md. I think you can tweak the settings so Claude reads both (I wonder if this is configurable by project setting or should be configured by per user - I don't know, I need to read more). But I think simplifying the setup-agent script doesn't really buy us anything, and keeping the agent-specific md file seems safer. It behaves as expected, without things slipping through unnoticed.
Yes! "If you need X, look in file Y" is the way to go. We give pointers, not the full context, unless something is very important and we don't want agents to miss it or skip it based on their own judgment call.
Yeah, I think I saw something similar a while ago where Boris said his own setup is surprisingly vanilla and he doesn't customize it much. Too much context degrades performance, so they keep it lean and only add small details gradually.
I do think it helps the LLM navigate a massive project. One improvement I've wanted to make, looking at this repo, is to keep a small AGENTS.md in the key subdirectories instead of having everything in a single AGENTS.md at the project root (the AGENTS.md at the project root would contain only the really important things, because it's loaded in every new session). Each one would be fairly small and focused on that directory only, and in Claude Code, it's loaded only when the agent reads a file there. It acts like a guidebook. I think of it like having a new hire on the team who doesn't have much context yet. When you introduce the codebase, you don't give every detail; you just give pointers on where to look. Without the pointers, the new hire will eventually find what they need, but it takes longer. And unlike a new hire, the agent starts from scratch every session, so it pays that exploration cost every time. |
|
https://code.claude.com/docs/en/memory#agents-md This doc is what I was referring to when I talked about agents.md vs claude.md in an earlier comment. The other sections are also a good read to understand how this AI tool works in general. |
Yeah, so I think you are saying: If we only have AGENTS.md, a developer may inadvertently create what they think is a local CLAUDE.md that adds, but in reality masks the material in AGENTS? Currently, the setup-agent script sets up a .claude/CLAUDE.md that contains
Grain of salt: I bet he is operating without any token limits. :-)
Claude now reads a top level AGENTS.md (if there is no CLAUDE.md or .claude/CLAUDE.md, but it's not clear to me if AGENTS anywhere else would be read. If this is an interest of yours, feel free to make a specific proposal as a PR. (I think we should merge this one first, with whatever minor revisions we agree on, because it does make a number of helpful fixes.) I'm sure there are some parts of what we have that can be dropped because it's totally general advice that the model doesn't need. And some parts are helpful but can be converted to "if you need X, look at Y" and put those instructions in other files (perhaps in docs/dev) that it will only read when on certain tasks. |
Turns out this is fine! I read the docs, and it does honor AGENTS.md at various subdirectories, as long as there isn't a CLAUDE.md at that particular subdirectory. |
Well, you can tweak the settings so that Claude Code reads both. What I am worried about is, what happens we have everything written in AGENTS.md in the project, but some user has CLAUDE.md somewhere in their local directory e.g., ~/.claude/CLAUDE.md and so on. And another question that follows is, if we set the settings so that Claude reads both, does it follow the project settings or the user's settings? I haven't fully read their docs about AGENTS vs CLAUDE md rules, so I don't know. But it seems like just supporting their native markdown files at all times makes things simpler and no need to worry about edge cases.
Haha! Fair enough. |
|
I believe I have addressed all of your comments and questions that can be addressed at this point in the discussion. Feel free to merge unless I have missed something. |
Wonder no more, because I do that! As long as the .claude/CLAUDE.md contains OIIO and OSL have a The project has AGENTS.md and some skills and other things in .agents. As long as you use one of the agent harnesses that use those, you are fine as-is. If you use Claude or some others that expect things in different places, we supply that setup-agent script that, depending on the agent you say you want to use, sets up a couple links and whatnot so it should all work. If you do The intent is for the out-of-the-box checkout to work with as many harnesses as possible, and the script can set you up to use one of the "nonstandard" ones. |
|
If there's a different way to set up what is checked in, what is linked, and what contains references, that is better, we can change. Checking a CLAUDE.md into the repo seemed too much like endorsing a particular product and I wanted to avoid that. It seemed to me like AGENTS.md and .agents, being generic and non-branded, were the best for what's checked in, and any adaptation for tools that expect something different can either handle it themselves or try our setup-agent script. |
|
I'll merge what you have now. But I do think there is merit to a much larger restructure of this file, as we have discussed. |
|
Ah, I think I explained it poorly. Yes, I think the current What I meant when I kept mentioning CLAUDE.md is that we should keep that mechanism and keep referencing AGENTS.md, instead of generating |
|
Oh, right, I think we're on the same page. I never thought we should generate .claude/AGENTS.md. I did hope (but you corrected me) that Claude would read the top level AGENTS.md unconditionally and that we could eliminate the part of setup-agents that writes I do hope that all the products coalesce on uniformly using AGENTS.md and .agents/skills, and only use other names for additional customizations specific to those tools. I find the "read AGENTS.md unless CLAUDE.md exists" non-intuitive and kind of hostile to projects that want a single setup that can work for developers using a number of agents of their choice. |
That would be the ideal scenario, but unfortunately we are not living in a perfect world! I think the setup-agent script works great for the time being. |
|
Perhaps there's already an agent shim somebody implemented as open source |
I came across some stale information in AGENTS.md while working on another PR. The first commit contains the necessary fixes: outdated minimum version info, and redundant or unclear instructions. Every wrong instruction misleads AI tools and wastes tokens, so this brings the file up to date. The second commit is a nice-to-have. It makes some of the wording more specific and adds an instruction about keeping AGENTS.md up to date.
Assisted-by: Claude Code / Opus-5.5
I initially noticed a couple of outdated details, then used AI to scan the file for more stale information and any wording that could confuse AI agents.
Checklist:
and if I used AI coding assistants, I have an
Assisted-by: TOOL / MODELline in the pull request description above.
behavior.
PR, by pushing the changes to my fork and seeing that the automated CI
passed there. (Exceptions: If most tests pass and you can't figure out why
the remaining ones fail, it's ok to submit the PR and ask for help. Or if
any failures seem entirely unrelated to your change; sometimes things break
on the GitHub runners.)
fixed any problems reported by the clang-format CI test.