samber/developer-relations-skillsチェック済み
SKILL DETAIL
oss-issue-triage
samber/developer-relations-skills/oss-issue-triage
Designs an issue and pull-request triage system a maintainer team can sustain - response targets sized against real capacity, the label taxonomy, intake cuts through structured forms and off-tracker routing, a separate security-report path, triage duty assignment, and the closing, staleness and volume-gating policy. Use whenever someone says "our issue tracker is out of control", "design an issue triage process", "set up labels for our repo", "we have 900 open issues", "should we run a stale bot", "PR backlog nobody reviews", "triage rotation", or "we are drowning in AI-generated reports" - even if they only say maintenance is overwhelming. Not good-first-issue curation - use samber/developer-relations-skills@oss-contributor-onboarding.
インストール · 417出典を見る
Installation
npx skills add https://github.com/samber/developer-relations-skills --skill oss-issue-triage
スキルファイル
SKILL.md
最終同期 · 2026/10/23
evals/evals.json›
{
"skill_name": "oss-issue-triage",
"evals": [
{
"id": 1,
"prompt": "I maintain quillsync, a Go CLI that syncs notes between apps. It has 912 open issues and 47 labels. I counted: 19 of those labels have been used exactly once, and there are three separate ones for bugs (`bug`, `Bug`, `type: bug`) that different people applied over the years. Can you redo the label set for me? There are two of us maintaining it and realistically we each get about two hours a week on this, in a good week. Roughly 35 new issues land per week. Just give me the final label list and I'll delete the old ones and apply the new ones tonight.",
"expected_output": "A response that measures the queue and does the capacity arithmetic before proposing any taxonomy, makes the per-item-versus-clustering call on a 912-issue backlog, states plainly that capacity is far below inflow so labels cannot fix the queue, and delivers a create/rename/merge/delete change list (each label with one definition and one axis) gated on maintainer approval and a single-pass migration.",
"files": [],
"expectations": [
"Produces a label-usage count and a queue baseline before proposing any label list",
"Reads the baseline numbers back in plain language before recommending changes",
"Makes an explicit per-item-versus-clustering call for the 912-issue backlog, naming grouping by shared root cause under a tracking issue as the alternative to item-by-item triage",
"Computes weekly triage capacity as triagers x sessions per week x minutes per session divided by minutes per item, using the stated two maintainers and two hours each",
"States that capacity is below the stated inflow of roughly 35 issues a week and that no label scheme closes that gap",
"Does not use RICE, ICE, or an impact-versus-effort scoring matrix anywhere in the answer",
"Organises labels into named orthogonal axes including at least type, state, priority and area, with at most one value per axis on any item",
"Recommends renaming labels into an axis before deleting any, and states that deleting erases the label from every past item and any saved view built on it",
"Recommends deleting the 19 single-use labels, on the grounds that a label nobody applies twice is a private note",
"Keeps priority to at most two or three values, or omits priority entirely, rather than importing a four-level scheme",
"Requires labels, intake forms, saved views and automation to be migrated in one pass rather than labels tonight and the rest later",
"Presents the taxonomy for the maintainers' approval before any tracker change is made",
"Delivers the labels as a change list where each entry carries one sentence of definition and names its axis"
]
},
{
"id": 2,
"prompt": "We run mapshard, a Rust geospatial indexing library. 1,400 open issues, the oldest from 2022. I want to turn on the stale bot with the standard 60-day setting and just let it run across the whole repo this weekend, which should clear most of that out. Anything that actually matters, people will reopen. What config should I put in the workflow file?",
"expected_output": "A configuration restricted to the waiting-on-reporter state only, with an explicit exempt list, a warning step, a dry run first, and a refusal to sweep items waiting on the maintainers, plus a warning about the notification storm a first live pass over a 2022-era backlog produces.",
"files": [],
"expectations": [
"Restricts the automated sweep to the waiting-on-reporter state only, rather than running it repository-wide",
"Refuses to auto-close items that are waiting on the maintainers, including confirmed bugs, and gives the reason that a bot-closed confirmed bug tells the reporter their evidence was worthless",
"Names an explicit exempt-state list covering at least confirmed or accepted, blocked, security and pinned items",
"Requires the first pass to run in dry-run or debug-only mode with the resulting list read before anything fires",
"Requires a warning comment before any close, with a stated warning-to-close window",
"Requires every close message to carry its reason and state how to reopen",
"Presents 60 days idle plus 7 days to close as the stale action's own shipped default rather than as a recommendation fitted to this project",
"Cites 20 days without a reporter response as a published Kubernetes clock",
"Anchors each clock to a named waiting party rather than to item age in the abstract",
"Warns that a first live sweep over a backlog reaching back to 2022 can close hundreds of items and produce a notification storm that costs the project contributors",
"Does not propose closing items solely because they are old",
"States that the sweep does not substitute for the capacity or intake decision",
"Does not describe reopening by the reporter as sufficient protection against wrongly closed items"
]
},
{
"id": 3,
"prompt": "Project is fernweb, a Python web framework, 2,100 open issues. My plan: close everything older than 18 months with a canned message, then do the same thing every January so it never builds up again. Sound reasonable? I want the message text and the script.",
"expected_output": "Acceptance of the one-time close as the opening move of a new policy, refusal of the annual repetition with the documented reason, and a set of preconditions (announcement, new policy, new intake live the same day, exempt states, baseline first, monthly re-measurement).",
"files": [],
"expectations": [
"States that a mass close is legitimate exactly once, as the opening move of a new policy",
"Refuses the proposed annual repetition and explains why a recurring bankruptcy is not a maintenance routine",
"Requires the announcement, the new triage policy and the new intake to go live on the same day as the mass close",
"Cites the documented outcome that mass-closing works once and the backlog returns to the same state within roughly six months",
"Names the underlying cause as the absence of a natural lifecycle for issues and pull requests, which the closure itself does not address",
"Requires the close message to carry an explanation and an open invitation to reopen",
"Pairs the mass close with an intake change rather than presenting it standalone",
"Excludes confirmed or accepted items and security items from the mass close",
"Warns that notification volume is the real cost of a bulk pass",
"Requires a baseline measurement of the queue before the mass close so the effect is measurable afterwards",
"Requires the new policy to name the published first-response target the project will hold after the close",
"Requires the baseline to be re-run monthly for the first quarter rather than checked once",
"Does not present a scheduled or calendared mass-close job as the deliverable"
]
},
{
"id": 4,
"prompt": "I help run security triage for tinderbox, an image-processing C library. Six of us, about four hours a week each. Since last year we've been buried: maybe one in fifteen reports is a real finding now, and each junk one still eats about 90 minutes because two of us have to look at it before we can dismiss it. We pay $500 a report through our bounty page. I want to build a classifier that scores incoming reports for AI generation and auto-rejects anything below a threshold. How do I train it and what threshold should I use?",
"expected_output": "A refusal to build a machine-generation detector, the arithmetic showing the invalid share already consumes most of the team's hours, and a gating design keyed on reputation and volume plus a change to the submission incentive, with a written trigger decided now.",
"files": [],
"expectations": [
"States that no reliable technical indicator for machine-generated submissions exists and declines to build or recommend one as the control",
"Gates on reputation and submission volume rather than on whether the content looks machine-generated",
"Warns that accusing a contributor of machine generation without proof creates a conduct incident on top of the original problem",
"Subtracts the hours consumed by never-actionable submissions before comparing capacity against inflow",
"Carries the prompt's own figures through the arithmetic, using six people at four hours each and 90 minutes with two people per junk report",
"States plainly that no published response target survives the current valid share",
"Names removing or changing the submission incentive, including the monetary reward, as a lever that acts on the economics of submitting rather than the cost of evaluating",
"Cites curl's confirmed-vulnerability rate falling from above 15% to below 5%, and the ending of its bounty programme, as the sourced precedent",
"Sets a written volume-gating trigger, expressed as an inflow rate or valid-report share, together with the restriction it fires, decided now rather than during a crisis",
"Lists gating options in rough order of reversibility, such as a reputation or contribution-history threshold, a cap on open submissions per author, and restricting who may create submissions",
"Rejects a stricter template, a better classifier and a faster rotation as answers to a collapsed valid share",
"States the asymmetry that generating a plausible-sounding report costs pennies in tokens while evaluating whether it is real still costs an hour of expert time",
"Does not propose machine duplicate detection as a working default"
]
},
{
"id": 5,
"prompt": "otterlink, an OSS reverse proxy. Someone just opened a public issue titled \"auth bypass in header parsing\" with a working proof of concept in the body, and it's been up for about 40 minutes. We don't have a security list or anything set up. Going forward I want to add a `security` label plus a security issue template so these get spotted and handled fast. What should the template ask for?",
"expected_output": "Immediate private routing of the live public report, and a security path that leaves the tracker entirely: private intake channel, named size-capped response team, embargo mechanism, with the destination published as a chooser contact link rather than an issue form.",
"files": [],
"expectations": [
"Treats the already-public report as an emergency to move to a private channel immediately rather than triaging it in the open",
"Moves security reports off the public tracker entirely rather than adding a security label or a security issue form",
"Names all three pipeline elements: a private intake channel, a named size-capped response team, and an embargo mechanism for coordinated disclosure",
"Recommends the code host's private vulnerability reporting as the default given that no security list exists",
"Cites the guidance never to enter details of security vulnerabilities in a project's public bug tracker",
"Keeps the response team size-capped and distinct from the general triage rota, citing the ten-member committee and three-working-day acknowledgement precedent",
"Places the security destination as a contact link in the template chooser rather than as an issue template",
"Exempts security items from any staleness sweep",
"Flags the isolation cost of security triage as a staffing consideration, since the work cannot be discussed with the wider project",
"Does not fold security response into the project's general first-response target",
"Does not recommend public triage or public discussion of unconfirmed vulnerabilities",
"Publishes the security destination in the triage policy's statement of what belongs in the tracker",
"States the coordinated-disclosure embargo as an explicit mechanism rather than an informal practice"
]
},
{
"id": 6,
"prompt": "We maintain slateform, a TypeScript form library. 230 open pull requests from outside contributors, oldest 14 months, four maintainers. A lot of the recent ones look like they came out of coding agents. My plan is to work oldest-first and actually review the code properly on every one, since that's the only fair way to do it. How long should I budget per PR?",
"expected_output": "A pull-request gate order distinct from the issue process, with sign-off and CI permission as automated preconditions and the direction check ahead of the size label and ahead of code review, justified by rejections settled per minute of reviewer time.",
"files": [],
"expectations": [
"Gives pull requests their own gate order rather than reusing the issue triage process",
"Treats sign-off (CLA or DCO) and CI permission for externally authored pull requests as automated preconditions rather than as ranked menu items",
"States that nothing merges without sign-off, tied to the licence provenance it records",
"Ranks the direction check ahead of the size-based risk label and ahead of code-correctness review",
"Justifies that order by rejections settled per minute of reviewer time rather than by which gate is cheapest",
"Cites the finding that roughly two thirds of rejected agentic pull requests never needed the code read at all, from 353 manually inspected rejections",
"Names code-correctness review as the only gate that can accept a change, and the expensive pass the order exists to protect",
"States the condition that promotes code review to first: a trusted repeat contributor sending a small change against an already-accepted issue",
"Gives pull requests a tighter first-response target than issues, justified by the asymmetric cost of an unanswered pull request against hours already spent",
"Counts externally authored pull requests as triage work while excluding the maintainers' own in-flight pull requests",
"Requires verifying the claim against the diff rather than against the description",
"Routes by ownership before review so the wait sits on a named person rather than on the queue",
"Does not present a raw diff-size or line-count threshold as an accept-or-reject rule"
]
},
{
"id": 7,
"prompt": "I'm the only maintainer of cindermark, a markdown renderer that around 40,000 projects depend on. New job, new baby. I get maybe 20 minutes a month on it now, and nobody else has commit rights or wants them. 380 open issues, about 12 new a week. Can you design me a proper triage system with labels and a rotation so this stays under control?",
"expected_output": "A refusal to design a taxonomy, with the arithmetic shown, and an honest deliverable of reduced intake plus a published maintenance-mode statement.",
"files": [],
"expectations": [
"Stops the design and says so rather than producing a label taxonomy",
"Names the honest deliverable as a reduced intake plus a published maintenance-mode statement for the project",
"Does not produce a label taxonomy, priority scheme or triage rotation for this project",
"Deletes adding triagers from the options entirely rather than listing it as a lower-ranked or future item, since nobody can be added",
"Refuses to publish a first-response target that 20 minutes a month cannot support",
"Shows the arithmetic that the stated capacity cannot cover the stated inflow of roughly 12 issues a week",
"Recommends narrowing intake, through disabling blank issues, restricting creation, or turning a submission type off, as the change that actually acts",
"Recommends routing usage questions off the tracker where a destination exists, or states plainly that none does",
"States that the maintenance-mode notice must be published where reporters see it before opening an item",
"Recommends the project state publicly what will and will not get a response rather than leaving the queue silently unanswered",
"Keeps every surviving recommendation to changes one person can make in a single sitting",
"Does not propose a stale bot as a substitute for the intake decision",
"Does not frame the situation as a failure of the maintainer's process discipline"
]
},
{
"id": 8,
"prompt": "Halbrecht Systems here. We open-sourced our pulsegrid scheduler last year. We hold 11 enterprise support contracts that guarantee a 4-hour response, and the same engineers also cover the public GitHub tracker. Two engineers, about 6 hours a week each on the open-source side. Leadership wants us to publish \"24-hour response on all GitHub issues\" in the README to show we're serious about the community. Draft that for me.",
"expected_output": "A target derived from the measured capacity rather than from leadership's number, the contractual customer path separated into its own intake, and a published policy that defines first response as a human reply and names the class of items given a tighter target.",
"files": [],
"expectations": [
"Derives the published first-response target from measured capacity rather than from the desired number",
"Refuses the blanket 24-hour target and names the figure the stated capacity actually supports",
"Separates the contractually obliged enterprise path into its own intake rather than letting it set the target for the public queue",
"States that shortening or changing an already-published target carries a contract-change cost where a contractual response obligation exists",
"Defines first response as a human reply, explicitly not a fix and not a bot acknowledgement",
"Sets the publicly visible target to one the team can hold in a bad month rather than a good one",
"Carries the prompt's figures, two engineers at six hours each, through the capacity formula",
"Writes the target into a published policy in the repository next to the contributing guide and links it from the intake forms",
"Names the class of items given a tighter target and the reason, rather than applying one target to everything",
"Adds a review trigger for any month where the target was missed more often than met",
"Does not redefine what counts as a response in order to meet the number",
"States that a published target nobody can meet is worse than publishing none",
"Does not count hours committed to the enterprise contracts as capacity available to the public queue"
]
},
{
"id": 9,
"prompt": "birchkit, a Node testing library. Our median time to first response now reads as 3 hours and our closure ratio is up 40% quarter over quarter, so the board slide looks great. Except both my triagers told me last week they're completely fried. We turned on a welcome bot in March that posts a thanks-for-filing comment on every new issue, and we added an LLM step that auto-closes anything it scores as a duplicate. What else can we automate to keep these numbers moving?",
"expected_output": "An explanation that the metrics are measuring the bot rather than the project, removal of the auto-greeting and the autonomous auto-close, a list of the mechanical automation that is safe, and a metric set pairing each count with a human-driven signal.",
"files": [],
"expectations": [
"Refuses the auto-greeting bot as a way to improve first-response numbers and recommends removing or discounting it",
"States that first-response metrics exclude bot replies and author self-replies by definition",
"Explains that an automated greeting followed by weeks of silence makes the wait worse for the reporter",
"Warns that closure ratio and time-to-close can improve while real maintainer burden rises, which matches the triagers' reported exhaustion",
"Requires every count metric to be paired with a human-driven signal",
"Names at least two human-driven signals, including first response with bots and author self-replies excluded, and how many people could cover triage if one person left",
"Rejects total open issue count as a health metric",
"Rejects closed-per-week or closure ratio as a productivity target",
"Lists the mechanical automation that is safe: area label from changed paths, type from the intake form, review request to the owning team, and the staleness clock",
"Removes the autonomous auto-close and keeps any model assistance as a scoring and suggestion layer with a human as final decision-maker on accept, reject and close",
"Requires disclosure inside any triage comment posted by an agent",
"Recommends segmenting first response by first-time versus returning reporter",
"Recommends tracking time to first response, untriaged share, queue age, and inflow versus outflow as the monthly set"
]
},
{
"id": 10,
"prompt": "quartzpipe, a data pipeline library. We have three markdown issue templates and people routinely delete the contents before submitting. About 70% of what comes in is bug reports, 20% feature requests, 10% people asking how to use the thing. I want to add a fourth markdown template for support questions so at least those arrive structured. We don't have a forum or a Discord and I'm not starting one. What fields should the support template have?",
"expected_output": "A refusal to build a support form, chooser routing deleted from this pass because no destination exists, validated forms promoted to first because bug reports dominate, and a bug form specified field by field with each field justified by the round-trip its absence forces.",
"files": [],
"expectations": [
"Refuses to create a usage-question issue form and states that usage questions should be routed rather than structured",
"Deletes chooser routing from this pass because no off-tracker destination exists, rather than listing it as pending or future work",
"Promotes validated forms to the first intake change because bug reports dominate the inflow at 70%",
"Replaces the markdown templates with validated forms, stating that markdown templates enforce nothing because the reporter can delete the whole body",
"Makes the version field a dropdown rather than free text, because free-text versions arrive as \"latest\"",
"Requires a minimal reproduction using the project's own APIs rather than a third-party wrapper",
"Requires logs or output rendered as a code block rather than screenshots of text",
"Includes a search-first confirmation checkbox",
"Orders the feature-request fields as problem statement, then proposed solution, then alternatives",
"Disables blank issues only once the forms cover the real categories",
"Auto-applies the type label from the form at submission",
"Justifies each required field by the round-trip its absence forces, and drops fields that do not change the outcome",
"States the triage minutes each intake change is expected to save"
]
},
{
"id": 11,
"prompt": "ashvane, a Kotlin serialization library. Our CONTRIBUTING.md has had a documented triage rota since 2024: six named volunteers, one week each, plus a biweekly triage call. I checked and nobody has actually done a shift since last March, and the call hasn't happened in about that long either. I'm going to re-announce it in the next release notes and remind everyone of their slot. Anything else I should put in the announcement?",
"expected_output": "A refusal to simply re-announce, the dormant-rotation failure named, a two-tier owner-plus-duty design whose mechanics are automated rather than documented, and a liveness check that makes a lapsed shift visible.",
"files": [],
"expectations": [
"Refuses a plain re-announcement of the existing rota as the fix",
"Names the failure that documentation still implying coverage is worse than having had no rotation, citing the dormant volunteer triage team precedent",
"Requires the rota to be retired loudly if it has lapsed, rather than left published while inactive",
"Designs a two-tier model: a long-term triage owner accountable per component plus a triage duty rotating on a stated shift length",
"Automates who is on duty rather than what to decide, by syncing the tracker's triage-owner field from a rotation calendar",
"Names a backup per shift",
"Adds a liveness check: one metric that would visibly move if the rotation stopped",
"Sets a stop rule where a session ends at its time box rather than at inbox zero",
"Sets the session order as newest untriaged first, then items where the reporter replied, then a fixed backlog quota",
"Grants triage-level permissions for labelling, closing and assigning rather than commit or push access",
"Defines the escalation rule: what a triager decides alone, what waits for a maintainer, and how long that wait may be",
"For the recurring call, requires a dedicated queue view, asynchronous pre-triage, and decisions recorded as label or state changes rather than only in minutes",
"Does not treat writing the rotation down or re-announcing it as the finish line"
]
},
{
"id": 12,
"prompt": "nordvex, an OSS analytics collector. Leadership wants me to justify hiring a second triager, so I need numbers. How much contributor retention do we lose per additional day of first response? If we go from 5 days to 3 days, what percentage gain is that? I've also budgeted 3 minutes per issue in the capacity model, that's the standard figure right? And what should I set as our target untriaged percentage so I can put it in the deck.",
"expected_output": "A refusal to supply a per-day retention curve, with the actual evidence described accurately and its limits stated, three minutes labelled as a planning placeholder to be replaced by a timed pilot, and the target figures labelled as starting baselines rather than benchmarks.",
"files": [],
"expectations": [
"Refuses to supply a per-day retention cost curve and states that no dose-response relationship between triage response time and retention is supported",
"States that the strongest available evidence measures pull-request first response, not issue triage",
"Describes that evidence as a binary at-or-under-one-day versus later split rather than a continuous relationship",
"Notes that the reported deltas are descriptive rather than significance-tested",
"Reports the newcomer effect as a median of roughly +15 percentage points and the existing-contributor effect as roughly +2 percentage points",
"Declines to extend the pull-request finding into a general issue-triage service-level promise",
"Does not supply an invented percentage, curve or benchmark in place of the unsupported figure",
"Labels three minutes per item as a planning placeholder rather than a standard or measured industry figure",
"Replaces it by timing a real pilot session and dividing, rather than by importing another constant",
"Sizes the pilot at roughly 15 to 20 real items and requires it to be run and timed against the new system",
"Labels the untriaged-share-below-10%, target-met-on-90%, and net-inflow-at-or-below-zero figures as this approach's own starting baselines to be replaced by the project's measured trend",
"Requires the baseline to be re-run monthly for the first quarter",
"Recommends lowering the published promise to one the team can hold if the targets do not hold, rather than leaving an unmet number published"
]
}
],
"trigger_queries": [
{ "query": "our issue tracker is out of control", "should_trigger": true },
{ "query": "design an issue triage process for our maintainer team", "should_trigger": true },
{ "query": "set up labels for our repo", "should_trigger": true },
{ "query": "we have 900 open issues and no idea where to start", "should_trigger": true },
{ "query": "should we run a stale bot", "should_trigger": true },
{ "query": "PR backlog nobody reviews", "should_trigger": true },
{ "query": "we need a triage rotation", "should_trigger": true },
{ "query": "we are drowning in AI-generated reports", "should_trigger": true },
{ "query": "maintaining this project is completely overwhelming me", "should_trigger": true },
{ "query": "how do we decide who answers new issues each week", "should_trigger": true },
{ "query": "our labels are a mess, redo them", "should_trigger": true },
{ "query": "what response time should we promise on GitHub issues", "should_trigger": true },
{ "query": "nobody on the team knows which issues are already handled", "should_trigger": true },
{ "query": "I want to mass-close everything older than a year, is that a bad idea", "should_trigger": true },
{ "query": "how many labels should an open source project have", "should_trigger": true },
{ "query": "write a triage policy for our repository", "should_trigger": true },
{ "query": "we get the same usage question over and over in the tracker and it's eating our week", "should_trigger": true },
{ "query": "how do I stop people opening support questions as bug reports", "should_trigger": true },
{ "query": "our contributors complain we never reply, what should we publish", "should_trigger": true },
{ "query": "help me size how many issues two volunteers can actually handle a week", "should_trigger": true },
{ "query": "security reports keep showing up in our public issue tracker", "should_trigger": true },
{ "query": "we need a process for external pull requests from strangers", "should_trigger": true },
{ "query": "set a first response SLA for our open source project", "should_trigger": true },
{ "query": "can we automate labeling issues", "should_trigger": true },
{ "query": "should a bot post a welcome comment on new issues", "should_trigger": true },
{ "query": "how do we know if our triage is actually working", "should_trigger": true },
{ "query": "what metrics should we track for our issue queue", "should_trigger": true },
{ "query": "design intake forms for our GitHub repo", "should_trigger": true },
{ "query": "our issue templates are ignored by everyone", "should_trigger": true },
{ "query": "I want to restrict who can open issues", "should_trigger": true },
{ "query": "our bug bounty is flooded with junk submissions", "should_trigger": true },
{ "query": "how do we stop low quality submissions without insulting people", "should_trigger": true },
{ "query": "the queue keeps growing no matter what we do", "should_trigger": true },
{ "query": "we have four maintainers and 60 new issues a week, what do we do", "should_trigger": true },
{ "query": "how should I structure priority labels", "should_trigger": true },
{ "query": "our stale bot closed a confirmed bug and people are angry", "should_trigger": true },
{ "query": "what should happen to issues nobody has touched in two years", "should_trigger": true },
{ "query": "build a label taxonomy for a multi-team monorepo", "should_trigger": true },
{ "query": "we want a weekly triage meeting, how should it run", "should_trigger": true },
{ "query": "who should be allowed to close issues on our repo", "should_trigger": true },
{ "query": "I keep spending my evenings answering the same three questions on GitHub", "should_trigger": true },
{ "query": "is it worth giving people triage permissions instead of write access", "should_trigger": true },
{ "query": "plan out how our project handles incoming reports end to end", "should_trigger": true },
{ "query": "our open source project is burning me out with notifications", "should_trigger": true },
{ "query": "we need to decide what is out of scope and stop re-arguing it", "should_trigger": true },
{ "query": "how should we handle feature requests we will never build", "should_trigger": true },
{ "query": "what goes in a security contact versus the issue tracker", "should_trigger": true },
{ "query": "our PRs from outside contributors sit for months", "should_trigger": true },
{ "query": "should issues or PRs get answered first when we only have time for one", "should_trigger": true },
{ "query": "we need a system that survives when the current maintainers leave", "should_trigger": true },
{ "query": "our tracker has 47 labels and half of them are used once", "should_trigger": true },
{ "query": "how do I clean up labels without losing history", "should_trigger": true },
{ "query": "write the rules for when we close things", "should_trigger": true },
{ "query": "the AI slop reports are costing us more time than the real bugs", "should_trigger": true },
{ "query": "how do we gate submission volume on our repo", "should_trigger": true },
{ "query": "set up a triage duty rotation with backups", "should_trigger": true },
{ "query": "our rota exists on paper but nobody has done a shift in a year", "should_trigger": true },
{ "query": "what is a reasonable untriaged percentage to aim for", "should_trigger": true },
{ "query": "measure our issue queue before we change anything", "should_trigger": true },
{ "query": "I want to understand our backlog before redesigning anything", "should_trigger": true },
{ "query": "we're a company-backed project with enterprise customers and a public tracker, how do we split them", "should_trigger": true },
{ "query": "how do we triage without it becoming a full time job", "should_trigger": true },
{ "query": "define states for issues like needs-info and blocked", "should_trigger": true },
{ "query": "make our issue process work for a project with one maintainer", "should_trigger": true },
{ "query": "what's the right way to run a first pass over a huge backlog", "should_trigger": true },
{ "query": "our issue numbers look healthy but everyone is exhausted", "should_trigger": true },
{ "query": "can we use an LLM to close issues automatically", "should_trigger": true },
{ "query": "plan a pilot session to test our new triage process", "should_trigger": true },
{ "query": "we published a 24 hour response promise and never hit it", "should_trigger": true },
{ "query": "how do we handle duplicate reports systematically", "should_trigger": true },
{ "query": "put together the rules for how our repo handles incoming work", "should_trigger": true },
{ "query": "every new contributor asks where they're supposed to ask questions", "should_trigger": true },
{ "query": "we need to stop people filing blank issues", "should_trigger": true },
{ "query": "how big should a security response team be for an open source project", "should_trigger": true },
{ "query": "what do we do about items waiting on us versus waiting on the reporter", "should_trigger": true },
{ "query": "write a CONTRIBUTING.md for our project", "should_trigger": false },
{ "query": "our good first issues get no takers", "should_trigger": false },
{ "query": "how do we get more open source contributors", "should_trigger": false },
{ "query": "design the first pull request experience for newcomers", "should_trigger": false },
{ "query": "nobody contributes to my project", "should_trigger": false },
{ "query": "curate a good-first-issue queue for beginners", "should_trigger": false },
{ "query": "write a GOVERNANCE.md for our project", "should_trigger": false },
{ "query": "how do we promote a contributor to maintainer", "should_trigger": false },
{ "query": "we have a bus factor of one, what governance do we need", "should_trigger": false },
{ "query": "should we donate the project to a foundation", "should_trigger": false },
{ "query": "our maintainers are deadlocked on a technical decision", "should_trigger": false },
{ "query": "write a code of conduct for our community", "should_trigger": false },
{ "query": "someone is harassing another contributor in our Discord", "should_trigger": false },
{ "query": "how do we ban a repeat offender from our Slack", "should_trigger": false },
{ "query": "recruit moderators for our community", "should_trigger": false },
{ "query": "our Discord is getting out of hand", "should_trigger": false },
{ "query": "build an error code reference page from our support tickets", "should_trigger": false },
{ "query": "write a known-issues page for our docs", "should_trigger": false },
{ "query": "users keep hitting this error, document it properly", "should_trigger": false },
{ "query": "turn our FAQ into proper troubleshooting docs", "should_trigger": false },
{ "query": "is our Discord community dying", "should_trigger": false },
{ "query": "build a community health dashboard", "should_trigger": false },
{ "query": "what community health metrics should we track", "should_trigger": false },
{ "query": "measure our contributor funnel from lurker to committer", "should_trigger": false },
{ "query": "what KPIs should our devrel team report to the exec team", "should_trigger": false },
{ "query": "build a devrel metrics dashboard for the quarter", "should_trigger": false },
{ "query": "should we use a CLA or a DCO", "should_trigger": false },
{ "query": "which open source license should we pick", "should_trigger": false },
{ "query": "help me choose between MIT and Apache 2.0", "should_trigger": false },
{ "query": "rewrite our README so it converts visitors", "should_trigger": false },
{ "query": "optimize my GitHub profile page", "should_trigger": false },
{ "query": "plan the launch of our open source project on Hacker News", "should_trigger": false },
{ "query": "get our library into homebrew and apt", "should_trigger": false },
{ "query": "triage this issue for me: users report the CLI hangs on Windows", "should_trigger": false },
{ "query": "label this GitHub issue correctly", "should_trigger": false },
{ "query": "read issue 4821 and tell me whether it's a duplicate", "should_trigger": false },
{ "query": "go through my open issues and tell me which to work on next", "should_trigger": false },
{ "query": "summarize the new issues filed on the repo since yesterday", "should_trigger": false },
{ "query": "triage my Gmail inbox", "should_trigger": false },
{ "query": "triage our customer support tickets in Zendesk", "should_trigger": false },
{ "query": "triage the Dependabot alerts in this repo", "should_trigger": false },
{ "query": "which CVEs from our scanner do we need to fix first", "should_trigger": false },
{ "query": "triage this crash dump and tell me the faulting frame", "should_trigger": false },
{ "query": "fix this Sentry issue", "should_trigger": false },
{ "query": "create GitHub issues from this implementation plan", "should_trigger": false },
{ "query": "open an issue for each unmet requirement in the spec", "should_trigger": false },
{ "query": "file a bug report for what I just described", "should_trigger": false },
{ "query": "debug why our CI is failing on main", "should_trigger": false },
{ "query": "we have a production incident, help me run the on-call triage", "should_trigger": false },
{ "query": "run sprint planning and groom our Jira backlog", "should_trigger": false },
{ "query": "estimate these tickets for the next sprint", "should_trigger": false },
{ "query": "prioritize our product roadmap for Q3", "should_trigger": false },
{ "query": "score our feature backlog with RICE", "should_trigger": false },
{ "query": "how much should we pay per severity tier in our bug bounty", "should_trigger": false },
{ "query": "write a disclosure timeline for a CVE we found in someone else's library", "should_trigger": false },
{ "query": "review this pull request and tell me if the code is correct", "should_trigger": false },
{ "query": "write the release notes for version 3.2", "should_trigger": false },
{ "query": "write a changelog entry for this release", "should_trigger": false },
{ "query": "set up a documentation site structure", "should_trigger": false },
{ "query": "make our docs rank better in search", "should_trigger": false },
{ "query": "write a quickstart guide for our SDK", "should_trigger": false },
{ "query": "design a tutorial for our API", "should_trigger": false },
{ "query": "plan our conference talk submissions for next year", "should_trigger": false },
{ "query": "find developer meetups we should sponsor", "should_trigger": false },
{ "query": "how do we run a developer champions program", "should_trigger": false },
{ "query": "set up GitHub Sponsors tiers for my project", "should_trigger": false },
{ "query": "how do I get a company to fund my maintenance work", "should_trigger": false },
{ "query": "build a pricing model for our developer tool", "should_trigger": false },
{ "query": "segment our developer audience", "should_trigger": false },
{ "query": "write an engineering blog post about our new architecture", "should_trigger": false },
{ "query": "plan a live demo for our product launch", "should_trigger": false },
{ "query": "hire a developer advocate", "should_trigger": false },
{ "query": "what does a devrel career path look like", "should_trigger": false },
{ "query": "how should we structure our devrel team", "should_trigger": false },
{ "query": "which podcasts and newsletters should I follow to keep up with devrel", "should_trigger": false }
]
}
references/issue-intake-and-routing.md›
# Issue intake and routing
Contents:
- what to ask on a form
- the template chooser as a routing layer
- tracker capabilities
- automation boundaries
## Ask only for what changes the outcome
A field earns its place when its absence forces a round-trip. Everything else trains reporters to skim the form.
**Bug report - the fields that decide the outcome**
| Field | Required | Why |
| ----------------------------------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| What happened / what you expected | yes | The report's only irreducible content |
| Minimal reproduction | yes | Node.js reduces its whole ask to this plus the description - a self-contained case using the project's own APIs, since a third-party wrapper hides whether the bug is even yours |
| Version | yes, as a dropdown | Free-text versions arrive as "latest" |
| Environment (OS, runtime, install method) | yes | Decides reproducibility |
| Logs / output | yes, rendered as a code block | Screenshots of text are unsearchable |
| Search-first confirmation | checkbox | Cuts duplicates measurably; costs the reporter one click |
**Feature request**
Order the fields:
1. Problem statement
2. Proposed solution
3. Alternatives
A form opening with "describe your solution" produces solutions to unstated problems, the hardest requests to triage.
**Usage question** - there should be no such form. Route it (below).
## The chooser is where routing happens
The list a reporter sees before typing is the highest-leverage surface in the system: it acts before any attention is spent.
- One entry per category the project wants, ordered deliberately (filename prefixes control order on most trackers: `01-bug`, `02-feature`).
- Off-tracker destinations for everything else: usage questions → forum, chat or discussions; security → the disclosure address; commercial or licensing → a contact address.
- Disable the blank/free-form option once the forms cover the real categories.
- Keep an explicit "something else" form if the project wants unclassifiable input - an undocumented dead end is worse than a wide door.
On GitHub this is `.github/ISSUE_TEMPLATE/config.yml` (`blank_issues_enabled`, `contact_links`); GitLab uses repository description templates. The mechanism differs, the design does not.
## Tracker capabilities to check before writing forms
- **Validated forms.** YAML issue forms (`.github/ISSUE_TEMPLATE/*.yml`) support text, textarea with code rendering, dropdown, multi-select, checkbox, file upload and markdown blocks, with `validations: required: true`. Markdown templates enforce nothing - the reporter can delete the whole body.
- **Auto-application.** A form can attach labels, an issue type, assignees and a title prefix on submission.
- **Org-wide defaults.** Default community health files ship one template set to every repository lacking its own - check before copying files by hand across repos.
- **Scripted fixes.** Some tracker CLIs cannot set every field (`gh issue create` cannot set an issue type; the REST API can). Check before scripting a bulk retrofit.
## The security path is not a routing entry
A contact link pointing at a security address is the visible part; the pipeline behind it is what matters, and every mature project runs the same three elements.
1. **A private intake channel** - a dedicated address, a bounty platform, or the code host's private vulnerability reporting; never the public tracker. With no security list, private vulnerability reporting is the default: it carries its own triage state and a private fork for collaborating on the fix.
2. **A named, size-capped response team**, distinct from general triage staff. Kubernetes caps its committee at ten members, nominated by sitting members, and acknowledges each report within three working days. Apache runs central triage across projects and lets mature projects request their own list once they can handle it.
3. **An embargo mechanism** for coordinated disclosure to downstream distributors before public release.
Two operational rules follow:
- A vulnerability reported publicly by mistake is an emergency: route it to the private channel immediately rather than triaging in the open.
- Staff it deliberately: security triage carries an isolation cost ordinary triage does not, because the work cannot be discussed with the wider project.
## Gating volume at the intake
When the valid share collapses, better forms do not help - the intake economics have to change. Options projects actually exercise, in rough order of reversibility:
- Require a reputation or contribution-history threshold before a submission is accepted.
- Cap the number of open submissions per author.
- Restrict issue or pull-request creation to collaborators, or turn a submission type off entirely - code hosts shipped these toggles in 2026 as a maintainer response to machine-generated contribution overload.
- Raise the submission cost: required reproduction, proof of concept, self-contained test case.
- Remove an incentive being farmed, including a monetary reward.
Gate on reputation and volume, not on whether content looks machine-generated. There is no reliable detector for the latter - OpenSSF's working group records that detection rests on maintainer intuition - and accusing a contributor without proof creates a conduct incident on top of the original problem.
## Automation boundaries
Automate classification and routing:
- path-based labelling fills the area axis from a PR's changed files
- owner files request review from the owning team, so the wait sits on a named group
- form-applied labels fill the type axis at submission
- the staleness sweep runs the waiting-on-reporter clock
Do not automate the first human sentence - the reasoning is in step 8.
## Retrofitting the existing queue
New intake only affects new items. For the existing backlog, either accept two vocabularies (old items keep their labels) or run a one-time bulk pass. Bulk-relabelling is scriptable through the tracker's API - do it in one batch, announce it, and expect notification volume to be the real cost.
references/label-taxonomy.md›
# Label taxonomy
Contents:
- the axes
- sized starter sets
- naming rules
- cleaning up an existing label mess
- tracker mechanics
## The axes
A queue is readable when each label belongs to one axis and an item carries at most one value per axis.
| Axis | Answers | Applied by | Required |
| ------------------ | ------------------------------- | ----------------------------------------- | ---------------------------------- |
| Type | What kind of work is this? | Intake form, or triager | Always |
| State | What is this waiting on? | Triager, automation | Always |
| Priority | What happens if nobody does it? | Triager or maintainer | Once the queue outgrows one person |
| Area | Which part of the project? | Automation from changed paths, or triager | Once there is more than one owner |
| Contributor-facing | Is this open to outside help? | Maintainer, deliberately | Only when the promise can be kept |
Two labels from the same axis on one item means nobody decided - the single most useful invariant to enforce, by convention or by tooling.
A minimum viable state set:
- `needs-triage` - nobody looked
- `needs-info` - waiting on the reporter
- `accepted` - real, will be worked
- `needs-owner` - real, unstaffed
- `blocked` - waiting on something external
Kubernetes runs `needs-triage` → `/triage accepted`; mattpocock's triage skill runs five states ending in `ready-for-agent` / `ready-for-human` / `wontfix`. Both work: pick one vocabulary and write it down.
## Sized starter sets
The three reference points from step 3:
- **ten** default GitHub labels (`bug`, `documentation`, `duplicate`, `enhancement`, `good first issue`, `help wanted`, `invalid`, `question`, `wontfix`, `accessibility`)
- **five state roles plus two categories** in the most-installed triage skill on skills.sh
- **six prefixed families** at Kubernetes (`kind/`, `priority/`, `sig/`, `area/`, `triage/`, `lifecycle/`)
The mapping below is this skill's judgment, not a sourced rule.
**Solo maintainer, small queue.** Four labels, state only:
- `needs-triage`
- `needs-info`
- `accepted`
- `blocked`
Type comes from the intake form. Priority honestly lives in the maintainer's head, and pretending otherwise creates unused labels. This sits below the shipped default of ten - the right direction: delete the defaults you will not apply.
**Small team.** Add:
- type: `type/bug`, `type/feature`, `type/docs`, `type/support`
- a two-value priority: `priority/now`, `priority/later` (three is already a debate)
- a contributor-facing pair: `help wanted`, `good first issue`
Near the five-plus-two shape.
**Multi-owner project.**
- Add `area/*` per owning team or subsystem.
- Split priority into the four Kubernetes levels if release planning depends on it.
- Add `lifecycle/*` for staleness.
This is the six-family shape, and the point at which the taxonomy needs an owner, a written definition file and the validation below, or it decays.
## Naming rules
- Prefix every multi-value axis: `kind/bug`, `priority/now`, `area/parser` - the prefix makes the axis visible in an alphabetical label list.
- Write one sentence of definition per label, in the repository, not only in the tracker's description field.
- Name the state from the waiting party's perspective (`needs-info` says who owes what; `pending` says nothing).
- Reuse conventional names where they exist (`good first issue`, `help wanted`) - trackers and third-party indexes surface them algorithmically, and a custom synonym loses that reach.
- Colour by axis, not by mood: one hue per family reads at a glance.
## Cleaning up an existing set
1. Export the label usage counts (`triage-baseline.py` prints them, including labels used exactly once).
2. Sort into: keep, rename into an axis, merge into another label, delete.
3. Delete unused and single-use labels - a label nobody applies twice is a private note.
4. Rename before deleting: deleting erases the label from every past item and any saved view built on it.
5. Migrate in one pass, announce it in one place, updating every saved view, form and automation in the same pass. A half-migrated taxonomy is worse than either version.
## Keeping a taxonomy from decaying: the allowlist pattern
A taxonomy with many labels and any automation touching it needs an enforcement layer. PyTorch runs one in production on its main repository - the clearest worked pattern available.
- **Hold the triage-eligible label set as a separate, versioned, machine-readable artifact**, not "whatever the tracker currently contains". Theirs carries a name and description per entry and deliberately excludes CI triggers, release-note labels, deprecated labels and anything needing a human decision; keeping it smaller than the live label set is the point.
- **Validate every write against it**, stripping labels not in the allowlist - which rejects invented or guessed names outright.
- **Collapse redundant pairs** - drop the general area label when a more specific one from the same family is present.
- **Merge with existing labels, never replace.** Most "set labels" operations are a full overwrite, so a write without a merge step silently erases what a human applied. The single highest-consequence detail in the pattern.
- **Fall back to a review state rather than doing nothing** when validation filters everything out, so the item still carries a status.
- **Stamp mutations for auditability** with a marker label, so triage actions can be filtered and reviewed later independently of the decision logic.
- Step 3's two PyTorch rules - confidence-gated labelling, top priority never auto-applied - are part of this same pattern.
Without automation the same discipline applies: an undefined label is applied inconsistently within a week, and an unvalidated taxonomy drifts into synonyms.
## Tracker mechanics worth knowing
- **Mutual exclusion.** GitLab's scoped labels (`priority::high`) enforce one value per key - a second value of the same key removes the first. Without that feature the invariant is a convention the triager holds.
- **Types vs labels.** Where the tracker has a first-class issue type (Bug, Feature, Task, Epic), use it for the type axis rather than duplicating it as labels.
- **Triage permission.** GitHub's `Triage` role grants labelling, closing, reopening, assigning, marking duplicates and requesting reviews, with no push or merge access - how a triage team grows without commit rights. Check the equivalent role on other trackers.
- **Automation surfaces.** Path-based auto-labelling fills `area/*` from the diff; owner files route review requests; intake forms apply the type label at submission. Every axis filled by a machine is one the triager does not spend attention on.
references/policy-and-metrics.md›
# Capacity, closing policy and metrics
Contents:
- capacity arithmetic
- the published policy document
- closing and staleness rules
- measurement
## Capacity arithmetic
Derive the response target from capacity; never the reverse.
```
weekly triage capacity (items) = triagers × sessions/week × minutes/session ÷ minutes per item
```
Subtract the hours consumed by items that will never be actionable before comparing against inflow - a queue where most submissions are invalid fails the arithmetic even when nominal capacity looks adequate. Then compare against inflow from the baseline (`triage-baseline.py` prints created vs closed over the window):
- Capacity ≥ inflow → the backlog can shrink; commit to a first-response target and add a backlog burn-down quota per session.
- Capacity < inflow → the queue grows however the labels are arranged. Pull one of step 2's three levers and say which, out loud, in the policy.
A company-backed project usually has paid capacity and possibly contractual response obligations for paying customers - separate those into their own intake path rather than letting them set the target for the volunteer-visible queue. A volunteer project should set a target it can hold during a bad month, not a good one.
## The published policy
Write it in the repository, next to the contributing guide, and link it from the intake forms. Keep it under a page.
```markdown
# How we triage
## What belongs here
Bugs and feature requests. Usage questions → [forum]. Security → [address].
## What to expect
First response: within [N business days]. Not a fix; a human reply.
[Class of items] gets [tighter target] because [reason].
## States
[label]: [what it means, who it waits on, what unblocks it]
## When we close
- Waiting on the reporter for [N] days with no reply.
- Out of scope (with the reason and a link).
- Cannot reproduce after [what was tried].
Closed is not final: comment with the missing detail and it reopens.
## Who triages
[Rotation, cadence, named backup, how to join.]
## If volume outruns us
[The trigger: inflow rate or valid share, and what we will restrict when it is hit.]
```
Publishing the target converts an unanswered issue from "abandoned project" into "known queue". A policy nobody can meet is worse than none - publish the number the baseline supports.
## Closing and staleness
A defensible configuration of the step-7 rule:
- Restrict the sweep to the waiting-on-reporter state (the `only-labels` mechanism in `actions/stale`; equivalent filters exist elsewhere).
- Exempt the confirmed, accepted, blocked, security and pinned states explicitly.
- Warn first, close later - 14 to 30 days from warning to close is common; `actions/stale` defaults to 60 days idle then 7 days to close.
- Run the first sweep in dry-run mode (`debug-only`) and read the list before it fires. A first live run on an old backlog can close hundreds of items and generate a notification storm that costs the project contributors.
- Make the close message state how to reopen.
Kubernetes' published clocks, each anchored to a named waiting party:
- 20 days without a reporter response → close
- 30 days without assignee action → ask them to produce a patch or release the assignment
- 30 days without owning-group movement → escalate to the group
- 90 days without any activity → mark stale
Backlog bankruptcy (mass-closing everything older than a threshold, with an explanation and an open invitation to reopen) is legitimate exactly once, as the opening move of a new policy. It requires the announcement, the new policy and the new intake to go live the same day. A named practitioner account independently confirms the "exactly once" limit: a maintainer who tried mass-closing everything older than six months found "it works once," then found the backlog back in the same state six months later, because the underlying issue - no natural lifecycle for issues and PRs - was never addressed by the closure itself.
## Metrics
Track four numbers monthly, from the same script that produced the baseline:
- **time to first response** - created → first reply from someone other than the author, bots excluded; whether the published promise is real
- **untriaged share** - open items with no state label ÷ open items; whether triage happens at all
- **queue age** - median and p90 of open items; whether the backlog ages or turns over
- **inflow vs outflow** - created vs closed over the window; whether capacity matches demand
Bot replies and author self-replies are excluded from first response by definition - CHAOSS's _Time to First Response_ and _Issue Response Time_ both make that exclusion explicit, because counting a greeting bot hides the exact delay being measured. Step 9 carries the thresholds and the rule against reading a count metric alone.
Segment first response by whether the opener is a first-time or returning contributor when the export allows it. A project that answers regulars in hours and newcomers in weeks has a contributor-funnel problem invisible in the median.
Don't track:
- total open issue count as a health metric (a growing project grows its queue)
- closed-per-week as a productivity target (it rewards closing hard items without deciding them)
## Review cadence
Re-read the policy against reality:
- quarterly
- immediately after a maintainer joins or leaves
- immediately after a release changes the inflow mix
- immediately after a month where the target was missed more often than met
Adjust the target or the intake, never the definition of what counts as a response.
references/published-findings.md›
# Published findings
This file covers:
- published figures with their sources
- the self-set baselines this skill uses instead, where no published figure exists
- the claims it refuses to make
Read it before quoting any number to a user: a maintainer will ask "says who?", and the answer differs by line.
## Sourced figures
| Figure or claim | Source |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| 20 days no reporter response → close; 30 days no assignee action → reassign; 30 days no group movement → escalate; 90 days idle → stale | Kubernetes contributor issue-triage guide |
| Four-tier priority vocabulary, with "drop what you're doing" as the top tier's own wording | Kubernetes contributor issue-triage guide |
| Stale-sweep defaults of 60 days idle then 7 days to close, plus label-restricted sweeps and a dry-run mode | The `actions/stale` action's own documentation |
| First response excludes bot replies and author self-replies | CHAOSS _Time to First Response_ and _Issue Response Time_ metric definitions |
| Count metrics apply only where activity is "primarily driven by humans" | CHAOSS metric guidance |
| Under machine-generated volume, event counts decouple from what they proxied for | Andrew Nesbitt, 2026-05-27 |
| Ten default labels on every new repository | GitHub documentation on managing labels |
| A triage permission tier that grants labelling, closing and assigning without push access | GitHub repository-roles documentation |
| Scoped labels enforcing one value per key | GitLab labels documentation |
| Five state roles plus two categories, in the most-installed triage skill on skills.sh | `mattpocock/skills@triage`, 596K installs |
| Long-term triage owner plus weekly rotating triage duty; duty routing synced from a rotation calendar | Mozilla bug-management policy and the `autonag` rotation announcement |
| Weekly compiler triage meeting with asynchronous pre-triage and priority set by label change | Rust compiler-team triage-meeting documentation |
| Security response committee capped at ten members, reports acknowledged within three working days | Kubernetes security-response committee repository |
| "Do not enter details of security vulnerabilities in a project's public bug tracker" | Apache Software Foundation security guidance |
| Label allowlist as a versioned artifact; writes validated, unknown labels stripped, existing labels merged not replaced; confidence-gated labelling; top priority never auto-applied | `pytorch/pytorch` triaging-issues skill, running in production on that repository |
| 35.7% of rejected agentic PRs were clear agentic failures, 31.2% workflow constraints, 33.1% no observable rationale, from 353 manually inspected PRs | MSR 2026 Mining Challenge empirical study |
| Agentic PRs show substantially longer pickup and wait times than unassisted PRs | LinearB 2026 Software Engineering Benchmarks Report |
| Roughly 20% of 2025 submissions AI slop; confirmed-vulnerability rate fell from above 15% to below 5%; bug bounty ended 2026-01-31 | Daniel Stenberg's own posts on curl's programme |
| "Generating a plausible-sounding vulnerability report now costs pennies in tokens. Evaluating whether it's real still costs an hour of expert time." | Kate Holterhoff, RedMonk, 2026-05-05 |
| "A sense of isolation due to the secretive nature of security reports" | Seth Larson, Python security developer-in-residence |
| Maintainer burnout named the top challenge by 45% of respondents | Intel open-source community survey, 2023 |
| A documented volunteer triage team can lapse into dormancy while its documentation still implies coverage | Fedora BugZappers' own joining page |
| Newcomers whose PR got a human first response within 1 day were 2.63%-37.44% more likely (median +15.14 pp) to submit another PR within a year than those who waited longer; existing contributors saw a smaller +0.06%-9.48% (median +2.25 pp) effect | Hasan et al., "Understanding the Time to First Response In GitHub Pull Requests," MSR 2023, 111,094 PRs across 10 projects |
## Self-set baselines
This skill's own defaults rather than industry benchmarks. Present them as starting points and replace each with the project's measured trend once a quarter of data exists.
| Baseline | Where it appears | How to replace it |
| ---------------------------------------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| Three minutes per item in the capacity formula | Step 2 | Time the step 9 pilot session and divide; use the project's own figure from then on |
| Untriaged share below 10% of open items | Step 9 | Set it from the first quarter's own distribution |
| Published target met on at least 90% of items | Step 9 | Lower it to whatever the team actually sustained in the first quarter, then improve deliberately |
| Net inflow at or below zero over a quarter | Step 9 | Reasonable as a direction; as a threshold it depends entirely on project growth |
| Two-minute per-item time box | The response playbook | Same pilot measurement as the capacity constant |
| The label-count sizing advice | Step 3 and the label taxonomy | Anchored to three real reference points (ten defaults, five-plus-two, six families) but the mapping to project size is this skill's judgment |
| 15-20 items in the pilot session | Step 9 | Enough to expose design gaps; not a validated sample size |
## Claims deliberately not made
- **No RICE, no impact-versus-effort matrix.** Kubernetes, Mozilla, Rust, Apache, GNOME and LLVM all triage on severity and priority labels instead. Naming a scoring framework would dress up a practice foundation-scale projects do not run.
- **No machine duplicate detection as a working default.** Manual search discipline dominates in practice; Mozilla's machine-learning triage tool ships component assignment, not duplicate detection, despite that being an announced plan.
- **No detector for machine-generated submissions.** OpenSSF's working-group issue records "no reliable technical indicator", with detection resting on maintainer intuition; which is why this skill gates on volume and reputation, not content authenticity.
- **No dose-response curve for issue-triage SLA and retention.** The strongest quantitative evidence (Hasan et al., MSR 2023, see above) measures PR first-response, not issue triage, as a binary ≤1-day-vs-later split with descriptive (not significance-tested) deltas — not a continuous "each extra day costs X% retention" relationship. Extending that PR finding to a general issue-triage-SLA promise is not supported by the source.
references/response-playbook.md›
# Response playbook
Contents:
- the per-item decision path
- reply templates with weak/strong pairs
- pull-request deltas
- the rejection knowledge base
## Per-item decision path
Run the same five questions on every item, stopping at the first that resolves it. The order is queue removed per minute spent, not cost: duplicates and already-shipped items lead because each closes an item outright for a search, while the scope decision costs the most thought and only hands the item onward.
- efficiency: duplicate > already answered or shipped > actionable as written > in scope > accept and classify
- effort: duplicate == already answered (one search each) > actionable as written (read the report against a required-fields list) > accept and classify (three axes to set) > in scope (the judgment that starts arguments, and the one that escalates)
The effort tie is real: both are a single search, and which runs first is a coin flip. Re-rank against the queue's shape - a tracker that is mostly usage questions resolves more items on question 2 than on question 1.
1. **Duplicate?** Search by concept, not by the reporter's wording. If it is one, link both directions and close the newer.
2. **Already answered or shipped?** Point at the release, docs page or code, then close.
3. **Reproducible / actionable as written?** A bug needs an environment and a reproduction; a feature request needs a problem statement. If a required element is missing, ask for exactly that one thing and set the waiting-on-reporter state.
4. **In scope?** Decide against the project's stated scope, not against how convincing the reporter is. Out of scope → decline, record why (rejection knowledge base below), close.
5. **In scope and real?** Set type, priority and area, then assign it or mark it unstaffed. "Accepted" without an owner is an honest state; "accepted" implying someone is working on it is not.
Time-box the pass. Most items resolve in under two minutes; the ones that do not are worth batching for a maintainer decision rather than blocking the queue.
## Reply templates
Fill the bracketed parts with specifics from the item. Generic replies read as form letters and produce another round-trip.
**Needs information**
```
Thanks for the report. To reproduce this I need [the exact missing element:
version + OS, or a minimal script using only project APIs].
What we know so far:
- [established fact 1]
- [established fact 2]
I have set [state label]; comment with that and it goes back in the queue.
Without it I will close this in [N] days, and you can reopen any time.
```
Weak: "Please provide more information." The reporter cannot tell what is missing, so the round-trip repeats.
Strong: "I need the output of `tool --version` and the config file with secrets removed." One specific ask, answerable in a minute.
Record what is already established, or a resumed conversation re-asks resolved questions - the fastest way to lose a reporter who was still willing to help.
**Support question routed off the tracker**
```
This is a usage question rather than a defect, so it belongs in [forum / chat /
discussions link] where more people will see it and the answer stays findable.
Short answer: [answer it anyway, in one or two lines].
```
Answering while redirecting makes the redirect land as help rather than a brush-off. A third arrival of the same question is a documentation defect - route it to a docs page instead of a fourth answer.
**Out of scope**
```
Thanks for taking the time to write this up. [Feature] is out of scope for the
project because [reason tied to stated scope, one sentence]. [Link to the
scope statement or the prior decision.] [Where to get it instead, if anywhere.]
Closing: the reasoning is recorded in [link] so it does not get re-litigated.
```
Weak: closing with no comment, or "not planned". Both read as dismissal and reliably generate a follow-up issue.
Strong: two sentences, a reason, a link, a door left open for a different framing of the problem.
**Confirmed bug, unstaffed**
```
Reproduced on [version/environment]: [what happens]. Setting [labels].
Nobody is working on this right now; [contribution pointer].
```
An honest "real, unstaffed, no date" ages far better than silence or an optimistic estimate.
## Pull requests
A pull request is an issue with code attached: same states, same clocks, three deltas.
- **Verify the claim against the diff**, not the description - check it out, run the tests, say what happened.
- **Route by ownership** (owner file, area label) before review, so the wait sits on a named person rather than the queue.
- **Stale-PR cost is asymmetric.** An unanswered issue costs the reporter nothing after they walk away; an unanswered PR cost someone hours already spent. Give PRs the tighter first-response target when capacity forces a choice.
- Externally-authored PRs are triage work; a maintainer's own in-flight PR is not.
## Rejection knowledge base
Keep one file per rejected _concept_ (not per issue) - `dark-mode.md`, `plugin-system.md` - each holding the decision, the reasoning and the prior issues that asked for it. When the request returns under a new name, link the file instead of re-arguing. The reasoning then survives the maintainer who made the call, and a contributor considering the same work reads the answer before writing code.
Rejected requests only. Something already implemented gets a pointer to where it lives.
## Tone rules that hold everywhere
- Thank first, decide second, close third - in that order, in two sentences.
- Never close silently; every close carries its reason, even one line.
- First-time reporters get the patient version, frequent contributors the terse one. The difference is deliberate.
- Disclose machine-generated triage comments; reporters can tell anyway, and undisclosed automation reads as contempt.
- Hostile or abusive behaviour is not a triage decision - hand it to the conduct process.
references/worked-examples.md›
# Worked examples
Contents:
- a capacity read that fails and what to do about it
- a rotation designed well and the same rotation designed badly
## Worked capacity read
From curl's publicly described security-triage situation - real and citable, not because most projects look like curl, but because the arithmetic shape transfers.
**What the baseline says.**
- A seven-person volunteer team.
- Each report costs 30 minutes to 3 hours and typically pulls in three to four people.
- Volunteers have around three hours a week.
- Roughly two machine-generated junk reports arrive weekly, and by mid-2025 only about one submission in twenty was genuine.
**The arithmetic.** Seven people × 3 hours = 21 person-hours a week at the optimistic end. Two junk reports a week, at the midpoint of the cost range with three people pulled into each, consume 2 × 1.75 × 3 ≈ 10.5 person-hours - half the team's weekly budget spent on submissions with no valid content, before a single real vulnerability is looked at.
**Reading it back to the user.** "Your capacity is not the problem and your labels are definitely not the problem. Half your available hours are consumed by submissions that were never valid. No response target you publish will survive that, so the design question is which lever you pull on intake."
**The lever chosen.** curl removed the monetary reward entirely, which changed the economics of submitting rather than the cost of evaluating.
This is not:
- a better classifier
- a stricter template
- a faster triage rotation
When the valid share collapses, the intervention has to act on inflow.
**The general form.**
1. Compute weekly person-hours.
2. Subtract the hours consumed by items that will never be actionable.
3. Compare what remains against genuine inflow.
If the remainder is negative, no amount of process design closes the gap: say so plainly and move to the intake and gating decisions in steps 2 and 4.
## Rotation design: a positive and a negative example
Both are documented projects. The negative one matters more: it is the failure a maintainer walks into while believing the problem is solved.
### Negative - the rotation that exists only on paper
Fedora's BugZappers were a real, well-built triage programme:
- a named volunteer and staff team
- a documented charter
- dedicated tooling
- a biweekly meeting with published minutes
Everything step 5 asks for was present. The project's own joining page now states the group is dormant and little triage is happening.
That is worse than never having had a rotation: the documentation still advertises coverage.
- Reporters expect a response that will not come.
- A contributor who wants to help finds a process to join rather than a gap to fill.
- Maintainers cannot see the failure from inside, because every artefact still exists.
**The mistake:** treating "we wrote down the rotation" as the finish line. A rotation decays silently because no error is raised when a shift is skipped.
### Positive - the rotation that cannot silently decay
Mozilla's two-tier model (step 5) survives because its mechanics are automated, not documented: the rotation lives on a shared calendar and a bot syncs the tracker's triage-owner field to whoever is on duty. Requests route to the person actually working this week, instead of accumulating on one name.
Three properties follow:
- A named owner per component prevents diffusion of responsibility.
- Owner-accountable/rotation-executing keeps the owner from becoming the single point of work.
- A named backup per shift stops a skipped week passing unnoticed.
**The contrast in one line:** Fedora documented who _should_ triage; Mozilla automated who _is_ triaging.
### Applying this
Add a liveness check the project can see: a named backup per shift, and one metric that would move if the rotation stopped. If nothing changes on the dashboard when the rota lapses, the rota is documentation, not a process.
scripts/triage-baseline.py›
#!/usr/bin/env python3
"""Measure an issue/PR queue before redesigning its triage system.
Reads an issue export (JSON array, JSON Lines, or CSV) and reports queue size,
age distribution, untriaged share, label usage, first-response time and
inflow/outflow balance. Expected fields per record, missing ones skipped rather
than fatal: number, title, state, createdAt, updatedAt, closedAt, labels,
author, comments. SKILL.md step 1 has the export and invocation commands.
"""
import argparse
import csv
import json
import sys
from datetime import datetime, timezone
def parse_time(value):
if not value:
return None
if isinstance(value, (int, float)):
return datetime.fromtimestamp(value, tz=timezone.utc)
text = str(value).strip().replace("Z", "+00:00")
if not text:
return None
try:
parsed = datetime.fromisoformat(text)
except ValueError:
for fmt in ("%Y-%m-%d %H:%M:%S", "%Y-%m-%d"):
try:
parsed = datetime.strptime(text, fmt)
break
except ValueError:
continue
else:
return None
return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc)
def load_records(path):
with open(path, encoding="utf-8") as handle:
head = handle.read(1)
handle.seek(0)
if head == "[":
return json.load(handle)
if head == "{":
return [json.loads(line) for line in handle if line.strip()]
return list(csv.DictReader(handle))
def normalize_labels(raw):
"""Accept [{'name': 'bug'}], ['bug'], or 'bug,enhancement'."""
if not raw:
return []
if isinstance(raw, str):
try:
raw = json.loads(raw)
except json.JSONDecodeError:
return [part.strip() for part in raw.split(",") if part.strip()]
if isinstance(raw, dict):
raw = raw.get("nodes", [])
out = []
for item in raw or []:
if isinstance(item, dict):
name = item.get("name") or item.get("title")
if name:
out.append(str(name))
elif item:
out.append(str(item))
return out
def author_login(raw):
if isinstance(raw, dict):
return raw.get("login") or raw.get("username") or raw.get("name")
return str(raw) if raw else None
def first_response_hours(record, created):
"""Hours until the first comment by someone other than the author.
Bot accounts are excluded: an automated greeting is not a response, and
counting it would hide exactly the delay this measures.
"""
comments = record.get("comments")
if isinstance(comments, dict):
comments = comments.get("nodes", [])
if not isinstance(comments, list) or not created:
return None
opener = (author_login(record.get("author")) or "").lower()
best = None
for comment in comments:
if not isinstance(comment, dict):
continue
login = (author_login(comment.get("author")) or "").lower()
if not login or login == opener or login.endswith("[bot]") or login.endswith("-bot"):
continue
at = parse_time(comment.get("createdAt") or comment.get("created_at"))
if at and (best is None or at < best):
best = at
if best is None:
return None
return max(0.0, (best - created).total_seconds() / 3600.0)
def percentile(values, share):
if not values:
return None
ordered = sorted(values)
index = min(len(ordered) - 1, int(round((len(ordered) - 1) * share)))
return ordered[index]
def build_report(records, now, stale_days, triage_prefixes, window_days):
open_ages, response_hours, label_counts = [], [], {}
untriaged, stale, no_response = [], [], []
open_count = closed_count = created_in_window = closed_in_window = 0
for record in records:
state = str(record.get("state") or record.get("status") or "open").lower()
created = parse_time(record.get("createdAt") or record.get("created_at"))
updated = parse_time(record.get("updatedAt") or record.get("updated_at")) or created
closed = parse_time(record.get("closedAt") or record.get("closed_at"))
labels = normalize_labels(record.get("labels"))
number = record.get("number") or record.get("iid") or record.get("id")
for label in labels:
label_counts[label] = label_counts.get(label, 0) + 1
if created and (now - created).days <= window_days:
created_in_window += 1
if closed and (now - closed).days <= window_days:
closed_in_window += 1
if state.startswith("closed") or state == "merged" or closed:
closed_count += 1
continue
open_count += 1
if created:
open_ages.append((now - created).days)
if updated and (now - updated).days >= stale_days:
stale.append(number)
classified = any(
label.startswith(prefix) or label == prefix.rstrip("/")
for label in labels
for prefix in triage_prefixes
) if triage_prefixes else bool(labels)
if not classified:
untriaged.append(number)
hours = first_response_hours(record, created)
if hours is None:
if isinstance(record.get("comments"), (list, dict)):
no_response.append(number)
else:
response_hours.append(hours)
return {
"generated_at": now.isoformat(),
"totals": {
"records": len(records),
"open": open_count,
"closed": closed_count,
},
"open_age_days": {
"median": percentile(open_ages, 0.5),
"p90": percentile(open_ages, 0.9),
"max": max(open_ages) if open_ages else None,
},
"untriaged": {
"count": len(untriaged),
"share_of_open": round(len(untriaged) / open_count, 3) if open_count else None,
"sample": untriaged[:15],
},
"stale": {
"threshold_days": stale_days,
"count": len(stale),
"share_of_open": round(len(stale) / open_count, 3) if open_count else None,
"sample": stale[:15],
},
"first_response_hours": {
"measured": len(response_hours),
"median": round(percentile(response_hours, 0.5), 1) if response_hours else None,
"p90": round(percentile(response_hours, 0.9), 1) if response_hours else None,
"never_answered_open": len(no_response),
},
"flow_window_days": window_days,
"flow": {
"created": created_in_window,
"closed": closed_in_window,
"net": created_in_window - closed_in_window,
},
"labels": {
"distinct": len(label_counts),
"top": sorted(label_counts.items(), key=lambda kv: -kv[1])[:20],
"used_once": sorted(name for name, count in label_counts.items() if count == 1),
},
}
def render(report):
totals, ages = report["totals"], report["open_age_days"]
lines = [
f"Records {totals['records']} open {totals['open']} closed {totals['closed']}",
f"Open age (days): median {ages['median']} p90 {ages['p90']} oldest {ages['max']}",
f"Untriaged: {report['untriaged']['count']} ({report['untriaged']['share_of_open']} of open)"
f" sample {report['untriaged']['sample']}",
f"Inactive >= {report['stale']['threshold_days']}d: {report['stale']['count']}"
f" ({report['stale']['share_of_open']} of open)",
f"First response (h): median {report['first_response_hours']['median']}"
f" p90 {report['first_response_hours']['p90']}"
f" measured on {report['first_response_hours']['measured']} items"
f" never answered {report['first_response_hours']['never_answered_open']}",
f"Last {report['flow_window_days']}d: +{report['flow']['created']} created"
f" -{report['flow']['closed']} closed net {report['flow']['net']:+d}",
f"Labels: {report['labels']['distinct']} distinct,"
f" {len(report['labels']['used_once'])} used exactly once",
]
for name, count in report["labels"]["top"]:
lines.append(f" {count:>5} {name}")
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("paths", nargs="+", help="issue export files (JSON array, JSONL or CSV)")
parser.add_argument("--stale-days", type=int, default=90, help="inactivity threshold (default 90)")
parser.add_argument("--window-days", type=int, default=90, help="inflow/outflow window (default 90)")
parser.add_argument(
"--triage-prefix",
default="",
help="comma-separated label prefixes that count as triaged, e.g. 'kind/,priority/'."
" Empty means any label counts.",
)
parser.add_argument("--format", choices=["text", "json"], default="text")
args = parser.parse_args()
records = []
for path in args.paths:
try:
records.extend(load_records(path))
except (OSError, json.JSONDecodeError) as error:
print(f"cannot read {path}: {error}", file=sys.stderr)
return 1
if not records:
print("no records found", file=sys.stderr)
return 1
prefixes = [p.strip() for p in args.triage_prefix.split(",") if p.strip()]
report = build_report(records, datetime.now(timezone.utc), args.stale_days, prefixes, args.window_days)
print(json.dumps(report, indent=2) if args.format == "json" else render(report))
return 0
if __name__ == "__main__":
sys.exit(main())
SKILL.md›
---
name: oss-issue-triage
description: Designs an issue and pull-request triage system a maintainer team can sustain - response targets sized against real capacity, the label taxonomy, intake cuts through structured forms and off-tracker routing, a separate security-report path, triage duty assignment, and the closing, staleness and volume-gating policy. Use whenever someone says "our issue tracker is out of control", "design an issue triage process", "set up labels for our repo", "we have 900 open issues", "should we run a stale bot", "PR backlog nobody reviews", "triage rotation", or "we are drowning in AI-generated reports" - even if they only say maintenance is overwhelming. Not good-first-issue curation - use samber/developer-relations-skills@oss-contributor-onboarding.
license: MIT
metadata:
author: Samuel Berthe
version: "1.0.1"
---
# OSS Issue Triage
You design the system that decides how a project's incoming issues and pull requests get handled - labels, intake, response promise, duty rotation, closing rules - sized to the capacity the maintainers actually have.
The queue is not out of control because the labels are wrong. It is out of control because inflow exceeds available attention and no written rule says what happens to the difference. Make that arithmetic explicit, then design around it.
Machine-generated submissions broke one side of it in 2025. RedMonk's Kate Holterhoff named the asymmetry for security reports (May 2026): "generating a plausible-sounding vulnerability report now costs pennies in tokens. Evaluating whether it's real still costs an hour of expert time." Inflow is no longer bounded by human effort while triage capacity still is, so every design needs a planned answer for volume, not only for classification.
Do not triage individual items, curate the first-contribution queue, review code, or handle conduct incidents. Route those elsewhere:
- First-contribution path, good first issues: samber/developer-relations-skills@oss-contributor-onboarding
- Repeat questions into docs: samber/developer-relations-skills@developer-troubleshooting-docs
- Hostile behaviour: samber/developer-relations-skills@developer-community-moderation
- Who decides scope: samber/developer-relations-skills@oss-governance
- The maintainer's own capacity, stepping back or handing over: samber/developer-relations-skills@oss-maintainer-sustainability
- Feature requests' product decision and the reply to requesters: samber/developer-relations-skills@developer-feedback-loop
- The wider measurement framework: samber/developer-relations-skills@developer-community-health
## Interview
Ask one at a time, multiple-choice where you can, skipping what the user already answered. Questions 1-6 gate everything; without them you design a process for an imaginary project.
**Memory: Store a few info about the current task or project context. When memory lives in a file, use `devrel-context.md`; if a different memory system is in use, rely on that instead. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.**
1. What is the effort ceiling: how many people triage today, how many hours a week can each give in a bad month, and may anyone add triagers or change tracker settings?
2. Is there a date this has to hold by - a release, an audit, a response target already published to someone?
3. Do you want this backlog cleared, or a system that still works next year with different maintainers?
4. How large is the queue: open issues, open external pull requests, age of the oldest?
5. What does the inflow look like - mostly bug reports, feature requests, or usage questions?
6. Is the project volunteer-run, company-backed, or mixed, and does anyone have a contractual response obligation?
7. Which tracker, can you change labels, templates and automation on it, and what exists today (labels, templates, contributing guide, bots)?
8. Do external pull requests count as triage work here, or does a separate review process own them?
9. Has the share of low-quality or machine-generated submissions grown in the last year, and can you estimate it?
10. Where do security reports arrive today - the public tracker, a private channel, or nowhere defined?
11. Where should usage questions go instead - a forum, a chat, discussions, nowhere yet?
12. Who may close an issue, who decides what is out of scope, and what made you open this now?
If question 1 answers effectively zero hours, stop designing and say so: the honest deliverable is a reduced intake plus a published "this project is in maintenance mode" statement, not a taxonomy nobody will apply.
Answers to 1, 2 and 3 re-rank every menu below; say which moved what.
- Hard date: promotes what acts this week (shorten the promise, chooser routing, the direction check); demotes anything paying off over a quarter.
- "Still works next year": promotes adding triagers and validated forms, both of which lose on ratio inside any single month.
- Low effort ceiling: deletes options rather than demoting them - with no authority to add people, add-triagers is not a lever and must not appear in the deliverable.
## Step 1 - Measure the queue before changing it
Export the tracker's issues and run the baseline; every later decision references these numbers.
```bash
gh issue list --state all --limit 2000 \
--json number,title,state,createdAt,updatedAt,closedAt,labels,author,comments > issues.json
python3 scripts/triage-baseline.py issues.json --triage-prefix "kind/,priority/"
python3 scripts/triage-baseline.py issues.json --stale-days 120 --format json > baseline.json
```
It reports, from JSON arrays, JSON Lines or CSV:
- open count
- age median/p90
- untriaged share
- inactive count
- median and p90 time to first human response
- created-versus-closed flow
- label usage, including single-use labels
Without a scriptable environment, sample 50 open items by hand and estimate the same numbers - an estimate beats designing blind, and without a baseline you cannot tell whether the redesign worked.
Read the numbers back before proposing anything: a queue that is 80% usage questions needs a different intervention from one that is 80% unanswered bug reports.
Then ask what the per-item framing hides: **do these items cluster?** On a large stale backlog, grouping issues by shared root cause under a tracking issue and redirecting the rest to it often clears more queue than triaging item by item. Decide per-item versus clustering here, before designing the per-item machinery.
## Step 2 - Do the capacity arithmetic
Multiply triagers × sessions per week × minutes per session ÷ minutes per item, and compare against the baseline's inflow.
Do not import a minutes-per-item constant. Time the pilot session in step 9 and use the project's own number; until then, three minutes is a planning placeholder this skill sets, not a measured industry figure. Deep investigation is not triage - it is work triage schedules.
When capacity is below inflow, no label scheme fixes it. Three levers close the gap; pick by queue relieved per maintainer hour, not by which is quickest to type:
- efficiency: cut intake > shorten the promise > add triagers
- effort: shorten the promise (rewrite one line, defend it once) > cut intake (a day of forms and chooser configuration) > add triagers (recruit, grant permissions, calibrate - a quarter before a new triager is net positive)
- value: cut intake (permanently lowers the left side of the arithmetic) > add triagers (the only lever that raises capacity at all) > shorten the promise (nothing improves; the promise merely stops being broken)
- compliance cost: cut intake == add triagers (tied because both are internal decisions with no external counterparty) > shorten the promise where question 6 found a contractual obligation, since there the published target is a commitment and shortening it is a contract change
Efficiency starves adding triagers - the only lever that raises capacity, and the slowest to pay. Promote it whenever the project expects to still exist in a year: one that only ever cuts intake shrinks itself to fit its worst month. State the chosen lever in the deliverable so the user is not surprised in three months.
Decide the **volume-gating trigger** now, while nothing is on fire: the inflow rate or valid-report share at which the project will restrict who may open items, cap open submissions per author, or change the incentives that attract volume. Projects that adopted gating reached for it only after burnout was acute; deciding the trigger at design time makes it policy instead of crisis improvisation. [./references/worked-examples.md](./references/worked-examples.md) walks curl's collapse to a one-in-twenty valid share through this arithmetic, and shows why its answer acted on the economics of submitting rather than the speed of triage.
Set the first-response target from the capacity number, not from ambition - a human reply, not a fix and not a bot greeting. The arithmetic, volunteer-versus-company-backed differences and contractual obligations: [./references/policy-and-metrics.md](./references/policy-and-metrics.md).
## Step 3 - Design the label taxonomy
Build labels as orthogonal axes - type, state, priority, area, contributor-facing - with at most one value per axis on any item. State moves the queue; priority starts arguments, so keep it as small as the project can bear.
Size the set against the reference points in [./references/published-findings.md](./references/published-findings.md):
- ten default labels on a new repository
- five state roles plus two categories in the most-installed triage skill
- six prefixed families at Kubernetes scale
Mapping those onto a project is this skill's judgment, not a sourced rule: the honest answer usually sits between the first two and approaches the third only when ownership is genuinely split.
Two rules come from PyTorch's taxonomy - the only documented running in production, on PyTorch's own main repository:
- **Gate labelling on confidence.** Apply a label at high or medium confidence; at low confidence apply a "needs review" state instead of guessing. A guessed area label sends the item to the wrong owner and costs more than no label.
- **Never let automation apply the top priority level.** Route every candidate for it to human confirmation instead. Priority is the axis with the most downstream consequence and the least machine-checkable definition.
Axes, sized starter sets, naming rules, cleaning up an existing label mess without erasing history, the allowlist-and-validation pattern, and tracker mechanics: [./references/label-taxonomy.md](./references/label-taxonomy.md).
Present the taxonomy for approval before touching the tracker - one imposed on maintainers who did not agree to it decays within a month.
## Step 4 - Cut the intake
Capacity spent on items that should never have arrived is the largest reclaimable block. Rank the four cuts by triage minutes reclaimed per hour of setup - no compliance-cost axis applies, since none of the four creates an obligation to anyone:
- efficiency: chooser routing > validated forms > disabling blank issues > auto-applied type label
- effort: disabling blank issues == auto-applied type label (one setting, one line of form YAML) > chooser routing (needs a destination that may not exist yet) > validated forms (one form per real category, written, tested and revised when reporters skip fields)
- value: chooser routing (removes whole categories instead of speeding them up) > validated forms (turns two round-trips into one submission on every bug report, forever) > disabling blank issues (closes the bypass that makes forms optional) > auto-applied type label (one axis, seconds an item)
1. **Chooser routing** - usage questions to a forum or chat, commercial requests to a sales contact. A stopping condition: a routed report never reaches the tracker, so steps 5-8 never touch it.
2. **Validated forms** replacing markdown templates. Require exactly the fields that change the outcome; every optional field a reporter skips teaches them the form is decorative.
3. **Disable blank issues** once the forms cover the real categories.
4. **Auto-applied type label** from the form, plus the issue type where the tracker has one.
Efficiency starves option 2: highest recurring value, most setup, so a maintainer with one free evening does the toggles instead and the round-trips continue. Promote it to first when bug reports dominate inflow, where that cost is paid on every item. A project with nowhere to route usage questions (question 12) has no option 1 - delete it from this pass rather than listing it as pending work.
**Security reports leave the tracker entirely.** Every mature project examined runs the same three-element pipeline, separate from general triage:
- a private intake channel
- a named, size-capped response team
- an embargo mechanism for coordinated disclosure
Apache instructs committers never to enter vulnerability details in a public tracker. With no security list, the code host's private vulnerability reporting is the default. Running the pipeline itself - SECURITY.md, embargo, advisory, CVE, patched release - belongs to samber/developer-relations-skills@oss-security-disclosure.
Form capabilities, chooser configuration, the full security path and the volume-gating options: [./references/issue-intake-and-routing.md](./references/issue-intake-and-routing.md) - read it before writing form YAML.
## Step 5 - Assign the duty
An unowned process is not a process.
1. Name who triages, in a rotation with a stated shift length (a week is the common unit) and a named backup. Mozilla's two-tier model is the most robust documented pattern: a long-term **triage owner** accountable per component, plus a **triage duty** rotating weekly that does the work.
2. Fix a cadence and a stop rule: a session ends at its time box, not at inbox zero, so triage survives a spike.
3. Set the session order: newest untriaged first (protects the response target), then items where the reporter replied, then a fixed backlog quota.
4. Grant triage-level permissions rather than commit access - most trackers have a role for labelling, closing and assigning without write access, which is how a triage team grows past the committers.
5. Define the escalation: what a triager decides alone, what waits for a maintainer, how long that wait may be.
Three things decide whether a recurring triage meeting survives:
- a queue view built for it
- asynchronous pre-triage, so the meeting is decision-making rather than first contact
- decisions recorded as label or state changes rather than only in minutes
Rust's compiler triage meeting is the clearest example: a Zulip stream pre-triages and priority is set by replacing a label, so the output stays queryable.
## Step 6 - Give pull requests their own gate order
A pull request carries hard automated checks an issue never has; do not reuse the issue process.
Two gates are not menu items, since both are automated preconditions that fire before a human looks:
- sign-off (CLA or DCO)
- CI permission for externally authored PRs
Nothing merges without sign-off, because the licence provenance it records is the project's only account of where the code came from.
Rank the three gates that spend maintainer attention by rejections settled per minute of reviewer time - not cheapest first, which buries the direction question under the machinery:
- efficiency: direction check ("does this belong in the project") > size-based risk label > code-correctness review
- effort: direction check (a minute on title, description and linked issue) > size-based risk label (computed from the diff, but it only sorts) > code-correctness review (the expensive pass this order exists to protect)
- value: code-correctness review (the only gate that can accept a change) > direction check (settles the rejections that were never about code) > size-based risk label (routes; decides nothing)
- compliance cost: direction check == size label (tied: a maintainer reading a description and a bot measuring a diff both leave the project owing nothing) > code-correctness review (accepting a change makes the project answerable for its provenance and security posture, which is what sign-off documents)
The direction check leads because roughly two-thirds of rejected agentic PRs never needed the code read at all - the MSR 2026 mining study's breakdown of 353 manually inspected rejections is in [./references/published-findings.md](./references/published-findings.md).
Efficiency starves the code-correctness review: highest value, highest effort, permanently last. Promote it to first when a trusted repeat contributor sends a small change against an already-accepted issue - the direction question is answered before the PR opens, and running the order costs more than the read.
The per-item decision path, reply templates with weak/strong pairs, the PR deltas and the rejection knowledge base are in [./references/response-playbook.md](./references/response-playbook.md).
## Step 7 - Write the closing and staleness policy
Decide explicitly what closes automatically and what never does. The defensible line: auto-close only items waiting on a reporter who stopped replying; never one waiting on the maintainers - a confirmed bug closed by a bot tells the reporter their evidence was worthless.
Configure the sweep:
- restrict it to the waiting-on-reporter state only
- exempt the confirmed and blocked states
- warn before closing
- run the first pass in dry-run mode, so the blast radius is known before it fires
Every close carries a reason and says how to reopen.
Anchor each clock to a named waiting party rather than to the item in the abstract. Kubernetes' published clocks, the sweep configuration, and when a one-time backlog bankruptcy is legitimate: [./references/policy-and-metrics.md](./references/policy-and-metrics.md).
## Step 8 - Automate classification, never the first reply
Automate the mechanical layer first, deterministically:
- fill the area label from changed paths
- apply the type from the form
- request review from the owning team
- run the staleness clock
Every documented machine-learning triage deployment keeps a human as final decision-maker: treat model assistance as a scoring and suggestion layer, never an autonomous accept, reject or close.
Copy two patterns before considering any classifier:
- **Automate who is on duty, not what to decide.** Mozilla syncs the triage-owner field to whoever is on the rotation calendar, so requests reach the person actually working rather than piling on one name.
- **Hold the label set as a validated artifact**, so automation cannot invent labels or overwrite human work - mechanics in `label-taxonomy.md`.
Leave the first human sentence to a human. Response metrics exclude bot replies by definition and reporters discount them the same way; an auto-greeting followed by three weeks of silence makes the wait worse. When an agent posts triage comments, disclose it in the comment.
## Step 9 - Publish, run a pilot pass, measure
Publish the policy:
- write it into the repository, next to the contributing guide
- link it from the intake forms
- announce the change once, in the project's own channel
Run one real triage session with the user against the new system on 15-20 items, and time it. That produces the minutes-per-item figure step 2 deferred and exposes what the design got wrong - a missing state, an ambiguous label, a reply template that does not fit. A policy never executed once is a draft.
Re-run `triage-baseline.py` monthly for the first quarter, and iterate against three starting targets. All three are this skill's self-set baselines, not industry benchmarks - replace each with the project's own measured trend once a quarter of data exists:
- Untriaged share below 10% of open items.
- Published first-response target met on at least 90% of items.
- Net inflow at or below zero over the quarter.
Iterate until they hold, or lower the published promise to one that does.
Never read a count metric alone. CHAOSS's guidance restricts these measures to activity "primarily driven by humans", and, under machine-generated volume, closure ratios and time-to-close can improve while real maintainer burden rises. Pair every count with a human-driven signal: first response with bots and author self-replies excluded, and how many people could cover triage if one person left.
With persistent memory available, store the durable decisions:
- capacity numbers
- the agreed response target
- the label axes and their definitions
- categories routed off the tracker
- the volume-gating trigger
- declared out-of-scope requests
The next review is a diff against those.
## Invocation examples
| The user says | You produce |
| ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| "We have 900 open issues and no idea where to start" | Run the baseline, read it back in plain language, then make the per-item-versus-clustering call - no taxonomy yet |
| "Design a triage process for our maintainer team" | All five deliverable artefacts, in order, validated one by one |
| "Our labels are a mess, redo them" | Label usage counts first, then a change list of create/rename/merge/delete with one definition and one axis per label |
| "Should we turn on a stale bot?" | The waiting-on-reporter-only rule, a dry-run first pass, and the exempt-state list - plus a refusal to sweep maintainer-owned items |
| "Nobody answers PRs from outside contributors" | The gate order from step 6 and a separate PR response target - not the issue process reused |
| "We are drowning in AI-generated reports" | The volume-gating trigger from step 2, intake economics, and the platform controls - before any classifier |
## Deliverable
Produce these five artefacts in order, each validated with the user before the next - never all in one silent pass:
1. **Baseline** - step 1's numbers, the one-line reading of them, and the per-item-versus-clustering call.
2. **Triage policy** - the published document: what belongs in the tracker, the response target, the states and their meanings, when items close, who triages, the volume-gating trigger. Template in [./references/policy-and-metrics.md](./references/policy-and-metrics.md).
3. **Label change list** - create, rename, merge, delete, each with a one-sentence definition and its axis.
4. **Intake change list** - forms to add or rewrite with required fields, chooser routing entries, the security path, automation to enable, each with the triage minutes it should save.
5. **Duty and gate design** - the rotation with named people and shift length, the escalation rule, the pull-request gate order.
Keep every artefact in the project's own vocabulary - real label strings, real channels, real maintainer names.
## Failure modes
Each row names a failure the steps do not already spell out; the fix is the whole row.
| Failure | Fix |
| ------------------------------------------------------------- | -------------------------------------------------------------------- |
| Aspirational target - "24-hour response" from two volunteers | Publish the number the baseline supports |
| Dormant rotation - a documented rota nobody has run in a year | Give it an owner and a liveness check; retire it loudly if it lapses |
| Priority theatre - everything is `priority/high` | Two or three levels, defined by consequence |
| Big-bang migration - new labels, old forms, old saved views | Migrate labels, forms, views and automation in one pass |
| Triage as fixing - a session spent debugging one item | Time-box the pass; triage schedules work; it is not the work |
| Unmeasured redesign - new process, no idea whether it helped | Re-run the baseline monthly |
## References
- [./references/label-taxonomy.md](./references/label-taxonomy.md) - axes, starter sets, cleanup procedure, tracker mechanics.
- [./references/issue-intake-and-routing.md](./references/issue-intake-and-routing.md) - form fields, chooser, off-tracker routing, security path.
- [./references/response-playbook.md](./references/response-playbook.md) - per-item decision path, reply templates, rejection knowledge base.
- [./references/policy-and-metrics.md](./references/policy-and-metrics.md) - capacity arithmetic, policy template, closing rules, measurement plan.
- [./references/worked-examples.md](./references/worked-examples.md) - a worked capacity read and a positive/negative rotation pair.
- [./references/published-findings.md](./references/published-findings.md) - which figures are sourced and which are self-set baselines.