Frame

Focus

Form

Forward

Frame

Focus

Form

Forward

A limit that couldn't tell a trusted stranger from a spam account

GitHub had already shipped a blunt fix for AI-driven PR noise: a cap on how many open pull requests a non-collaborator could have, plus a manual way for maintainers to search for and exempt specific users. It worked, where maintainers put in one name at a time. Yet, feedback kept pointing that: the judging if a user is trustworthy based on write access was the wrong signal.


That distinction broke immediately at scale. For example, Rust has contributors who aren't org members but have 10 to 30 legit PRs open at once. They are long-time-trusted contributors. However, the cap couldn't tell them apart from a first-time account flooding the repo with AI-generated noise. Charlie Marsh at OpenAI and Ivorforce at Godot said versions of the same thing on the public issue. Four maintainers, four major projects, and the same objection- independently.


I set out to design a rule builder maintainers could configure once, correctly, with the logic available exactly when they needed it, and validate it with real maintainers before it shipped. My job was the fix: let maintainers define trust themselves, instead of GitHub defining it for them with a single access flag. I owned usability testing on PlaybookUX, AI prototyping with Copilot, and cross-functional green lights on the engineering and accessibility after.

GitHub had already shipped a blunt fix for AI-driven PR noise: a cap on how many open pull requests a non-collaborator could have, plus a manual way for maintainers to search for and exempt specific users. It worked, where maintainers put in one name at a time. Yet, feedback kept pointing that: the judging if a user is trustworthy based on write access was the wrong signal.


That distinction broke immediately at scale. For example, Rust has contributors who aren't org members but have 10 to 30 legit PRs open at once. They are long-time-trusted contributors. However, the cap couldn't tell them apart from a first-time account flooding the repo with AI-generated noise. Charlie Marsh at OpenAI and Ivorforce at Godot said versions of the same thing on the public issue. Four maintainers, four major projects, and the same objection- independently.


We set out to design a rule builder maintainers could configure once, correctly, with the logic available exactly when they needed it, and validate it with real maintainers before it shipped. My job was the fix: let maintainers define trust themselves, instead of GitHub defining it for them with a single access flag. I owned usability testing on PlaybookUX, AI prototyping with Copilot, and green lights on the engineering and accessibility after.

Things to keep in mind

Reasoning

As open-source projects scale, so does the noise, more contributors, more AI-generated PRs, more bots. While maintainers manually exempt trusted users, this doesn't scale once every new PR gets capped by default.

Double-edged sword: apply the wrong exemption and you either give access to unwanted users, or throttle the people keeping your project alive.

Goal

Maintainers can configure exemptions once and walk away, not babysit the list.


When two maintainers want opposite behavior from the same setting, GitHub doesn't pick a side.

Dynamics

Temporary interactions exists and overlaps in terms of logic. Nobody ever defined whether one overrides the other.

Never conducted user research at GitHub.


The logic had to work without anyone reading a manual. My research question was built around that constraint directly.

Context

Every open source project live on GitHub is extremely different. The size and shape of project, GitHub Actions, and its purpose.

Maintainers had very different ideas of what trust signals works for them.


Heavy system interdependencies across interaction tools (eg: temporary limits, block)

Formation

Choose which trust signals were worth building

'Auto-exempt trusted contributors' sounded clear, however it wasn't buildable until someone decided what "trusted" actually meant. Six signals were on the table: merged PRs, account age, org membership, issue assignment, a cap-of-zero option, and team or org-level bypasses. I picked three, based on what maintainers had already told us:

On merged PRs

"someone who has already landed PRs has demonstrated good faith and shouldn't be throttled " - P4

On account age

"spam and automated AI PR floods frequently come from newly created throwaway accounts " - P3

On organization membership

"…we have relationships with other organizations, so allowing them based on it would be easier" - P1

The goal underneath both was the same: make trust automatic and configurable so maintainers stop hand-maintaining allowlists. That set the build order: merged PRs and account age first, org member.

Form 03

Visual exploration to answer: is this the right granularity offered?

I needed one pattern to balance criterion complexity and simplicity, so I built four clickable prototypes with GitHub's internal AI prototyping tool: a toggle, a dropdown, a checkbox, and a toggle that kept its inputs visible but disabled until switched on. Three trust signals were already on the table, and from the roadmap there were going to be more. In envisioning scalability grounds, keeping every criterion permanently visible crowds the page as the ruleset grows. This is what got me to the final interaction model, with one caveat: AND/OR logic.

UI exploration with Copilot: Checkboxes, Toggles, and visibility

Form 04

Writing down what could prove me wrong

Before testing, I bet on two things. One: maintainers can build rules without understanding AND/OR, as long as the summary banner reflects their intent. Two: they'll disagree about how exemptions behave during temporary limits. My disproof bar was that if most people got stuck adding a second rule set, the OR model was wrong.


I was half right. Maintainers built the rules and found the OR, so the model held. But the banner didn't carry it. The inline confirmation and visual cues did. Temporary limits split exactly as predicted: one maintainer saw an emergency brake, another wanted "just a knob." The banner stepped back, and AND/OR moved into the layout.

Form 05

Moderated usability testing with maintainers to verify AND/OR logic, complexity, and temporary limits.

Four maintainers, five tasks, one question: can maintainers control which trusted contributors skip the PR limit, without having to understand AND/OR logic? I recruited directly in the maintainer Slack channel and GitHub maintainer community. Participants built rules live in the prototype, then answered scenario questions about temporary limits, which we couldn't prototype yet.

Do maintainers actually need this much control?

Yes. Nobody asked for a simpler system. The real blocker was outside my design: telling first-time contributors apart from returning.

Changed:

AND/OR stayed. First-time vs. returning contributors became the #1 adoption blocker and the first follow-up study.

Can they build rules without help?

Eventually, but through the help text, not the layout. Then the OR indicator appeared: "Chef's kiss on that one." (P2)

Changed:

Renamed to bypass rules.

AND lines inside each rule set.

An always-visible OR, including between the rules and the bypass list.

What do they expect from temporary limits?

Conflicted answers. Most don't use it now.

"An emergency blocker" (P2) vs. "It just needs to be a knob" (P3)

Changed:

An explicit override setting (defaulting to override), clear on/off states, and a recommendation to fold temporary limits into PR limits.

Do they trust what the rule will do?

They trusted the inline confirmation line. The most important sentence was buried.

Changed:

The override warning moved to the top, and the banner was demoted.

Copy fixes: "At least," day-based durations, correct plurals.

A changelog went on the roadmap.

Impact

Impact

Ten findings, ten changes in the redesign. Every pre-set decision got an answer. Six weeks later, org-level limits shipped with the same hidden-override bug.

Form 06

Unshipping temporary interaction limits

Temporary interaction limits had shared a page with PR limits for years, and nobody had defined how they relate. I turned that unanswered question into part of the study and got three incompatible answers. A maintainer repurposed temporary limits to something permanent and another expected to be an emergency override.

They weren't even the same shape. Temporary limits block comments, issues and PRs, while the cap touches only PRs. Every participant had set a manual calendar reminder for when a limit expired. I recommended an explicit on/off setting for how they interact, and ranked three options for deprioritizing or consolidating temporary limits. Unshipping is as real a decision as shipping.

Results

Maintainers no longer have to hand-pick trusted contributors one name at a time. With Bypass rules, they define trust themselves. They build AND/OR logic from signals they already rely on: merged PRs, account age and org membership. The relationship between rules is visible by default. All 10 research findings made it into the redesign spec, which ships in October 2026.

A tested prototype ready for engineering. Engineering and accessibility issues were flagged before implementation, not after launch.


A roadmap from the research. Telling first-time contributors apart from returning ones became the #1 adoption blocker and the topic of the first follow-up study. I also recommended folding temporary limits into PR limits instead of keeping two overlapping features.

"Everyone I speak to in the Rust project is amazed... I am extremely glad your team exists."

— P2 (Rust maintainer)

"This will let us implement interaction limits and ease a bit the work of our moderators, so that would be amazing."

— P3 (Apache maintainer)

A reusable pattern for embedding a modern data table inside an older Rails page, beyond this

Next

Now that maintainers can define trust through rules instead of a single access flag, the next constraint is cross-repository trust. Maintainers repeatedly said, unprompted, that knowing a contributor was already trusted, or already blocked, somewhere else on GitHub would be the strongest signal available. That comes with its own privacy and abuse risks, and it wasn't scoped into this project.

Takeaways

The blunt version of a feature always ships first, because it's the fastest thing to build. The real work is watching where it breaks in the hands of the people who actually use it, and having the discipline to let that evidence decide what you build next, even when the answer is less, not more.

Collaborators

Stephanie Hong - SWE for Rails and React constraints

Catharine McNally - Staff program manager

Camilla Moraes- Product manager

Next project

My UN App

Lead product designer

United Nations

My UN App

Lead product designer

United Nations

Contact

About

Projects

Play

Resume

LinkedIn

devyanijain4@gmail.com

Contact

About

Projects

Play

Resume

LinkedIn

devyanijain4@gmail.com