samber/developer-relations-skills已通过检查
SKILL DETAIL
devrel-analytics
samber/developer-relations-skills/devrel-analytics
Builds the tracking plan that instruments developer-relations surfaces - docs, blog, repositories, package registries, community venues, and off-web appearances - with an event taxonomy, an identity spine, link-tagging discipline, source-confidence labelling, and funnel views. Use when the user says "devrel tracking plan", "docs analytics", "utm discipline", "event taxonomy for our docs", "our GitHub numbers don't match analytics", "how do we instrument the developer funnel", "join docs traffic to signups", or wants to know why devrel numbers disagree between tools. Instrumentation layer only - which KPIs deserve a target belongs to samber/developer-relations-skills@devrel-metrics.
安装量 · 408查看来源
Installation
npx skills add https://github.com/samber/developer-relations-skills --skill devrel-analytics
技能文件
SKILL.md
最近同步 · 2026年10月23日
evals/evals.json›
{
"skill_name": "devrel-analytics",
"evals": [
{
"id": 1,
"prompt": "I run developer marketing at Velmont. We ship a Python SDK and a hosted API. Google Analytics says we had 4,200 docs sessions last month. PyPI says 61,000 downloads. Our CDN logs show about 19,000 unique IPs hitting /docs. My VP wants one headline 'developer reach' number for the board deck next week, and honestly the three numbers being this far apart makes us look like we don't know our own product. Can you reconcile these into something I can present?",
"expected_output": "A refusal to blend the three sources into one reach figure, an explanation of client-side blocking on a developer audience with the published measurement attached, each number labelled with what it actually counts, and a recommendation to move docs collection first-party or proxied.",
"files": [],
"expectations": [
"States that a third-party client-side analytics script is blocked by roughly 58% of a tech audience, and attaches that figure to a published measurement rather than presenting it as its own estimate.",
"Names at least one per-segment blocking rate (68.2% desktop, 88.3% Firefox, or 82.3% Linux) when explaining that the miss rate is not uniform.",
"Refuses to average or blend the client-side session count with the server-side CDN log count into a single figure.",
"States that the blocking bias varies by page, so it cannot be corrected by applying one multiplier.",
"Declines to produce a single headline 'developer reach' number spanning the three sources.",
"Labels each of the three numbers with the source it came from and what that source actually counts.",
"Recommends first-party or proxied collection on the docs site as the fix that recovers the missing share, rather than switching to a different third-party tool.",
"Advises trusting ratios inside one source, such as step-to-step drop-off, over absolute counts compared across sources.",
"States that the PyPI figure is a request count and cannot be converted into a count of people or users.",
"Reconciles the gap between the sources explicitly instead of splitting the difference.",
"Does not offer bot traffic or a tracking misconfiguration as the leading explanation for the gap between analytics and the CDN logs."
]
},
{
"id": 2,
"prompt": "Kestrelbase here. Our open-source repo has been going since March and we're putting together a nine-month retrospective for our investors. I want a chart of unique visitors and clones per week since launch, plus top referrers over that whole period. Nobody has exported anything before, we just look at the Insights tab when we remember. Can you tell me how to pull that history, and what else I should be grabbing from the repo while I'm in there?",
"expected_output": "A statement that the nine months of repository traffic history is unrecoverable, a daily snapshot job with owner and schedule, a dated baseline captured today, and the bias warnings on clones, stars and referrers.",
"files": [],
"expectations": [
"States that the forge's traffic endpoints retain only 14 days of data.",
"States plainly that the nine months of history requested cannot be recovered and that backfill is impossible.",
"Instructs scheduling a daily export appended into the team's own store, and explains that a monthly or weekly export loses most of the window.",
"Notes that traffic endpoint data is readable only for repositories where the caller holds write access.",
"Warns that clone counts include CI systems and mirrors and are not people.",
"Warns that stars only ever increase, and recommends converting to stars-per-period if reported at all, never as a headline.",
"Notes that referrers into a repository are largely stripped, the same way web referrers are.",
"Names issue or pull-request API signals worth collecting alongside traffic, such as first-time contributor count, time to first response, or external pull requests merged.",
"Records today's values as a dated baseline before any further instrumentation work.",
"Does not propose reconstructing the missing traffic history from git history, a third-party mirror, or an archive as an equivalent substitute.",
"Frames the deliverable as a collection recipe plus a snapshot job with a named owner and schedule, rather than as a traffic analysis."
]
},
{
"id": 3,
"prompt": "Orbitlane. We publish the same client library on npm and on PyPI. Last month npm reported 412,000 downloads and PyPI reported 88,000. We're raising a seed round and the deck has a slide that says '500K developers using Orbitlane'. The investor's associate asked where the number comes from and I need to answer them today. Also our npm number dropped 30% last week and I want to know whether we should be worried.",
"expected_output": "A refusal of the summed developer count, the two registries' counting definitions side by side, a per-registry series against its own baseline, and a first-party alternative for the adoption claim.",
"files": [],
"expectations": [
"Refuses to present the summed 500,000 figure as a count of developers or users.",
"States that npm counts HTTP 200 responses serving tarball files, so build servers, mirrors and analysis bots are counted deliberately.",
"States that PyPI excludes known mirrors by default and exposes with-mirrors and without-mirrors as separate series.",
"States explicitly that the two registries measure different things and that their figures must never be summed.",
"Recommends reporting each registry's trend against its own baseline, one series per registry with its counting definition written beside it.",
"Notes npm's retention limits when planning collection: 18 months of history, 365 days for bulk queries, 7 days per version.",
"Notes PyPI's 180-day time-series retention and recommends snapshotting into the team's own store.",
"Mentions the guidance that below roughly 50 downloads per day the npm signal is mostly automation.",
"Considers CI, mirror or automation behaviour as candidate causes of the 30% drop before drawing any conclusion about adoption.",
"Does not offer a conversion ratio or divisor for turning downloads into a developer headcount.",
"Proposes a defensible alternative number grounded in a first-party signal instead of accepting a registry figure as the adoption claim."
]
},
{
"id": 4,
"prompt": "Thornwick AI publishes six models and three datasets on a public model hub. Our board slide currently says '1.2M downloads', which is our model downloads plus our dataset downloads added together. One model has 40,000 downloads and 12 likes; another has 900 downloads and 310 likes, and we've been telling people the second one is our breakout. We also want to start tracking our trending rank weekly, and I'd like to know how many distinct people have pulled each model. We poll the hub API from a cron job without an API token.",
"expected_output": "A rejection of the combined download figure on counting-definition grounds, a correction of the likes-as-adoption reading, trending rejected as a tracked metric, the unique-downloader question declared unanswerable on the free tier, and the untokenized polling job flagged.",
"files": [],
"expectations": [
"States that a model download on that hub counts every HTTP GET or HEAD against a per-library query file such as config.json, so repeated automated access counts every time.",
"States that a dataset download collapses every file one IP fetches from one repository inside a 5-minute window into a single download.",
"Concludes that model and dataset counts are not summable, and rejects the combined 1.2M figure.",
"States that unique-downloader figures are not available by default and require a paid enterprise add-on, so 'how many people' is unanswerable from the public surface.",
"Corrects the likes-versus-downloads reading: likes are a recency signal concentrating on new releases while downloads concentrate on older dependencies, and neither proxies the other.",
"Instructs keeping likes and downloads as two separate series, each labelled with what it means.",
"Rejects trending rank as a tracked metric because the score has no published formula and no stability guarantee, allowing it only as a snapshot observation.",
"Flags that the untokenized polling job falls under a lower anonymous rate limit enforced over fixed 5-minute windows, and instructs sending a token.",
"Warns that a collection job silently returning rate-limit errors produces a flat line that reads as lost adoption.",
"Instructs snapshotting totals daily into the team's own store and reporting deltas, because the public API returns current totals rather than history.",
"Does not treat this hub's download counts as comparable to a package registry's download counts."
]
},
{
"id": 5,
"prompt": "Marrowfield Data. Our CEO asked me to put a number in the OKRs: docs visitor to signup conversion rate. I've been reading around and the figure people keep citing for developer tools is 20-40% activation, so I was going to set 25% as our Q3 target and back into how much docs traffic we need to hit it. We've had analytics on the docs for five weeks. Can you help me build the tracking plan that proves we hit it?",
"expected_output": "A refusal of the borrowed target, the activation band named and rejected specifically, target-setting routed to the metrics sibling skill, and a baseline-capture plan built from a decision list offered instead.",
"files": [],
"expectations": [
"Declines to supply or endorse a docs-conversion or activation target borrowed from other companies.",
"Identifies the 20-40% activation band as resting on repetition rather than on a measured cohort, and refuses it by name.",
"States that a healthy conversion rate is a property of one program's own audience and surface mix, so any target must come from the program's own trailing baseline.",
"Routes the target-setting question to samber/developer-relations-skills@devrel-metrics rather than answering it here.",
"Offers a baseline-capture plan that would produce a real target, instead of refusing without an alternative.",
"Starts the plan from a decision list, naming the decisions the data should change and the question that settles each one.",
"States that an event tracing to no decision on that list is deleted rather than deferred.",
"Warns that five weeks of client-side docs data on a developer audience is a biased and partial denominator for any conversion rate.",
"Does not invent a substitute benchmark figure of its own and present it as typical for the category.",
"Labels any threshold it does propose explicitly as a self-set starting point the user may move.",
"Declines to frame the deliverable as a plan that proves a pre-chosen number was hit."
]
},
{
"id": 6,
"prompt": "Delacroix Systems. Quarterly review is Thursday. Our join coverage is at 74% and I want to write 'below the industry standard of 80%' in the deck so the web team takes the remediation seriously. It was 89% in June. Our head of data will read this. Is that wording fine, and what should I put next to it?",
"expected_output": "The phrase refused, the 80% identified as a self-set floor, corrected wording supplied, and the writeup redirected onto which surface stopped resolving and when, with an owner and a re-trace.",
"files": [],
"expectations": [
"Refuses the phrase 'industry standard' applied to the 80% figure.",
"States that 80% is a floor the plan set for itself, not a published or measured benchmark, and that no such benchmark exists to cite.",
"Warns that a reader who catches a self-set number dressed as a benchmark discounts the rest of the plan.",
"Supplies corrected wording that presents the floor as one the team set in its own plan.",
"Redirects the writeup toward which surface stopped resolving and when, rather than toward the distance from a threshold.",
"Uses the drop from 89% in June as the comparison that matters, measuring the program against its own past.",
"Names the identity spine, the anonymous-ID cookie or cross-subdomain sharing as the mechanism to investigate for a coverage drop.",
"Recommends replacing the 80% floor with the program's own trailing rate once enough of its own data exists.",
"Attaches a named owner and a re-trace action to the finding.",
"Explains that join coverage is the preferred standing health check because it is more diagnostic than reconciling absolute counts between two tools.",
"Does not cite any external company's or vendor's join-coverage figure as a comparison point."
]
},
{
"id": 7,
"prompt": "Palewood. We inherited a tracking plan with about 140 events, all named in Proper Case like 'Docs Page Viewed' and 'Python Quickstart Step 3 Done'. I'm cleaning it up this sprint: converting everything to snake_case, renaming 'Python Quickstart Step 3 Done' to 'quickstart_step_3_completed', and splitting 'Docs Page Viewed' into 'reference_page_viewed', 'guide_page_viewed' and 'blog_page_viewed' so we can query them separately. Sanity-check my plan?",
"expected_output": "The casing conversion advised against on consistency grounds, the per-section split and the baked-in step number rejected in favour of properties, the history-splitting cost of renames stated, and a property registry required.",
"files": [],
"expectations": [
"Advises against converting the existing plan's casing, and instructs matching the convention already in place.",
"States that consistency is the sourced rule and that no particular casing is mandated by the naming framework.",
"Names the Object-Action naming framework as the convention the plan should follow: object then action, past tense.",
"Warns that renaming a shipped event splits its history in two.",
"Offers the alternative to renaming: add a property, or version the plan and note the break.",
"Rejects splitting the page-view event into three per-section events, because that pushes context into the event name.",
"Instructs carrying the section as a property on one event instead.",
"Rejects the step number baked into the event name and instructs carrying step_number as a property.",
"Notes that 140 events is past the readability limit, and that the list should be small enough to fit on one screen.",
"Requires a property registry giving a type and an allowed-value list per property.",
"Bans PII and PII-shaped identifiers from every property."
]
},
{
"id": 8,
"prompt": "Hollybrook. Two problems. First, nearly every session in our reports shows the source as our own docs domain. I assumed that was good news until someone pointed out it can't be. We do tag internal docs links with utm_source=docs so we can see which docs pages send people to the pricing page. Second, launch week was 78% 'direct' and our CMO thinks the tracking is broken. We use a link shortener for conference links and everything goes through an OAuth login before signup. Tell me what to fix.",
"expected_output": "Internal tagging named as the cause of the self-referral and stripped, the direct bucket explained as normal with its referrer-stripping causes, the query string checked through shortener and OAuth hops, and a closed tagging vocabulary plus a link register set up.",
"files": [],
"expectations": [
"Identifies tagging internal links as the cause of the self-referral, and instructs stripping the tags from internal links.",
"Explains that an internal tag restarts the session and overwrites the real inbound source with the team's own.",
"Instructs excluding the team's own domains from source classification.",
"States that a large direct bucket during launch week is the normal case rather than an instrumentation defect.",
"Names referrer-stripping causes for the direct bucket, such as aggregators, chat clients, privacy browsers, PDFs or slide decks.",
"Cites that roughly 70% of AI-assistant-driven traffic arrives with no referrer at all.",
"Warns that shorteners, redirect chains and OAuth hops kill the tag unless they preserve the query string, and instructs verifying that they do.",
"Answers the internal docs-page question with a first-party mechanism such as an event property or click property, rather than with campaign tags.",
"Requires a closed allowed-list vocabulary for source and medium, fixed before the next tagged link is published.",
"Requires every published tagged link to be registered in one table with owner and publish date.",
"Recommends tagging per artifact rather than per campaign when the point is comparing artifacts."
]
},
{
"id": 9,
"prompt": "Nordvale. We have eight months of signups where the 'channel' field on the record is a mess: about 40% blank, the rest filled in by whoever noticed at the time. I want to run a backfill job so that for every signup we look at which campaign was live that week and write that campaign's channel into the field, making the whole history consistent so the board chart has no gaps. We ran three to five campaigns at a time most months and they mostly landed on /get-started. Then I want to chart the last eight months as a trend.",
"expected_output": "The overwrite refused, the campaign-window heuristic's three pinning conditions stated and found unmet, three source fields required instead of one, backfilled rows kept visually separate, and the eight-month trend chart declined.",
"files": [],
"expectations": [
"Restricts backfill to records whose source is blank, and forbids overwriting any record already carrying a journey-linked or self-reported value.",
"Rejects the proposed job specifically because it would overwrite populated rows.",
"States the three conditions the campaign-window heuristic requires: the date falls inside exactly one campaign's window, the landing page belongs to that campaign, and that campaign's tags were live.",
"Concludes that three to five concurrent campaigns sharing the /get-started landing page break the pin, so those records must be written as unknown.",
"Instructs recording three fields rather than a bare channel: source, source_basis and source_confidence.",
"Defines the source_basis values as journey_linked, self_reported and campaign_window.",
"States that confidence is judged on the quality of the evidence and is never read off the basis.",
"States the fail-closed rule: when the merge is ambiguous, record nothing, because a gap is visible and fixable while a wrong merge is corruption nobody later detects.",
"Requires backfilled history to be kept visually separate so it is not read as a trend.",
"Declines to present the eight-month backfilled series as a trend chart.",
"Does not propose a probabilistic or fractional allocation of ambiguous conversions across the concurrent campaigns."
]
},
{
"id": 10,
"prompt": "Cinderbank, series A devtool. I'm the only DevRel person. Board meeting in three weeks and I have to show something about whether our content is working. Engineering will not give me a sprint; best case I get one web-team ticket done in a month, and there's no data team. Right now I have nothing: no analytics on the docs, no tagging, nothing. I own the docs site, the blog, two repos, a Discord, and we publish on npm. Product analytics exist but belong to the product team and they've said no twice. What do I build?",
"expected_output": "An explicitly stated build order by value per hour, the near-zero rungs first, the rungs the constraints rule out deleted from the plan with the deletion written down, and baselines captured on the expiring surfaces before anything else.",
"files": [],
"expectations": [
"Sequences the work by value returned per hour and states the ranking out loud rather than implying it through row order.",
"Puts the tagging vocabulary and link registry first, as an hour of decisions requiring no engineering.",
"Puts snapshot jobs for the expiring surfaces second, justified by history that cannot be bought later.",
"Puts the self-reported source field at signup third, as near-zero effort.",
"States that links published before the tagging vocabulary exists stay unmergeable forever.",
"Deletes the rungs the stated constraints rule out, such as log analysis, product-side events and CLI or SDK telemetry, rather than parking them at the bottom of the list.",
"Writes the deletion and its reason into the plan document, because a parked rung returns next quarter as unbudgeted scope.",
"Names what the efficiency order starves, the high-value high-effort work such as the identity spine and product-side events, and states the condition that promotes it anyway.",
"States that the identity spine is promoted ahead of everything once a decision on the list asks whether a surface produces signups.",
"Ties the three-week deadline explicitly to promoting the near-zero rungs and deferring the identity spine.",
"Captures baselines on the expiring surfaces before any other instrumentation work begins."
]
},
{
"id": 11,
"prompt": "Ravenstock. Big week: we shipped cross-domain identity stitching on Monday and attributed signups went from 4 in the previous week to 261 this week. The analytics debugger is green on every event and our tag manager shows everything firing. I'm writing the monthly update. Can you help me frame this as the win it is, and tell me whether there's anything left to check before I hit send?",
"expected_output": "The jump identified as instrumentation rather than growth and annotated, the green debugger rejected as evidence, and an end-to-end trace plus internal-traffic exclusion required before the update goes out.",
"files": [],
"expectations": [
"States that the jump is instrumentation, not growth, and must be annotated as such before anyone reports it.",
"Instructs comparing only within the post-stitch period until a full cycle has passed.",
"Explains that first-party attribution reads near zero until the cross-domain stitch is verified in production, then flips the week it lands.",
"States that a green debugger proves something is configured, not that anything is measured correctly, and is not evidence.",
"Requires an end-to-end trace of each primary conversion event, walked as a real user in a fresh browser profile arriving through a tagged external link.",
"Requires confirming that the landing URL keeps its query string through redirects and OAuth hops.",
"Requires finding the stored record and confirming the identity fields resolved, with anonymous_id present and user_id attached on the signup event.",
"Requires finding the row in the report the funnel view will actually read, and names it as the step teams skip.",
"Requires a duplicate check by refreshing the confirmation page and re-entering through the back button, confirming the deduplication key holds.",
"Requires excluding internal, office, VPN, CI and preview-deployment traffic, and notes that exclusion filters are not retroactive.",
"Requires confirming the exclusion filter is active rather than merely defined."
]
},
{
"id": 12,
"prompt": "Quillon. Last quarter we did 19 conference talks and 11 of the people we met later opened an account, so I want to report a 58% talk-to-signup conversion rate. It's our strongest number. I also want to start a public leaderboard in our community server showing the top 20 answerers each month with their handles and answer counts, and set every advocate a target of four published blog posts per month. Does that all hold up?",
"expected_output": "The 58% rate refused on small-N grounds and replaced by a dated raw list, the off-web attribution basis named and framed as a floor, the public leaderboard refused without consent, and the posts target paired with a counter-metric.",
"files": [],
"expectations": [
"Rejects the 58% rate because it is computed on fewer than roughly 30 events.",
"Instructs publishing the raw list with dates instead of a percentage at that volume.",
"Notes that DevRel counts such as talks, enterprise conversations and external pull requests are naturally small, so this is the normal case rather than an exception.",
"Flags that the talk-to-signup link needs a stated evidence basis, since off-web appearances cannot be tagged.",
"Names the substitutes for off-web attribution: a short speakable vanity path per talk, a QR code per event resolving to a tagged URL, or a self-reported source field at signup.",
"States that those substitutes read as a floor on the surface's contribution, never as a total.",
"Refuses the public per-member leaderboard without consent from the members it would name.",
"Pairs the four-posts-per-month target with a quality counter-metric, and explains that an incentivised volume metric gets gamed once it becomes a target.",
"Attributes the gaming dynamic to the published measure-becomes-a-target principle rather than presenting it as an original observation.",
"Requires person-level community data to be aggregated before anything is published.",
"Keeps the self-reported source field's answer list short and stable across quarters so periods stay comparable."
]
},
{
"id": 13,
"prompt": "Arbordyne. We sell a platform to enterprises, but individual engineers sign up for the free tier first and then a procurement conversation starts months later. We built account rollup by matching on email domain, and now we have one 'account' holding 4,100 people, plus a long tail of accounts with exactly one person each. Sales wants a report of 'accounts that had a devrel touch before they became an opportunity'. Our CRM has a lead source field and nothing else. What do we fix?",
"expected_output": "Free-mail domains identified as the cause of the giant account and excluded from matching, the one-person accounts diagnosed as unresolved, the resolved share required in any account-level rate, and the influenced-pipeline report gated on whether the CRM can carry an account-level touch record.",
"files": [],
"expectations": [
"Identifies free-mail domains as the cause of the 4,100-person account, and instructs excluding free-mail-class domains from account matching.",
"Explains that free-mail domains identify no company, so matching on them collapses strangers into one account.",
"Diagnoses the one-person accounts as signup domains that never resolved to a company, and recommends enrichment or manual matching as the fallback.",
"Requires reporting what share of the population actually resolved to an account.",
"Forbids presenting an account-level rate computed from a partially resolved population without stating that share.",
"States that in a company-buying motion every event must carry account_id alongside user_id.",
"Identifies this as the both-at-once motion, instrumenting person-level everywhere and rolling up to account only where an org object genuinely exists.",
"States that the influenced-pipeline report depends on the CRM being able to carry an account-level touch record, and that a lead source field alone cannot honestly support it.",
"Settles that dependency before the report is promised to sales, rather than after.",
"Resolves account_id from the signup domain or the product's own org object, not from a person-level identifier."
]
},
{
"id": 14,
"prompt": "Brightmoor CLI, about 30,000 installs. Leadership wants usage data, and my plan is to turn telemetry on by default in the next minor release, send the full command line the user typed plus the working directory path so we can see what kinds of projects people use it in, hash the machine ID, and mention it in the release notes. We also want a Slack alert per event. How do I position this to the community?",
"expected_output": "The command line and working directory rejected outright, the documented collection floor named, a visible first-run notice and a documented disable path required, and the whole proposal checked against the decision list before it is built.",
"files": [],
"expectations": [
"Rejects sending the full command line and the working directory path.",
"States the rule that code, paths and arguments are never sent.",
"Names the documented floor of what may be collected, such as the command invoked, the tool or framework version, the OS, plugins, and build duration or size.",
"Requires a visible first-run notice rather than a mention in release notes.",
"Requires opt-out at minimum, with a documented environment variable or command that disables collection.",
"Requires documenting exactly what is sent.",
"States that hashing is not consent.",
"States that a telemetry surprise costs more community trust than the data returns.",
"Places CLI or SDK telemetry last in the build order on value per hour, and notes that its cost is paid in community trust rather than in hours.",
"Names the stricter opt-in model, where uploadable counters are approved through a public proposal process and results are published back openly, as an option that trades reach for trust and carries real governance overhead.",
"Checks the proposal against the decision list, naming the decision the telemetry would change or cutting it."
]
}
],
"trigger_queries": [
{ "query": "devrel tracking plan", "should_trigger": true },
{ "query": "we need an event taxonomy for our docs site", "should_trigger": true },
{ "query": "our GitHub traffic numbers don't match what analytics says", "should_trigger": true },
{ "query": "how do we instrument the developer funnel", "should_trigger": true },
{ "query": "utm discipline for our developer marketing links", "should_trigger": true },
{ "query": "how do I join docs traffic to signups", "should_trigger": true },
{ "query": "why do our devrel numbers disagree between tools", "should_trigger": true },
{ "query": "Can you write a tracking plan for our documentation site and our open-source repo?", "should_trigger": true },
{ "query": "every session in our analytics says the source is our own docs domain", "should_trigger": true },
{ "query": "we lost all our repo traffic history, what do we do now", "should_trigger": true },
{ "query": "should we add telemetry to our CLI", "should_trigger": true },
{ "query": "npm says 400k downloads, how many users is that", "should_trigger": true },
{ "query": "what events should we fire on our quickstart page", "should_trigger": true },
{ "query": "our docs sessions dropped 40% after the redesign, is the tracking broken", "should_trigger": true },
{ "query": "how do I tag conference links so I can tell which talk sent people", "should_trigger": true },
{ "query": "we can't tell which blog posts drive signups", "should_trigger": true },
{ "query": "half our traffic shows up as direct and I don't know why", "should_trigger": true },
{ "query": "set up utm conventions before we start publishing", "should_trigger": true },
{ "query": "how do I know if my analytics is actually recording the right thing", "should_trigger": true },
{ "query": "developers block our analytics script, what can we even measure", "should_trigger": true },
{ "query": "we need a property registry for our events", "should_trigger": true },
{ "query": "someone renamed an event and now the chart is broken", "should_trigger": true },
{ "query": "we're about to launch, what should I measure first", "should_trigger": true },
{ "query": "our attributed signups jumped 60x last week and I'm suspicious", "should_trigger": true },
{ "query": "I want to backfill the channel field on eight months of old signups", "should_trigger": true },
{ "query": "one account in our CRM has 4000 people in it", "should_trigger": true },
{ "query": "how should I snapshot PyPI download data", "should_trigger": true },
{ "query": "can I add our model hub downloads and our dataset downloads together", "should_trigger": true },
{ "query": "what should a docs analytics setup actually collect", "should_trigger": true },
{ "query": "I want to stitch anonymous ids to user ids across our marketing site and our app", "should_trigger": true },
{ "query": "instrumentation plan for a developer tool", "should_trigger": true },
{ "query": "do I tag internal links on my docs site", "should_trigger": true },
{ "query": "we have 140 events and nobody understands them", "should_trigger": true },
{ "query": "write the identity spine for our signup flow", "should_trigger": true },
{ "query": "which of our surfaces can we actually measure and which ones can't we", "should_trigger": true },
{ "query": "how do I prove a conference talk produced anything", "should_trigger": true },
{ "query": "our community numbers are inflated by bots and duplicate handles", "should_trigger": true },
{ "query": "I need to verify our conversion event is firing correctly end to end", "should_trigger": true },
{ "query": "we don't own the product event stream, what do we do about measuring activation", "should_trigger": true },
{ "query": "documentation analytics", "should_trigger": true },
{ "query": "measurement plan for our SDK adoption", "should_trigger": true },
{ "query": "what should I capture before we ship the new docs site", "should_trigger": true },
{ "query": "our chat venue numbers and our forum numbers can't be compared", "should_trigger": true },
{ "query": "help me design the events for our developer portal", "should_trigger": true },
{ "query": "do our docs page views mean anything if half our audience runs an ad blocker", "should_trigger": true },
{ "query": "how do we count downloads consistently across two registries", "should_trigger": true },
{ "query": "our two analytics tools give different session counts for the same page", "should_trigger": true },
{ "query": "should I use snake_case or Proper Case for event names", "should_trigger": true },
{ "query": "I want to know where our signups actually came from", "should_trigger": true },
{ "query": "we never recorded a baseline and now we can't tell if anything changed", "should_trigger": true },
{ "query": "what goes in a tracking plan document", "should_trigger": true },
{ "query": "make our developer funnel measurable", "should_trigger": true },
{ "query": "our reference docs have huge time-on-page, is that a good sign", "should_trigger": true },
{ "query": "how do we exclude our own team's traffic from the numbers", "should_trigger": true },
{ "query": "plan for collecting repo, registry and docs data into one place", "should_trigger": true },
{ "query": "where do people drop off in our quickstart, and how would I even know", "should_trigger": true },
{ "query": "we want to compare our sample repos by which one actually drives usage", "should_trigger": true },
{ "query": "our connector marketplace shows 8000 installs, is that adoption", "should_trigger": true },
{ "query": "help me write the events and properties for our docs search box", "should_trigger": true },
{ "query": "what's the right way to record how we know where a signup came from", "should_trigger": true },
{ "query": "we keep double counting conversions on the thank-you page", "should_trigger": true },
{ "query": "do a review of our developer analytics setup", "should_trigger": true },
{ "query": "how often should we re-check that our tracking still works", "should_trigger": true },
{ "query": "we changed our consent banner and all the numbers moved", "should_trigger": true },
{ "query": "what's a sane way to measure an open-source project's reach", "should_trigger": true },
{ "query": "I need to explain to my VP why Google Analytics and our server logs disagree", "should_trigger": true },
{ "query": "stars, downloads and sessions, which of these can I actually add together", "should_trigger": true },
{ "query": "our event names have the language baked into them and it's a nightmare to query", "should_trigger": true },
{ "query": "we're adding a new docs subdomain, what breaks in our tracking", "should_trigger": true },
{ "query": "which devrel KPIs should we report to our exec team", "should_trigger": false },
{ "query": "how do I prove devrel ROI to my CFO", "should_trigger": false },
{ "query": "what belongs in a devrel quarterly scorecard", "should_trigger": false },
{ "query": "our numbers look good but leadership still isn't convinced", "should_trigger": false },
{ "query": "we only report stars and impressions, what should replace them", "should_trigger": false },
{ "query": "set a target for our developer activation rate this quarter", "should_trigger": false },
{ "query": "build a devrel measurement framework with an owner and an action per metric", "should_trigger": false },
{ "query": "cut the vanity metrics out of our devrel report", "should_trigger": false },
{ "query": "is our Discord dying", "should_trigger": false },
{ "query": "which community health metrics should we track", "should_trigger": false },
{ "query": "design a contributor funnel for our community", "should_trigger": false },
{ "query": "our community feels quiet, how do we tell whether that's real", "should_trigger": false },
{ "query": "map the developer journey for our enterprise segment", "should_trigger": false },
{ "query": "where does adoption drop off between trial and production", "should_trigger": false },
{ "query": "who owns each stage of our developer experience", "should_trigger": false },
{ "query": "developers try our product but never ship with it", "should_trigger": false },
{ "query": "our docs don't rank on Google", "should_trigger": false },
{ "query": "fix the canonicals on our versioned docs", "should_trigger": false },
{ "query": "our preview docs got indexed, how do we get them out", "should_trigger": false },
{ "query": "hreflang setup for our translated documentation", "should_trigger": false },
{ "query": "the old version of our docs outranks the current one", "should_trigger": false },
{ "query": "what do developers actually search for around our API", "should_trigger": false },
{ "query": "build a keyword list from our error strings", "should_trigger": false },
{ "query": "which docs pages should we create based on search demand", "should_trigger": false },
{ "query": "how does our developer experience compare to our biggest competitor's", "should_trigger": false },
{ "query": "what is our competitor doing in devrel right now", "should_trigger": false },
{ "query": "benchmark our community size against two rivals", "should_trigger": false },
{ "query": "which developer audience should we serve first", "should_trigger": false },
{ "query": "how big is the Rust ecosystem for a product like ours", "should_trigger": false },
{ "query": "who signs the contract when the developer isn't the buyer", "should_trigger": false },
{ "query": "how should we split next year's devrel budget across conferences and content", "should_trigger": false },
{ "query": "is sponsoring this conference worth the money", "should_trigger": false },
{ "query": "rewrite our quickstart so people reach a first successful call faster", "should_trigger": false },
{ "query": "what should the first five minutes of our getting-started page do", "should_trigger": false },
{ "query": "audit the information architecture of our documentation", "should_trigger": false },
{ "query": "our docs are organized by team instead of by task", "should_trigger": false },
{ "query": "turn these 40 commits into release notes", "should_trigger": false },
{ "query": "write a migration guide for our v3 breaking changes", "should_trigger": false },
{ "query": "should we publish our revenue publicly", "should_trigger": false },
{ "query": "start a weekly devlog for our open-source project", "should_trigger": false },
{ "query": "which open-source license should we pick", "should_trigger": false },
{ "query": "write a governance model for our project", "should_trigger": false },
{ "query": "triage the 300 open issues on our repo", "should_trigger": false },
{ "query": "plan the launch of our open-source project on Hacker News", "should_trigger": false },
{ "query": "improve our repo README so it converts better", "should_trigger": false },
{ "query": "optimize our GitHub organization profile", "should_trigger": false },
{ "query": "should we sponsor the big cloud-native conference this year", "should_trigger": false },
{ "query": "start a local meetup program for our users", "should_trigger": false },
{ "query": "outline a 25-minute conference talk about our architecture", "should_trigger": false },
{ "query": "write a CFP submission for a Python conference", "should_trigger": false },
{ "query": "how should we price our API by usage", "should_trigger": false },
{ "query": "open core or hosted, which business model fits us", "should_trigger": false },
{ "query": "how many devrel people do we need at series B", "should_trigger": false },
{ "query": "write a job description for a developer advocate", "should_trigger": false },
{ "query": "how do I get promoted from advocate to head of devrel", "should_trigger": false },
{ "query": "set up server-side conversion tracking for our Google Ads campaigns", "should_trigger": false },
{ "query": "query the Google Analytics Data API and build me a report", "should_trigger": false },
{ "query": "configure property settings through the Google Analytics Admin API", "should_trigger": false },
{ "query": "build a rank tracker for our target keywords", "should_trigger": false },
{ "query": "track our AWS cost by service each month", "should_trigger": false },
{ "query": "set up mobile app analytics for our iOS app", "should_trigger": false },
{ "query": "wire up attribution for our paid social ad spend", "should_trigger": false },
{ "query": "instrument our production services with OpenTelemetry traces and spans", "should_trigger": false },
{ "query": "add structured logging and alerting to our backend API", "should_trigger": false },
{ "query": "build a Grafana dashboard for our Kubernetes cluster", "should_trigger": false },
{ "query": "set up frontend error tracking with source maps", "should_trigger": false },
{ "query": "write dbt models to turn our raw event tables into marts", "should_trigger": false },
{ "query": "review the data processing agreement with our analytics vendor", "should_trigger": false },
{ "query": "A/B test our pricing page headline", "should_trigger": false }
]
}
references/published-findings.md›
# Published findings and self-set baselines
Where every number, rule and named framework in this skill comes from, and which ones are this skill's own defaults. Read it before quoting a figure to a user or defending a threshold.
The short version: the _mechanics_ of this field are documented - retention windows, download-counting policies, blocking rates, naming conventions. The _targets_ are not: a healthy join-coverage rate, event count or docs-funnel conversion is a property of one program's audience and surface mix, not of the field. Every target in a plan comes from the program's own trailing baseline.
## Contents
- [Sourced claims and their sources](#sourced-claims-and-their-sources)
- [Named frameworks and what they actually mandate](#named-frameworks-and-what-they-actually-mandate)
- [Self-set baselines](#self-set-baselines)
- [Claims to refuse](#claims-to-refuse)
- [Negative example: a threshold presented as a standard](#negative-example-a-threshold-presented-as-a-standard)
## Sourced claims and their sources
| Claim | Source |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| 58% of a tech audience blocks a third-party analytics script - 68.2% desktop, 49.9% mobile, Firefox 88.3%, Chrome 50.4%, Safari 41.4%, Linux 82.3%, Windows 74.4%, macOS 61.7% | Plausible Analytics, "Google Analytics is blocked by adblockers", August 2021, measured against its own proxied script on a site carrying Hacker News and Reddit traffic |
| Repository traffic endpoints (views, clones, referrers, popular paths) retain **14 days**, and are readable only for repositories you hold write access to | GitHub REST API, "Metrics / Traffic" documentation |
| One major package registry keeps 18 months of download data, 365 days for bulk queries, 7 days per version, earliest data 2015-01-10 | npm `download-counts.md` and the npm blog post "Numeric precision matters: how npm download counts work" |
| That registry counts "the number of HTTP 200 responses we served that were tarball files" - build servers, mirrors and analysis bots included deliberately - and treats below ~50 downloads/day as mostly automation | Same npm sources |
| The other keeps 180 days of time series and excludes known mirrors by default, exposing with-mirrors and without-mirrors as separate series | pypistats.org API documentation |
| Roughly 70% of AI-assistant-driven traffic arrives with no referrer and lands in "direct" | Demand Curve newsletter #331 |
| Observational attribution overstates true lift by roughly 7-9x for paid media; a properly powered advertising experiment "can easily require more than ten million person-weeks" | Gordon et al., _Marketing Science_, 2019; Lewis & Rao, 2015 - the reason a DevRel-scale holdout reads direction, never a point estimate |
| "When a measure becomes a target, it ceases to be a good measure" | Marilyn Strathern, 1997, generalising Charles Goodhart, 1975 |
| Counter-metrics ("guardrails") published as a required field of a metric definition, alongside pre-conditions and obstacles | DevRel Foundation Metrics Index, `metrics/activity/blog-post-publish-rate/`, contributed by Jayson DeLancey |
| Volatility in a rate is expected and the fix is a longer window, not a redefinition | Same DRF page: "The measure is a rate, so extend the period of time to a larger frame to smooth the value out" |
| A coverage rate on the identity stitch is the standing health metric: "the `journey_linked: false` rate tells you how many conversions bypassed the stitch" | Production practice credited to Tessa Kriesel (DevRel practitioner; "Developer GTM Expert", previously platform DevRel lead at Snap) |
| "When identity is ambiguous, send nothing. A missing journey is a data gap; a wrong merge is corruption." | Same source, Kriesel-credited |
| First-party attribution reads near zero until the cross-domain stitch is verified in production, then flips to real numbers the week it ships | Same source, Kriesel-credited - the reason a launch-week jump is not a trend |
| Account rollup must exclude free-mail domains from domain matching, because they cannot identify a company | Same source, Kriesel-credited |
| Write `source`, `source_confidence` and `source_basis` (`journey_linked` / `self_reported` / `campaign_window`) rather than a bare channel | Same source, Kriesel-credited |
| "Telemetry without a question is noise" - write the 2-4 questions before instrumenting | `addyosmani/agent-skills@observability-and-instrumentation` (production telemetry, but the same discipline) |
| Stripe emits one canonical log line per request per service, asynchronously to its data warehouse, and the same schema powers the charts on Stripe's own customer-facing Developer Dashboard | Stripe Engineering blog, "Canonical log lines" |
| Stripe's Request Logs surface every API call a developer makes through the Dashboard UI itself; Integration Insights then analyzes error patterns in those logs and suggests fixes automatically | Kenneth Auchenberg (Stripe platform engineer), personal account, not an official Stripe page |
| Stripe's Workbench (2023) unifies debug, monitor and grow into one surface, tested a year with users including Slack and Notion before rollout | Stripe Engineering blog, "Workbench: a new way to debug, monitor, and grow your Stripe integration" |
| ReadMe's Metrics API/Developer Dashboard treats "Time to First Call" as the API team's north-star metric; a per-language Metrics SDK sits as request/response middleware (with redaction) to build a per-developer API log, while the same product exposes per-page docs view counts, top search terms and a per-page "was this helpful" score | ReadMe, "Customize your DX with Developer Dashboard" |
| Next.js's CLI telemetry collects the invoked command, framework version, OS, plugins and build duration/size by default, explicitly excludes anything that could carry secrets (env vars, paths, file contents, logs), and is disabled with one documented command | Next.js documentation, "Telemetry" |
| Go's own toolchain telemetry (`golang.org/x/telemetry`) is opt-in, restricted to Go-team-maintained tools, uploads only counters approved through a public telemetry proposal process, and publishes results openly | Go telemetry module source (`go.googlesource.com/telemetry`) and its `pkg.go.dev` documentation |
| Datadog uses its own RUM, logs and notebooks internally - explicitly calling this "dogfooding" - for visibility into the user experience its product produces | Datadog Engineering blog, "How Datadog uses Datadog to gain visibility into the Datadog user experience" |
| Datadog's Internal Developer Portal (2025) builds on its APM suite to auto-map services and dependencies, then layers self-service actions and reliability/security scorecards on the live telemetry | Datadog product documentation and its company-issued launch announcement |
## Named frameworks and what they actually mandate
**Object-Action** (Twilio Segment, "Naming conventions for clean data", undated page) is the only published naming framework in this space, and the origin of the `object_action` shape every analytics skill reuses.
- Mandated:
- Choose objects, then the actions customers perform on them.
- Use past tense.
- Carry the same properties with every event about the same object.
- Enforce through a centralised tracking plan with validation of required properties, allowed values and data types.
- **Not mandated: casing.** The source lists five options, recommends Proper Case for events with snake_case for properties, and rules that "the only thing that really matters is that you keep it consistent!" This skill's lowercase-snake_case-for-everything rule is therefore a house convention, chosen because one casing across events and properties removes a whole class of hand-typing mismatch. Say so if a user's existing plan uses another case - consistency beats conversion.
- "Context in properties, never in the name" is not stated by the source, though its examples obey it. A strong convention, not a citation.
**Goodhart / Strathern** justifies the counter-metric pairing; the DRF Metrics Index gives it an institutional form ("guardrails"). Treat the point at which a metric counts as gamed as a judgement call, never as a published threshold.
## Self-set baselines
These are this skill's own defaults, chosen so a plan is decidable, and a user may move any of them. Present them as starting points, never as measured norms.
| Baseline | Value | Why this value |
| ----------------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Join coverage | ≥ 80% of primary conversion events resolve to a first touch | 80% leaves room for the genuinely unresolvable (off-web, consent-denied) while still failing a broken stitch loudly. Replace it with the program's own trailing rate after two months of data. |
| Decision coverage | 100% of events trace to a decision | Not a measurement, a definition - an event with no decision behind it has nothing to be right about. |
| Verified traces | 100% of primary conversion events traced end to end | Same: a primary event nobody has traced is unverified, whatever the debugger says. |
| Tag well-formedness | 100% of published external links validate | Cheap to enforce, and a single stray tag is unrecoverable six months later. |
| Small-N floor | below ~30 events in a period, publish the raw list with dates | Statistical convention, not a DevRel finding. DevRel counts (talks, enterprise conversations, external PRs) are naturally small. |
| Event list size | small enough to read on one screen | A taxonomy nobody reads is a taxonomy nobody follows. The DRF's own advice to extend the window rather than add metrics points the same way. |
| Re-verification cadence | quarterly, plus the trigger list | Silent breakage is the default failure mode and nobody files a bug for a missing event. |
## Claims to refuse
- Any docs-funnel, activation or join-coverage benchmark borrowed from another company. Audience, product maturity and surface mix change what normal looks like; compare the program to its own past.
- The 20-40% activation band that circulates in developer-marketing content - it rests on repetition rather than on a measured cohort.
- A "users" figure derived from registry downloads, and any sum across registries.
- Any number that averages a client-side and a server-side count of the same thing.
- A percentage computed on fewer than ~30 events.
## Negative example: a threshold presented as a standard
Wrong, and tempting because it sounds authoritative:
> Our join coverage is 74%, below the industry standard of 80%. We need to fix the stitch before the quarterly review.
Two defects. The 80% is this skill's default, and the sentence launders it into a benchmark the reader cannot check. And "below standard" hides the only question worth asking: which surface stopped resolving, and when.
Right:
> Join coverage is 74%, against the 80% floor we set ourselves in the plan. It was 89% in June; the drop is entirely in blog-sourced signups and starts the week of the docs redesign, which is consistent with the anonymous-ID cookie no longer being shared across the blog subdomain. Owner: web team, re-trace scheduled.
references/surface-register.md›
# Surface register
One row per surface in the tracking plan. Fill owner, data source, retention, bias and collection recipe before writing a single event.
## Contents
- [The register table](#the-register-table)
- [Docs, blog and marketing site](#docs-blog-and-marketing-site)
- [Repositories](#repositories)
- [Package registries](#package-registries)
- [Artifact and connector hubs](#artifact-and-connector-hubs)
- [Community venues](#community-venues)
- [Off-web surfaces](#off-web-surfaces)
- [Product and CLI](#product-and-cli)
- [The account roll-up (B2B) versus the individual motion (B2C-shaped)](#the-account-roll-up-b2b-versus-the-individual-motion-b2c-shaped)
## The register table
| Surface | Source that exists | Retention | Baked-in bias |
| ------------------------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| Docs / blog / marketing site | client event collection, server or CDN logs, search-console-class data | your tool's setting; search console data commonly 16 months | client collection under-counts developers heavily |
| Repository | forge traffic endpoints (views, clones, referrers, popular paths), git history, issue/PR API | **14 days** on traffic endpoints, and only for repos you hold write access to | clones include CI and mirrors; stars only ever go up |
| Package registry | registry download API | npm: 18 months, 365 days for bulk queries, 7 days per version; PyPI: 180 days of time series, longer only via its public data warehouse | see the two definitions below |
| Artifact / connector hub | hub API fields (downloads, likes, installs), or a paid publisher-analytics export | full history on the paid export of one major hub; the public API returns current totals, not series | every hub defines "download" differently, and the default counts deduplicate nothing |
| Community venue | venue analytics panel or export API | free plans cap message history | bots, staff accounts and duplicate identities inflate every count |
| Talks, podcasts, print, hallway | nothing automatic | n/a | invisible unless you plant a vanity URL, a short link or a self-reported field |
| Product / CLI / SDK | your own event stream, API request logs | yours | the only surface that proves behaviour changed; usually owned by another team |
Sources for the retention figures: the forge's REST metrics/traffic documentation, and each registry's own stats documentation. Re-check them when the plan is revised - they change without notice.
## Docs, blog and marketing site
Collect: page view with a `section` property (`docs`, `reference`, `blog`, `marketing`), `docs_search_performed` with `zero_results`, `sample_copied`, quickstart step events, outbound clicks to the repository and to signup.
Do not collect: scroll depth on reference pages, time-on-page as a quality signal. Reference pages are scanned, not read; both metrics report the opposite of what they seem to.
Search-console-class data is the only source that sees the real queries reaching a docs site. Segment it by path prefix - a sitewide average hides that reference pages earn impressions and no clicks while guides earn the reverse.
ReadMe's own Developer Dashboard product binds this to the API side directly: a per-language Metrics SDK sits as request/response middleware on the API server, forwarding request/response details (with optional redaction of private fields) to build a per-developer API log, while the same system exposes per-page doc view counts, the top search terms typed into docs search, and a per-page "was this helpful" score. ReadMe frames "Time to First Call" as the north-star metric this instrumentation optimizes toward - the API-side equivalent of the time-to-first-success events this register already recommends for a quickstart.
## Repositories
Collect daily via the traffic endpoints and append to your own store: unique visitors, unique cloners, top referrers, top paths. Add from the issue/PR API: first-time contributor count, time to first response, open-to-close ratio, external pull requests merged.
Traps:
- The 14-day window means a monthly export loses half the data. Schedule daily.
- Clones are not people; CI clones on every build.
- Stars are monotonic and gameable. Convert to stars-per-period if you report them at all, and never as a headline.
- Referrers into a repository are mostly stripped, exactly like web referrers.
Sample and template repositories have no instrumentation pattern of their own; no dedicated named framework exists for sample-app telemetry (forks, clones, template-repo usage). Read the same traffic endpoints, stars-per-period and clone counts as any other repository row - forks are the one signal specific to a template repo, and inherit the same CI/mirror-inflation trap as clones.
## Package registries
A registry's counting policy is a published property of that platform, not a vendor feature, so the two dominant ones are named here - an unnamed retention figure cannot be acted on. Read whichever registry the product actually ships on; the pattern generalises, the numbers do not.
- **npm** counts "the number of HTTP 200 responses we served that were tarball files". Build servers, mirrors and analysis bots are all counted deliberately; npm's own guidance is that below roughly 50 downloads/day the signal is mostly automation. It is a request count, not a person count.
- **PyPI** excludes known mirrors by default and exposes with-mirrors and without-mirrors as separate series.
So an npm figure and a PyPI figure for the same product measure different things and must never be summed as "users". Report the trend of one registry against its own baseline. If the product ships on several, keep one series per registry, each with its definition written next to it.
## Artifact and connector hubs
A hub where the product ships as a model, dataset, connector, plugin or app is a fourth counting regime, not a variant of the package-registry one. Two rules decide everything you can say from it.
**The same hub can count two of its own surfaces incompatibly.** On Hugging Face, a model download is every HTTP `GET` or `HEAD` against a per-library query file - `config.json` by default - so a CI job that touches the file on every build counts every time, and self-contained formats such as GGUF double-count a whole-repository clone. A dataset download on the same hub is the opposite: every file one IP fetches from one repository inside a 5-minute window collapses to a single download. Model and dataset counts from that hub are therefore not summable and not comparable, and neither is comparable to a package registry.
**Unique downloaders are a paid feature, not a default.** The hub's own organisation dashboard states its figures are "not deduplicated by user"; request-level logs carrying a hashed user or IP, country and user agent exist only as an add-on to the top enterprise tier. Until that is bought, no "how many people" question is answerable from the surface - only "how many requests".
Collect: per-artifact downloads, likes, and the hub's own trending or usability score where one exists; per-connector installs. Snapshot daily into your own store and report deltas - the public API returns current totals, not history, so a series you did not save is gone.
Traps:
- **Likes are not stars, and not downloads either.** On the ML hubs they are a recency signal that concentrates on new releases, while downloads concentrate on old dependencies. The hub's own published analysis calls treating either as a proxy for the other "the most common mistake we see in coverage of the Hub". Keep them as two series and label what each one means.
- **Trending scores are opaque.** They are exposed as an API sort key with no published formula and no stability guarantee. Usable as a snapshot observation, never as a tracked metric.
- **Polling is rate-limited per plan.** Limits are enforced over 5-minute fixed windows and differ by tier - an anonymous caller gets a fraction of what an authenticated one does. Always send a token; a collection job that silently starts 429-ing produces a flat line that reads as lost adoption.
- **Installs on a connector hub are not usage.** Hosts publish install counts and almost never publish uninstalls or active-workspace counts, so the number is monotonic by construction. Pair it with a first-party signal from the connector itself if you need to claim adoption.
## Community venues
Collect: monthly active members, messages by channel, questions asked versus questions answered, median time to first human response, share of answers written by non-staff, new-member retention at 7 and 30 days.
Four things break these numbers before you even read them:
- **Bots.** A welcome bot answering in four seconds makes time-to-first-response look excellent while no human ever replied. Keep an explicit bot list and filter at query time.
- **Identity.** The same person is a forge handle, a chat handle, a commit email and a forum account. Without a merge step, "unique contributors" is inflated and every concentration measure is wrong.
- **Retention caps.** A metric defined today may be uncomputable for last quarter. Snapshot monthly into your own store from day one.
- **Staff flag.** "Share of answers written by non-staff" and "moderator share of messages" separate a community from a support queue, and both need the flag.
## Off-web surfaces
There is no passive collection here. Plant one of these before the appearance, never after:
- A short, speakable vanity path per talk or episode - the only clean per-appearance click signal.
- A QR code per event resolving to a tagged URL.
- One self-reported source field at signup, short, with a stable answer list so quarters stay comparable.
Read these as a floor on the surface's contribution, never a total, and pair them with a branded-search or direct-traffic read over the 7-14 days after publication against the preceding baseline.
## Product and CLI
The product event stream is where enablement and behaviour-change claims are actually settled: activation, time to first success, second-project rate, cohort retention. Getting a devrel event into it usually means a negotiation with whoever owns it - budget for that, and settle on one shared `user_id` rather than a parallel devrel-only identity.
Stripe's own engineering blog describes a structural way to make that negotiation unnecessary: a canonical log line, one structured line per request per service, emitted asynchronously to the data warehouse, whose schema also powers the charts on Stripe's customer-facing Developer Dashboard. The observability pipeline and the enablement surface are the same pipeline there, not two a team has to keep in sync.
CLI and SDK telemetry is a separate decision with a real trust cost in developer audiences. If you propose it:
- Make it opt-out at minimum, with a visible first-run notice.
- Document exactly what is sent.
- Ship a documented environment variable to disable it.
- Never send code, paths, or arguments.
Next.js's own CLI telemetry documents that floor concretely: command invoked, framework version, OS, plugins, build duration/size, explicitly excluding anything that could carry secrets (env vars, paths, file contents, logs), disabled with one documented command. A stricter shape sits above the floor: Go's own toolchain telemetry (`golang.org/x/telemetry`) is opt-in, scoped to tools the Go team itself maintains, and only uploads counters approved through a public proposal process, with every result published back openly. That model trades reach for trust and costs real governance overhead - a proposal process and a review body - worth adopting only where that overhead already exists, not a bar every CLI needs to clear.
A telemetry surprise costs more community trust than the data is worth.
## The account roll-up (B2B) versus the individual motion (B2C-shaped)
- **Individual self-serve adoption**: the spine stops at `user_id`. Funnel views run person by person, and the success event is a product action.
- **Company buying**: every event must also carry `account_id`, resolved from the signup domain or the product's own org object. The buying decision, the renewal and the churn all live on the account, so a person-level-only plan cannot answer any question the funder asks.
- **Both at once** (the common devtool case): instrument person-level everywhere, roll up to account only where an org object genuinely exists, and never present an account-level rate computed from a partially resolved population without saying what share resolved.
An account-level touch record is also what an influenced-pipeline rule depends on. If the CRM cannot carry "this account had a devrel touch before this stage", that rule cannot be honestly reported - decide that here, before promising it.
references/tracking-plan-template.md›
# Tracking plan template
Copy the structure, fill it with the user's real surfaces and events, and keep the version block current. The plan is a contract between whoever defines the metric and whoever implements it - everything ambiguous in it becomes a defect three months later.
## Contents
- [Template](#template)
- [Worked example: a devtool docs and repository plan](#worked-example-a-devtool-docs-and-repository-plan)
- [Event naming: good vs bad](#event-naming-good-vs-bad)
- [Two rows compared](#two-rows-compared)
- [Two tagging vocabularies compared](#two-tagging-vocabularies-compared)
## Template
```markdown
# <Product> DevRel tracking plan
Version: 1.0 · Owner: <name> · Last verified: <date> · Review trigger: <redesign, new surface, quarterly>
## Decisions this plan serves
| # | Decision | Question that settles it | Funnel view |
| --- | -------- | ------------------------ | ----------- |
## Surface register
| Surface | Owner | Data source | Retention | Known bias | Snapshot schedule |
| ------- | ----- | ----------- | --------- | ---------- | ----------------- |
## Build order
| Rank | What gets built | Effort (order of magnitude) | What it buys | Deleted / deferred, and why |
| ---- | --------------- | --------------------------- | ------------ | --------------------------- |
## Identity spine
- anonymous_id: <where set, which domains carry it>
- user_id: <when assigned, which event stitches it>
- account_id: <resolution rule, or "not applicable - individual motion">
- Unresolvable surfaces: <list, and the substitute signal for each>
## Events
| Event | Surface | Trigger | Properties | Decision # | Verified |
| ----- | ------- | ------- | ---------- | ---------- | -------- |
## Property registry
| Property | Type | Allowed values | Notes |
| -------- | ---- | -------------- | ----- |
## Link tagging vocabulary
| Parameter | Allowed values / pattern |
| --------- | ------------------------ |
## Funnel views
| View | Entry event | Steps | Success event | Segment |
| ---- | ----------- | ----- | ------------- | ------- |
## Baselines captured
| Metric | Value | Date | Source |
| ------ | ----- | ---- | ------ |
## Quality gate
| Check | Threshold | Current | Status |
| ----- | --------- | ------- | ------ |
## Known gaps
| Gap | Impact on which number | Owner | Decision |
| --- | ---------------------- | ----- | -------- |
```
## Worked example: a devtool docs and repository plan
Decision list (abridged):
| # | Decision | Question | Funnel view |
| --- | ---------------------------------- | --------------------------------------------------- | ------------------ |
| 1 | Rewrite the quickstart or leave it | Where do first-time readers stop? | Quickstart funnel |
| 2 | Keep or drop the weekly blog | Do blog readers ever reach a first successful call? | Blog-to-activation |
| 3 | Renew the conference sponsorship | Did the event produce anything past a badge scan? | Event-tagged entry |
Events (abridged):
| Event | Surface | Trigger | Properties | Decision # |
| --------------------------- | ------- | --------------------------------- | ----------------------------------------------- | ---------- |
| `docs_page_viewed` | docs | page render | `section`, `path`, `version`, `language` | 1, 2 |
| `quickstart_step_completed` | docs | step checkpoint reached | `step_number`, `step_name`, `language` | 1 |
| `sample_copied` | docs | copy control used on a code block | `path`, `block_id`, `language` | 1 |
| `docs_search_performed` | docs | search submitted | `query_length`, `results_count`, `zero_results` | 1 |
| `signup_completed` | app | account created | `self_reported_source`, `anonymous_id` | 1, 2, 3 |
| `first_api_call_succeeded` | product | first 2xx from a new account | `endpoint`, `sdk_version`, `hours_since_signup` | 1, 2, 3 |
Quickstart funnel view: entry `docs_page_viewed {path: /quickstart}` → `quickstart_step_completed {step_number: 1..n}` → `signup_completed` → `first_api_call_succeeded`, segmented by `language` and by inbound source.
Known gap worth writing down explicitly: docs page views are collected client-side and a majority of this audience blocks that collection, so the funnel's absolute entry count is a floor. Step-to-step ratios inside the funnel are unaffected as long as the blocking is not correlated with the step - record that assumption, and sanity-check it once against server logs.
## Event naming: good vs bad
```
// ✓ Good - object then action, context lives in properties
quickstart_step_completed {step_number: 3, step_name: "install", language: "python"}
sample_copied {page, block_id, language}
docs_search_performed {query_length, results_count, zero_results}
first_api_call_succeeded {endpoint, sdk_version}
// ✗ Bad - context baked into the name, or no properties at all
reference_page_viewed
python_quickstart_step_3_done
button_clicked // no properties
```
The first two bad names push context into the name, so every future query has to enumerate names. The third pushes everything into one bucket you can never split.
## Two rows compared
Good row:
| Event | Surface | Trigger | Properties | Decision # |
| --------------------------- | ------- | ------------------------------------------------------ | ---------------------------------------------------------- | ---------- |
| `quickstart_step_completed` | docs | reader reaches the checkpoint that proves the step ran | `step_number` (int), `step_name` (enum), `language` (enum) | 1 |
Bad row, and why:
| Event | Surface | Trigger | Properties | Decision # |
| ------------------------------ | ------- | ---------------------------- | ------------------- | ---------- |
| `python_quickstart_step3_done` | docs | scrolled past the code block | `label` (free text) | - |
- The name hard-codes language and step, so every future query must enumerate event names and a fifth step means a fifth event.
- "Scrolled past" measures the page, not the reader. A step event has to fire on evidence the step worked, otherwise the funnel measures scrolling.
- A free-text property is un-queryable within a quarter.
- No decision number: nothing changes when this number moves, so the row should not exist.
## Two tagging vocabularies compared
Good - closed lists, one casing, a pattern the author can follow without asking:
| Parameter | Allowed values / pattern |
| --------- | ----------------------------------------------------------------------------------------------------------------------------- |
| source | `x`, `linkedin`, `hn`, `reddit`, `newsletter`, `conf`, `podcast`, `partner` (closed list; additions need a plan version bump) |
| medium | `social`, `email`, `referral`, `talk`, `sponsorship` (closed list) |
| campaign | `<yyyy-mm>-<artifact-slug>`, lowercase, hyphen-separated |
Bad, and why every line of it breaks a report:
| Parameter | Allowed values / pattern |
| --------- | ---------------------------------------------------------------------------------------------------------------------------- |
| source | anything descriptive (`Twitter`, `twitter`, `X`, `x.com` all appear within a month, and no report can merge them afterwards) |
| medium | `social media`, `Email Newsletter` (spaces and mixed case survive some tools and get mangled by others) |
| campaign | `launch` (reused for three launches in eighteen months, so the series is meaningless) |
Two more mistakes belong here rather than in the vocabulary:
- Tagging internal docs links, which restarts the session and overwrites the real source with your own.
- Leaving the published-link register unwritten, which makes every historical tag unreadable once its author forgets what it meant.
references/verification-playbook.md›
# Verification playbook
The plan is unproven until one real event has been traced end to end on each surface. A green debugger, a live tag, or a configured API connection proves that something is configured - not that anything is measured correctly.
## Contents
- [The trace](#the-trace)
- [Per-surface trace recipes](#per-surface-trace-recipes)
- [Exclude your own traffic](#exclude-your-own-traffic)
- [Comparability gate](#comparability-gate)
- [Re-verification triggers](#re-verification-triggers)
- [Finding schema](#finding-schema)
## The trace
For each primary conversion event, walk the real user path and confirm every hop, in this order:
1. **Act as a real user.** Fresh browser profile, no cached session, arriving through a tagged external link. Accept or deny consent deliberately - run the trace twice if both states matter.
2. **Confirm the landing URL keeps its query string.** Redirects, shorteners, OAuth hops and locale rewrites are where tags die.
3. **Inspect the outgoing request payload.** It must carry the event name, a unique event ID, the identity fields the spine defines, and the properties in the registry - with the registry's types, not stringified numbers.
4. **Confirm receipt in the collector**, not just in the browser's network tab.
5. **Find the stored record** and confirm the identity fields resolved: `anonymous_id` present, `user_id` attached on the signup event, `account_id` populated if the motion needs it.
6. **Find the row in the report** the funnel view will actually read. This is the step teams skip, and it is where a correct event silently lands in the wrong bucket.
7. **Verify no duplicate.** Refresh the confirmation page and re-enter through the back button; the deduplication key must hold.
8. **Record the evidence** - surface, event, timestamp, consent state, payload excerpt, report row - in the plan document. An untraced event ships as "unverified", never as done.
## Per-surface trace recipes
| Surface | The one trace that matters |
| ---------- | ----------------------------------------------------------------------------------------------------------------------------- |
| Docs | tagged inbound link → quickstart page → `quickstart_step_completed` → success event → the funnel view's step-2 count |
| Blog | tagged inbound link → outbound click to signup → signup event carrying the original source |
| Repository | export the traffic endpoints twice 24 hours apart and confirm the append landed without gaps or duplicate days |
| Registry | pull the same day's download figure twice a week apart; if it moved, you are reading a revised or differently-filtered series |
| Community | post as a test member, confirm the export sees the message, the bot filter excludes bots, and the staff flag marks staff |
| Off-web | load the vanity URL from a phone on mobile data and confirm it resolves, tags, and lands in the source report |
| Product | fire the activation event from a test account and confirm the devrel-side identity resolves to it |
## Exclude your own traffic
Do this before the plan goes live; exclusion filters are not retroactive.
- Office and VPN ranges, the team's own accounts, CI, preview/staging deployments, and docs-preview URLs.
- Confirm the filter is _active_, not merely defined - collection tools commonly ship a new filter in a testing state that tags internal traffic without excluding it.
- Retract test conversions and delete test records once the trace is recorded.
- Internal and test traffic is a large fraction of sessions on a small docs site; unfiltered, it makes the team's own reading habits look like adoption.
## Comparability gate
Refuse to sum or average numbers from two sources until they share:
- The event definition.
- The attribution window.
- The counting rule (once per session versus every occurrence).
- The deduplication identity.
- The timezone.
- The bot/mirror filter.
Two download series with different mirror policies, or two conversion counts with different windows, are not addable - publish both with their definitions and explain the delta.
## Re-verification triggers
Re-run the full trace after any of these, because each one silently strips instrumentation:
- A site redesign or a docs-generator major upgrade.
- A consent-banner or consent-mode change.
- A new domain, subdomain or CDN in front of an existing surface.
- A new surface, a new registry, or a migrated community venue.
- Any change to the identity spine.
Schedule a standing re-trace at least quarterly even when nothing above happened. Silent breakage is the default failure mode; nobody files a bug for a missing event.
## Finding schema
Report each verification result in one shape so a reviewer can act without re-reading the plan:
| Field | Values |
| ------------------ | ---------------------------------------------------------------------------------------- |
| Result | pass / fail / unknown / not applicable |
| Severity | critical (a primary event is missing, double-counted, or wrongly valued) / major / minor |
| Confidence | high / medium / low, with the reason |
| Observation | what was seen |
| Evidence | payload excerpt, report row, timestamps |
| Owner and due date | a named person |
Reserve _critical_ for a confirmed defect with observed impact. A surface that is simply not instrumented yet is a coverage gap, not a critical finding - labelling everything critical is how a verification report gets ignored.
A verification pass never mutates production tracking. It returns findings; someone else applies them, and the trace is re-run afterwards.
SKILL.md›
---
name: devrel-analytics
description: Builds the tracking plan that instruments developer-relations surfaces - docs, blog, repositories, package registries, community venues, and off-web appearances - with an event taxonomy, an identity spine, link-tagging discipline, source-confidence labelling, and funnel views. Use when the user says "devrel tracking plan", "docs analytics", "utm discipline", "event taxonomy for our docs", "our GitHub numbers don't match analytics", "how do we instrument the developer funnel", "join docs traffic to signups", or wants to know why devrel numbers disagree between tools. Instrumentation layer only - which KPIs deserve a target belongs to samber/developer-relations-skills@devrel-metrics.
license: MIT
metadata:
author: Samuel Berthe
version: "1.0.1"
---
# DevRel Analytics
You are a developer-relations measurement engineer. You design the tracking plan that makes a DevRel program's numbers real: which event fires on which surface, how a person is recognized across surfaces that share no identifier, how links get tagged, and which funnel views the plan is built to serve.
This skill produces a written tracking plan, not a dashboard and not a metric framework. If the user still has to decide _which_ numbers matter, route them to `samber/developer-relations-skills@devrel-metrics` first and come back.
## Which numbers you may cite
The mechanics of this field are documented: retention windows, download-counting policies, blocking rates, the one published naming framework. The targets are not - a healthy join-coverage rate or docs-funnel conversion is a property of one program's audience and surfaces, so every target in a real plan comes from the program's own trailing baseline.
Hold that line in everything you write. The first reviewer who catches a self-set number dressed as a benchmark stops trusting the rest of the plan.
- **Quote freely, source attached**: the 58% blocked-analytics measurement, the 14-day repository-traffic window, each registry's retention and counting rule, the Object-Action naming framework, the case for a coverage rate as the standing health metric. Full list with citations in [references/published-findings.md](references/published-findings.md).
- **Quote as this skill's own defaults, movable by the user**: the ≥80% join-coverage floor, the ~30-event small-N floor, the one-screen event list, the quarterly re-verification cadence.
- **Refuse**: a docs-funnel or activation benchmark borrowed from another company, a "users" count derived from registry downloads, any average of a client-side and a server-side count of the same thing.
## Interview
Ask these one at a time, multiple-choice where you can. Stop as soon as you can name the surfaces and the decisions; do not run the whole list for a single-surface request.
**Memory: Store a few info about the current task or project context. When memory lives in a file, use `devrel-context.md`; if a different memory system is in use, rely on that instead. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.**
1. Which decisions should this data change? Name two or three real ones (kill a content format, rewrite a quickstart, renew a sponsorship).
2. Which surfaces do you own? Docs site, blog, marketing site, one or more repositories, package registries, a community venue, a CLI or SDK, the product itself.
3. Which surfaces do you _not_ own but still get traffic from? Talks, podcasts, other people's posts, aggregators.
4. What is already instrumented, and who owns it?
5. Is the adoption motion individual self-serve, company buying, or both? This decides whether events roll up to an account.
6. What is the consent regime for your traffic (EU/UK/CA users, an existing consent banner)?
7. Who implements - the docs/web team, a data team, or you?
8. By what date must the first real numbers land, and is there a launch, conference or migration before it? A hard date promotes the near-zero rungs of "Which instrumentation to build first" and pushes the identity spine to phase two; a launch inside the window makes baseline capture step one regardless of order.
9. Do you want a one-off answer to one question, or an asset that keeps paying? A one-off answer promotes tagging and a self-reported field; a compounding mandate promotes the snapshot jobs and the identity spine, which are worthless in week one and irreplaceable in month six.
10. What is the honest ceiling on effort - a spreadsheet and a monthly export, an occasional web-team ticket, or a pipeline someone maintains? A spreadsheet ceiling deletes log analysis, product-side events and CLI telemetry from the plan; a maintained pipeline promotes them.
If your harness has persistent memory, store the answers plus the final surface register and event list, so a later run edits the plan instead of re-deriving it.
## Workflow
1. **Write the decision list first.** One line per decision, with the question that would settle it. Every event in the plan must trace back to a line here. An event with no decision behind it is deleted, not deferred - that is the only reliable defence against a taxonomy that doubles every quarter.
2. **Build the surface register.** One row per surface: owner, data source that actually exists, retention window, known bias, and what you will collect. Read [references/surface-register.md](references/surface-register.md) - it holds the per-surface source, retention and bias for docs, repositories, registries, community venues, off-web appearances and the product.
3. **Apply the developer-audience correction.** Decide, per surface, whether the number you plan to collect survives a developer audience blocking client-side analytics. See "The blocked-analytics correction" below; this usually rewrites step 2's data-source column before anything is implemented.
4. **Order what gets built.** The register names more instrumentation than any team ships at once. Sequence it with "Which instrumentation to build first" below, delete the rungs the interview's effort ceiling rules out, and write the surviving order into the plan so the next reader inherits the reasoning instead of the list.
5. **Set the identity spine.** Define:
- `anonymous_id`: first visit, first-party, carried across every owned subdomain.
- `user_id`: assigned at signup, emitted alongside `anonymous_id` on the signup event so history stitches.
- `account_id` (company-buying motion only): resolved from the signup domain with free-mail domains excluded, since those identify no company and would collapse strangers into one account.
Mark the surfaces that can never be resolved to a person. They get a spoken vanity URL, a per-event short link, or a self-reported source field instead. See "Recording how you know" below for what each conversion carries out of the spine.
6. **Write the event taxonomy.**
- Object then action, past tense: Segment's Object-Action framework, the one published naming convention in this space.
- Put context in properties, never in the name. The framework's own examples follow this without stating it as a rule.
- Keep a property registry with a type and an allowed-value list per property.
- Ban PII and PII-shaped identifiers from every property.
7. **Write the link-tagging convention.** Fix the controlled vocabulary for source and medium before the first link exists, and the pattern for campaign. See "Link tagging" below.
8. **Define the funnel views.** One view per question from step 1, each naming its entry event, its steps, its success event, and the surface each step lives on. A view that cannot be built from the events in step 6 sends you back to step 6.
9. **Capture baselines and start snapshotting.** Three of the six surfaces expire their own history (repository traffic after 14 days, one major registry's time series after 180 days, chat venues per plan). Record today's values and schedule the export before instrumenting anything else - backfill is impossible.
10. **Verify each surface with one end-to-end trace,** then exclude internal traffic. Follow [references/verification-playbook.md](references/verification-playbook.md). A green debugger is not evidence; the trace is.
11. **Ship the plan document** using the template in [references/tracking-plan-template.md](references/tracking-plan-template.md), with a version, an owner, and a review trigger. Re-run step 10 after any site redesign, docs-generator upgrade, consent-banner change, or new surface - all four silently strip instrumentation.
## The blocked-analytics correction
Developer audiences block client-side analytics at rates no marketing playbook assumes. Plausible's August 2021 study, comparing its own proxied script against Google Analytics on a site carrying Hacker News and Reddit traffic, measured **58% of tech-audience visitors blocking Google Analytics** - 68.2% on desktop, 88.3% on Firefox, and 82.3% on Linux.
What that forces into the plan:
- A third-party analytics script on a docs site measures roughly the non-developer half of a developer audience, and the miss rate varies by page. A Linux install page is blocked far harder than a pricing page, so the bias is not even a constant you can divide out.
- Trust ratios inside one source over absolutes across sources. Step-to-step drop-off on the quickstart survives the correction; "monthly docs sessions" does not.
- Never average a blocked client-side number against a server-side one to "split the difference". Label each number with the source it came from and reconcile the gap explicitly.
- Consent denial removes measurement one-for-one on the surfaces it covers; treat the consented share as a sample, and say so in every report built on it.
## Which instrumentation to build first
The register names eight buildable things, and they differ by two orders of magnitude in what they cost to stand up. Rank them by what each returns per hour spent - not by what is cheapest, which is a different order and answers a different question.
- efficiency (value per hour - the default build order): tagging vocabulary > snapshot jobs > self-reported source field > docs event collection > identity spine > log analysis > product-side events > CLI/SDK telemetry
- effort (heaviest first): CLI/SDK telemetry > product-side events > identity spine > docs event collection == log analysis > snapshot jobs > tagging vocabulary == self-reported source field
- value (strongest evidence first): identity spine > product-side events > docs event collection > snapshot jobs > tagging vocabulary > log analysis > CLI/SDK telemetry > self-reported source field
- compliance cost (heaviest review first): CLI/SDK telemetry > product-side events > identity spine > docs event collection > self-reported source field > log analysis > snapshot jobs == tagging vocabulary
The three ties are genuine ones:
- Tagging vocabulary and the self-reported field: both are a decision written down rather than code, so neither can be made cheaper than the other.
- Docs event collection and log analysis: each is about a week of somebody else's sprint, one on the site and one on the pipeline.
- Snapshot jobs and the tagging vocabulary: both touch only public counts and query strings, so neither reaches a person and neither triggers a review.
In efficiency order, with the trade-off folded into each rung:
1. **Tagging vocabulary and link registry** - an hour of decisions, no engineering. Every future external link becomes readable, and the decision is unrepeatable: links published before the vocabulary exists stay unmergeable forever.
2. **Snapshot jobs for the expiring surfaces** - an hour to write, then a standing job. Buys the only history that cannot be bought later; repository traffic is gone in 14 days and one registry's series in 180.
3. **Self-reported source field at signup** - near-zero. The only signal that exists at all for talks, podcasts and hallway conversations, and weak everywhere those are not the question.
4. **Docs event collection on a first-party or proxied script** - about a week of web-team work. Recovers the developer half of the audience that a third-party script never saw, and makes quickstart drop-off measurable.
5. **Identity spine and the cross-domain stitch** - a week of engineering if you own both domains, a quarter if you do not. The strongest evidence in the plan and the only thing that answers whether a surface produces signups.
6. **Server or CDN log analysis** - about a week of data work. Raw unblockable reach, no behaviour, no funnel; a sanity check on the client numbers rather than a replacement.
7. **Product-side events** - a quarter, most of it negotiating with the team that owns the stream. Settles activation and behaviour-change claims that nothing upstream can.
8. **CLI/SDK telemetry** - a standing job plus a trust cost paid in the community, not in hours. Answers real-usage questions no other surface reaches, and a surprise here costs more goodwill than the data returns.
Default: ship rungs 1-3 in the first week, then stop and re-read the decision list. Move to rung 4 when a decision needs page-level behaviour, and promote the identity spine ahead of everything when any decision on the list asks whether a surface produces signups - until the stitch lands, every number above it is decorative.
What this order starves is deep instrumentation: the identity spine, product-side events and CLI/SDK telemetry all sit high on value and high on effort, so an efficiency ranking demotes them every quarter and the program keeps re-buying cheap reach it already has. Promote them the moment the decision list contains a question only post-signup behaviour can settle, or when a funder asks for influenced pipeline - that question has no cheap answer, and a plan without the spine has to answer it with a guess.
Delete rather than demote. A spreadsheet-and-monthly-export effort ceiling deletes rungs 6, 7 and 8 from the plan outright - say so in the document, because a rung parked at the bottom returns next quarter as unbudgeted scope. No consent basis for client-side collection on the majority of traffic deletes rung 4, and the docs read is rebuilt from logs instead.
This order is a default, not a law: it shifts with context and with who executes it. Re-rank it against what you already know about this team.
A first-party analytics proxy already running, or a product event stream this team owns, has had its effort paid already - both jump to the top. A team whose only real surface is a repository has no rung 4 at all.
## Recording how you know
Step 5 decides who a conversion belongs to; this section decides what you can claim about where it came from. A bare channel written onto a record gets treated as fact by everyone downstream. Carry three fields instead of one on every conversion the plan reports - a practice from the production runbook of DevRel practitioner Tessa Kriesel (citations in [references/published-findings.md](references/published-findings.md)):
- `source` - the channel.
- `source_basis` - the evidence type: `journey_linked` (the identity chain resolved), `self_reported` (the human answered), `campaign_window` (a heuristic guess).
- `source_confidence` - judged on the quality of the evidence, never read off the basis. A journey-linked touch that arrived with a malformed tag is lower confidence than a specific self-report; a vague self-report is lower than both. Report basis and confidence together - either one alone gets the whole record distrusted.
Three rules from the same runbook keep the fields honest:
- **Fail closed.** When the merge is ambiguous, record nothing. A missing journey is a gap someone can see and fix; a wrong merge is corruption nobody later detects.
- **Guess narrowly.** The campaign-window heuristic - attributing an unlinked conversion to whatever was running in its window - is a real signal only when three conditions pin it to a single source: the date falls inside exactly one campaign's window, the landing page belongs to that campaign, and its tags were live. Overlapping campaigns or a shared landing page break the pin; write `unknown`. Used bluntly, the heuristic manufactures attribution instead of recovering it.
- **Never overwrite.** Backfill only records whose source is blank, never one already carrying a journey-linked or self-reported value, and keep backfilled history visually separate so it is not read as a trend.
Expect near-zero attributed conversions until the cross-domain stitch is verified in production, then a jump the week it lands. That jump is instrumentation, not growth - annotate it before someone reports it as a win.
## Event taxonomy
For the good/bad naming pattern, see [references/tracking-plan-template.md](references/tracking-plan-template.md#event-naming-good-vs-bad).
Rules that survive a year:
- Write events and properties in lowercase snake_case, so a hand-typed identifier can never mismatch on case. That is this skill's convention, not the framework's - Segment prescribes no casing and rules only that consistency matters.
- Match an existing plan's casing instead of converting it. Consistency is the sourced rule; any particular case never was.
- One event per user intent, never per UI element - renaming a button must not fork the event's history.
- Renaming a shipped event splits its history in two. Add a property instead, or version the plan and note the break.
- Keep the event count small enough that the whole list fits on one screen. A taxonomy nobody can read is a taxonomy nobody follows.
- Emit the same event name for the same intent across surfaces, and let a `surface` property carry the difference.
## Link tagging
- Fix an allowed list for source and medium before the first tagged link. An open vocabulary produces `Twitter`, `twitter`, `X` and `x.com` as four sources inside a month, and no report can merge them afterwards.
- Lowercase everything, one separator character, no spaces.
- Tag only links a third party clicks _into_ your site. Tagging an internal docs link restarts the session and overwrites the real source with your own - the most common self-inflicted attribution wound on a docs site.
- Tag per artifact, not per campaign, whenever the point is comparing artifacts: which post, which sample repo, which event page.
- Register every published tagged link in one table with owner and publish date. An unregistered tag is unreadable six months later.
- Redirect chains, shorteners and OAuth hops must preserve the query string, or the tag dies at the hop.
- Off-web surfaces cannot be tagged at all. Give each talk or episode a short speakable vanity path, each event a QR code resolving to a tagged URL, and keep one short self-reported source field at signup whose answer list stays stable across quarters.
Expect "direct" to remain the largest bucket regardless. Aggregators, chat clients, privacy browsers, PDFs, and slide decks all strip referrers, and roughly 70% of AI-assistant traffic arrives with no referrer at all (Demand Curve #331). A large direct bucket during a launch week is the normal case, not an instrumentation defect.
## Quality gate
The plan is not done when it is written; it is done when it passes. Iterate until all four hold, and report the numbers alongside the plan.
| Check | Threshold | Where the number comes from |
| ------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------- |
| Decision coverage | 100% of events trace to a line on the decision list; anything else is cut | definitional |
| Verified traces | 100% of primary conversion events traced end to end on their real surface | definitional |
| Join coverage | ≥ 80% of primary conversion events resolve to a first touch through the identity spine | **this skill's baseline**, movable by the user |
| Tag well-formedness | 100% of published external links validate against the controlled vocabulary | definitional |
Treat join coverage as the plan's primary health check - simpler and far more diagnostic than reconciling absolute counts between two tools. Measure it monthly; a sudden drop names the surface that broke.
The metric comes from practitioner production use (the same runbook's `journey_linked: false` rate), but no threshold is published anywhere: the 80% is this plan's own floor. State it as such, and replace it with the program's trailing rate once two months of data exist.
Two guardrails on the numbers the plan produces:
- Below roughly 30 events in a period, publish the raw list with dates instead of a rate. Small-N percentages swing wildly, and DevRel counts (talks, external pull requests, enterprise conversations) are naturally small.
- Pair any incentivised volume metric with a quality counter-metric - published posts with organic entries per post, signups with 30-day retention of that cohort - or the number gets gamed the moment it becomes a target.
## Invocation examples
| The user says | What you do | What you hand back |
| ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| "We need a tracking plan for our docs and repo before the launch" | Full workflow; capture baselines before anything else, since the launch destroys the pre-period | The plan document, with the surface register, event table, funnel views and a dated baseline table |
| "Our GitHub traffic numbers vanished" | Skip to the surface register; explain the 14-day window and schedule a daily export | A one-page collection recipe plus the snapshot job's schedule and owner |
| "Docs sessions dropped 40% after the redesign" | Re-verification trace first, before any analysis | A findings table (result, severity, confidence, evidence, owner) - not a traffic explanation |
| "Which blog posts drive signups?" | Link-tagging convention, per-artifact tags, plus the self-reported field; state the direct-bucket ceiling upfront | The tagging vocabulary, the registry table, and the one funnel view that answers it |
One request gets a refusal instead of a deliverable: "set us a target for docs conversion rate." A conversion target borrowed from another program's funnel measures that program's audience, not this one, so decline the borrowed number, route target-setting to `samber/developer-relations-skills@devrel-metrics`, and offer the baseline-capture plan that would produce a real target.
The default deliverable is one markdown document following [references/tracking-plan-template.md](references/tracking-plan-template.md): decisions, surface register, build order, identity spine, event table, property registry, tagging vocabulary, funnel views, baselines, quality-gate results, known gaps. A single-surface request gets the same shape with the irrelevant sections dropped, never a different shape.
## Failure modes
| Symptom | Likely cause | Fix |
| ----------------------------------------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Docs traffic looks tiny next to registry downloads | client-side script blocked by the audience | proxied/first-party collection, or read server/CDN logs |
| Registry downloads spike with no other movement | CI, mirrors and analysis bots counted as installs | switch to the registry's mirror-excluded series where one exists; compare to its own baseline only |
| Repository traffic history gone | forge traffic endpoints keep 14 days | schedule an export from day one; the gap is unrecoverable |
| Every session's source is your own docs | UTM tags on internal links | strip tags from internal links, exclude your own domains from source classification |
| Conversions counted twice | confirmation page reachable by refresh, or two collectors firing | unique event ID per conversion, deduplicate on it |
| Signups have no history before signup | `anonymous_id` not emitted with the signup event | fix the stitch first; every upstream number is unusable until it lands |
| Two tools disagree on the same metric | different definitions, windows, timezones or counting rules | refuse to sum them; publish both with their definitions and reconcile the delta |
| Numbers look great, nobody acts on them | events were chosen from what was easy to collect | rebuild from the decision list in step 1 |
| Attributed conversions jump the week the stitch ships | the spine only started resolving then | annotate the date; compare only within the post-stitch period until a full cycle has passed |
| A verified source was replaced by a channel guess | a backfill job overwrote populated rows | restore, then restrict backfill to blank sources and tag every backfilled row with its basis |
| Strangers merged into one giant account | domain matching included free-mail domains, which identify no company | exclude free-mail-class domains from account matching |
| Every account holds exactly one person | signup domains never resolved to a company | fall back to enrichment or manual matching, and report what share resolved |
## Privacy gate
Clear this before shipping, not after:
- Aggregate anything person-level.
- Keep PII out of properties (hashing is not consent).
- Publish no per-member leaderboard without consent.
- Check a venue's terms before bulk-exporting its content.
- Honour data-retention and deletion obligations on every store you create.
Community and contributor data is person-level by default, which is exactly where this bites.
## Reference
- [references/surface-register.md](references/surface-register.md) - per-surface data source, retention window, bias and collection recipe, plus the B2B account roll-up.
- [references/verification-playbook.md](references/verification-playbook.md) - the end-to-end trace procedure, internal-traffic exclusion, and the re-verification triggers.
- [references/tracking-plan-template.md](references/tracking-plan-template.md) - the output document template with a worked example and a good/bad row pair.
- [references/published-findings.md](references/published-findings.md) - every sourced figure with its citation, this skill's own baselines, the claims to refuse, and a negative example of a baseline laundered into a standard.
- `samber/developer-relations-skills@developer-journey-map` - for the stage model the funnel views should mirror.
- `samber/developer-relations-skills@developer-community-health` - for community metric definitions; instrument them here, define them there.
- `samber/developer-relations-skills@docs-seo` - for search-side measurement of a docs site, which uses a source this plan does not cover.
- `samber/developer-relations-skills@developer-quickstart-guide` - for the first-success funnel this plan's enablement events feed.