Internal Developer Platform

What Developers Actually Ask For

The developer survey came back and the top answer was better documentation. The platform team spent a quarter on documentation. The next survey came back and the top answer was better documentation.


The Problem at Scale

Two sources of truth about what developers need, and they disagree:

  the survey        what people say when asked in the abstract
  the ticket queue  what people were blocked on badly enough
                    to ask a human for help

The survey is a popularity contest among complaints. The ticket queue is a record of actual blockage, timestamped and costed. When they disagree, the queue is the better specification, and it is the one nobody reads systematically because reading three hundred tickets is tedious in a way that reading a survey summary is not.

KEY CONCEPT

Requests for documentation are usually not requests for documentation. They are requests to not have to know something, arriving in the only vocabulary the developer has for that. A team asking how to configure a readiness probe correctly is telling you the probe should have had a sensible default, and answering with a better written page about probes leaves the underlying request unserved while appearing to address it.


How It Works

Classify by what the request implies

The useful move is to translate each request into the platform change that would prevent it.

  "how do I set memory limits"
      -> a default that is right for most services

  "my deploy is stuck"
      -> the status is not visible where they are looking

  "who owns the payments service"
      -> the catalog is missing, wrong, or unreachable

  "can you give me a staging database"
      -> a self service path does not exist

  "how do I do X"  (asked five times by five teams)
      -> X should not require knowing

The last pattern is the highest value signal in the queue. A question asked once is a gap in someone's knowledge. The same question asked by five separate teams is a design defect, and the number of askers is the evidence.

Volume times cost, again

From the previous lesson, but applied to prioritisation rather than to the build decision:

  request type        volume   hours each   total
  ---------------------------------------------------
  new service          40        4.0        160
  namespace            90        0.3         27
  database             12        8.0         96
  access change       120        0.2         24
  one off debugging     25       3.0         75

New service onboarding is the golden path, and this table is how you find that out rather than by assuming it. Notice that namespaces are the highest volume item and nearly the lowest total cost, which means automating them feels productive and moves very little.

What the queue does not contain

Two categories are systematically missing, and both matter.

Requests nobody bothered to make. Teams that worked around the platform instead of asking. The evidence is elsewhere: a team running its own database in a namespace, a service deployed by a pipeline nobody registered. Those are unserved needs that never reached the queue.

Requests from the teams who can help themselves. Your most senior teams do not file tickets, so the queue over represents the needs of teams new to the platform. Building only from the queue produces a platform that serves beginners well and gives experienced teams nothing, which is how you lose the people whose adoption would persuade everyone else.

The loudest team is not the median team

One team with a difficult requirement and a persistent tech lead can consume a disproportionate share of a roadmap. The check is whether the request is common or merely loud:

  how many distinct teams have asked for this
  what fraction of services would use it
  what happens to them if you do not build it

A requirement from one team that no other team shares is a case for helping that team directly, not for putting it in the platform. Platform features have a permanent maintenance cost and a one off benefit if only one consumer exists.


Building and Operating It

Make the queue analysable, which mostly means tagging it at intake.

# Intake form fields that make the queue a specification rather than
# a pile of prose. Without the type and the team, the analysis in
# this lesson is a manual read of three hundred tickets.
fields:
  - name: request_type
    type: select
    options: [new_service, namespace, database, access, deploy_issue,
              cost_question, other]
  - name: team
    type: select
    source: service_catalog.teams
  - name: blocked
    type: boolean
    label: "Is this blocking work right now?"

Then read it as a specification.

-- The question asked by many distinct teams is a design defect.
-- The number of distinct askers is the evidence, and it separates
-- a knowledge gap from a missing default.
SELECT request_type,
       count()                         AS volume,
       count(DISTINCT team)            AS teams_asking,
       sum(hours_spent)                AS total_hours,
       countIf(blocked) / count()      AS blocked_share
FROM platform_requests
WHERE created_at > now() - INTERVAL 90 DAY
GROUP BY request_type
HAVING teams_asking > 3
ORDER BY total_hours DESC;

Look for the work that bypassed you entirely.

# Services running in the cluster that the catalog does not know
# about. Each one is a need that was met without you, which is a
# requirement the ticket queue will never show.
kubectl get deploy -A -o json \
| jq -r '.items[] | "\(.metadata.namespace)/\(.metadata.name)"' \
| sort > /tmp/running.txt
catalog list services --format=namespace/name | sort > /tmp/known.txt
comm -23 /tmp/running.txt /tmp/known.txt

And check whether a request is common or loud before it reaches a roadmap.

-- One team asking ten times is a support problem. Ten teams asking
-- once is a platform requirement.
SELECT team, count() AS asks
FROM platform_requests
WHERE request_type = '{{TYPE}}' AND created_at > now() - INTERVAL 180 DAY
GROUP BY team ORDER BY asks DESC;
WAR STORY

A platform team ran the same developer survey twice a year and documentation was the top complaint both times, so the second quarter went into a documentation rewrite that changed nothing. Tagging the ticket queue afterwards showed that the largest single category was questions about resource limits and probe configuration, asked by eleven different teams, none of whom wanted to learn the answer. The eventual fix was a set of defaults applied at scaffolding time plus a validating admission policy that rejected the four configurations that actually caused incidents. Question volume in that category dropped by roughly three quarters, and the next survey no longer had documentation at the top, despite no further documentation being written.


Tradeoffs and Decision Framework

Request patternWhat it meansResponse
Same question, many teamsA default is missingChange the default
Same question, one teamA knowledge gapAnswer it, do not build
"How do I configure X"X should be configured for themSensible default plus policy
"Where is X"Catalog gapFix the catalog, not the docs
"Can you provision X"No self service pathAutomate if volume justifies
Work done outside the platformAn unmet need that never filedGo and ask them
SourceStrengthWeakness
Ticket queueReal blockage, costedMisses teams who work around you
SurveyBroad reachMeasures annoyance, not blockage
Shadow infrastructureReveals unmet needsRequires going to look
Senior teamsHighest leverage adoptionThey never file tickets

Three questions when reading demand. How many distinct teams asked, because that separates a design defect from a knowledge gap. What is the volume times cost, since that orders the work. And what is running that you do not know about, as that is the demand that never reached you.

Default: tag requests by type and team at intake, prioritise by total hours with at least four distinct teams asking, translate each recurring question into the default or path that would prevent it, and periodically diff what is running against what the catalog knows.


Failure Modes and Common Mistakes

Building from the survey. It measures annoyance, not blockage.

Answering documentation requests with documentation. The request is usually to not have to know.

Automating the highest volume item. Volume without cost moves very little.

Serving the loudest team. Check how many other teams share the requirement.

Reading only the queue. The teams who work around you never appear in it.

Ignoring senior teams. They file no tickets and their adoption persuades everyone else.

KNOWLEDGE CHECK

Eleven separate teams have each asked how to configure resource limits and readiness probes correctly. What does the number of distinct teams tell you that the volume alone does not?

INTERVIEW QUESTION

Your developer survey says the biggest problem is documentation. What do you do with that?