We Ran Our Own Validation Tool Against Itself — Here's Who Actually Needs It

Published May 1, 2026 · Modest Idea · 9 min read · View the full segment data →

We built a tool that tells founders who needs their product. So we asked: who needs our tool?

The obvious answer — "founders" — is useless. There are solo founders on their first idea with three months of savings and no engineering background. There are VC-backed teams with 10 engineers, a product manager, and a structured discovery process. Both are "founders." Their need for pre-build audience validation couldn't be more different.

So we did what the tool is supposed to do. We described Modest Idea as a product concept, ran it through 250 Census-grounded synthetic personas, and got segment-level PSF scores back. The results confirmed some assumptions and inverted others.

How We Did It

The product description we submitted: "A pre-launch audience discovery tool that helps founders identify which demographic segments experience the problems their product solves — using 250 Census-grounded synthetic personas and multi-model AI evaluation to produce segment-level PSF scores with reasoning."

The analysis ran 250 personas through the full pipeline: problem recognition scoring, pain severity scoring, solution gap assessment, and final PSF calculation. Each persona was evaluated by multiple AI models at varying temperatures to reduce systematic bias. The 250 personas were IPF-weighted to match US Census distributions, so the results reflect the real distribution of founder types — not just the ones most visible in startup media.

The analysis clustered results into 8 segments based on shared characteristics and PSF patterns. Here's what came back.

The Results: Who Scores What

Segment PSF Score
Solo Non-Technical Founders (first build) 87
Side-Project Builders (employed, validating weekends) 78
Indie Hackers / Solo SaaS Builders 72
Non-Technical Founder Teams (2–3 people) 65
Student / Recent Grad Founders 52
Corporate Innovation Teams 44
Technical Solo Founders (can code) 38
VC-Backed Teams (5+ engineers) 31

See the full segment analysis page for detailed reasoning on each score.

What's Driving the Top Scores

The three highest-scoring segments share one structural characteristic: the cost of being wrong is high and the ability to course-correct is low.

Solo non-technical founders (87): No engineering background means no ability to prototype in a weekend. They can't build a quick version, watch how people use it, and pivot. Every decision to build something is a weeks-long commitment involving either learning new skills or paying contractors. Being wrong about who the customer is doesn't cost a sprint — it costs months of runway and money they can't recover. The existential math makes validation worth almost any price. Maximum need, minimum course-correction capacity.

Side-project builders (78): The constraint isn't money — it's time. Someone with a day job and a family gets 10–15 hours a week on their idea. Spending three months building for the wrong segment is three months of those scarce hours permanently gone. Validation is decision gating: "Is this worth my next 200 hours?" They also face a threshold decision — at some point, the evidence has to be good enough to quit the job. A segment-level PSF analysis gives them something more concrete than "people on Twitter said it sounds cool."

Indie hackers / solo SaaS builders (72): Revenue-dependent. Wrong product choice = no revenue. The indie hacker culture already understands this framing — they've read the niche selection literature, they know that picking the right problem is the highest-leverage decision, and they actively seek validation frameworks. Strong willingness to pay for niche clarity. They can pivot faster than non-technical founders, but the revenue stakes are direct.

The Pattern

High PSF in all three cases traces to the same root: limited resources (time, money, or both) create a high cost of being wrong. Validation has maximum ROI when the downside of the alternative — just building and seeing — is highest. These are precisely the founders who can least afford to learn by doing.

The Counterintuitive Finding: The Inversion Problem

Here's the finding that complicates Modest Idea's own go-to-market.

VC-backed teams score 31. Corporate innovation teams score 44. Technical solo founders score 38. These are the segments with the highest capacity to pay — they have budgets, expense accounts, or at minimum stable income while they build. But they score lowest on PSF.

Why? VC-backed teams can afford to build, launch, and learn. They have engineers who can prototype in a week, a product manager running structured discovery, and investors who explicitly expect them to iterate toward PMF rather than getting it right on the first try. Pre-build audience validation doesn't fit their process — they'd rather ship and see. The risk that makes validation valuable (being wrong is very expensive) simply isn't present.

Technical founders who can code are similar. They can build a version in a weekend. The cost of being wrong is low — they'll know quickly, and they'll just build something else. Validation competes with their natural feedback loop of "ship and measure."

56pt
PSF gap between solo non-technical founders (87) and VC-backed teams (31). Both are "founders." They are not the same market.

The inversion: the people who need validation most are the ones least able to afford being wrong — and the ones who can afford being wrong already have validation processes. Maximum need does not correlate with maximum ability to pay.

This is the central tension in Modest Idea's business model. The tool's value is highest for people with limited runway who can't experiment freely. But those people also have the lowest ability to pay for tools. The customers who could comfortably pay $200/month for a validation tool are the ones who don't really need it.

What This Means for Founders (Not Just for Modest Idea)

The broader lesson here applies to any product targeting "founders" or "builders" as an audience.

The word "founder" spans an enormous range of situations. A 22-year-old with a YC acceptance letter and $500K in the bank is a founder. A 38-year-old working nights on a SaaS idea while keeping their day job is also a founder. A former consultant building their first startup at 45 with personal savings is a founder. The problems they need solved, the urgency they feel, and the price they can pay are completely different.

The most common mistake when building for "founders" is targeting the most visible subgroup — the VC-funded, conference-attending, Twitter-posting startup ecosystem — because they're the easiest to reach and they sound like the market. But this analysis suggests that group has the lowest PSF for a validation tool. They have money, engineers, time, and processes. They don't need to validate before they build because the downside of being wrong is manageable.

The founder who needs validation most is invisible in that ecosystem. They're working a day job and building on weekends. They're a first-time non-technical founder with six months of savings. They're a solo SaaS builder who's learned that niche selection is the hardest part. They're not at the conference. They're in the Discord server. They're on the subreddit. They're reading this blog post.

If you're building for founders, this inversion applies to you too. Who specifically? At what stage? With what constraints? The segment matters more than the category.

Honest Limitations

This is a synthetic analysis. It shows who has the problem — not who will pay, not who will convert, not what price point they'll accept.

The PSF scores reflect problem recognition and solution gap as modeled by 250 synthetic personas grounded in Census data. Real founders have specific behaviors, community affiliations, and discovery paths that the personas approximate but don't capture perfectly. A solo non-technical founder in, say, a healthcare services business might be meaningfully different from one building a consumer app — they're both "solo non-technical founders" but the context shapes their validation needs differently.

We haven't validated this with real customer interviews yet. That comes next. The analysis tells us who to talk to — the top three segments. It tells us what to listen for — existential risk, time scarcity, niche selection anxiety. But it doesn't replace the conversation.

The inversion finding especially needs validation. The claim that maximum-need customers have minimum willingness-to-pay is a structural argument, not a measured one. It could be wrong: maybe solo non-technical founders are willing to pay more than we think because the stakes are so high. Maybe there's a corporate segment we've underscored. Customer interviews with the top two or three segments will either confirm the inversion or reveal where the model's reasoning breaks down.

We're publishing this now, before those interviews, because the analysis itself is useful — and because watching a product analyze itself has some entertainment value. The full segment analysis is available if you want to dig into the detailed reasoning for each score.

And if you're a solo non-technical founder building your first product — the tool works. Try it.

Get the Free PSF Framework

A 5-step process for evaluating problem-solution fit, with scoring templates and real case studies from 250-persona analyses.

Get the Free Guide →

Run your own analysis

Get PSF scores for your product concept across 250 Census-grounded personas and 6–8 demographic segments.

Start your analysis →
← Back to Blog