Same prompt. Specific exemplars. Human preference labels. Transductive evaluation.
Independent research + product architecture / Eduardo Cortes / July 2026
The Selector Remains Human
Tools should help people curate exemplars, preserve context, express disagreement, compare concrete alternatives, and monitor drift. That is the main engineering implication of this project. I reached it after an AI-native media pipeline exposed selection as the bottleneck, then generic mechanical proxies failed to replace human-shaped judgment across poetry and LitBench.
The 87.7% result shown below is a transductive, same-prompt probe built from other test labels. It is evidence of a strong local boundary—not a transferable creative evaluator.
MiniLM against same-prompt preference pools · n=1,155
Averaging the same local domain erased the usable boundary.
Preference-labeled domains did not transfer into a general selector.
Evidence / what the experiments changed
The metric was not the miracle. The comparison set was.
A conditional language-model readout initially looked like a strong compression-based selector. A crossed operator control changed the interpretation: directional MiniLM performed better on the identical rows, while centroid pooling failed.
The strong boundary requires prompt-local preference labels.
Randomizing labels within the same prompt collapses to chance. Moving real preference labels across prompts also collapses toward chance. The result is local and label-defined, not a free-standing value function.
The surface-format baseline is 60.8%. The 87.7% row is not comparable to an inductive reward-model score because its domains contain labels from other held-out test items.
Can compression replace appraisal?
On human-rated poetry, generic literary compression did not recover human ratings and could anti-align under matched controls.
Was the strong LitBench result uniquely linguistic?
No. Directional MiniLM reached 87.7% on the same 1,155 rows where the conditional-LM readout reached 79.7%.
Was sharing a prompt sufficient?
No. Random same-prompt domains fell to 48.9%. The preference labels—not prompt membership alone—were load-bearing.
Did human-shaped domains generalize?
Not across prompts. Train- and test-labeled cross-prompt domains stayed near 53% and did not become a general selector.
Framework / anatomy of a selector
Creativity as search without a writable value function.
This is an engineering model, not a universal definition: a generator proposes; a selector supplies value-ordering. The selector is defined by where its authority comes from and how it compares—not by whether it emits a score.
01 / Decorrelation
Independent errors
The selector is useful when its mistakes are not the generator’s mistakes. Judging a realized artifact creates some informational independence even when architecture is shared.
02 / Unfrozenness
Alive at the frontier
A learned judge freezes yesterday’s labels. Creative value moves. Frontier decisions require a source of judgment that can still change with the audience, context, and moment.
03 / Grounding
Where order comes from
The world can ground proofs, compilers, and win conditions. Internal proxies measure features. In open-ended aesthetics, minds remain the tested source of value-ordering.
04 / Reference set
The contrast pool matters
In these experiments, constructing what a candidate was compared against mattered more than the readout family. Human-shaped pools carried the strongest signal; generic domains did not.
05 / Form
Specific and directional
Candidate-to-exemplar relations preserved structure that a pooled centroid destroyed. Taste behaved more like a set of particular comparisons than a single prototype.
06 / Locality
Always for a situation
The prompt-local boundary was strong; the transferred boundary was not. A selector is indexed to a property, population, context, and time—not assumed to be a global aesthetic oracle.
The practical consequence
Human work narrows—and moves upward.
The result is not that people must judge every output. It is that automation did not remove the source of value. Human contribution relocates from per-candidate labor to the conditions under which candidates are judged.
Construct the reference set
Choose the exemplars, contrasts, and specific comparisons that constitute a local taste.
Adjudicate the frontier
Intervene where a frozen proxy is least reliable and novelty makes automation uncertain.
Make high-leverage commitments
Set premises, direction, and path-dependent decisions that downstream execution inherits.
Certify non-drift
Check that the machine’s ordering still resembles the intended human one.
Machine work: generation, diversity, revision execution, inexpensive readout, and the bulk of low-novelty scoring.
Founder path / from production bottleneck to company architecture
I built this because generation stopped being the bottleneck.
This was not research invented after the company. It records the sequence of systems I built, the shortcut I hoped would work, the controls that broke that interpretation, and the product decision that followed.
Abundance exposed the selection problem.
I assembled a reusable pipeline for research, music, image and video synthesis, editing, captioning, and release packaging. Generators could produce many plausible directions; the expensive decision became which direction deserved another round of work.
I separated proposing from judging.
I built a private selection system around concrete candidates, pairwise judgments, remembered constraints, and mutation. The public Idea Search artifact ↗ shows the method without exposing the private application.
I tried to remove the remaining judgment cost.
I tested compression progress, model surprise, conditional language-model readouts, surface controls, TF-IDF, MiniLM, centroid pooling, directional instance-level operators, random labels, cross-prompt transfer, and bootstrap uncertainty.
The selector became a social layer.
The product leap was not “build a better score.” It was to let creators define bounded decisions, let audiences compare actual possibilities, record commitments and fulfillment, and make each consequence part of a persistent property history.
What I personally built
An auditable research system, not a single result.
The repository preserves the path from processed inputs to controls, uncertainty estimates, figures, manuscript, PDF, and this self-contained publication bundle. The strongest number is useful because the failed interpretations around it are also recorded.
Normalized human ratings and pairwise comparisons, tracked frozen inputs, manifests, and machine-readable figure data.
Compression, item surprise, conditional LM, TF-IDF, MiniLM, centroid, and directional per-instance operators.
Identical-row operator crosses, random labels, surface baselines, prompt-local and cross-prompt domains, and bootstrap intervals.
Generated figures, reproducible tables, working manuscript and PDF, public essay, validation scripts, and static deployment bundle.
Research → product / what changed in Manifold
Build after the generator.
Generators are becoming abundant and interchangeable. Manifold is designed around the layer they do not own: who has authority, which possibilities a community can influence, what was promised, and what the eventual release proves happened.
Judge realized alternatives, not an abstract prompt.
Audience decisions operate on specific candidate scenes, songs, characters, or directions. This preserves the instance-level, local structure that pooling erased in the experiments.
Model decision rights instead of treating every vote as truth.
Creators define what is open, the resolution rule, whether the result is binding, and which approval boundaries remain. Audience preference is local governance—not universal artistic quality.
Turn selection into persistent property history.
A Loop Contract records the options, commitment, decision, fulfillment state, and descendant release. The audience can see what its action changed instead of disappearing into a metric.
Evidence boundary: the research explains these architectural choices. It does not prove that audiences will join, return, pay, or prefer Manifold to an incumbent platform. Those are separate product experiments.
Scientific boundary / evidence before mythology
A useful negative—not an impossibility proof.
The framework is a disciplined interpretation of two datasets and a family of tested mechanisms. The site keeps established results, engineering implications, and open philosophical claims separate.
Established here
The tested generic proxies were insufficient.
Generic compression and surprise did not recover appraisal. The strongest LitBench boundary depended on prompt-local human labels and per-instance comparison.
Engineering implication
Invest in the conditions of judgment.
Tools should help people curate exemplars, preserve context, express disagreement, compare concrete alternatives, and monitor drift.
Not claimed
No mechanical value signal can ever exist.
The experiments reject the tested mechanisms as sufficient label-free selectors. They do not prove that every possible world-trained or population-grounded observer must fail.
Working paper / methods, controls, and reproducibility
Read the full argument.
The manuscript documents the compression hypothesis, the negative poetry result, the LitBench operator confound, the local-label boundary, crossed controls, limitations, the founder-to-product interpretation, and the exact reproduction path.