+1

Coordination.md

reviewed

··· 59 59 - Cannot be detected by looking at actions alone, as the same actions can be legitimate or collusive depending on context and intentions. 60 60 - Requires specific countermeasures like deliberate decentralization, skin-in-the-game mechanisms, and fostering counter-coordination (e.g., forking in blockchain systems). 61 61 - Examples include price fixing between sellers, vote buying, and 51% attacks on blockchains. 62 62 + - Any mechanism that helps under-coordinated parties coordinate will also help already-coordinated parties over-coordinate to extract value. 62 63 - Explore [Cooperative Governance](https://nadia.xyz/voting) ideas. It offers an elegant compromise between on-chain and off-chain governance: you have a failure outcome that's enforced and "real", but the process of consensus remains firmly in the hands of people. 63 64 - There are [interventions](https://www.prosocialdesign.org/) to help online spaces bring out the best in human nature online (deleting a comment that breaks a forum's rules, a reminder of the community rules, ...). This helps build trust and coordination.

+133

Impact Evaluators.md

reviewed

··· 1 1 + # Impact Evaluators 2 2 + 3 3 + It's hard to do [[Public Goods Funding]], open-source software, research, etc. that don't have a clear, immediate financial return, especially high-risk/high-reward projects. 4 4 + 5 5 + Traditional funding often fails here. Instead of just giving money upfront (prospectively), Impact Evaluators create systems that look back at what work was actually done and what impact it actually had (retrospectively). It's much easier to judge the impact in a retrospective way! 6 6 + 7 7 + - The setup is similar to Control Theory. Based on measuring and evaluating this impact against predefined goals, the system then distributes rewards (e.g: similar to how BitCoin block rewards do it). 8 8 + - The Impact Evaluator goal is to **create strong incentives for people/teams to work on valuable, uncertain things** by promising a reward if they succeed in creating demonstrable impact. 9 9 + - They work well on concrete things that you can turn into measurable stuff. They are powerful and will always overfit. When the goal is not exactly aligned, they can be harmful. Eg. Bitcoin wasn't created to maximize the energy consumption. 10 10 + - They should be flexible as it's hard to predict ways the evaluation metrics will be gamed. 11 11 + - Allow Community Feedback Mechanisms. Implement robust feedback systems that allow participants to report and address concerns about the integrity of the metrics or behaviors in the community. This feedback can be used to refine and improve the system continuously. 12 12 + - Use hard to game metrics. Metrics that are hard, or expensive, to game, will be resistant (but not immune) to goodharts law. 13 13 + - Use diversified metrics. Rather than relying on a single metric or indicator, use a diverse set of metrics to assess performance or impact. 14 14 + - Dynamic and Adaptive Metrics. Implement mechanisms that allow for metrics to be adjusted or replaced as the system evolves. This adaptability can help prevent the gaming of static metrics and ensure that measures continue to align with the underlying goals of the IE. 15 15 + 16 16 + Impact Evaluators are frameworks for [[Coordination|coordinating]] work and aligning [[Incentives]] in complex [[Systems]]. They provide mechanisms for retrospectively evaluating and rewarding contributions based on actual impact, helping solve coordination problems in [[Public Goods Funding]], research evaluation, and decentralized systems. 17 17 + 18 18 + ## Core Principles 19 19 + 20 20 + - **Retrospective evaluation beats prospective promises**. Judge work by its actual impact, not promised outcomes. This reduces speculation and gaming while rewarding genuine value creation. 21 21 + - **Credible neutrality through simplicity**. [The simpler a mechanism, the less space for hidden privilege](https://vitalik.eth.limo/general/2020/09/11/coordination.html). Fewer parameters mean more resistance to corruption and overfit. 22 22 + - **Comparative truth over absolute metrics**. [Pairwise comparisons and rankings are more reliable than absolute scoring](https://anishathalye.com/designing-a-better-judging-system/). Humans excel at relative judgments. 23 23 + - **Permissionless scalability**. Anyone should be able to [fork the evaluation system with their own criteria](https://vitalik.eth.limo/general/2024/09/28/alignment.html), preventing capture and enabling experimentation. 24 24 + - **Legible impact attribution**. Make contributions and their value visible. [Transform vague notions of "alignment" into measurable criteria](https://vitalik.eth.limo/general/2024/09/28/alignment.html) that projects can compete on. 25 25 + 26 26 + ## Design Considerations 27 27 + 28 28 + - **Start local and iterate**. Begin with small communities defining their own [[Metrics]] and evaluation criteria. Use rapid [[Feedback Loops]] to learn what works before scaling up. 29 29 + - Each community understands its context better than outsiders ([seeing like a state blinds you to local realities](https://slatestarcodex.com/2017/03/16/book-review-seeing-like-a-state/)) 30 30 + - Local experiments surface patterns for higher-level systems 31 31 + - Small groups enable iterated games that reward trust and penalize defection 32 32 + - **Build anti-Goodhart resilience**. Any metric used for decisions [becomes subject to gaming pressures](https://en.wikipedia.org/wiki/Campbell%27s_law). Design for evolution: 33 33 + - Run multiple evaluation algorithms in parallel and let humans choose 34 34 + - Use exploration/exploitation trade-offs (like multi-armed bandits) to test new metrics 35 35 + - Make the meta-layer for evaluating evaluators explicit 36 36 + - **Separate data from judgment**. [Impact Evaluators work like data-driven organizations](https://handbook.davidgasquez.com/data/data-culture): 37 37 + - Gather objective attestations about work (commits, usage stats, dependencies) 38 38 + - Apply multiple "evaluation lenses" to interpret the data 39 39 + - Let funders choose which lenses align with their values 40 40 + - **Design for composability**. Define clear data structures (graphs, weight vectors) as APIs between layers. This enables: 41 41 + - Multiple communities to share measurement infrastructure 42 42 + - Different evaluation methods to operate on the same data 43 43 + - Evolution through recombination rather than redesign 44 44 + 45 45 + ## Implementation Challenges 46 46 + 47 47 + - **Incomplete contracts problem**. [It's expensive to measure what really matters](https://meaningalignment.substack.com/p/market-intermediaries-a-post-agi), so we optimize proxies that drift from true goals. 48 48 + - Current markets optimize clicks and engagement over human flourishing 49 49 + - The more powerful the optimization, the more dangerous the misalignment 50 50 + - **Information elicitation without verification**. Getting truthful data from subjective evaluation when you can't verify it requires clever [[Mechanism Design]]: 51 51 + - [Peer prediction mechanisms](https://jonathanwarden.com/information-elicitation-mechanisms/) that reward agreement with hidden samples 52 52 + - [Bayesian Truth Serum](https://www.science.org/doi/10.1126/science.1102081) that uses both answers and predictions 53 53 + - Coordination games where truth serves as a Schelling point 54 54 + - **Collusion resistance**. Any mechanism helping under-coordinated parties will also help [over-coordinated parties extract value](https://vitalik.eth.limo/general/2019/04/03/collusion.html). Countermeasures include: 55 55 + - Identity-free incentives (like proof-of-work) 56 56 + - Fork-and-exit rights for minorities 57 57 + - Privacy pools that exclude provably malicious actors 58 58 + - Multiple independent "dashboard organizations" preventing capture 59 59 + 60 60 + ## Mechanism Toolkit 61 61 + 62 62 + - **Staking and slashing**. Require deposits that get burned for misbehavior. Simple but requires upfront capital. 63 63 + - **Pairwise comparison engines**. Convert human judgments into weights using [Elo ratings or Bradley-Terry models](https://www.keiruaprod.fr/blog/2021/06/02/elo-vs-bradley-terry-model.html). 64 64 + - **Quadratic mechanisms**. [Funding](https://vitalik.eth.limo/general/2019/12/07/quadratic.html) and voting that make influence proportional to square root of resources, reducing plutocracy. 65 65 + - **Prediction and decision markets**. ["Vote values, bet beliefs"](https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c) - use conditional markets to choose policies maximizing agreed metrics. 66 66 + - **Token-curated registries**. Stakeholders deposit tokens to curate lists, with challenges and slashing for bad entries. 67 67 + - **Augmented bonding curves**. Automated market makers that route fees to upstream dependencies based on usage. 68 68 + - **Deliberative protocols**. [Structured discussion processes](https://jonathanwarden.com/deliberative-consensus-protocols/) that surface information before voting. 69 69 + 70 70 + ## Related Fields 71 71 + 72 72 + - Social Choice Theory 73 73 + - Mechanism Design 74 74 + - Computational Social Choice 75 75 + - Game Theory 76 76 + - Machine Learning 77 77 + - Voting Theory 78 78 + - Process Control Theory 79 79 + - LLM Evals 80 80 + 81 81 + ## The Path Forward 82 82 + 83 83 + - **Embrace plurality over perfection**. [No single mechanism can satisfy all desirable properties](https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem) (efficiency, fairness, incentive compatibility, budget balance). Different contexts need different trade-offs. 84 84 + - **Make evaluation infrastructure permissionless**. Just as anyone can fork code, anyone should be able to fork evaluation criteria. This prevents capture and enables innovation. 85 85 + - **Focus on error analysis**. Like in [LLM evaluations](https://hamel.dev/blog/posts/evals-faq/), understanding failure modes matters more than optimizing metrics. Study what breaks and why. 86 86 + - **Layer human judgment on algorithmic engines**. The ["engine and steering wheel" pattern](https://vitalik.eth.limo/general/2025/02/28/aihumans.html) - let algorithms handle scale while humans set direction and audit results. 87 87 + 88 88 + Impact Evaluators are powerful but dangerous. Like nuclear reactors, they can solve major [[Coordination]] problems when designed well, but cascade failures are catastrophic. Start small, fail safely, and always maintain [credible exit options](https://newsletter.squishy.computer/p/soulbinding-like-a-state). 89 89 + 90 90 + ## Resources 91 91 + 92 92 + - [Generalized Impact Evaluators](https://research.protocol.ai/publications/generalized-impact-evaluators/ngwhitepaper2.pdf) 93 93 + - [A Flexible Design for Funding Public Goods](https://arxiv.org/pdf/1809.06421) 94 94 + - [A Budget-balanced, Incentive-compatible Scheme for Social Choice](https://infoscience.epfl.ch/server/api/core/bitstreams/e929c9a6-d7d4-453b-a200-20ad93b64052/content) 95 95 + - [Introduction to Impact Evaluators](https://youtu.be/TdDHWv00Z4E) 96 96 + - [Impact Evaluator Design](https://youtu.be/1soPQ31ZHkQ) 97 97 + - [Impact Evaluators](https://youtu.be/dpLtrugjfMc) 98 98 + - [Generalized Impact Evaluators, A year of experiments and theory](https://research.protocol.ai/blog/2023/generalized-impact-evaluators-a-year-of-experiments-and-theory/) 99 99 + - [Elo vs Bradley-Terry Model](https://www.keiruaprod.fr/blog/2021/06/02/elo-vs-bradley-terry-model.html) 100 100 + - [Designing a Better Judging System](https://anishathalye.com/designing-a-better-judging-system/) 101 101 + - [Deliberative Consensus Protocols](https://jonathanwarden.com/deliberative-consensus-protocols/) 102 102 + - [Credible Neutrality](https://balajis.com/p/credible-neutrality) 103 103 + - [Quadratic Payments: A Primer](https://vitalik.eth.limo/general/2019/12/07/quadratic.html) 104 104 + - [Quadratic vs Pairwise](https://blog.zaratan.world/p/quadratic-v-pairwise) 105 105 + - [Quadratic Funding is Not Optimal](https://jonathanwarden.com/quadratic-funding-is-not-optimal/) 106 106 + - [A Mild Critique of Quadratic Funding](https://kronosapiens.github.io/blog/2019/12/13/mild-critique-qf.html) 107 107 + - [Funding impact via milestone markets](https://docs.fileverse.io/0x0D97273dee4D1010321f9eBa2e9eaB135C17D6dE/0#key=5GgcacTDy2h1QwWV9vJqGD-YzwomzuIOueMACpjghbJLxfG3ZqbWl1qDC1Le04uR) 108 108 + - [Kafka Index](https://summerofprotocols.com/wp-content/uploads/2024/04/Kafka-Index-Nadia-Asparouhova-1.pdf) 109 109 + - [The Unreasonable Sufficiency of Protocols](https://summerofprotocols.com/the-unreasonable-sufficiency-of-protocols-web) 110 110 + - [Good Death](https://summerofprotocols.com/research/good-death) 111 111 + - [Retroactive Public Goods Funding](https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c) 112 112 + - [The Public Goods Funding Landscape](https://splittinginfinity.substack.com/p/the-public-goods-funding-landscape) 113 113 + - [Coordination, Good and Bad](https://vitalik.eth.limo/general/2020/09/11/coordination.html) 114 114 + - [On Collusion](https://vitalik.eth.limo/general/2019/04/03/collusion.html) 115 115 + - [Remuneration Rights](https://openrevolution.net/remuneration-rights) 116 116 + - [AI as the engine, humans as the steering wheel](https://vitalik.eth.limo/general/2025/02/28/aihumans.html) 117 117 + - [Central Planning as Overfitting](https://vitalik.eth.limo/general/2018/11/25/central_planning.html) 118 118 + - [Info Finance](https://vitalik.eth.limo/general/2024/11/09/infofinance.html) 119 119 + - [Plurality and Alignment](https://vitalik.eth.limo/general/2024/09/28/alignment.html) 120 120 + - [Privacy in Evaluation Systems](https://vitalik.eth.limo/general/2025/04/14/privacy.html) 121 121 + - [The Case for Plurality](https://vitalik.eth.limo/general/2024/08/21/plurality.html) 122 122 + - [Community Notes: An Example That Works](https://vitalik.eth.limo/general/2023/08/16/communitynotes.html) 123 123 + - [Open Source Funding](https://vitalik.eth.limo/general/2025/03/29/pubos.html) 124 124 + - [Soulbinding Like a State](https://newsletter.squishy.computer/p/soulbinding-like-a-state) 125 125 + - [Market Intermediaries in a Post-AGI World](https://meaningalignment.substack.com/p/market-intermediaries-a-post-agi) 126 126 + - [Goodhart's Law Not Useful](https://commoncog.com/goodharts-law-not-useful/) 127 127 + - [Ten Kilograms of Chocolate](https://medium.com/@florian_32814/ten-kilograms-of-chocolate-75c4fa3492b6) 128 128 + - [Bittensor's Anatomy of Incentive Mechanism](https://docs.bittensor.com/learn/anatomy-of-incentive-mechanism) 129 129 + - [Frequently Asked Questions (And Answers) About AI Evals](https://hamel.dev/blog/posts/evals-faq/) 130 130 + - [Proportionally fair online allocation of public goods with predictions](https://dl.acm.org/doi/abs/10.24963/ijcai.2023/3) 131 131 + - [A natural adaptive process for collective decision-making](https://onlinelibrary.wiley.com/doi/10.3982/TE5380) 132 132 + - [Tournament Theory: Thirty Years of Contests and Competitions](https://www.researchgate.net/publication/275441821_Tournament_Theory_Thirty_Years_of_Contests_and_Competitions) 133 133 + - [Are We Learning Yet? A Meta-Review of Evaluation Failures Across Machine Learning](https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/file/757b505cfd34c64c85ca5b5690ee5293-Paper-round2.pdf)

+2

Mechanism Design.md

reviewed

··· 36 36 - [Sam Harsimony's Substack](https://substack.com/@splittinginfinity) 37 37 - [Victor Sint Nicolaas' blog](https://victorsintnicolaas.com/) 38 38 - [Allocation Mechanisms](https://www.allo.expert/mechanisms) 39 39 + - [Generalized Impact Evaluators](https://research.protocol.ai/publications/generalized-impact-evaluators/ngwhitepaper2.pdf) - Framework for retrospective reward mechanisms 40 40 + - [Info Finance](https://vitalik.eth.limo/general/2024/11/09/infofinance.html) - Using information aggregation for social decisions

-15

Public Goods Funding.md

reviewed

··· 27 27 - [Publig Goods Funding Mechanism List](https://docs.google.com/document/d/1n8fP3tWfLBIa-w4wC0rbOw2wo7odWvwseWptbuBdg6c/edit?tab=t.0) 28 28 - [List of Public Goods Funding Mechanisms](https://harsimony.wordpress.com/2022/02/10/list-of-public-goods-funding-mechanisms/) 29 29 - [Funding public goods using the Nash product rule](https://victorsintnicolaas.com/funding-public-goods-using-the-nash-product-rule/) 30 30 - 31 31 - ## Impact Evaluators 32 32 - 33 33 - It's hard to fund important things like public goods, open-source software, research, etc. that don't have a clear, immediate financial return, especially high-risk/high-reward projects. 34 34 - 35 35 - Traditional funding often fails here. Instead of just giving money upfront (prospectively), Impact Evaluators create systems that look back at what work was actually done and what impact it actually had (retrospectively). It's much easier to judge the impact in a retrospective way! 36 36 - 37 37 - - The setup is similar to Control Theory. Based on measuring and evaluating this impact against predefined goals, the system then distributes rewards (e.g: similar to how BitCoin block rewards do it). 38 38 - - The Impact Evaluator goal is to **create strong incentives for people/teams to work on valuable, uncertain things** by promising a reward if they succeed in creating demonstrable impact. 39 39 - - They work well on concrete things that you can turn into measurable stuff. They are powerful and will always overfit. When the goal is not exactly aligned, they can be harmful. Eg. Bitcoin wasn't created to maximize the energy consumption. 40 40 - - They should be flexible as it's hard to predict ways the evaluation metrics will be gamed. 41 41 - - Allow Community Feedback Mechanisms. Implement robust feedback systems that allow participants to report and address concerns about the integrity of the metrics or behaviors in the community. This feedback can be used to refine and improve the system continuously. 42 42 - - Use hard to game metrics. Metrics that are hard, or expensive, to game, will be resistant (but not immune) to goodharts law. 43 43 - - Use diversified metrics. Rather than relying on a single metric or indicator, use a diverse set of metrics to assess performance or impact. 44 44 - - Dynamic and Adaptive Metrics. Implement mechanisms that allow for metrics to be adjusted or replaced as the system evolves. This adaptability can help prevent the gaming of static metrics and ensure that measures continue to align with the underlying goals of the IE.