-1

optimism.json

reviewed

··· 1 1 - {"post_stream":{"posts":[{"id":41452,"name":"carl","username":"ccerv1","avatar_template":"/user_avatar/gov.optimism.io/ccerv1/{size}/6309_2.png","created_at":"2024-11-05T12:22:42.346Z","cooked":"\u003cp\u003etl;dr-\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003eIn 2023, Optimism ran two megarounds. In 2024, Optimism ran one round per domain per year. We’ve learned that mega rounds devolve into popularity contests and annual feedback loops are too slow. In 2025, Optimism should focus on fewer domains, iterate more rapidly, and refine what works.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eIn 2023, Optimism had everyone vote on everything. In 2024, Optimism ran experiments around expertise-based voting and metrics-based voting. We’ve learned what humans are good at - and what data is good at. In 2025, Optimism should take the best of both (and set aside the rest).\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eIn both 2023 and 2024, we struggled at measuring the success of Retro Funding itself. The time for not knowing how well this works is over.\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eFull post…\u003c/p\u003e\n\u003ch3\u003e\u003ca name=\"p-41452-this-was-supposed-to-be-simple-1\" class=\"anchor\" href=\"#p-41452-this-was-supposed-to-be-simple-1\"\u003e\u003c/a\u003eThis was supposed to be simple\u003c/h3\u003e\n\u003cp\u003eAs Vitalik \u003ca href=\"https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c\" rel=\"noopener nofollow ugc\"\u003ewrote\u003c/a\u003e when initially describing the mechanism: “The core principle behind the concept of retroactive public goods funding is simple: it’s easier to agree on what was useful than what will be useful.”\u003c/p\u003e\n\u003cp\u003eNow, nearly two years into this experiment, we should be able to look back at the mechanism of Retro Funding itself and apply this same analysis.\u003c/p\u003e\n\u003cp\u003eSpecifically, we want to understand:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eCan people actually agree on what was useful?\u003c/li\u003e\n\u003cli\u003eUnder what circumstances does retroactive funding produce superior outcomes?\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eSimple, right? Not entirely.\u003c/p\u003e\n\u003cp\u003eMy team at OSO has been trying to help Optimism answer some of these questions since \u003ca href=\"https://gov.optimism.io/t/a-review-of-rpgf2-and-ideas-for-how-data-can-improve-future-iterations/6438\"\u003eRound 2\u003c/a\u003e. In this post, we’ll look at a very high level at what’s happened across five rounds of Retro Funding (Rounds 2-5) and share some of our observations.\u003c/p\u003e\n\u003cp\u003eThe good news is there are some clear learnings about what people are good at agreeing on and how to create the right conditions for consensus. These learnings are the result of deliberate experiments that were undertaken in 2024, for instance, experimenting with metrics-based voting in Round 4 and expertise-based voting in Round 5.\u003c/p\u003e\n\u003cp\u003eThe not-so-good news is that we don’t (yet) have the data to show that retroactive funding produces superior outcomes.\u003c/p\u003e\n\u003cp\u003eIn our view, solving the measurement problem is the most important thing to get right in 2025. We need to build an engine for measuring the impact of each allocation cycle—one that gives us more than a vague sense that it’s working and one that gives the collective more than a single round (per domain) per year to see what’s working.\u003c/p\u003e\n\u003cp\u003eOur recommendations include:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eContinuing to move away from project-based voting\u003c/li\u003e\n\u003cli\u003eFinding the right balance of metrics and experts’ subjective assessments\u003c/li\u003e\n\u003cli\u003eFocusing on a relatively narrow set of domains but with more rapid feedback cycles\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eBasically, if 2024 was all about experimenting on expertise-based voting and metrics-based voting, then 2025 should be about combining the best of both. Once we discover the optimal combinations, we can expand the scope and complexity of rounds.\u003c/p\u003e\n\u003ch3\u003e\u003ca name=\"p-41452-what-humans-are-good-at-and-what-data-is-good-at-2\" class=\"anchor\" href=\"#p-41452-what-humans-are-good-at-and-what-data-is-good-at-2\"\u003e\u003c/a\u003eWhat humans are good at (and what data is good at)\u003c/h3\u003e\n\u003cp\u003eOptimism has now completed five rounds that use badgeholder-based voting to allocate tokens.\u003c/p\u003e\n\u003cp\u003eEach round had different design parameters and these have allowed us to learn different things about what humans and data are good at.\u003c/p\u003e\n\u003cp\u003eHere’s a quick summary:\u003c/p\u003e\n\u003cdiv class=\"md-table\"\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eRound\u003c/th\u003e\n\u003cth\u003eDesign Parameters\u003c/th\u003e\n\u003cth\u003eKey Lessons Learned\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003eRound 2\u003c/td\u003e\n\u003ctd\u003eProjects had to be nominated by a badgeholder in order to make the round. Badgeholders had to determine outright token allocations for each of the 200 projects with minimal tooling.\u003c/td\u003e\n\u003ctd\u003eProject nomination process was awkward and time-consuming for badgeholders. Badgeholders needed more structure in the voting process; everyone came up with their own scattershot method (eg, sharing spreadsheets to rate their favorite projects).\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eRound 3\u003c/td\u003e\n\u003ctd\u003eAny project could sign-up for one or more domains and self-report their metrics. Badgeholders would do a light review to filter out spam, but left everything else intact. Projects had to get at least 17 votes in order to receive rewards. Over 600 projects ended up getting approved. Badgeholders still had to determine outright token allocations for each of the 600+ projects. Badgeholders could also create “lists” recommending projects and outright allocations.\u003c/td\u003e\n\u003ctd\u003eVoters felt overwhelmed. It was very difficult to differentiate between weak projects and good ones with little reputation; borderline projects led campaigns to reach the quorum line. The domain categories were not strictly enforced and thus mostly useless to badgeholders. Impact metrics were not comparable. Every list was different and there was no quality control. Onchain projects received a smaller share of the token pool than most badgeholders felt they deserved.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eRound 4\u003c/td\u003e\n\u003ctd\u003eFocused only on onchain builders with strict eligibility requirements (based on onchain activity). Badgeholders were provided metrics instead of projects to vote on. Over 200 projects were approved out of 400+ applicants. Projects could receive a multiplier if they were open source.\u003c/td\u003e\n\u003ctd\u003eBadgeholders found voting much easier. Results had a steep power law distribution, but voters generally felt the top projects were rewarded fairly. Metrics alone couldn’t capture quality, momentum, and other nuances, highlighting the need for more complex evaluation signals. Metrics did not work as well for certain “cutting-edge” sub-domains (eg, 4337 related apps). The open source multiplier was too complex to enforce consistently.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003eRound 5\u003c/td\u003e\n\u003ctd\u003eFocused only on OP Stack contributions with strict eligibility requirements (enforced by a small review team). Returned to project-based voting, grouping voters by expertise and assigning each to a single category of 20-30 projects, rather than hundreds.\u003c/td\u003e\n\u003ctd\u003eResults were very flat, and voters felt the top projects were not rewarded sufficiently. Perverse incentive to divide work across multiple smaller contributions / teams in order to receive more tokens. Grouping by expertise revealed significant differences between experts and non-experts both in project selection and allocation strategy. After seeing both allocations, voters preferred experts’ selections.\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\u003cp\u003eIn Round 6, which is currently underway, Optimism is experimenting with impact attestations and a more aggressive set of distribution options for badgeholders.\u003c/p\u003e\n\u003cp\u003eClearly, the optimal Retro Funding design hasn’t been found yet. But we do see some recurring themes:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eHumans are good at relative comparisons (what’s more valuable)\u003c/li\u003e\n\u003cli\u003eHumans are bad at outright comparisons (how much more valuable)\u003c/li\u003e\n\u003cli\u003eData is good at providing comprehensive coverage of things that are countable\u003c/li\u003e\n\u003cli\u003eData is bad at dealing with nuances and qualitative concepts that experts intuitively understand\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eWe’ve also learned that people only reveal their true opinions after seeing the result. This follows basic product theory: you need to show people something and iterate based on their reactions in order to build something they actually want.\u003c/p\u003e\n\u003ch3\u003e\u003ca name=\"p-41452-metrics-driven-experts-in-the-loop-3\" class=\"anchor\" href=\"#p-41452-metrics-driven-experts-in-the-loop-3\"\u003e\u003c/a\u003eMetrics-driven, experts in the loop\u003c/h3\u003e\n\u003cp\u003eThese hard-earned lessons inform our recommendation for how Optimism approaches future round designs. The goal should be to combine the best parts of what humans and data are good at.\u003c/p\u003e\n\u003cp\u003eHere is the basic framework we propose:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eWe use metrics-based evaluations to propose initial token allocations within a domain.\u003c/li\u003e\n\u003cli\u003eWe let subject matter experts review, fine-tune the metrics, and choose the best allocations.\u003c/li\u003e\n\u003cli\u003eOver time, we identify which metrics best align with experts’ qualitative assessments, refining models through consistent backtesting.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis approach leverages data’s systematic reach with human intuition’s nuanced adjustments. Metrics establish a quantitative foundation, ensuring that projects are assessed objectively and fairly, while expert review adds layers of qualitative nuance, including quality, innovation, and momentum. An iterative feedback loop lets us adjust metrics based on expert insights—particularly valuable when experts consistently revise scores or highlight lower-scoring projects. This process is similar to RLHF (Reinforcement Learning with Human Feedback) in machine learning, but with an emphasis on retaining clear, interpretable inputs for expert adjustments.\u003c/p\u003e\n\u003cp\u003ePractically, we implement this by establishing metrics within a domain, generating proposals for initial allocations (by weighting metrics into an evaluation algorithm), and refining the allocations with expert input. Such an approach should work best in domains with lots of verifiable data, e.g., onchain builders and software dependencies.\u003c/p\u003e\n\u003cp\u003eThis framework should also perform well in fast-evolving domains. Experts can adjust allocations to reflect Optimism’s shifting priorities (e.g., prioritizing interoperability transactions over standard transactions), address data blind spots (e.g., fine-tuning metrics for 4337-related projects), and reward innovation (e.g., favoring fast-growing, high-potential projects over more established but static ones).\u003c/p\u003e\n\u003cp\u003eOne essential element is running funding rounds even more frequently and continuously. Doing so lets us pinpoint metrics most correlated with desirable outcomes and backtest these metrics and evaluation algorithms against historical data. Each round remains an experiment, but the cumulative impact across rounds should reveal a clear, positive trend over time.\u003c/p\u003e\n\u003ch3\u003e\u003ca name=\"p-41452-hard-choices-easy-life-4\" class=\"anchor\" href=\"#p-41452-hard-choices-easy-life-4\"\u003e\u003c/a\u003eHard Choices → Easy Life\u003c/h3\u003e\n\u003cp\u003eThe end goal remains super ambitious: to develop predictive models that guide economic policy for the collective. For instance, in 2025, we may want to discover that incentivizing certain behaviors predictably leads to increased interoper transaction volume.\u003c/p\u003e\n\u003cp\u003eReaching this goal won’t be easy. It requires focus, as the outcomes are highly path-dependent.\u003c/p\u003e\n\u003cp\u003eCurrently, hard choices need to be made around domain scopes. Any changes may be unpopular, especially among community members accustomed to existing Retro Funding patterns. However, narrowing the scope and committing to continuous improvement within scopes are essential steps in reaching the top of the mountain.\u003c/p\u003e\n\u003cp\u003e\u003cdiv class=\"lightbox-wrapper\"\u003e\u003ca class=\"lightbox\" href=\"https://europe1.discourse-cdn.com/bc41dd/original/2X/e/e8c2684c34954a269f489a8549d29f4e6abe8a64.png\" data-download-href=\"/uploads/short-url/xd5hVjQX46YkPSmZgXfvABC8Nrm.png?dl=1\" title=\"image\" rel=\"noopener nofollow ugc\"\u003e\u003cimg src=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/e/e8c2684c34954a269f489a8549d29f4e6abe8a64_2_605x499.png\" alt=\"image\" data-base62-sha1=\"xd5hVjQX46YkPSmZgXfvABC8Nrm\" width=\"605\" height=\"499\" srcset=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/e/e8c2684c34954a269f489a8549d29f4e6abe8a64_2_605x499.png, https://europe1.discourse-cdn.com/bc41dd/original/2X/e/e8c2684c34954a269f489a8549d29f4e6abe8a64.png 1.5x, https://europe1.discourse-cdn.com/bc41dd/original/2X/e/e8c2684c34954a269f489a8549d29f4e6abe8a64.png 2x\" data-dominant-color=\"040404\"\u003e\u003cdiv class=\"meta\"\u003e\u003csvg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#far-image\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003cspan class=\"filename\"\u003eimage\u003c/span\u003e\u003cspan class=\"informations\"\u003e800×661 27 KB\u003c/span\u003e\u003csvg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#discourse-expand\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003c/div\u003e\u003c/a\u003e\u003c/div\u003e\u003c/p\u003e\n\u003cp\u003eLooking back, we’ll likely see some initial assumptions were overly optimistic or naive. But we can’t improve by continuing to “spray and pray.” Governance is ultimately about making hard choices with limited resources.\u003c/p\u003e\n\u003cp\u003eOptimism has spent two important years learning, but it’s time to double-down on what works in 2025.\u003c/p\u003e","post_number":1,"post_type":1,"posts_count":14,"updated_at":"2024-11-05T13:45:34.463Z","reply_count":0,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":139,"reads":106,"readers_count":105,"score":1121.2,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"carl","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":2,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c","internal":false,"reflection":false,"title":"Retroactive Public Goods Funding. Note: The Optimism team has long been… | by Optimism | Optimism PBC Blog | Medium","clicks":5},{"url":"https://gov.optimism.io/t/a-review-of-rpgf2-and-ideas-for-how-data-can-improve-future-iterations/6438","internal":true,"reflection":false,"title":"A review of RPGF2 and ideas for how data can improve future iterations","clicks":1},{"url":"https://gov.optimism.io/t/governance-weekly-recap/3352/97","internal":true,"reflection":true,"title":"Governance Weekly Recap","clicks":0},{"url":"https://gov.optimism.io/t/governance-weekly-recap/3352/99","internal":true,"reflection":true,"title":"Governance Weekly Recap","clicks":0}],"read":true,"user_title":"","bookmarked":false,"actions_summary":[{"id":2,"count":21}],"moderator":false,"admin":false,"staff":false,"user_id":9106,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/1","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null,"can_vote":false},{"id":41460,"name":"Gonna.eth (Dhannte)","username":"Gonna.eth","avatar_template":"/user_avatar/gov.optimism.io/gonna.eth/{size}/3379_2.png","created_at":"2024-11-06T00:19:01.390Z","cooked":"\u003cp\u003eGreat article Carl as always super helpful!\u003c/p\u003e\n\u003cp\u003eAre there any concrete steps for building “an engine” to track impact? It Would be nice to have a clear picture of how to define, track, and iterate on these metrics, and love the idea of aggregating nuanced human impact over time.\u003c/p\u003e\n\u003cp\u003eNarrowing domains and increasing feedback frequency is great, but it would be helpful to clarify which domains would be prioritized and how to handle potential trade-offs (e.g., does narrowing the scope risk excluding high-impact outlier projects?). Or at least define a few “long-term” metrics from a bigger list, Why?\u003c/p\u003e\n\u003cp\u003eAn interesting effect of these metrics in the past is that they’re starting to serve as a guide for projects on what The Foundation, and hopefully The Collective in the future, considers worth funding retroactively. This dynamic shows the influence of Retro Funding metrics as a signaling tool to foster long-term commitment from builders, as they gain confidence that their contributions align with recognized priorities and are likely to be rewarded. Defining long-term metrics could help attract and retain talent, as builders see a clear pathway to recognition and support for their efforts.\u003c/p\u003e\n\u003cp\u003eNarrowing domains may be unpopular, but involving The Collective as much as The Foundation can in decision-making and processes will probably mitigate resistance.\u003c/p\u003e\n\u003cp\u003eAfter participating in the OP-Stack round, the ‘Metrics-Driven, Experts in the Loop’ approach feels especially compelling and separating voting from judging. I’m inclined to a model where badgeholders delegate their vote to professionals with specific expertise to asses impact and vote on behalf of the Citizen House. I would even prefer Citizens determine the budget and make OP Stack community members do a peer review between themselves for budget allocation.\u003c/p\u003e\n\u003cp\u003eLastly, about the ‘Guest voter’ experiment: while I believe it’s good to experiment, after the OP Stack report, I think we can all agree that a high schooler who just passed chemistry can’t choose the Nobel Prize winner in chemistry. We should be working on Badgeholder accountability and expansion.\u003c/p\u003e","post_number":2,"post_type":1,"posts_count":14,"updated_at":"2024-11-06T00:36:55.799Z","reply_count":1,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":2,"reads":83,"readers_count":82,"score":226.6,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"Gonna.eth (Dhannte)","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":2,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":"","bookmarked":false,"actions_summary":[{"id":2,"count":9}],"moderator":false,"admin":false,"staff":false,"user_id":50,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/2","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41466,"name":"carl","username":"ccerv1","avatar_template":"/user_avatar/gov.optimism.io/ccerv1/{size}/6309_2.png","created_at":"2024-11-06T05:06:06.814Z","cooked":"\u003cp\u003eThanks for the feedback Gonna. Will respond in line to several of the topics you raised.\u003c/p\u003e\n\u003caside class=\"quote group-govNERDs\" data-username=\"Gonna.eth\" data-post=\"2\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/gonna.eth/48/3379_2.png\" class=\"avatar\"\u003e Gonna.eth:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAre there any concrete steps for building “an engine” to track impact?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eNot yet. But if there’s support for this general direction, I’d love to help prototype something for RF7 (dev tooling).\u003c/p\u003e\n\u003caside class=\"quote group-govNERDs\" data-username=\"Gonna.eth\" data-post=\"2\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/gonna.eth/48/3379_2.png\" class=\"avatar\"\u003e Gonna.eth:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eNarrowing domains and increasing feedback frequency is great, but it would be helpful to clarify which domains would be prioritized\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eSome domains would definitely be better suited than others, onchain builders being the most obvious, but in principle this could be done for any domain with public / verifiable data.\u003c/p\u003e\n\u003caside class=\"quote group-govNERDs\" data-username=\"Gonna.eth\" data-post=\"2\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/gonna.eth/48/3379_2.png\" class=\"avatar\"\u003e Gonna.eth:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003edoes narrowing the scope risk excluding high-impact outlier projects?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eDefinitely a risk. Hopefully this is a gap that mission requests can fill.\u003c/p\u003e\n\u003caside class=\"quote group-govNERDs\" data-username=\"Gonna.eth\" data-post=\"2\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/gonna.eth/48/3379_2.png\" class=\"avatar\"\u003e Gonna.eth:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eOr at least define a few “long-term” metrics from a bigger list\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eThe \u003ca href=\"https://docs.google.com/spreadsheets/d/1f-uIW_PzlGQ_XFAmsf9FYiUf0N9l_nePwDVrw0D5MXY/edit?pli=1\u0026amp;gid=584971628#gid=584971628\" rel=\"noopener nofollow ugc\"\u003eSuperchain Health dashboard\u003c/a\u003e seems like a good place to start.\u003c/p\u003e\n\u003caside class=\"quote group-govNERDs\" data-username=\"Gonna.eth\" data-post=\"2\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/gonna.eth/48/3379_2.png\" class=\"avatar\"\u003e Gonna.eth:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eDefining long-term metrics could help attract and retain talent, as builders see a clear pathway to recognition and support for their efforts.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003e100%\u003c/p\u003e\n\u003caside class=\"quote group-govNERDs\" data-username=\"Gonna.eth\" data-post=\"2\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/gonna.eth/48/3379_2.png\" class=\"avatar\"\u003e Gonna.eth:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eNarrowing domains may be unpopular, but involving The Collective as much as The Foundation can in decision-making and processes will probably mitigate resistance.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eAgreed - this is an important function of governance. BUT also retro funding shouldn’t be expected to do everything for everyone. I hope mission requests remain broad and adaptable.\u003c/p\u003e\n\u003caside class=\"quote group-govNERDs\" data-username=\"Gonna.eth\" data-post=\"2\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/gonna.eth/48/3379_2.png\" class=\"avatar\"\u003e Gonna.eth:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAfter participating in the OP-Stack round, the ‘Metrics-Driven, Experts in the Loop’ approach feels especially compelling and separating voting from judging.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eSame here - the findings from the OP Stack round were what really stood out to me. I would also add that subject matter expertise isn’t the only kind of expertise that’s relevant to the job of reviewing projects. Some people bring an expert craft to the process of researching projects, mitigating biases, asking good questions, etc.\u003c/p\u003e","post_number":3,"post_type":1,"posts_count":14,"updated_at":"2024-11-06T05:06:06.814Z","reply_count":1,"reply_to_post_number":2,"quote_count":1,"incoming_link_count":3,"reads":72,"readers_count":71,"score":109.4,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"carl","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://docs.google.com/spreadsheets/d/1f-uIW_PzlGQ_XFAmsf9FYiUf0N9l_nePwDVrw0D5MXY/edit?pli=1\u0026gid=584971628#gid=584971628","internal":false,"reflection":false,"title":"Superchain Health Dashboard - Google Sheets","clicks":9}],"read":true,"user_title":"","bookmarked":false,"actions_summary":[{"id":2,"count":3}],"moderator":false,"admin":false,"staff":false,"user_id":9106,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/3","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41467,"name":"Joan","username":"joanbp","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/{size}.png","created_at":"2024-11-06T10:23:49.361Z","cooked":"\u003caside class=\"quote no-group\" data-username=\"ccerv1\" data-post=\"3\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/ccerv1/48/6309_2.png\" class=\"avatar\"\u003e ccerv1:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eI would also add that subject matter expertise isn’t the only kind of expertise that’s relevant to the job of reviewing projects. Some people bring an expert craft to the process of researching projects, mitigating biases, asking good questions, etc.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eThank you for putting this into words - I had been looking for a way to communicate exactly this sentiment.\u003c/p\u003e\n\u003cp\u003eAlso, while I believe strongly in listening to experts, there is also the matter of building a strong democratic foundation. To me, this is what citizenship is all about: Listening to everyone, discussing any concerns raised, considering pros and cons, and trying to get to a point of rough consensus where there are (ideally) no real losers.\u003c/p\u003e\n\u003cp\u003eI kind of like \u003ca class=\"mention\" href=\"/u/gonna.eth\"\u003e@Gonna.eth\u003c/a\u003e’s idea about having citizens delegate to experts - but I would like it to only be an option, not a requirement. And it should be possible to only delegate for a round at a time.\u003c/p\u003e\n\u003cp\u003eThat way, I might decide to delegate my citizen’s vote to, say, Carl in a round such as RF5, but I might decide to delegate it to LauNaMu or myself in RF6. That way, I, as the citizen, get to decide in each round what \u003cem\u003ekind\u003c/em\u003e (kinds) of expertise I value for the task at hand.\u003c/p\u003e\n\u003cp\u003eApart from the matter of decentralizing and democratizing power by distributing it among (in the future, I hope, many more!) citizens rather than a few fixed ‘badgeholders’ or ‘experts’, I think this has a crucial psychological/pedagogical/social rationale:\u003c/p\u003e\n\u003cp\u003eWhen you get to hold power, as a citizen, you also get responsibility. Without that, you naturally lack incentive to learn and discuss and ask questions and point out problems and come up with solutions. A certain apathy arises.\u003c/p\u003e\n\u003cp\u003eThe past few rounds have brought important insights. I think a major issue with them, though, has been the lack of much debate, both internally among citizens, among citizens and guest voters (experts in RF5 and ‘random’ in RF6) and among citizens and other groups of the Collective in general.\u003c/p\u003e\n\u003cp\u003eAs for citizens vs guest voters, debate has in fact been actively discouraged!\u003c/p\u003e\n\u003cp\u003eI understand why, in the name of the experiment. But I think we should also be clear that a lack of communication seriously handicaps the democratic process that might have been.\u003c/p\u003e\n\u003cp\u003eOf course there will always be experts who know more than random citizens about any specific domain. But no group of experts will be better voters \u003cem\u003ein general\u003c/em\u003e than a well informed and caring group of citizens who are strong communicators and know how to pull in the experts where it makes sense.\u003c/p\u003e\n\u003cp\u003eThe issue is: It takes time and effort to grow well informed and caring and communicating citizens of that kind. It is a set of skills that needs to be learned, through meaningful practice and application to real-world problems that genuinely concern you. And it is relationships and shared frames of reference that have to be seeded and nurtured.\u003c/p\u003e\n\u003cp\u003eYou can’t just give random people power, tell them not to talk to the experts and see how they handle it, and then decide that they are not up to the task.\u003c/p\u003e\n\u003cp\u003eWell, you can, but you are not going to learn what a truly informed citizenship might be able to acomplish.\u003c/p\u003e\n\u003cp\u003eIf the Citizens House is supposed to be all about focussing on the long-term sustainability of Optimism (and the Superchain), then I believe in the model of informed citizenship where it might be possible to delegate power to subject matter experts on a case by case basis, but where it is equally possible to vote yourself, and where there is always an incentive to learn and discuss and share context and consider many different perspectives and kinds of expertise - and then make up your mind.\u003c/p\u003e\n\u003cp\u003eAnything less than that would be a missed opportunity, the way I see it.\u003c/p\u003e","post_number":4,"post_type":1,"posts_count":14,"updated_at":"2024-11-13T07:16:07.294Z","reply_count":2,"reply_to_post_number":3,"quote_count":1,"incoming_link_count":13,"reads":59,"readers_count":58,"score":191.8,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"Joan","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":6,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://gov.optimism.io/t/joans-rpgf6-reflections/9261","internal":true,"reflection":true,"title":"Joan's RPGF6 Reflections","clicks":0}],"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":5}],"moderator":false,"admin":false,"staff":false,"user_id":10522,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/4","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41537,"name":"LauNaMu","username":"LauNaMu","avatar_template":"/user_avatar/gov.optimism.io/launamu/{size}/6092_2.png","created_at":"2024-11-10T10:04:36.326Z","cooked":"\u003cp\u003eI’ve read through this and started analyzing it, however, as I shared with Jonas (who asked me to post it here) before I go into details it would be very valuable to get the direct sources of the summary \u003cimg src=\"https://emoji.discourse-cdn.com/twitter/slight_smile.png?v=12\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\" loading=\"lazy\" width=\"20\" height=\"20\"\u003e particularly for the “Key Lessons Learned” section. It’s currently unclear if each of the conclusions presented is derived from a significant amount of data.\u003c/p\u003e\n\u003cp\u003eI think this would additionally strengthen the analysis and the hypothesis that follow. Could these sources please be included?\u003c/p\u003e","post_number":5,"post_type":1,"posts_count":14,"updated_at":"2024-11-10T10:05:12.176Z","reply_count":2,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":0,"reads":40,"readers_count":39,"score":63.0,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"LauNaMu","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":"","bookmarked":false,"actions_summary":[{"id":2,"count":3}],"moderator":false,"admin":false,"staff":false,"user_id":10342,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/5","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41538,"name":"VibesTheAlchemist","username":"Gneric","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/g/71c47a/{size}.png","created_at":"2024-11-10T10:41:49.613Z","cooked":"\u003cp\u003eAs a citizen being able to allocate to experts and several well informed citizens seems smart IMO, but what could be the downside of such decision?\u003c/p\u003e","post_number":6,"post_type":1,"posts_count":14,"updated_at":"2024-11-10T10:41:49.613Z","reply_count":0,"reply_to_post_number":4,"quote_count":0,"incoming_link_count":0,"reads":43,"readers_count":42,"score":8.6,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"VibesTheAlchemist","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"reply_to_user":{"id":10522,"username":"joanbp","name":"Joan","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/{size}.png"},"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":13961,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/6","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41540,"name":"carl","username":"ccerv1","avatar_template":"/user_avatar/gov.optimism.io/ccerv1/{size}/6309_2.png","created_at":"2024-11-10T10:51:58.785Z","cooked":"\u003cp\u003eHi Lau,\u003cbr\u003e\nI don’t have access to the raw survey results, but here are some screenshots from the post-round badgeholder retros.\u003cbr\u003e\n\u003cdiv class=\"lightbox-wrapper\"\u003e\u003ca class=\"lightbox\" href=\"https://europe1.discourse-cdn.com/bc41dd/original/2X/a/ad60837708f27d5ebff13418a9f35ed356f2fdd2.png\" data-download-href=\"/uploads/short-url/oJLnFbJkpdeiNRmUlLWO0Qcx626.png?dl=1\" title=\"image\" rel=\"noopener nofollow ugc\"\u003e\u003cimg src=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/a/ad60837708f27d5ebff13418a9f35ed356f2fdd2_2_690x353.png\" alt=\"image\" data-base62-sha1=\"oJLnFbJkpdeiNRmUlLWO0Qcx626\" width=\"690\" height=\"353\" srcset=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/a/ad60837708f27d5ebff13418a9f35ed356f2fdd2_2_690x353.png, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/a/ad60837708f27d5ebff13418a9f35ed356f2fdd2_2_1035x529.png 1.5x, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/a/ad60837708f27d5ebff13418a9f35ed356f2fdd2_2_1380x706.png 2x\" data-dominant-color=\"DDD9D7\"\u003e\u003cdiv class=\"meta\"\u003e\u003csvg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#far-image\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003cspan class=\"filename\"\u003eimage\u003c/span\u003e\u003cspan class=\"informations\"\u003e2028×1038 133 KB\u003c/span\u003e\u003csvg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#discourse-expand\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003c/div\u003e\u003c/a\u003e\u003c/div\u003e\u003c/p\u003e\n\u003cp\u003e\u003cdiv class=\"lightbox-wrapper\"\u003e\u003ca class=\"lightbox\" href=\"https://europe1.discourse-cdn.com/bc41dd/original/2X/e/eca2171ad66779b39569d636be826310530f6a50.png\" data-download-href=\"/uploads/short-url/xLlXvrit3Ix3bchrvpYetAVaiBi.png?dl=1\" title=\"image\" rel=\"noopener nofollow ugc\"\u003e\u003cimg src=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/e/eca2171ad66779b39569d636be826310530f6a50_2_690x369.png\" alt=\"image\" data-base62-sha1=\"xLlXvrit3Ix3bchrvpYetAVaiBi\" width=\"690\" height=\"369\" srcset=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/e/eca2171ad66779b39569d636be826310530f6a50_2_690x369.png, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/e/eca2171ad66779b39569d636be826310530f6a50_2_1035x553.png 1.5x, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/e/eca2171ad66779b39569d636be826310530f6a50_2_1380x738.png 2x\" data-dominant-color=\"E5E5D5\"\u003e\u003cdiv class=\"meta\"\u003e\u003csvg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#far-image\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003cspan class=\"filename\"\u003eimage\u003c/span\u003e\u003cspan class=\"informations\"\u003e1526×818 14.7 KB\u003c/span\u003e\u003csvg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#discourse-expand\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003c/div\u003e\u003c/a\u003e\u003c/div\u003e\u003c/p\u003e\n\u003cp\u003e\u003cdiv class=\"lightbox-wrapper\"\u003e\u003ca class=\"lightbox\" href=\"https://europe1.discourse-cdn.com/bc41dd/original/2X/d/df5a2a1dd141591dd104ce5435240ff510980e27.png\" data-download-href=\"/uploads/short-url/vRREscwphkZWvWCABvDlCDte7Y3.png?dl=1\" title=\"image\" rel=\"noopener nofollow ugc\"\u003e\u003cimg src=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/df5a2a1dd141591dd104ce5435240ff510980e27_2_690x389.png\" alt=\"image\" data-base62-sha1=\"vRREscwphkZWvWCABvDlCDte7Y3\" width=\"690\" height=\"389\" srcset=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/df5a2a1dd141591dd104ce5435240ff510980e27_2_690x389.png, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/df5a2a1dd141591dd104ce5435240ff510980e27_2_1035x583.png 1.5x, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/df5a2a1dd141591dd104ce5435240ff510980e27_2_1380x778.png 2x\" data-dominant-color=\"D6CFD7\"\u003e\u003cdiv class=\"meta\"\u003e\u003csvg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#far-image\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003cspan class=\"filename\"\u003eimage\u003c/span\u003e\u003cspan class=\"informations\"\u003e1500×846 36.8 KB\u003c/span\u003e\u003csvg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#discourse-expand\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003c/div\u003e\u003c/a\u003e\u003c/div\u003e\u003c/p\u003e\n\u003cp\u003e\u003cdiv class=\"lightbox-wrapper\"\u003e\u003ca class=\"lightbox\" href=\"https://europe1.discourse-cdn.com/bc41dd/original/2X/d/d3a97db9af99b7c593e52f52783378fd431b594e.png\" data-download-href=\"/uploads/short-url/ucrSw2wLCYz25F9Hk6oqrHvuf2S.png?dl=1\" title=\"image\" rel=\"noopener nofollow ugc\"\u003e\u003cimg src=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/d3a97db9af99b7c593e52f52783378fd431b594e_2_690x397.png\" alt=\"image\" data-base62-sha1=\"ucrSw2wLCYz25F9Hk6oqrHvuf2S\" width=\"690\" height=\"397\" srcset=\"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/d3a97db9af99b7c593e52f52783378fd431b594e_2_690x397.png, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/d3a97db9af99b7c593e52f52783378fd431b594e_2_1035x595.png 1.5x, https://europe1.discourse-cdn.com/bc41dd/optimized/2X/d/d3a97db9af99b7c593e52f52783378fd431b594e_2_1380x794.png 2x\" data-dominant-color=\"E9E9DD\"\u003e\u003cdiv class=\"meta\"\u003e\u003csvg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#far-image\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003cspan class=\"filename\"\u003eimage\u003c/span\u003e\u003cspan class=\"informations\"\u003e1652×952 15 KB\u003c/span\u003e\u003csvg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"\u003e\u003cuse href=\"#discourse-expand\"\u003e\u003c/use\u003e\u003c/svg\u003e\u003c/div\u003e\u003c/a\u003e\u003c/div\u003e\u003c/p\u003e\n\u003cp\u003eRegarding the expert vs non-expert allocations, I have in my notes from the RF5 retro that ~80% of survey respondents preferred the expert allocations from RF5.\u003c/p\u003e\n\u003cp\u003eThis \u003ca href=\"https://gov.optimism.io/t/retropgf-round-3-feedback-thread/6177/43\"\u003ethread\u003c/a\u003e also has a lot of feedback on RF3, including issues with lists and the relatively low allocation to onchain projects. Trent from Protocol Guild also had a good \u003ca href=\"https://trent.mirror.xyz/ia1sSXWw6Q_0gseWhPDpt0WbsOadCfQ-23yAxNn4sXA\" rel=\"noopener nofollow ugc\"\u003eessay\u003c/a\u003e about the perverse incentive to “atomize” work in RF5.\u003c/p\u003e\n\u003cp\u003eWere there other sources you had in mind?\u003c/p\u003e","post_number":7,"post_type":1,"posts_count":14,"updated_at":"2024-11-10T10:51:58.785Z","reply_count":0,"reply_to_post_number":5,"quote_count":0,"incoming_link_count":0,"reads":47,"readers_count":46,"score":54.4,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"carl","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"link_counts":[{"url":"https://trent.mirror.xyz/ia1sSXWw6Q_0gseWhPDpt0WbsOadCfQ-23yAxNn4sXA","internal":false,"reflection":false,"title":"Aggregation \u0026 atomization: dependency funding round dynamics — Trent","clicks":2},{"url":"https://gov.optimism.io/t/retropgf-round-3-feedback-thread/6177/43","internal":true,"reflection":false,"title":"RetroPGF Round 3 Feedback Thread","clicks":0}],"read":true,"user_title":"","reply_to_user":{"id":10342,"username":"LauNaMu","name":"LauNaMu","avatar_template":"/user_avatar/gov.optimism.io/launamu/{size}/6092_2.png"},"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":9106,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/7","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41541,"name":"Arnold#1a35e1","username":"1a35e1","avatar_template":"/user_avatar/gov.optimism.io/1a35e1/{size}/8658_2.png","created_at":"2024-11-10T11:42:39.783Z","cooked":"\u003cp\u003eAgree that there needs to be more data oriented processes. The more public data is available the easier it will be to explore and make judgements.\u003c/p\u003e\n\u003cp\u003eCan we work to build out a \u003ccode\u003ehttps://data.optimism.io/\u003c/code\u003e or similar \u003cimg src=\"https://emoji.discourse-cdn.com/twitter/slight_smile.png?v=12\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\" loading=\"lazy\" width=\"20\" height=\"20\"\u003e\u003c/p\u003e","post_number":8,"post_type":1,"posts_count":14,"updated_at":"2024-11-10T11:42:39.783Z","reply_count":1,"reply_to_post_number":5,"quote_count":0,"incoming_link_count":0,"reads":42,"readers_count":41,"score":88.4,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"Arnold#1a35e1","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":"","reply_to_user":{"id":10342,"username":"LauNaMu","name":"LauNaMu","avatar_template":"/user_avatar/gov.optimism.io/launamu/{size}/6092_2.png"},"bookmarked":false,"actions_summary":[{"id":2,"count":3}],"moderator":false,"admin":false,"staff":false,"user_id":11978,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/8","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41543,"name":"","username":"Chain_L","avatar_template":"/user_avatar/gov.optimism.io/chain_l/{size}/7930_2.png","created_at":"2024-11-10T18:33:45.408Z","cooked":"\u003cp\u003eThat’s a great idea.\u003cbr\u003e\nCurrently there is public data around, but scattered everywhere.\u003c/p\u003e\n\u003cp\u003enumbaNERDs can be the path via which this can be maintained.\u003c/p\u003e","post_number":9,"post_type":1,"posts_count":14,"updated_at":"2024-11-10T18:33:45.408Z","reply_count":0,"reply_to_post_number":8,"quote_count":0,"incoming_link_count":0,"reads":41,"readers_count":40,"score":68.2,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":"","reply_to_user":{"id":11978,"username":"1a35e1","name":"Arnold#1a35e1","avatar_template":"/user_avatar/gov.optimism.io/1a35e1/{size}/8658_2.png"},"bookmarked":false,"actions_summary":[{"id":2,"count":2}],"moderator":false,"admin":false,"staff":false,"user_id":10260,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/9","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41552,"name":"Deelabs","username":"Deelabs","avatar_template":"/user_avatar/gov.optimism.io/deelabs/{size}/10235_2.png","created_at":"2024-11-11T11:47:43.162Z","cooked":"\u003cp\u003eWe find this retrospective analysis of Optimism’s Retro Funding mechanism to be thorough and insightful, particularly in its systematic evaluation of the evolving funding approaches and their respective outcomes across different rounds.\u003c/p\u003e\n\u003cp\u003eThe analysis is proficient in identifying key learnings from each implementation phase, and effectively outlines the strengths and limitations of both human judgment and data-driven approaches.\u003c/p\u003e\n\u003cp\u003eParticularly compelling is the proposed framework that combines metrics-based evaluation with expert oversight. This hybrid approach arguably addresses many of the limitations observed in previous rounds:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eLeverages data’s systematic reach while maintaining human intuition for nuanced decisions\u003c/li\u003e\n\u003cli\u003eEnables iterative improvement through faster feedback cycles\u003c/li\u003e\n\u003cli\u003eCreates a more sustainable and scalable evaluation process\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe emphasis on narrowing domain scope while increasing iteration frequency is a pragmatic evolution and in our view, a suitable approach. Rather than attempting to solve everything at once, this approach allows for meaningful refinement of the mechanism within specific contexts.\u003c/p\u003e\n\u003cp\u003eIt would be efficient to specify and concretize the domains worthy of further attention and prioritization.\u003c/p\u003e\n\u003cp\u003eWe’re supportive of the recommendation to move toward a more focused, metrics-driven approach with expert oversight. The emphasis on continuous improvement and faster iteration cycles should enable more effective resource allocation while building stronger empirical evidence for what works.\u003c/p\u003e","post_number":10,"post_type":1,"posts_count":14,"updated_at":"2024-11-11T11:47:43.162Z","reply_count":0,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":0,"reads":44,"readers_count":43,"score":8.8,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"Deelabs","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":"","bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":12907,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/10","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41621,"name":"","username":"noturhandle","avatar_template":"/user_avatar/gov.optimism.io/noturhandle/{size}/10216_2.png","created_at":"2024-11-18T12:53:29.740Z","cooked":"\u003cp\u003eHey Carl, incredible summary. Thanks for sharing.\u003c/p\u003e\n\u003cp\u003eIn the lessons learned for Round 5, you mentioned that teams had perverse incentives to break the work into smaller contributions, but I can’t determine the cause.\u003c/p\u003e\n\u003cp\u003eCan you expand at all? Were projects simply trying to submit to more categories, i.e., spread betting? Or was there some element of the voting design that preferred smaller projects?\u003c/p\u003e","post_number":11,"post_type":1,"posts_count":14,"updated_at":"2024-11-18T12:53:29.740Z","reply_count":1,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":0,"reads":39,"readers_count":38,"score":27.8,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":6844,"hidden":false,"trust_level":1,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/11","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41623,"name":"İsmail Emin Erdoğdu","username":"ismailemin","avatar_template":"/user_avatar/gov.optimism.io/ismailemin/{size}/5610_2.png","created_at":"2024-11-18T13:08:55.863Z","cooked":"\u003cp\u003eI really liked the article thanks for it Carl.\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"joanbp\" data-post=\"4\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/48.png\" class=\"avatar\"\u003e joanbp:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThat way, I might decide to delegate my citizen’s vote to, say, Carl in a round such as RF5, but I might decide to delegate it to LauNaMu or myself in RF6. That way, I, as the citizen, get to decide in each round what \u003cem\u003ekind\u003c/em\u003e (kinds) of expertise I value for the task at hand.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eAlso liked the idea of delegating the VP in the Citizens’ House. Maybe we should create profiles for every badgeholder so anyone can know what are their expertise areas. This also enables the selection of new citizens based on gaps in expertise areas.\u003c/p\u003e","post_number":12,"post_type":1,"posts_count":14,"updated_at":"2024-11-18T13:08:55.863Z","reply_count":0,"reply_to_post_number":null,"quote_count":1,"incoming_link_count":0,"reads":40,"readers_count":39,"score":23.0,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"İsmail Emin Erdoğdu","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":6691,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/12","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41626,"name":"carl","username":"ccerv1","avatar_template":"/user_avatar/gov.optimism.io/ccerv1/{size}/6309_2.png","created_at":"2024-11-18T17:59:14.036Z","cooked":"\u003cp\u003eHey \u003ca class=\"mention\" href=\"/u/noturhandle\"\u003e@noturhandle\u003c/a\u003e\u003c/p\u003e\n\u003caside class=\"quote no-group\" data-username=\"noturhandle\" data-post=\"11\" data-topic=\"9239\"\u003e\n\u003cdiv class=\"title\"\u003e\n\u003cdiv class=\"quote-controls\"\u003e\u003c/div\u003e\n\u003cimg loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://dub1.discourse-cdn.com/bc41dd/user_avatar/gov.optimism.io/noturhandle/48/10216_2.png\" class=\"avatar\"\u003e noturhandle:\u003c/div\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIn the lessons learned for Round 5, you mentioned that teams had perverse incentives to break the work into smaller contributions, but I can’t determine the cause.\u003c/p\u003e\n\u003cp\u003eCan you expand at all? Were projects simply trying to submit to more categories, i.e., spread betting? Or was there some element of the voting design that preferred smaller projects?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003c/aside\u003e\n\u003cp\u003eI think the primary reason is that many badgeholders want to give something to everyone who has had \u003cem\u003esome impact\u003c/em\u003e, however small. So they give 1-2% to lots of small projects. Even if there are some zero votes, the median is still likely to be around 1-2%.\u003c/p\u003e\n\u003cp\u003eAt the same time, many badgeholders have historically been wary of giving very large allocations to projects they feel are high impact, eg, over 10%. So large projects will also get a lot of 1-2% votes and then a few \u0026gt;10%, but not enough to significantly skew the median.\u003c/p\u003e\n\u003cp\u003eThe result is a median of 1-2% for a small project and maybe 3-5% for a large project.\u003c/p\u003e\n\u003cp\u003eIf a well-known large project wanted to take advantage of this tendency and optimize its retro funding, it would almost certainly do better by submitting 5 projects with a combined expected value of 5-10% (versus the current 3-5% for a single project).\u003c/p\u003e\n\u003cp\u003eNote: I’m writing this from recall - I’m not looking at actual numbers.\u003c/p\u003e","post_number":13,"post_type":1,"posts_count":14,"updated_at":"2024-11-18T17:59:14.036Z","reply_count":0,"reply_to_post_number":11,"quote_count":1,"incoming_link_count":0,"reads":40,"readers_count":39,"score":53.0,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"carl","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":"","bookmarked":false,"actions_summary":[{"id":2,"count":1}],"moderator":false,"admin":false,"staff":false,"user_id":9106,"hidden":false,"trust_level":2,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/13","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null},{"id":41889,"name":"happy","username":"happy","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/h/f05b48/{size}.png","created_at":"2024-11-28T22:50:08.115Z","cooked":"\u003cp\u003egreat info and super useful content\u003c/p\u003e","post_number":14,"post_type":1,"posts_count":14,"updated_at":"2024-11-28T22:50:08.115Z","reply_count":0,"reply_to_post_number":null,"quote_count":0,"incoming_link_count":0,"reads":31,"readers_count":30,"score":6.2,"yours":false,"topic_id":9239,"topic_slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","display_username":"happy","primary_group_name":null,"flair_name":null,"flair_url":null,"flair_bg_color":null,"flair_color":null,"flair_group_id":null,"badges_granted":[],"version":1,"can_edit":false,"can_delete":false,"can_recover":false,"can_see_hidden_post":false,"can_wiki":false,"read":true,"user_title":null,"bookmarked":false,"actions_summary":[],"moderator":false,"admin":false,"staff":false,"user_id":14124,"hidden":false,"trust_level":0,"deleted_at":null,"user_deleted":false,"edit_reason":null,"can_view_edit_history":true,"wiki":false,"post_url":"/t/lessons-learned-from-two-years-of-retroactive-public-goods-funding/9239/14","can_accept_answer":false,"can_unaccept_answer":false,"accepted_answer":false,"topic_accepted_answer":null}],"stream":[41452,41460,41466,41467,41537,41538,41540,41541,41543,41552,41621,41623,41626,41889]},"timeline_lookup":[[1,377],[5,373],[8,372],[10,371],[11,364],[14,354]],"suggested_topics":[],"tags":[],"tags_descriptions":{},"fancy_title":"Lessons learned from two years of Retroactive Public Goods Funding","id":9239,"title":"Lessons learned from two years of Retroactive Public Goods Funding","posts_count":14,"created_at":"2024-11-05T12:22:42.272Z","views":697,"reply_count":7,"like_count":50,"last_posted_at":"2024-11-28T22:50:08.115Z","visible":true,"closed":false,"archived":false,"has_summary":false,"archetype":"regular","slug":"lessons-learned-from-two-years-of-retroactive-public-goods-funding","category_id":46,"word_count":3959,"deleted_at":null,"user_id":9106,"featured_link":null,"pinned_globally":false,"pinned_at":null,"pinned_until":null,"image_url":"https://europe1.discourse-cdn.com/bc41dd/original/2X/e/e8c2684c34954a269f489a8549d29f4e6abe8a64.png","slow_mode_seconds":0,"draft":null,"draft_key":"topic_9239","draft_sequence":null,"unpinned":null,"pinned":false,"current_post_number":1,"highest_post_number":14,"deleted_by":null,"actions_summary":[{"id":4,"count":0,"hidden":false,"can_act":false},{"id":8,"count":0,"hidden":false,"can_act":false},{"id":10,"count":0,"hidden":false,"can_act":false},{"id":7,"count":0,"hidden":false,"can_act":false}],"chunk_size":20,"bookmarked":false,"topic_timer":null,"message_bus_last_id":0,"participant_count":11,"show_read_indicator":false,"thumbnails":[{"max_width":null,"max_height":null,"width":800,"height":661,"url":"https://europe1.discourse-cdn.com/bc41dd/original/2X/e/e8c2684c34954a269f489a8549d29f4e6abe8a64.png"}],"slow_mode_enabled_until":null,"tags_disable_ads":false,"related_topics":[{"fancy_title":"Upcoming Retro rounds and their design","id":7861,"title":"Upcoming Retro rounds and their design","slug":"upcoming-retro-rounds-and-their-design","posts_count":55,"reply_count":15,"highest_post_number":57,"image_url":"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/6/6dd5eaa2ae0d3a509d3d850ecfb870f226bae8b5_2_1024x156.png","created_at":"2024-03-26T14:14:03.968Z","last_posted_at":"2025-08-23T10:26:25.217Z","bumped":true,"bumped_at":"2025-08-23T10:26:25.217Z","archetype":"regular","unseen":false,"pinned":false,"unpinned":null,"excerpt":"Based on Round 3 learnings, Retroactive Public Goods Funding (Retro Funding) is transitioning from broad scoped rounds to more narrowly scoped rounds; you can find out more about this transition here. \nTo provide more tr\u0026hellip;","visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"tags":[],"tags_descriptions":{},"like_count":397,"views":9252,"category_id":46,"featured_link":null,"has_accepted_answer":false,"posters":[{"extras":null,"description":"Original Poster","user":{"id":-1,"username":"system","name":"system","avatar_template":"https://europe1.discourse-cdn.com/bc41dd/original/1X/391d6e71a279a9f881f75bfa7df5cdf3cf905b3f.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":10522,"username":"joanbp","name":"Joan","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/{size}.png","trust_level":2}},{"extras":null,"description":"Frequent Poster","user":{"id":218,"username":"lefterisjp","name":"Lefteris Karapetsas","avatar_template":"/user_avatar/gov.optimism.io/lefterisjp/{size}/9240_2.png","trust_level":2}},{"extras":null,"description":"Frequent Poster","user":{"id":60,"username":"Jonas","name":"","avatar_template":"/user_avatar/gov.optimism.io/jonas/{size}/11574_2.png","admin":true,"moderator":true,"trust_level":4}},{"extras":"latest","description":"Most Recent Poster","user":{"id":16137,"username":"stephanschwab","name":"stephanschwab","avatar_template":"/user_avatar/gov.optimism.io/stephanschwab/{size}/12434_2.png","trust_level":2}}]},{"fancy_title":"Retro Funding 6: Badgeholder Manual","id":9109,"title":"Retro Funding 6: Badgeholder Manual","slug":"retro-funding-6-badgeholder-manual","posts_count":15,"reply_count":7,"highest_post_number":16,"image_url":null,"created_at":"2024-10-22T10:52:35.018Z","last_posted_at":"2025-03-22T07:23:20.790Z","bumped":true,"bumped_at":"2025-03-22T07:23:20.790Z","archetype":"regular","unseen":false,"pinned":false,"unpinned":null,"excerpt":"Retro Funding: 6 Badgeholder Manual\n\nVoting period: Oct 31st - Nov 13th 18:00 UTC\nParticipation: Selected badgeholders and guest voters are expected to allocate up to 10 hours to participate in round 6\nReward Amount: Max\u0026hellip;","visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"visibility_reason_id":4,"tags":["round-6"],"tags_descriptions":{},"like_count":46,"views":758,"category_id":46,"featured_link":null,"has_accepted_answer":false,"posters":[{"extras":null,"description":"Original Poster","user":{"id":-1,"username":"system","name":"system","avatar_template":"https://europe1.discourse-cdn.com/bc41dd/original/1X/391d6e71a279a9f881f75bfa7df5cdf3cf905b3f.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":60,"username":"Jonas","name":"","avatar_template":"/user_avatar/gov.optimism.io/jonas/{size}/11574_2.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":10522,"username":"joanbp","name":"Joan","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/{size}.png","trust_level":2}},{"extras":null,"description":"Frequent Poster","user":{"id":12276,"username":"TEKNNUKU","name":"Naza Raymond","avatar_template":"/user_avatar/gov.optimism.io/teknnuku/{size}/9064_2.png","trust_level":2}},{"extras":"latest","description":"Most Recent Poster","user":{"id":14511,"username":"chibuike58","name":null,"avatar_template":"/user_avatar/gov.optimism.io/chibuike58/{size}/11869_2.png","trust_level":1}}]},{"fancy_title":"Retro Funding: Moving from broad to narrow scoped rounds","id":7759,"title":"Retro Funding: Moving from broad to narrow scoped rounds","slug":"retro-funding-moving-from-broad-to-narrow-scoped-rounds","posts_count":24,"reply_count":4,"highest_post_number":24,"image_url":null,"created_at":"2024-03-07T12:43:44.430Z","last_posted_at":"2024-03-26T02:59:16.996Z","bumped":true,"bumped_at":"2024-03-26T02:59:16.996Z","archetype":"regular","unseen":false,"pinned":false,"unpinned":null,"excerpt":"Recently the learnings \u0026amp; reflections of Retroactive Public Goods Funding 3 have been published, featuring detailed accounts of badgeholder feedback and key learnings which the Collective is aiming to address in the next \u0026hellip;","visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"tags":[],"tags_descriptions":{},"like_count":92,"views":2976,"category_id":46,"featured_link":null,"has_accepted_answer":false,"posters":[{"extras":null,"description":"Original Poster","user":{"id":-1,"username":"system","name":"system","avatar_template":"https://europe1.discourse-cdn.com/bc41dd/original/1X/391d6e71a279a9f881f75bfa7df5cdf3cf905b3f.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":60,"username":"Jonas","name":"","avatar_template":"/user_avatar/gov.optimism.io/jonas/{size}/11574_2.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":5696,"username":"Carlosjmelgar","name":"Carlos Melgar","avatar_template":"/user_avatar/gov.optimism.io/carlosjmelgar/{size}/7795_2.png","trust_level":2}},{"extras":null,"description":"Frequent Poster","user":{"id":11654,"username":"Hellofady","name":"Fady","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/h/b5e925/{size}.png","trust_level":1}},{"extras":"latest","description":"Most Recent Poster","user":{"id":9939,"username":"santicristobal","name":"","avatar_template":"/user_avatar/gov.optimism.io/santicristobal/{size}/5516_2.png","trust_level":2}}]},{"fancy_title":"RetroPGF Round 2 Voting Rationale","id":5570,"title":"RetroPGF Round 2 Voting Rationale","slug":"retropgf-round-2-voting-rationale","posts_count":25,"reply_count":9,"highest_post_number":26,"image_url":null,"created_at":"2023-03-11T00:00:19.460Z","last_posted_at":"2023-11-09T15:14:49.908Z","bumped":true,"bumped_at":"2023-11-09T15:14:49.908Z","archetype":"regular","unseen":false,"pinned":false,"unpinned":null,"excerpt":"Badgeholders are not required to disclose how they’ve voted but may share their voting rationale here if they choose.","visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"tags":["retropgf-2"],"tags_descriptions":{},"like_count":190,"views":6871,"category_id":46,"featured_link":null,"has_accepted_answer":false,"posters":[{"extras":null,"description":"Original Poster","user":{"id":-1,"username":"system","name":"system","avatar_template":"https://europe1.discourse-cdn.com/bc41dd/original/1X/391d6e71a279a9f881f75bfa7df5cdf3cf905b3f.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":9095,"username":"tjayrush","name":"Thomas Jay Rush","avatar_template":"/user_avatar/gov.optimism.io/tjayrush/{size}/9834_2.png","trust_level":2}},{"extras":null,"description":"Frequent Poster","user":{"id":9143,"username":"cryptoAYA","name":"Yashar Baradarvar","avatar_template":"/user_avatar/gov.optimism.io/cryptoaya/{size}/5125_2.png","trust_level":2}},{"extras":null,"description":"Frequent Poster","user":{"id":263,"username":"Joxes","name":"Joxes | SEED Latam","avatar_template":"/user_avatar/gov.optimism.io/joxes/{size}/5814_2.png","trust_level":2}},{"extras":"latest","description":"Most Recent Poster","user":{"id":10895,"username":"Jilt","name":"","avatar_template":"/user_avatar/gov.optimism.io/jilt/{size}/7207_2.png","trust_level":0}}]},{"fancy_title":"Retro Funding 4: Impact Metrics, a Collective Experiment","id":8226,"title":"Retro Funding 4: Impact Metrics, a Collective Experiment","slug":"retro-funding-4-impact-metrics-a-collective-experiment","posts_count":9,"reply_count":0,"highest_post_number":9,"image_url":"https://europe1.discourse-cdn.com/bc41dd/optimized/2X/3/319c8b72e08e73c76d0fb4d84ac3daf79c95a48b_2_1024x488.png","created_at":"2024-05-29T11:59:28.126Z","last_posted_at":"2024-07-22T13:08:49.859Z","bumped":true,"bumped_at":"2024-07-22T13:08:49.859Z","archetype":"regular","unseen":false,"pinned":false,"unpinned":null,"excerpt":"Since one of our Collective Values is Iterative Innovation, we are continuing to refine retroactive public goods funding (retro funding) with round 4. The focus here is on collaboratively curating impact metrics, guided \u0026hellip;","visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"tags":["round-4"],"tags_descriptions":{},"like_count":38,"views":1509,"category_id":46,"featured_link":null,"has_accepted_answer":false,"posters":[{"extras":null,"description":"Original Poster","user":{"id":-1,"username":"system","name":"system","avatar_template":"https://europe1.discourse-cdn.com/bc41dd/original/1X/391d6e71a279a9f881f75bfa7df5cdf3cf905b3f.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":60,"username":"Jonas","name":"","avatar_template":"/user_avatar/gov.optimism.io/jonas/{size}/11574_2.png","admin":true,"moderator":true,"trust_level":4}},{"extras":null,"description":"Frequent Poster","user":{"id":50,"username":"Gonna.eth","name":"Gonna.eth (Dhannte)","avatar_template":"/user_avatar/gov.optimism.io/gonna.eth/{size}/3379_2.png","trust_level":2}},{"extras":null,"description":"Frequent Poster","user":{"id":9106,"username":"ccerv1","name":"carl","avatar_template":"/user_avatar/gov.optimism.io/ccerv1/{size}/6309_2.png","trust_level":2}},{"extras":"latest","description":"Most Recent Poster","user":{"id":12940,"username":"Takeshi","name":"Takeshi","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/t/57b2e6/{size}.png","trust_level":2}}]}],"summarizable":false,"can_vote":false,"vote_count":0,"user_voted":false,"discourse_zendesk_plugin_zendesk_id":null,"discourse_zendesk_plugin_zendesk_url":"https://your-url.zendesk.com/agent/tickets/","details":{"can_edit":false,"notification_level":1,"participants":[{"id":9106,"username":"ccerv1","name":"carl","avatar_template":"/user_avatar/gov.optimism.io/ccerv1/{size}/6309_2.png","post_count":4,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":6691,"username":"ismailemin","name":"İsmail Emin Erdoğdu","avatar_template":"/user_avatar/gov.optimism.io/ismailemin/{size}/5610_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":6844,"username":"noturhandle","name":"","avatar_template":"/user_avatar/gov.optimism.io/noturhandle/{size}/10216_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":1},{"id":10260,"username":"Chain_L","name":"","avatar_template":"/user_avatar/gov.optimism.io/chain_l/{size}/7930_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":10342,"username":"LauNaMu","name":"LauNaMu","avatar_template":"/user_avatar/gov.optimism.io/launamu/{size}/6092_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":10522,"username":"joanbp","name":"Joan","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/{size}.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":11978,"username":"1a35e1","name":"Arnold#1a35e1","avatar_template":"/user_avatar/gov.optimism.io/1a35e1/{size}/8658_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":12907,"username":"Deelabs","name":"Deelabs","avatar_template":"/user_avatar/gov.optimism.io/deelabs/{size}/10235_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":13961,"username":"Gneric","name":"VibesTheAlchemist","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/g/71c47a/{size}.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":50,"username":"Gonna.eth","name":"Gonna.eth (Dhannte)","avatar_template":"/user_avatar/gov.optimism.io/gonna.eth/{size}/3379_2.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":2},{"id":14124,"username":"happy","name":"happy","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/h/f05b48/{size}.png","post_count":1,"primary_group_name":null,"flair_name":null,"flair_url":null,"flair_color":null,"flair_bg_color":null,"flair_group_id":null,"trust_level":0}],"created_by":{"id":9106,"username":"ccerv1","name":"carl","avatar_template":"/user_avatar/gov.optimism.io/ccerv1/{size}/6309_2.png"},"last_poster":{"id":14124,"username":"happy","name":"happy","avatar_template":"https://avatars.discourse-cdn.com/v4/letter/h/f05b48/{size}.png"},"links":[{"url":"https://docs.google.com/spreadsheets/d/1f-uIW_PzlGQ_XFAmsf9FYiUf0N9l_nePwDVrw0D5MXY/edit?pli=1\u0026gid=584971628#gid=584971628","title":"Superchain Health Dashboard - Google Sheets","internal":false,"attachment":false,"reflection":false,"clicks":9,"user_id":9106,"domain":"docs.google.com","root_domain":"google.com"},{"url":"https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c","title":"Retroactive Public Goods Funding. Note: The Optimism team has long been… | by Optimism | Optimism PBC Blog | Medium","internal":false,"attachment":false,"reflection":false,"clicks":5,"user_id":9106,"domain":"medium.com","root_domain":"medium.com"},{"url":"https://trent.mirror.xyz/ia1sSXWw6Q_0gseWhPDpt0WbsOadCfQ-23yAxNn4sXA","title":"Aggregation \u0026 atomization: dependency funding round dynamics — Trent","internal":false,"attachment":false,"reflection":false,"clicks":2,"user_id":9106,"domain":"trent.mirror.xyz","root_domain":"mirror.xyz"},{"url":"https://gov.optimism.io/t/a-review-of-rpgf2-and-ideas-for-how-data-can-improve-future-iterations/6438","title":"A review of RPGF2 and ideas for how data can improve future iterations","internal":true,"attachment":false,"reflection":false,"clicks":1,"user_id":9106,"domain":"gov.optimism.io","root_domain":"optimism.io"}]},"bookmarks":[]}

-50

post1.txt

reviewed

··· 1 1 - tl;dr- 2 2 - In 2023, Optimism ran two megarounds. In 2024, Optimism ran one round per domain per year. We’ve learned that mega rounds devolve into popularity contests and annual feedback loops are too slow. In 2025, Optimism should focus on fewer domains, iterate more rapidly, and refine what works. 3 3 - In 2023, Optimism had everyone vote on everything. In 2024, Optimism ran experiments around expertise-based voting and metrics-based voting. We’ve learned what humans are good at - and what data is good at. In 2025, Optimism should take the best of both (and set aside the rest). 4 4 - In both 2023 and 2024, we struggled at measuring the success of Retro Funding itself. The time for not knowing how well this works is over. 5 5 - Full post… 6 6 - This was supposed to be simple 7 7 - As Vitalik wrote when initially describing the mechanism: “The core principle behind the concept of retroactive public goods funding is simple: it’s easier to agree on what was useful than what will be useful.” 8 8 - Now, nearly two years into this experiment, we should be able to look back at the mechanism of Retro Funding itself and apply this same analysis. 9 9 - Specifically, we want to understand: 10 10 - Can people actually agree on what was useful? 11 11 - Under what circumstances does retroactive funding produce superior outcomes? 12 12 - Simple, right? Not entirely. 13 13 - My team at OSO has been trying to help Optimism answer some of these questions since Round 2 . In this post, we’ll look at a very high level at what’s happened across five rounds of Retro Funding (Rounds 2-5) and share some of our observations. 14 14 - The good news is there are some clear learnings about what people are good at agreeing on and how to create the right conditions for consensus. These learnings are the result of deliberate experiments that were undertaken in 2024, for instance, experimenting with metrics-based voting in Round 4 and expertise-based voting in Round 5. 15 15 - The not-so-good news is that we don’t (yet) have the data to show that retroactive funding produces superior outcomes. 16 16 - In our view, solving the measurement problem is the most important thing to get right in 2025. We need to build an engine for measuring the impact of each allocation cycle—one that gives us more than a vague sense that it’s working and one that gives the collective more than a single round (per domain) per year to see what’s working. 17 17 - Our recommendations include: 18 18 - Continuing to move away from project-based voting 19 19 - Finding the right balance of metrics and experts’ subjective assessments 20 20 - Focusing on a relatively narrow set of domains but with more rapid feedback cycles 21 21 - Basically, if 2024 was all about experimenting on expertise-based voting and metrics-based voting, then 2025 should be about combining the best of both. Once we discover the optimal combinations, we can expand the scope and complexity of rounds. 22 22 - What humans are good at (and what data is good at) 23 23 - Optimism has now completed five rounds that use badgeholder-based voting to allocate tokens. 24 24 - Each round had different design parameters and these have allowed us to learn different things about what humans and data are good at. 25 25 - Here’s a quick summary: 26 26 - Round Design Parameters Key Lessons Learned Round 2 Projects had to be nominated by a badgeholder in order to make the round. Badgeholders had to determine outright token allocations for each of the 200 projects with minimal tooling. Project nomination process was awkward and time-consuming for badgeholders. Badgeholders needed more structure in the voting process; everyone came up with their own scattershot method (eg, sharing spreadsheets to rate their favorite projects). Round 3 Any project could sign-up for one or more domains and self-report their metrics. Badgeholders would do a light review to filter out spam, but left everything else intact. Projects had to get at least 17 votes in order to receive rewards. Over 600 projects ended up getting approved. Badgeholders still had to determine outright token allocations for each of the 600+ projects. Badgeholders could also create “lists” recommending projects and outright allocations. Voters felt overwhelmed. It was very difficult to differentiate between weak projects and good ones with little reputation; borderline projects led campaigns to reach the quorum line. The domain categories were not strictly enforced and thus mostly useless to badgeholders. Impact metrics were not comparable. Every list was different and there was no quality control. Onchain projects received a smaller share of the token pool than most badgeholders felt they deserved. Round 4 Focused only on onchain builders with strict eligibility requirements (based on onchain activity). Badgeholders were provided metrics instead of projects to vote on. Over 200 projects were approved out of 400+ applicants. Projects could receive a multiplier if they were open source. Badgeholders found voting much easier. Results had a steep power law distribution, but voters generally felt the top projects were rewarded fairly. Metrics alone couldn’t capture quality, momentum, and other nuances, highlighting the need for more complex evaluation signals. Metrics did not work as well for certain “cutting-edge” sub-domains (eg, 4337 related apps). The open source multiplier was too complex to enforce consistently. Round 5 Focused only on OP Stack contributions with strict eligibility requirements (enforced by a small review team). Returned to project-based voting, grouping voters by expertise and assigning each to a single category of 20-30 projects, rather than hundreds. Results were very flat, and voters felt the top projects were not rewarded sufficiently. Perverse incentive to divide work across multiple smaller contributions / teams in order to receive more tokens. Grouping by expertise revealed significant differences between experts and non-experts both in project selection and allocation strategy. After seeing both allocations, voters preferred experts’ selections. 27 27 - In Round 6, which is currently underway, Optimism is experimenting with impact attestations and a more aggressive set of distribution options for badgeholders. 28 28 - Clearly, the optimal Retro Funding design hasn’t been found yet. But we do see some recurring themes: 29 29 - Humans are good at relative comparisons (what’s more valuable) 30 30 - Humans are bad at outright comparisons (how much more valuable) 31 31 - Data is good at providing comprehensive coverage of things that are countable 32 32 - Data is bad at dealing with nuances and qualitative concepts that experts intuitively understand 33 33 - We’ve also learned that people only reveal their true opinions after seeing the result. This follows basic product theory: you need to show people something and iterate based on their reactions in order to build something they actually want. 34 34 - Metrics-driven, experts in the loop 35 35 - These hard-earned lessons inform our recommendation for how Optimism approaches future round designs. The goal should be to combine the best parts of what humans and data are good at. 36 36 - Here is the basic framework we propose: 37 37 - We use metrics-based evaluations to propose initial token allocations within a domain. 38 38 - We let subject matter experts review, fine-tune the metrics, and choose the best allocations. 39 39 - Over time, we identify which metrics best align with experts’ qualitative assessments, refining models through consistent backtesting. 40 40 - This approach leverages data’s systematic reach with human intuition’s nuanced adjustments. Metrics establish a quantitative foundation, ensuring that projects are assessed objectively and fairly, while expert review adds layers of qualitative nuance, including quality, innovation, and momentum. An iterative feedback loop lets us adjust metrics based on expert insights—particularly valuable when experts consistently revise scores or highlight lower-scoring projects. This process is similar to RLHF (Reinforcement Learning with Human Feedback) in machine learning, but with an emphasis on retaining clear, interpretable inputs for expert adjustments. 41 41 - Practically, we implement this by establishing metrics within a domain, generating proposals for initial allocations (by weighting metrics into an evaluation algorithm), and refining the allocations with expert input. Such an approach should work best in domains with lots of verifiable data, e.g., onchain builders and software dependencies. 42 42 - This framework should also perform well in fast-evolving domains. Experts can adjust allocations to reflect Optimism’s shifting priorities (e.g., prioritizing interoperability transactions over standard transactions), address data blind spots (e.g., fine-tuning metrics for 4337-related projects), and reward innovation (e.g., favoring fast-growing, high-potential projects over more established but static ones). 43 43 - One essential element is running funding rounds even more frequently and continuously. Doing so lets us pinpoint metrics most correlated with desirable outcomes and backtest these metrics and evaluation algorithms against historical data. Each round remains an experiment, but the cumulative impact across rounds should reveal a clear, positive trend over time. 44 44 - Hard Choices → Easy Life 45 45 - The end goal remains super ambitious: to develop predictive models that guide economic policy for the collective. For instance, in 2025, we may want to discover that incentivizing certain behaviors predictably leads to increased interoper transaction volume. 46 46 - Reaching this goal won’t be easy. It requires focus, as the outcomes are highly path-dependent. 47 47 - Currently, hard choices need to be made around domain scopes. Any changes may be unpopular, especially among community members accustomed to existing Retro Funding patterns. However, narrowing the scope and committing to continuous improvement within scopes are essential steps in reaching the top of the mountain. 48 48 - image 800×661 27 KB 49 49 - Looking back, we’ll likely see some initial assumptions were overly optimistic or naive. But we can’t improve by continuing to “spray and pray.” Governance is ultimately about making hard choices with limited resources. 50 50 - Optimism has spent two important years learning, but it’s time to double-down on what works in 2025.

-210

rpgf.txt

reviewed

··· 1 1 - 2 2 - --- POST 41452 BY ccerv1 --- 3 3 - tl;dr- 4 4 - In 2023, Optimism ran two megarounds. In 2024, Optimism ran one round per domain per year. We’ve learned that mega rounds devolve into popularity contests and annual feedback loops are too slow. In 2025, Optimism should focus on fewer domains, iterate more rapidly, and refine what works. 5 5 - In 2023, Optimism had everyone vote on everything. In 2024, Optimism ran experiments around expertise-based voting and metrics-based voting. We’ve learned what humans are good at - and what data is good at. In 2025, Optimism should take the best of both (and set aside the rest). 6 6 - In both 2023 and 2024, we struggled at measuring the success of Retro Funding itself. The time for not knowing how well this works is over. 7 7 - Full post… 8 8 - This was supposed to be simple 9 9 - As Vitalik 10 10 - wrote 11 11 - when initially describing the mechanism: “The core principle behind the concept of retroactive public goods funding is simple: it’s easier to agree on what was useful than what will be useful.” 12 12 - Now, nearly two years into this experiment, we should be able to look back at the mechanism of Retro Funding itself and apply this same analysis. 13 13 - Specifically, we want to understand: 14 14 - Can people actually agree on what was useful? 15 15 - Under what circumstances does retroactive funding produce superior outcomes? 16 16 - Simple, right? Not entirely. 17 17 - My team at OSO has been trying to help Optimism answer some of these questions since 18 18 - Round 2 19 19 - . In this post, we’ll look at a very high level at what’s happened across five rounds of Retro Funding (Rounds 2-5) and share some of our observations. 20 20 - The good news is there are some clear learnings about what people are good at agreeing on and how to create the right conditions for consensus. These learnings are the result of deliberate experiments that were undertaken in 2024, for instance, experimenting with metrics-based voting in Round 4 and expertise-based voting in Round 5. 21 21 - The not-so-good news is that we don’t (yet) have the data to show that retroactive funding produces superior outcomes. 22 22 - In our view, solving the measurement problem is the most important thing to get right in 2025. We need to build an engine for measuring the impact of each allocation cycle—one that gives us more than a vague sense that it’s working and one that gives the collective more than a single round (per domain) per year to see what’s working. 23 23 - Our recommendations include: 24 24 - Continuing to move away from project-based voting 25 25 - Finding the right balance of metrics and experts’ subjective assessments 26 26 - Focusing on a relatively narrow set of domains but with more rapid feedback cycles 27 27 - Basically, if 2024 was all about experimenting on expertise-based voting and metrics-based voting, then 2025 should be about combining the best of both. Once we discover the optimal combinations, we can expand the scope and complexity of rounds. 28 28 - What humans are good at (and what data is good at) 29 29 - Optimism has now completed five rounds that use badgeholder-based voting to allocate tokens. 30 30 - Each round had different design parameters and these have allowed us to learn different things about what humans and data are good at. 31 31 - Here’s a quick summary: 32 32 - Round 33 33 - Design Parameters 34 34 - Key Lessons Learned 35 35 - Round 2 36 36 - Projects had to be nominated by a badgeholder in order to make the round. Badgeholders had to determine outright token allocations for each of the 200 projects with minimal tooling. 37 37 - Project nomination process was awkward and time-consuming for badgeholders. Badgeholders needed more structure in the voting process; everyone came up with their own scattershot method (eg, sharing spreadsheets to rate their favorite projects). 38 38 - Round 3 39 39 - Any project could sign-up for one or more domains and self-report their metrics. Badgeholders would do a light review to filter out spam, but left everything else intact. Projects had to get at least 17 votes in order to receive rewards. Over 600 projects ended up getting approved. Badgeholders still had to determine outright token allocations for each of the 600+ projects. Badgeholders could also create “lists” recommending projects and outright allocations. 40 40 - Voters felt overwhelmed. It was very difficult to differentiate between weak projects and good ones with little reputation; borderline projects led campaigns to reach the quorum line. The domain categories were not strictly enforced and thus mostly useless to badgeholders. Impact metrics were not comparable. Every list was different and there was no quality control. Onchain projects received a smaller share of the token pool than most badgeholders felt they deserved. 41 41 - Round 4 42 42 - Focused only on onchain builders with strict eligibility requirements (based on onchain activity). Badgeholders were provided metrics instead of projects to vote on. Over 200 projects were approved out of 400+ applicants. Projects could receive a multiplier if they were open source. 43 43 - Badgeholders found voting much easier. Results had a steep power law distribution, but voters generally felt the top projects were rewarded fairly. Metrics alone couldn’t capture quality, momentum, and other nuances, highlighting the need for more complex evaluation signals. Metrics did not work as well for certain “cutting-edge” sub-domains (eg, 4337 related apps). The open source multiplier was too complex to enforce consistently. 44 44 - Round 5 45 45 - Focused only on OP Stack contributions with strict eligibility requirements (enforced by a small review team). Returned to project-based voting, grouping voters by expertise and assigning each to a single category of 20-30 projects, rather than hundreds. 46 46 - Results were very flat, and voters felt the top projects were not rewarded sufficiently. Perverse incentive to divide work across multiple smaller contributions / teams in order to receive more tokens. Grouping by expertise revealed significant differences between experts and non-experts both in project selection and allocation strategy. After seeing both allocations, voters preferred experts’ selections. 47 47 - In Round 6, which is currently underway, Optimism is experimenting with impact attestations and a more aggressive set of distribution options for badgeholders. 48 48 - Clearly, the optimal Retro Funding design hasn’t been found yet. But we do see some recurring themes: 49 49 - Humans are good at relative comparisons (what’s more valuable) 50 50 - Humans are bad at outright comparisons (how much more valuable) 51 51 - Data is good at providing comprehensive coverage of things that are countable 52 52 - Data is bad at dealing with nuances and qualitative concepts that experts intuitively understand 53 53 - We’ve also learned that people only reveal their true opinions after seeing the result. This follows basic product theory: you need to show people something and iterate based on their reactions in order to build something they actually want. 54 54 - Metrics-driven, experts in the loop 55 55 - These hard-earned lessons inform our recommendation for how Optimism approaches future round designs. The goal should be to combine the best parts of what humans and data are good at. 56 56 - Here is the basic framework we propose: 57 57 - We use metrics-based evaluations to propose initial token allocations within a domain. 58 58 - We let subject matter experts review, fine-tune the metrics, and choose the best allocations. 59 59 - Over time, we identify which metrics best align with experts’ qualitative assessments, refining models through consistent backtesting. 60 60 - This approach leverages data’s systematic reach with human intuition’s nuanced adjustments. Metrics establish a quantitative foundation, ensuring that projects are assessed objectively and fairly, while expert review adds layers of qualitative nuance, including quality, innovation, and momentum. An iterative feedback loop lets us adjust metrics based on expert insights—particularly valuable when experts consistently revise scores or highlight lower-scoring projects. This process is similar to RLHF (Reinforcement Learning with Human Feedback) in machine learning, but with an emphasis on retaining clear, interpretable inputs for expert adjustments. 61 61 - Practically, we implement this by establishing metrics within a domain, generating proposals for initial allocations (by weighting metrics into an evaluation algorithm), and refining the allocations with expert input. Such an approach should work best in domains with lots of verifiable data, e.g., onchain builders and software dependencies. 62 62 - This framework should also perform well in fast-evolving domains. Experts can adjust allocations to reflect Optimism’s shifting priorities (e.g., prioritizing interoperability transactions over standard transactions), address data blind spots (e.g., fine-tuning metrics for 4337-related projects), and reward innovation (e.g., favoring fast-growing, high-potential projects over more established but static ones). 63 63 - One essential element is running funding rounds even more frequently and continuously. Doing so lets us pinpoint metrics most correlated with desirable outcomes and backtest these metrics and evaluation algorithms against historical data. Each round remains an experiment, but the cumulative impact across rounds should reveal a clear, positive trend over time. 64 64 - Hard Choices → Easy Life 65 65 - The end goal remains super ambitious: to develop predictive models that guide economic policy for the collective. For instance, in 2025, we may want to discover that incentivizing certain behaviors predictably leads to increased interoper transaction volume. 66 66 - Reaching this goal won’t be easy. It requires focus, as the outcomes are highly path-dependent. 67 67 - Currently, hard choices need to be made around domain scopes. Any changes may be unpopular, especially among community members accustomed to existing Retro Funding patterns. However, narrowing the scope and committing to continuous improvement within scopes are essential steps in reaching the top of the mountain. 68 68 - image 69 69 - 800×661 27 KB 70 70 - Looking back, we’ll likely see some initial assumptions were overly optimistic or naive. But we can’t improve by continuing to “spray and pray.” Governance is ultimately about making hard choices with limited resources. 71 71 - Optimism has spent two important years learning, but it’s time to double-down on what works in 2025. 72 72 - 73 73 - --- POST 41460 BY Gonna.eth --- 74 74 - Great article Carl as always super helpful! 75 75 - Are there any concrete steps for building “an engine” to track impact? It Would be nice to have a clear picture of how to define, track, and iterate on these metrics, and love the idea of aggregating nuanced human impact over time. 76 76 - Narrowing domains and increasing feedback frequency is great, but it would be helpful to clarify which domains would be prioritized and how to handle potential trade-offs (e.g., does narrowing the scope risk excluding high-impact outlier projects?). Or at least define a few “long-term” metrics from a bigger list, Why? 77 77 - An interesting effect of these metrics in the past is that they’re starting to serve as a guide for projects on what The Foundation, and hopefully The Collective in the future, considers worth funding retroactively. This dynamic shows the influence of Retro Funding metrics as a signaling tool to foster long-term commitment from builders, as they gain confidence that their contributions align with recognized priorities and are likely to be rewarded. Defining long-term metrics could help attract and retain talent, as builders see a clear pathway to recognition and support for their efforts. 78 78 - Narrowing domains may be unpopular, but involving The Collective as much as The Foundation can in decision-making and processes will probably mitigate resistance. 79 79 - After participating in the OP-Stack round, the ‘Metrics-Driven, Experts in the Loop’ approach feels especially compelling and separating voting from judging. I’m inclined to a model where badgeholders delegate their vote to professionals with specific expertise to asses impact and vote on behalf of the Citizen House. I would even prefer Citizens determine the budget and make OP Stack community members do a peer review between themselves for budget allocation. 80 80 - Lastly, about the ‘Guest voter’ experiment: while I believe it’s good to experiment, after the OP Stack report, I think we can all agree that a high schooler who just passed chemistry can’t choose the Nobel Prize winner in chemistry. We should be working on Badgeholder accountability and expansion. 81 81 - 82 82 - --- POST 41466 BY ccerv1 --- 83 83 - Thanks for the feedback Gonna. Will respond in line to several of the topics you raised. 84 84 - Gonna.eth: 85 85 - Are there any concrete steps for building “an engine” to track impact? 86 86 - Not yet. But if there’s support for this general direction, I’d love to help prototype something for RF7 (dev tooling). 87 87 - Gonna.eth: 88 88 - Narrowing domains and increasing feedback frequency is great, but it would be helpful to clarify which domains would be prioritized 89 89 - Some domains would definitely be better suited than others, onchain builders being the most obvious, but in principle this could be done for any domain with public / verifiable data. 90 90 - Gonna.eth: 91 91 - does narrowing the scope risk excluding high-impact outlier projects? 92 92 - Definitely a risk. Hopefully this is a gap that mission requests can fill. 93 93 - Gonna.eth: 94 94 - Or at least define a few “long-term” metrics from a bigger list 95 95 - The 96 96 - Superchain Health dashboard 97 97 - seems like a good place to start. 98 98 - Gonna.eth: 99 99 - Defining long-term metrics could help attract and retain talent, as builders see a clear pathway to recognition and support for their efforts. 100 100 - 100% 101 101 - Gonna.eth: 102 102 - Narrowing domains may be unpopular, but involving The Collective as much as The Foundation can in decision-making and processes will probably mitigate resistance. 103 103 - Agreed - this is an important function of governance. BUT also retro funding shouldn’t be expected to do everything for everyone. I hope mission requests remain broad and adaptable. 104 104 - Gonna.eth: 105 105 - After participating in the OP-Stack round, the ‘Metrics-Driven, Experts in the Loop’ approach feels especially compelling and separating voting from judging. 106 106 - Same here - the findings from the OP Stack round were what really stood out to me. I would also add that subject matter expertise isn’t the only kind of expertise that’s relevant to the job of reviewing projects. Some people bring an expert craft to the process of researching projects, mitigating biases, asking good questions, etc. 107 107 - 108 108 - --- POST 41467 BY joanbp --- 109 109 - ccerv1: 110 110 - I would also add that subject matter expertise isn’t the only kind of expertise that’s relevant to the job of reviewing projects. Some people bring an expert craft to the process of researching projects, mitigating biases, asking good questions, etc. 111 111 - Thank you for putting this into words - I had been looking for a way to communicate exactly this sentiment. 112 112 - Also, while I believe strongly in listening to experts, there is also the matter of building a strong democratic foundation. To me, this is what citizenship is all about: Listening to everyone, discussing any concerns raised, considering pros and cons, and trying to get to a point of rough consensus where there are (ideally) no real losers. 113 113 - I kind of like 114 114 - @Gonna.eth 115 115 - ’s idea about having citizens delegate to experts - but I would like it to only be an option, not a requirement. And it should be possible to only delegate for a round at a time. 116 116 - That way, I might decide to delegate my citizen’s vote to, say, Carl in a round such as RF5, but I might decide to delegate it to LauNaMu or myself in RF6. That way, I, as the citizen, get to decide in each round what 117 117 - kind 118 118 - (kinds) of expertise I value for the task at hand. 119 119 - Apart from the matter of decentralizing and democratizing power by distributing it among (in the future, I hope, many more!) citizens rather than a few fixed ‘badgeholders’ or ‘experts’, I think this has a crucial psychological/pedagogical/social rationale: 120 120 - When you get to hold power, as a citizen, you also get responsibility. Without that, you naturally lack incentive to learn and discuss and ask questions and point out problems and come up with solutions. A certain apathy arises. 121 121 - The past few rounds have brought important insights. I think a major issue with them, though, has been the lack of much debate, both internally among citizens, among citizens and guest voters (experts in RF5 and ‘random’ in RF6) and among citizens and other groups of the Collective in general. 122 122 - As for citizens vs guest voters, debate has in fact been actively discouraged! 123 123 - I understand why, in the name of the experiment. But I think we should also be clear that a lack of communication seriously handicaps the democratic process that might have been. 124 124 - Of course there will always be experts who know more than random citizens about any specific domain. But no group of experts will be better voters 125 125 - in general 126 126 - than a well informed and caring group of citizens who are strong communicators and know how to pull in the experts where it makes sense. 127 127 - The issue is: It takes time and effort to grow well informed and caring and communicating citizens of that kind. It is a set of skills that needs to be learned, through meaningful practice and application to real-world problems that genuinely concern you. And it is relationships and shared frames of reference that have to be seeded and nurtured. 128 128 - You can’t just give random people power, tell them not to talk to the experts and see how they handle it, and then decide that they are not up to the task. 129 129 - Well, you can, but you are not going to learn what a truly informed citizenship might be able to acomplish. 130 130 - If the Citizens House is supposed to be all about focussing on the long-term sustainability of Optimism (and the Superchain), then I believe in the model of informed citizenship where it might be possible to delegate power to subject matter experts on a case by case basis, but where it is equally possible to vote yourself, and where there is always an incentive to learn and discuss and share context and consider many different perspectives and kinds of expertise - and then make up your mind. 131 131 - Anything less than that would be a missed opportunity, the way I see it. 132 132 - 133 133 - --- POST 41537 BY LauNaMu --- 134 134 - I’ve read through this and started analyzing it, however, as I shared with Jonas (who asked me to post it here) before I go into details it would be very valuable to get the direct sources of the summary 135 135 - particularly for the “Key Lessons Learned” section. It’s currently unclear if each of the conclusions presented is derived from a significant amount of data. 136 136 - I think this would additionally strengthen the analysis and the hypothesis that follow. Could these sources please be included? 137 137 - 138 138 - --- POST 41538 BY Gneric --- 139 139 - As a citizen being able to allocate to experts and several well informed citizens seems smart IMO, but what could be the downside of such decision? 140 140 - 141 141 - --- POST 41540 BY ccerv1 --- 142 142 - Hi Lau, 143 143 - I don’t have access to the raw survey results, but here are some screenshots from the post-round badgeholder retros. 144 144 - image 145 145 - 2028×1038 133 KB 146 146 - image 147 147 - 1526×818 14.7 KB 148 148 - image 149 149 - 1500×846 36.8 KB 150 150 - image 151 151 - 1652×952 15 KB 152 152 - Regarding the expert vs non-expert allocations, I have in my notes from the RF5 retro that ~80% of survey respondents preferred the expert allocations from RF5. 153 153 - This 154 154 - thread 155 155 - also has a lot of feedback on RF3, including issues with lists and the relatively low allocation to onchain projects. Trent from Protocol Guild also had a good 156 156 - essay 157 157 - about the perverse incentive to “atomize” work in RF5. 158 158 - Were there other sources you had in mind? 159 159 - 160 160 - --- POST 41541 BY 1a35e1 --- 161 161 - Agree that there needs to be more data oriented processes. The more public data is available the easier it will be to explore and make judgements. 162 162 - Can we work to build out a 163 163 - https://data.optimism.io/ 164 164 - or similar 165 165 - 166 166 - --- POST 41543 BY Chain_L --- 167 167 - That’s a great idea. 168 168 - Currently there is public data around, but scattered everywhere. 169 169 - numbaNERDs can be the path via which this can be maintained. 170 170 - 171 171 - --- POST 41552 BY Deelabs --- 172 172 - We find this retrospective analysis of Optimism’s Retro Funding mechanism to be thorough and insightful, particularly in its systematic evaluation of the evolving funding approaches and their respective outcomes across different rounds. 173 173 - The analysis is proficient in identifying key learnings from each implementation phase, and effectively outlines the strengths and limitations of both human judgment and data-driven approaches. 174 174 - Particularly compelling is the proposed framework that combines metrics-based evaluation with expert oversight. This hybrid approach arguably addresses many of the limitations observed in previous rounds: 175 175 - Leverages data’s systematic reach while maintaining human intuition for nuanced decisions 176 176 - Enables iterative improvement through faster feedback cycles 177 177 - Creates a more sustainable and scalable evaluation process 178 178 - The emphasis on narrowing domain scope while increasing iteration frequency is a pragmatic evolution and in our view, a suitable approach. Rather than attempting to solve everything at once, this approach allows for meaningful refinement of the mechanism within specific contexts. 179 179 - It would be efficient to specify and concretize the domains worthy of further attention and prioritization. 180 180 - We’re supportive of the recommendation to move toward a more focused, metrics-driven approach with expert oversight. The emphasis on continuous improvement and faster iteration cycles should enable more effective resource allocation while building stronger empirical evidence for what works. 181 181 - 182 182 - --- POST 41621 BY noturhandle --- 183 183 - Hey Carl, incredible summary. Thanks for sharing. 184 184 - In the lessons learned for Round 5, you mentioned that teams had perverse incentives to break the work into smaller contributions, but I can’t determine the cause. 185 185 - Can you expand at all? Were projects simply trying to submit to more categories, i.e., spread betting? Or was there some element of the voting design that preferred smaller projects? 186 186 - 187 187 - --- POST 41623 BY ismailemin --- 188 188 - I really liked the article thanks for it Carl. 189 189 - joanbp: 190 190 - That way, I might decide to delegate my citizen’s vote to, say, Carl in a round such as RF5, but I might decide to delegate it to LauNaMu or myself in RF6. That way, I, as the citizen, get to decide in each round what 191 191 - kind 192 192 - (kinds) of expertise I value for the task at hand. 193 193 - Also liked the idea of delegating the VP in the Citizens’ House. Maybe we should create profiles for every badgeholder so anyone can know what are their expertise areas. This also enables the selection of new citizens based on gaps in expertise areas. 194 194 - 195 195 - --- POST 41626 BY ccerv1 --- 196 196 - Hey 197 197 - @noturhandle 198 198 - noturhandle: 199 199 - In the lessons learned for Round 5, you mentioned that teams had perverse incentives to break the work into smaller contributions, but I can’t determine the cause. 200 200 - Can you expand at all? Were projects simply trying to submit to more categories, i.e., spread betting? Or was there some element of the voting design that preferred smaller projects? 201 201 - I think the primary reason is that many badgeholders want to give something to everyone who has had 202 202 - some impact 203 203 - , however small. So they give 1-2% to lots of small projects. Even if there are some zero votes, the median is still likely to be around 1-2%. 204 204 - At the same time, many badgeholders have historically been wary of giving very large allocations to projects they feel are high impact, eg, over 10%. So large projects will also get a lot of 1-2% votes and then a few >10%, but not enough to significantly skew the median. 205 205 - The result is a median of 1-2% for a small project and maybe 3-5% for a large project. 206 206 - If a well-known large project wanted to take advantage of this tendency and optimize its retro funding, it would almost certainly do better by submitting 5 projects with a combined expected value of 5-10% (versus the current 3-5% for a single project). 207 207 - Note: I’m writing this from recall - I’m not looking at actual numbers. 208 208 - 209 209 - --- POST 41889 BY happy --- 210 210 - great info and super useful content