+1 -1

Coordination.md

reviewed

··· 25 25 - The requirements to govern a commons without tragedy: 26 26 - Clear boundaries. 27 27 - Managed by locals. 28 28 - - In a small community, everybody knows everybody, and can keep track of what they do. This makes small groups iterated games which rewards trust and penalizes sociopathic behavior. 28 28 + - In a small community, everybody knows everybody, and can keep track of what they do. This makes small groups iterated games which rewards trust and penalizes sociopathic behavior. 29 29 - Community makes its own rules. 30 30 - Community can monitor behavior. 31 31 - Graduated sanctions for those who violate community rules.

+7 -7

Data/Data Culture.md

reviewed

··· 9 9 - Data is fundamentally a collaborative design process rather than a tool, an analysis, or even a product. [Data works best when the entire feedback loop from ideation to production is an iterative process](https://pedram.substack.com/p/data-can-learn-from-design). 10 10 - [To get buy in, explain how the business could benefit from better data](https://youtu.be/Mlz1VwxZuDs) (e.g: more and better insights). Start small and show value. 11 11 - Run *[Purpose Meetings](https://www.avo.app/blog/tracking-the-right-product-metrics)* or [Business Metrics Review](https://youtu.be/nlMn572Dabc). 12 12 - - Purpose Meetings are 30 min meetings in which stakeholders, engineers and data align on the goal of a release and what is the best way to evaluate the impact and understand its success. Align on the goal, commit on metrics and design the data. 13 13 - - Business Metrics Review is a 30 to 60 minutes meeting to chat and explore key metrics and teach how to think with data. 12 12 + - Purpose Meetings are 30 min meetings in which stakeholders, engineers and data align on the goal of a release and what is the best way to evaluate the impact and understand its success. Align on the goal, commit on metrics and design the data. 13 13 + - Business Metrics Review is a 30 to 60 minutes meeting to chat and explore key metrics and teach how to think with data. 14 14 - Value of clear goals and expectations. Validate what you think your job is with your manager and stakeholders, repeatedly. 15 15 - [While the output of your team is what you want to maximize, you'll need some indicators that will help guide you day-to-day](https://data-columns.hightouch.io/your-first-60-days-as-a-first-data-hire-weeks-3-4/). Decide what's important to you (test coverage, documentation missing, queries run, models created, ...), and generate some internal reports for yourself. 16 16 - [Data teams should be a part of the business conversations from the beginning](https://cultivating-algos.stitchfix.com/). Get the data team involved early, have open discussions with them about the existing work, and how to prioritize new work against the existing backlog. Don’t accept new work without addressing the existing bottlenecks, and don’t accept new work without requirements. **Organizational [[politics]] matter way more than any data methods or technical knowledge**. ··· 19 19 - The modern data team needs to have *real organizational power*—it needs to be able to say "no” and mean it. If your data team does not truly have the power to say no to stakeholders, it will get sent on all kinds of wild goose chases, be unproductive, experience employee churn, etc. 20 20 - Data should report to the CEO. Ideally at least with some weekly metrics split into (a) notable trends, (b) watching close, and (c) business as usual. 21 21 - If data is the most precious asset in a company, does it make sense to have only one team responsible for it? 22 22 - - [People talk about data as the new oil but for most companies it’s a lot closer uranium](https://news.ycombinator.com/item?id=27781286). Hard to find people who can to handle or process it correctly, nontrivial security/liabilities if PII is involved, expensive to store and a generally underwhelming return on effort relative to the anticipated utility. 22 22 + - [People talk about data as the new oil but for most companies it’s a lot closer uranium](https://news.ycombinator.com/item?id=27781286). Hard to find people who can to handle or process it correctly, nontrivial security/liabilities if PII is involved, expensive to store and a generally underwhelming return on effort relative to the anticipated utility. 23 23 - [The pain in data teams come from needing to influence PMs/peers with having little control of them. Data teams need to become really great internal marketers/persuaders](https://anchor.fm/census/episodes/The-evolution-of-the-data-industry--data-jobs-w-Avo-CEO-and-Co-founder-Stefania-Olafsdottir-e16hu1l). That said, it shouldn't be the data team job to convince the organization to be data driven. That's not an effective way of spending resources. 24 24 - People problems are orders of magnitude more difficult to solve than data problems. 25 25 - **Integrate data where the decision is made**. E.g: Google showing restaurant scores when you're looking something for dinner. ··· 42 42 - You won't have the best allocation of resources in a reactive team. Data teams need extra [[slack]]. [Balance user requests with actual needs](https://scientistemily.substack.com/p/product-management-skills-for-data). 43 43 - How can we measure the data team impact? 44 44 - Making a [[Writing a Roadmap|roadmap]] can help you telling if you are hitting milestone deadlines or letting them slip. 45 45 - - Embedded data team members need to help other teams build their roadmap too. 45 45 + - Embedded data team members need to help other teams build their roadmap too. 46 46 - Also, having a changelog ([do releases!](https://betterprogramming.pub/great-data-platforms-use-conventional-commits-51fc22a7417c)) will help show the team impact on the data product across time. 47 47 - [Push for a *centralization of the reporting structure*, but keeping the *work management decentralized*](https://erikbern.com/2021/07/07/the-data-team-a-short-story.html). 48 48 - Unify resources (datasets, entities, definitions, metrics). Have one source of truth for each one and make that clear to everyone. That source of truth needs heavy curation. Poor curation leads to confusion, distrust and…. lots of wasted effort. ··· 84 84 - [Data ownership is a hard problem](https://www.linkedin.com/posts/chad-sanderson_heres-why-data-ownership-is-an-incredibly-activity-6904107936533114880-gw8n/). Data is fundamentally generated by services (or front-end instrumentation) which is managed by engineers. CDC and other pipelines are built by data engineers. The delineation of ownership responsibilities is very rarely established, with each group wanting to push 'ownership' onto someone else so they can do the jobs they were hired for. 85 85 - [Becoming a data-driven organization is a journey, which unfolds over time and requires critical thinking, human judgement, and experimentation](https://hbr.org/2022/02/why-becoming-a-data-driven-organization-is-so-hard). Fail fast, learn faster. 86 86 - [Path to create a data-driven organization](https://twitter.com/_abhisivasailam/status/1520274838450888704): 87 87 - - 1. Get a well-placed leader with influence to message, model, and demand data-driven execution. 88 88 - - 2. Hire/fire based on data aptitude and usage. 89 89 - - 3. Create mechanisms that force analytical conversations. Sometimes there is no way around spending an afternoon breaking down metrics by different segments until you find The Thing. 87 87 + - 1. Get a well-placed leader with influence to message, model, and demand data-driven execution. 88 88 + - 2. Hire/fire based on data aptitude and usage. 89 89 + - 3. Create mechanisms that force analytical conversations. Sometimes there is no way around spending an afternoon breaking down metrics by different segments until you find The Thing. 90 90 - [Start small. Don't try to wrangle data for the entire company until you have the tools and process down for one team](https://data-columns.hightouch.io/your-first-60-days-as-a-first-data-hire-weeks-3-4/). 91 91 - Difficulty to work with data scales exponentially with size. 92 92 - [Rule of thumb; your first customer as a data person should be growth](https://twitter.com/josh_wills/status/1577699871335010304).

+2 -2

Data/Data Engineering.md

reviewed

··· 36 36 - Decouple producers and consumers adding a layer in between. That can be something as simple as a text file or complex as a [[Databases|database]]. 37 37 - **Schemas changes**. Most of the time you won't be there at the exact time of the change so aim to save everything. 38 38 - Ideally, the schema will evolve in a backward compatible way: 39 39 - - Data types don't change in the same column. 40 40 - - Columns are either deleted or added but never renamed. 39 39 + - Data types don't change in the same column. 40 40 + - Columns are either deleted or added but never renamed. 41 41 - Create a few extra columns like `processed_at` or `schema_version`. 42 42 - Generate stats to provide the operator with feedback. 43 43 - Data coming from pipelines should be easily reproducible. If you want to re-run a process, you should ensure that it will produce always the same result. This can be achieved by enforcing the [Functional Data Engineering Paradigm](https://medium.com/@maximebeauchemin/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a).

+1 -1

Decentralized Protocols.md

reviewed

··· 4 4 5 5 - Decentralized protocols become [fat protocols](https://www.usv.com/writing/2016/08/fat-protocols/). 6 6 - On the internet, the main protocols take care of communications (HTTPS, SSH, ...) and apps are built on top. These apps and services store our data in silos. These protocols are necessary but not valuable. Value is captured by the apps. 7 7 - - A great example of a modern open source protocol is [[IPFS]]. 7 7 + - A great example of a modern open source protocol is [[IPFS]]. 8 8 - Fat protocols will use tools like blockchain to store the data. With open protocols and decentralized data ([[Web3]]), apps are only the frontend of the services. 9 9 - [Protocol and Open Source Funding](https://youtu.be/few99D5WnRg?list=WL). It'll add to the current ways to to fund open source projects: 10 10 - Consulting: open source the code, sell consulting.

+1 -1

Feedback Loops.md

reviewed

··· 9 9 - Feedback loops vary in their accuracy. 10 10 - Accurate feedback means that it **reliably** and **clearly** tells you when you do something right. If you get the quadratic formula wrong, you can check the right formula and know what was wrong. 11 11 - Inaccurate feedback loop means that the results of the evaluation phase are “**noisy**” and contain significant variance, so the next cycle will need to take that into account. E.g: playing bowls without a coach. 12 12 - - Learning under conditions of noisy data starts with world construction. Imagine a possible future, and repeat this to generate hundreds of possible future worlds. The main skills and resources required are creativity, [[slack]], and [equanimity](https://en.wikipedia.org/wiki/Equanimity). Creativity leads to a higher rate of idea generation and [[slack]] gives us more time to generate ideas. Equanimity is important because it allows us to persevere in the absence of tangible feedback. 12 12 + - Learning under conditions of noisy data starts with world construction. Imagine a possible future, and repeat this to generate hundreds of possible future worlds. The main skills and resources required are creativity, [[slack]], and [equanimity](https://en.wikipedia.org/wiki/Equanimity). Creativity leads to a higher rate of idea generation and [[slack]] gives us more time to generate ideas. Equanimity is important because it allows us to persevere in the absence of tangible feedback. 13 13 - [The shorter and more accurate the feedback loop, the easier it is to learn](https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loops/). The **tighter** your feedback loop, the better your work. 14 14 - Fast and accurate loops might get you on a local maximum or might not work when the underlying system is noisy. 15 15

+1 -1

Identity.md

reviewed

··· 2 2 3 3 - [Maintain a very small identity](http://www.paulgraham.com/identity.html). The act of labeling yourself is the act of restricting yourself to what you think fits that label. Don't have opinions on everything. Avoid forming an opinion at all about things that are not evident. Do not affiliate your identity with anything extrinsic - such as a religion, political party, country, company, profession, [[Programming]] language, social class, etc. 4 4 - Identity can be helpful in some cases. When we identity as something aligned with our [[Values]] and that can self correct (e.g: rationalism), it encourages you to behave better! 5 5 - - [Try to affiliate more strongly with the communities whose core beliefs would be less dangerous if they turned out to be wrong](https://economicsdetective.com/2016/10/identity-mind-killer/). 5 5 + - [Try to affiliate more strongly with the communities whose core beliefs would be less dangerous if they turned out to be wrong](https://economicsdetective.com/2016/10/identity-mind-killer/). 6 6 - Identity labels are a way of [[Conceptual Compression]]. They help you infer some things about people that identify as something. 7 7 - You're not your opinions. Don't define yourself by what you work on or what you hate. Once a belief becomes part of your identity, any evidence that threatens the belief is a personal attack. 8 8 - The only constant in the world is that it changes. Identify as someone that changes their mind when the data changes!

+3 -3

Journaling.md

reviewed

··· 14 14 - Review a set of recurrent prompts. Tweak them over time. For example: 15 15 - Consistency at your core [[habits]] this week ([[Fitness]], [[Routine]], [[Productivity]], etc.). How can you tweak them to be more consistent or more useful? 16 16 - What did you do this week that was a mistake and how can I avoid repeating it? 17 17 - - What would you like to accomplish next week? 18 18 - - Do you need to clarify something? 19 19 - - Which actions will you move closer to your [[goals]]? 17 17 + - What would you like to accomplish next week? 18 18 + - Do you need to clarify something? 19 19 + - Which actions will you move closer to your [[goals]]?

+1 -1

Mindfulness.md

reviewed

··· 6 6 - One task at a time. [[Focus|No distractions]]. 7 7 - **KISS**. What would less/simple look like? 8 8 - When [[Communication|communicating]], do it in a clear and concise way. 9 9 - - Sometimes, even if your intention is good, signal might turn into noise and won't be interpreted the way you expect. 9 9 + - Sometimes, even if your intention is good, signal might turn into noise and won't be interpreted the way you expect. 10 10 - When [[Problem Solving |facing a problem]], prefer a lean approach with a simple solution and built upon it. Re-framing problems will make easy to give simpler solutions. How would it look like if it was simple? 11 11 - Remove friction. Focus on essentials. Complexity itself has costs. It makes life harder to manage, reduces our degrees of freedom, and so forth. Often people do not factor those costs into their decisions as they incrementally and inattentively complexify their lives. A person with the virtue of simplicity asks, of any decision they make, "does this make my life more complex, and if so is that worth it?" 12 12 - Live smarter, not harder. Don't complain about stuff you can easily fix, [[Automation|automate]], or delegate. Money can buy [[time]].

+1

Open Questions.md

reviewed

··· 1 1 # Open Questions 2 2 3 3 Some questions which I'm intrigued about and haven't researched enough to add their own entry on this handbook. 4 4 + 4 5 Inspired by [Gwern Questions](https://www.gwern.net/Questions), [Patrick Collison](https://patrickcollison.com/questions) and [Alexey Guzey](https://guzey.com/personal/research-ideas/). 5 6 6 7 - [What is a good approach to effective education?](https://www.lesswrong.com/posts/mMKKsbxGiNirGjsA9/effective-children-education).

+2 -2

Organizations.md

reviewed

··· 14 14 - The purpose for which the company exists. 15 15 - The way people within a company do things—how they communicate, how they create, how they decide—is that company's operating system. 16 16 - All organizations have their own way of doing things ([[Culture]]) and their own particular views of how the world should work (vision). 17 17 - - **Vision** is the change in the world you want to see. 18 18 - - **Mission** defines the approach that you will execute to make your vision a reality. The **strategy** further refines the mission into concrete steps. 17 17 + - **Vision** is the change in the world you want to see. 18 18 + - **Mission** defines the approach that you will execute to make your vision a reality. The **strategy** further refines the mission into concrete steps. 19 19 - There are two core processes that take place within any company: making decisions and doing work. 20 20 - Align [[incentives]]. Make the right thing to do the easiest thing to do. 21 21 - The layout of the organization impacts how many hops the information has to do.

+2

Parenting.md

reviewed

··· 19 19 - Clearly communicated consequences ahead of time works much better than punishment. 20 20 - Raising responsible humans requires giving them responsibility, the opportunity for soft failures to learn from and having those awkward long talks and figuring out how to set healthy boundaries. 21 21 - Allowing kids to have lots of small failures. Whenever you can, let them experience a small consequence that won't be too bad. 22 22 + 22 23 * Allow them to experience the natural consequences of their actions as much as possible, both good and bad. Even if it has a negative consequence for them, as long as it's not too much. 24 24 + 23 25 * Aim to be as predictable as possible. The brain is a prediction engine and hates surprises, so minimize the surprises they have to deal with. 24 26 25 27 ## Resources

+2 -2

Reducing Environmental Impact.md

reviewed

··· 15 15 - Donate to birth control projects. 16 16 - [[Systems|Systemic problems need systemic solutions]]. We can't get there if everyone thinks they already "did their part" (turning off light switches, turning the thermostat down a degree, paper/metal straws, sorting recycling). 17 17 - [Individual actions aren't enough to improve the current state](https://www.youtube.com/watch?v=DYLWZPFEWTw). They give a false sense of progress [while not doing much progress](https://criticalscience.medium.com/climate-change-on-a-little-planet-b859721767d5). E.g: recycling doesn't seem to make a dent in climate change and [recycled plastics are often actively harmful, poisoning people or ending up in the sea](https://hwfo.substack.com/p/an-illustrated-guide-to-plastic-straws). Choices like your diet or how much you take the car have a much bigger impact. 18 18 - - [The cultural dynamic in many places of serving more food than your guests can possibly eat–as a form of status or generosity–is persistent and wasteful. But it's just a small part of a system that needs fixing](https://seths.blog/2023/04/profiting-from-food-waste-confusion/). 18 18 + - [The cultural dynamic in many places of serving more food than your guests can possibly eat–as a form of status or generosity–is persistent and wasteful. But it's just a small part of a system that needs fixing](https://seths.blog/2023/04/profiting-from-food-waste-confusion/). 19 19 - [By focusing on the individuals, we miss the much bigger problem of climate change.](https://youtu.be/RSgXcFdHxFI?list=WL). Around 70% of emissions come for 100 companies. Push governments to transform the economy so individuals don't have all the pressure. 20 20 - It's hard to make changes since climate change happens slowly and doesn't trigger our fight and fly response. We need [[Incentives]] that makes buying and using the good things the selfish thing to do. 21 21 - What's good for the individual is harming the collective and no one is going to concede because is not [[Thinking|rational]]. E.g: it's cheaper to buy an foul car than an electric one. 22 22 - - Tesla is changing the incentives so even if you don't care about environmental change, you'd buy a Tesla. 22 22 + - Tesla is changing the incentives so even if you don't care about environmental change, you'd buy a Tesla.