+439

__tests__/standardizeToAtProto.test.ts

reviewed

··· 1 1 + import { expect, test, describe } from 'vitest' 2 2 + import { SiteStandardDocument } from '../src/standardizeToAtProto' 3 3 + 4 4 + let markdownText = "> When I envision the web, I picture an infinite expanse of empty space that stretches as far as the eye can see. It's full of fertile soil, but no seeds have taken root. That is, except for about an acre of it.\n - Molly White, [We can have a different web](/noteworthy/we-can-have-a-different-web/)\n\nThe future of the internet seems up in the air. Consumed by rotting behemoths. What we have now is failing, but it is also part of our every-day life, our politics, our society, our communities and our friendships. All of those are at risk, in part because the ways we communicate are under attack. The fate of the open web is inextricable from the other ways our world is in crisis. \n\nWe face down a seemingly endless series of political, social, economic, and medical disasters on a melting planet. There is no doubt that the internet is deeply integrated into everything that is going both right and wrong.\n\nAt this moment, [we are actively discussing how to build a future for our shared global telecommunications network](https://aramzs.xyz/lists/a-future-for-the-web/). This is good. We need this discussion. We also need to acknowledge that the wide open platform for infinite data, innovation and communication isn't a shelter from the disaster, but it *is* at the heart of everything we do. The day we could go back from that, even if we wanted to, is long gone. That means [rewilding the web](https://www.noemamag.com/we-need-to-rewild-the-internet/) isn't **just** a matter of a better internet. Our way to a better web can't be purely technical. It's the matter of being part of the making of a better world. \n\nWith the problem so large, how do we approach a solution? If we want to plant a forest how do we grow the ground cover? \n\nDe-abstract. What is the internet really? It's wires, and waves (and occasionally, joking aside, [pipes](https://www.windsystemsmag.com/the-latest-advancements-in-submarine-cables-protection/)) all put to the purpose of connecting people to each other. \n\nThe internet then is the series of connections between [people](https://adactio.com/journal/18337), at a huge scale. Not everyone is connected to everyone, but if you're on the internet, you're connecting to *someone*. So it isn't so much a world wide web as it is world wide **webs**. \n\nThe future core of a [humane](https://humanewebmanifesto.com/) internet, if we are going to be able to use it to do the essential work of surviving this century, is a series of linked routes that stick us all together and supports our communities.\n\nI don't think there is only one solution, I couldn't possibly know them all, but I have one approach that has led me to launch this particular website (and [some others](/projects/)). \n\nThe thing about webs is [they are failure resistent, one strand falling won't take the whole thing down](https://news.mit.edu/2012/spider-web-strength-0202). You can punch a hole through them, but webs are also rebuilt regularly, they can fail and come back quickly. \n\nWe need to think of our connections and how we transmit information to each other in the same way. An important message should be able to travel edge to edge without interference and make it from web to web successfully. We need to think about *links*. Not just in the literal sense. We weave our communities together by establishing bedrocks of shared knowledge. \n\nI think what really put this into sharp focus for me is [the long death of Twitter](https://www.schizochronotopia.com/p/on-twitter-we-look-down). Twitter is still a platform where, for professional reasons, I'm quite active, but its biggest utility--directing me to information that I'm interested in or that is important to me--that's gone. \n\nWe need to rethink how amplification works without requiring big tech or even a steady internet connection. Here are three big assumptions sitting inside this larger question of how best to build our webs. \n\n1. Amplification through cross-linking between individual entities and feeds is essentially rejuvenating for the web. \n2. We [can't](https://ahrefs.com/blog/link-rot-study/) [take the life of those links for granted](https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/) and an essential part of this rejuvenation process is archiving links that are important enough to amplify.\n3. In the current world [it is nearly impossible](https://www.nytimes.com/2020/07/31/technology/blocking-the-tech-giants.html) to build without touching or using the technology of a mega-corporation in some way so we should expect to build on things that will fail, enshittify, or rot, and need to be moved off from quickly.\n\n## To that end I've considered some principles for my own projects.\n\nThese (and the project of crafting them) are heavily inspired by [the IndieWeb principles](https://indieweb.org/principles). \n\n### Principles of Amplification for better webs\n\n- Aggregation is amplification.\n- In a post-Twitter world, reach requires coordination. \n- If it has to query a backend to load it will one day die. \n- Amplification is power, use it responsibly. \n- Massive replication of your own work is good. Massive aggregation between members of a community is even better. \n- [Amplification](/thoughts/using-amplify-as-a-type/) is a method to build community\n- You can always [check out](https://dataprivacymanager.net/what-is-data-subject-access-request-dsar/), copy, or scrape your work from other platforms and take ownership.\n\n### Principles of Archiving for better webs\n\n- Using the web downloads it to your computer, you should get to keep it for your personal use.\n- [Kill the app](https://tweets.aramzs.com/1390071898202120192/). They are black boxes filled with content and knowledge that will one day be lost forever.\n- Everyone should support the Internet Archive, but we [shouldn't](https://www.forbes.com/sites/mattnovak/2023/05/28/internet-archive-suffers-major-global-outage/) assume it is invulnerable. \n- Own your own archive. \n- Single points of failure are dangerous and, as an expected operating tool, destroy innovation.\n- Anything that can't fit on its own offline drive will die.\n\n### Principles of Web Weaving\n\n- Connecting to each other connects our audiences.\n- Curated connections form bridges out of the walled gardens that hurt us and our society. \n- Accept the rotting giant corporations that harvest human connections for profit, realize you must exist in their context to help others, and use their platforms and technology as a vaulting poll to bring yourself and others outside the walls.\n- The [baseline has shifted](/glossary/shifting-baselines/) away from an open web, but if we act like we are already in the future we want to build we can shift them again.\n\n## The Long Next\n\nWith these concepts in mind, I think of a lot of acronyms I've been considering as part of the future of the web: \n\n- [ESC](https://esc.fyi/) - End Surveillance Capitalism\n- [POSSE](https://indieweb.org/POSSE) - Publish (on your) Own Site, Syndicate Elsewhere\n- [PESOS](https://indieweb.org/PESOS) - Publish Elsewhere, Syndicate (to your) Own Site\n\nI want to add one more to the list. \n\nWe need to think about how to [ENTER](/glossary/enter/) the wild web.\n\n### Extract\n\nFor now it's [impossible to interact with others and move through this world without touching big tech](https://www.nytimes.com/2020/07/31/technology/blocking-the-tech-giants.html). With this in mind, we are all going to end up using some of those platforms. So what we need are methods to extract the data off those platforms. There are blunt ways like scraping, and potentially captured ways like platform APIs. However, thanks to the explosion of privacy and interoperability-related legislation around the world, [most sites have an option to check out all your data](https://dataprivacymanager.net/what-is-data-subject-access-request-dsar/). \n\nOnce you have reclaimed your own works, you can do all sorts of things with it. Eventually you can even leave the platform, delete everything, and take canonical status for that work with your own sites. [I'm just getting started](https://aramzs.xyz/tutorials/how-to-export-your-links-from-pocket/), but I plan on writing more about how to do this.\n\n### Nullify \n\nRedirect or cross link your own sites and platform over the big tech ones whenever possible. Once you've linked off a enshittified platform and brought someone out of the walled garden and into your web, it's important to give them ways to interact with the internet that don't just push them right back into the hands of big tech. Whenever possible, push readers to open and wild web alternatives. \n\nCreate sites and networks that aren't just informative, but also fun to explore. \n\n### Transform \n\nTake the posts you've checked out or otherwise extracted and turn them into something new and cool. Everything we post on social media platforms owned by big tech companies is limited by the business models and requirement to render ad targeting data. It shapes our participation in ways we aren't aware of. Once we free our *own* works from those platforms, take the time and effort to turn them into something cool and new, no longer limited by the constraints from inside the wall. \n\nOne of my main projects for this site is building tools for transforming data extractions from [Letterboxd](/lists/film-and-tv/), [Pocket](/amplify/), and [Kindle](/resources/quotes/).\n\n### Exchange \n\nBuild community though connecting. Link to other people, this is hardly a new idea, but it does seem to have fallen by the wayside at various periods of the social web. \n\nTruly fair exchanges between members of the (or a) web means more than that though, it means actively getting permission from others to remix their work and doing it. It means using and building on top of the work of others in all formats, not just code.\n\nIt also means: [copyleft](https://en.wikipedia.org/wiki/Copyleft) your work. The future of the web is not copyrighted. The future of the web is post-capitalism. Trying to play copyright games with big tech is a losing game, all we get is siloed culture that big tech steals anyway, either directly or in forms of aggregation or by bundling it into AI which spits it back out as plagiarism and bullshit. The future of the web isn't just open source code, it's [open creativity](https://adactio.com/articles/1522/). It's is [share and share alike](https://creativecommons.org/share-your-work/cclicenses/) to build, remix and replicate the best things on the internet. \n\n### Replicate \n\nWe should understand that there is no \"Long Now\". VCs, big clocks and big money are a fleeting phenomenon that is more likely to self-destruct or destroy us than it is to save us. The foundation of projects like The Long Now is built on the resources of people who think we need to maintain the present, scrolled out into an eternally status quo-maintaining future. \n\nLet me be blunt: The present isn't worth preserving. \n\nLet's think instead about **The Long Next**, a future where we build sustainably; live with the knowledge that [we can't be always-online](/resources/bookmarks/low-tech-magazine/); and consider a world that **we** sustain in our projects, groups and communities. The Long Next isn't about preserving wealth and capital long enough to catapult the rich to Mars, it is about holistic rewilding of the web, the Earth, the future. A world we can live in, not one we are going to sacrifice on the alter of wealthy egos. \n\nCentral archives are incredibly important, but they're also central points of failure, vulnerable not just to physical attacks, but to legal ones as well. Worse, it is at the mercy of big donors who have proved time and again that they'd rather burn the present to build their personal vision of the future than sustain it. It's hard to believe such people will forever consider supporting projects like The Internet Archive in their best interest, especially as they sit above companies whose works are focused on essentially in-housing that work *as a competitive advantage*. \n\nWe should continue to support big important archival projects. I think their work is amazing and deserves all our support to keep them going as long as we can. \n\nA long next, a sustainable future, means that we need to do more than that, we need to think about how to keep the archives of what is important to us close to home, replicate them across our local networks and retain their information in case of the worst. \n\nThe parts of the web that are most important to you should be available to you, even if the internet isn't.\n\nSo use your newly wild websites to preserve the parts of the web that matter to you. Host site-local archives with indicators that point back to and [300s](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection) back to the originals. Keep the option to take them with you, or to drop redirects if they die so you'll have them *just in case*. Make sure to set up those local archives so they can be accessed by you, by others, and using your own search tools. [Submit](https://web.archive.org/save) [those](https://chromewebstore.google.com/detail/wayback-machine/fpnmgdkabkmnadcjpehmlllkndpkmiak) [links](https://archive.org/details/introduction-to-the-warc) [to](https://help.archive.org/help/uploading-a-basic-guide/) [The Internet Archive](https://help.archive.org/help/save-pages-in-the-wayback-machine/) at the same time. \n\n[Make sure what you build is easy for others to archive](https://jeffhuang.com/designed_to_last/).\n\n## The Future Soon \n\nThis is a summary. I think I could explore any one of these points more deeply and maybe I will. This is a good start at trying to describe what I want to do with this site and the other sites I'm working on.\n\nI think that working in public and thinking in public keeps me honest, so that's what I've got here. If it's interesting to you, maybe it is something you can explore as well, try to follow through with on your own websites. Increasingly I wish for a **zine-web** filled with tiny one-off websites and fun projects, a million well-tended webs intersecting with each other, and archived by their peers. \n\n\nWe can join our own little webs together. One thing is certain, making the web a better place isn't a project to undertake alone or in private. \n\n[Reach out on Mastodon to talk](https://indieweb.social/@Chronotope). \n\n~~\n\nSelected Responses: \n\n- [Don Marti](https://federate.social/@dmarti/112587045311696156)\n- I really like [the idea of bringing back Diskmags](https://elk.zone/indieweb.social/@teajaygrey@rap.social/112606872290801809). I was not familiar with the concept at all, but ended up [finding out about it when I researched the response](https://diskmags.de/index.php?title=Introduction) and I like the idea of removable media used as a potential model for a sometimes-on web. \n- There is [a really cool list of articles that talk about this topic of the future of a better web which you can check out](https://projects.kwon.nyc/internet-is-fun/). \n\n~~\n\nCover Image by <a href=\"https://commons.wikimedia.org/wiki/User:Chen-Pan_Liao\" title=\"User:Chen-Pan Liao\">Chen-Pan Liao</a> - <span class=\"int-own-work\" lang=\"en\">Own work</span>, <a href=\"https://creativecommons.org/licenses/by-sa/3.0\" title=\"Creative Commons Attribution-Share Alike 3.0\">CC BY-SA 3.0</a>, <a href=\"https://commons.wikimedia.org/w/index.php?curid=15926393\">Link</a>" 5 5 + let plainText = "When I envision the web, I picture an infinite expanse of empty space that stretches as far as the eye can see. It's full of fertile soil, but no seeds have taken root. That is, except for about an acre of it.\nMolly White, We can have a different web\n\nThe future of the internet seems up in the air. Consumed by rotting behemoths. What we have now is failing, but it is also part of our every-day life, our politics, our society, our communities and our friendships. All of those are at risk, in part because the ways we communicate are under attack. The fate of the open web is inextricable from the other ways our world is in crisis.\n\nWe face down a seemingly endless series of political, social, economic, and medical disasters on a melting planet. There is no doubt that the internet is deeply integrated into everything that is going both right and wrong.\n\nAt this moment, we are actively discussing how to build a future for our shared global telecommunications network. This is good. We need this discussion. We also need to acknowledge that the wide open platform for infinite data, innovation and communication isn't a shelter from the disaster, but it is at the heart of everything we do. The day we could go back from that, even if we wanted to, is long gone. That means rewilding the web isn't just a matter of a better internet. Our way to a better web can't be purely technical. It's the matter of being part of the making of a better world.\n\nWith the problem so large, how do we approach a solution? If we want to plant a forest how do we grow the ground cover?\n\nDe-abstract. What is the internet really? It's wires, and waves (and occasionally, joking aside, pipes) all put to the purpose of connecting people to each other.\n\nThe internet then is the series of connections between people, at a huge scale. Not everyone is connected to everyone, but if you're on the internet, you're connecting to someone. So it isn't so much a world wide web as it is world wide webs.\n\nThe future core of a humane internet, if we are going to be able to use it to do the essential work of surviving this century, is a series of linked routes that stick us all together and supports our communities.\n\nI don't think there is only one solution, I couldn't possibly know them all, but I have one approach that has led me to launch this particular website (and some others).\n\nThe thing about webs is they are failure resistent, one strand falling won't take the whole thing down. You can punch a hole through them, but webs are also rebuilt regularly, they can fail and come back quickly.\n\nWe need to think of our connections and how we transmit information to each other in the same way. An important message should be able to travel edge to edge without interference and make it from web to web successfully. We need to think about links. Not just in the literal sense. We weave our communities together by establishing bedrocks of shared knowledge.\n\nI think what really put this into sharp focus for me is the long death of Twitter. Twitter is still a platform where, for professional reasons, I'm quite active, but its biggest utility--directing me to information that I'm interested in or that is important to me--that's gone.\n\nWe need to rethink how amplification works without requiring big tech or even a steady internet connection. Here are three big assumptions sitting inside this larger question of how best to build our webs.\nAmplification through cross-linking between individual entities and feeds is essentially rejuvenating for the web.\nWe can't take the life of those links for granted and an essential part of this rejuvenation process is archiving links that are important enough to amplify.\nIn the current world it is nearly impossible to build without touching or using the technology of a mega-corporation in some way so we should expect to build on things that will fail, enshittify, or rot, and need to be moved off from quickly.\n\nTo that end I've considered some principles for my own projects.\n\nThese (and the project of crafting them) are heavily inspired by the IndieWeb principles.\n\nPrinciples of Amplification for better webs\nAggregation is amplification.\nIn a post-Twitter world, reach requires coordination.\nIf it has to query a backend to load it will one day die.\nAmplification is power, use it responsibly.\nMassive replication of your own work is good. Massive aggregation between members of a community is even better.\nAmplification is a method to build community\nYou can always check out, copy, or scrape your work from other platforms and take ownership.\n\nPrinciples of Archiving for better webs\nUsing the web downloads it to your computer, you should get to keep it for your personal use.\nKill the app. They are black boxes filled with content and knowledge that will one day be lost forever.\nEveryone should support the Internet Archive, but we shouldn't assume it is invulnerable.\nOwn your own archive.\nSingle points of failure are dangerous and, as an expected operating tool, destroy innovation.\nAnything that can't fit on its own offline drive will die.\n\nPrinciples of Web Weaving\nConnecting to each other connects our audiences.\nCurated connections form bridges out of the walled gardens that hurt us and our society.\nAccept the rotting giant corporations that harvest human connections for profit, realize you must exist in their context to help others, and use their platforms and technology as a vaulting poll to bring yourself and others outside the walls.\nThe baseline has shifted away from an open web, but if we act like we are already in the future we want to build we can shift them again.\n\nThe Long Next\n\nWith these concepts in mind, I think of a lot of acronyms I've been considering as part of the future of the web:\nESC - End Surveillance Capitalism\nPOSSE - Publish (on your) Own Site, Syndicate Elsewhere\nPESOS - Publish Elsewhere, Syndicate (to your) Own Site\n\nI want to add one more to the list.\n\nWe need to think about how to ENTER the wild web.\n\nExtract\n\nFor now it's impossible to interact with others and move through this world without touching big tech. With this in mind, we are all going to end up using some of those platforms. So what we need are methods to extract the data off those platforms. There are blunt ways like scraping, and potentially captured ways like platform APIs. However, thanks to the explosion of privacy and interoperability-related legislation around the world, most sites have an option to check out all your data.\n\nOnce you have reclaimed your own works, you can do all sorts of things with it. Eventually you can even leave the platform, delete everything, and take canonical status for that work with your own sites. I'm just getting started, but I plan on writing more about how to do this.\n\nNullify\n\nRedirect or cross link your own sites and platform over the big tech ones whenever possible. Once you've linked off a enshittified platform and brought someone out of the walled garden and into your web, it's important to give them ways to interact with the internet that don't just push them right back into the hands of big tech. Whenever possible, push readers to open and wild web alternatives.\n\nCreate sites and networks that aren't just informative, but also fun to explore.\n\nTransform\n\nTake the posts you've checked out or otherwise extracted and turn them into something new and cool. Everything we post on social media platforms owned by big tech companies is limited by the business models and requirement to render ad targeting data. It shapes our participation in ways we aren't aware of. Once we free our own works from those platforms, take the time and effort to turn them into something cool and new, no longer limited by the constraints from inside the wall.\n\nOne of my main projects for this site is building tools for transforming data extractions from Letterboxd, Pocket, and Kindle.\n\nExchange\n\nBuild community though connecting. Link to other people, this is hardly a new idea, but it does seem to have fallen by the wayside at various periods of the social web.\n\nTruly fair exchanges between members of the (or a) web means more than that though, it means actively getting permission from others to remix their work and doing it. It means using and building on top of the work of others in all formats, not just code.\n\nIt also means: copyleft your work. The future of the web is not copyrighted. The future of the web is post-capitalism. Trying to play copyright games with big tech is a losing game, all we get is siloed culture that big tech steals anyway, either directly or in forms of aggregation or by bundling it into AI which spits it back out as plagiarism and bullshit. The future of the web isn't just open source code, it's open creativity. It's is share and share alike to build, remix and replicate the best things on the internet.\n\nReplicate\n\nWe should understand that there is no \"Long Now\". VCs, big clocks and big money are a fleeting phenomenon that is more likely to self-destruct or destroy us than it is to save us. The foundation of projects like The Long Now is built on the resources of people who think we need to maintain the present, scrolled out into an eternally status quo-maintaining future.\n\nLet me be blunt: The present isn't worth preserving.\n\nLet's think instead about The Long Next, a future where we build sustainably; live with the knowledge that we can't be always-online; and consider a world that we sustain in our projects, groups and communities. The Long Next isn't about preserving wealth and capital long enough to catapult the rich to Mars, it is about holistic rewilding of the web, the Earth, the future. A world we can live in, not one we are going to sacrifice on the alter of wealthy egos.\n\nCentral archives are incredibly important, but they're also central points of failure, vulnerable not just to physical attacks, but to legal ones as well. Worse, it is at the mercy of big donors who have proved time and again that they'd rather burn the present to build their personal vision of the future than sustain it. It's hard to believe such people will forever consider supporting projects like The Internet Archive in their best interest, especially as they sit above companies whose works are focused on essentially in-housing that work as a competitive advantage.\n\nWe should continue to support big important archival projects. I think their work is amazing and deserves all our support to keep them going as long as we can.\n\nA long next, a sustainable future, means that we need to do more than that, we need to think about how to keep the archives of what is important to us close to home, replicate them across our local networks and retain their information in case of the worst.\n\nThe parts of the web that are most important to you should be available to you, even if the internet isn't.\n\nSo use your newly wild websites to preserve the parts of the web that matter to you. Host site-local archives with indicators that point back to and 300s back to the originals. Keep the option to take them with you, or to drop redirects if they die so you'll have them just in case. Make sure to set up those local archives so they can be accessed by you, by others, and using your own search tools. Submit those links to The Internet Archive at the same time.\n\nMake sure what you build is easy for others to archive.\n\nThe Future Soon\n\nThis is a summary. I think I could explore any one of these points more deeply and maybe I will. This is a good start at trying to describe what I want to do with this site and the other sites I'm working on.\n\nI think that working in public and thinking in public keeps me honest, so that's what I've got here. If it's interesting to you, maybe it is something you can explore as well, try to follow through with on your own websites. Increasingly I wish for a zine-web filled with tiny one-off websites and fun projects, a million well-tended webs intersecting with each other, and archived by their peers.\n\nWe can join our own little webs together. One thing is certain, making the web a better place isn't a project to undertake alone or in private.\n\nReach out on Mastodon to talk.\n\n~~\n\nSelected Responses:\nDon Marti\nI really like the idea of bringing back Diskmags. I was not familiar with the concept at all, but ended up finding out about it when I researched the response and I like the idea of removable media used as a potential model for a sometimes-on web.\nThere is a really cool list of articles that talk about this topic of the future of a better web which you can check out.\n\n~~\n\nCover Image by Chen-Pan Liao - Own work, CC BY-SA 3.0, Link" 6 6 + 7 7 + describe('SiteStandardDocument', () => { 8 8 + // Test basic instantiation 9 9 + test('should create a document with required fields', () => { 10 10 + const doc = new SiteStandardDocument({ 11 11 + site: 'https://example.com', 12 12 + path: '/my-post', 13 13 + title: 'Test Title', 14 14 + description: 'Test description', 15 15 + markdownContent: '# Hello World', 16 16 + publishedAt: new Date('2024-01-01'), 17 17 + updatedAt: new Date('2024-01-02'), 18 18 + tags: ['test', 'example'] 19 19 + }); 20 20 + 21 21 + expect(doc.$type).toBe('site.standard.document'); 22 22 + expect(doc.site).toBe('https://example.com'); 23 23 + expect(doc.path).toBe('/my-post'); 24 24 + expect(doc.title).toBe('Test Title'); 25 25 + expect(doc.description).toBe('Test description'); 26 26 + expect(doc.tags).toEqual(['test', 'example']); 27 27 + }); 28 28 + 29 29 + test('should accept at:// protocol URLs', () => { 30 30 + const doc = new SiteStandardDocument({ 31 31 + site: 'at://did:plc:abc123', 32 32 + path: '/post', 33 33 + title: 'AT Protocol Post', 34 34 + description: 'A post on AT Protocol', 35 35 + markdownContent: 'Content', 36 36 + publishedAt: new Date(), 37 37 + updatedAt: new Date(), 38 38 + tags: [] 39 39 + }); 40 40 + 41 41 + expect(doc.site).toBe('at://did:plc:abc123'); 42 42 + }); 43 43 + 44 44 + // Test markdown conversion 45 45 + test('should convert markdown to content object', () => { 46 46 + const markdown = '# Title\n\nThis is content.'; 47 47 + const doc = new SiteStandardDocument({ 48 48 + site: 'https://example.com', 49 49 + path: '/post', 50 50 + title: 'Title', 51 51 + description: 'Description', 52 52 + markdownContent: markdown, 53 53 + publishedAt: new Date(), 54 54 + updatedAt: new Date(), 55 55 + tags: [] 56 56 + }); 57 57 + 58 58 + expect(doc.content).toEqual({ 59 59 + $type: 'site.standard.content.markdown', 60 60 + text: markdown, 61 61 + version: '1.0' 62 62 + }); 63 63 + }); 64 64 + 65 65 + test('should use provided content object instead of generating from markdown', () => { 66 66 + const customContent = { 67 67 + $type: 'site.standard.content.html', 68 68 + text: '<h1>HTML Content</h1>', 69 69 + version: '2.0' 70 70 + }; 71 71 + 72 72 + const doc = new SiteStandardDocument({ 73 73 + site: 'https://example.com', 74 74 + path: '/post', 75 75 + title: 'Title', 76 76 + description: 'Description', 77 77 + markdownContent: '# Markdown', 78 78 + publishedAt: new Date(), 79 79 + updatedAt: new Date(), 80 80 + tags: [], 81 81 + content: customContent 82 82 + }); 83 83 + 84 84 + expect(doc.content).toEqual(customContent); 85 85 + }); 86 86 + 87 87 + // Test optional fields 88 88 + test('should handle optional rkey field', () => { 89 89 + const doc = new SiteStandardDocument({ 90 90 + site: 'https://example.com', 91 91 + path: '/post', 92 92 + title: 'Title', 93 93 + description: 'Description', 94 94 + markdownContent: 'Content', 95 95 + publishedAt: new Date(), 96 96 + updatedAt: new Date(), 97 97 + tags: [], 98 98 + rkey: 'custom-rkey-123' 99 99 + }); 100 100 + 101 101 + expect(doc.rkey).toBe('custom-rkey-123'); 102 102 + }); 103 103 + 104 104 + test('should handle optional coverImage field', () => { 105 105 + const coverImage = { 106 106 + $type: 'blob' as const, 107 107 + ref: { $link: 'bafyrei...' }, 108 108 + mimeType: 'image/jpeg' as any 109 109 + }; 110 110 + 111 111 + const doc = new SiteStandardDocument({ 112 112 + site: 'https://example.com', 113 113 + path: '/post', 114 114 + title: 'Title', 115 115 + description: 'Description', 116 116 + markdownContent: 'Content', 117 117 + publishedAt: new Date(), 118 118 + updatedAt: new Date(), 119 119 + tags: [], 120 120 + coverImage 121 121 + }); 122 122 + 123 123 + expect(doc.coverImage).toEqual(coverImage); 124 124 + }); 125 125 + 126 126 + test('should handle optional bskyPostRef field', () => { 127 127 + const bskyPostRef = { 128 128 + $type: 'com.atproto.repo.strongRef' as const, 129 129 + uri: 'at://did:plc:abc123' as const, 130 130 + cid: 'bafyrei123' 131 131 + }; 132 132 + 133 133 + const doc = new SiteStandardDocument({ 134 134 + site: 'https://example.com', 135 135 + path: '/post', 136 136 + title: 'Title', 137 137 + description: 'Description', 138 138 + markdownContent: 'Content', 139 139 + publishedAt: new Date(), 140 140 + updatedAt: new Date(), 141 141 + tags: [], 142 142 + bskyPostRef 143 143 + }); 144 144 + 145 145 + expect(doc.bskyPostRef).toEqual(bskyPostRef); 146 146 + }); 147 147 + 148 148 + // Test content update methods 149 149 + test('should update content and timestamp', () => { 150 150 + const doc = new SiteStandardDocument({ 151 151 + site: 'https://example.com', 152 152 + path: '/post', 153 153 + title: 'Title', 154 154 + description: 'Description', 155 155 + markdownContent: 'Original content', 156 156 + publishedAt: new Date('2024-01-01'), 157 157 + updatedAt: new Date('2024-01-01'), 158 158 + tags: [] 159 159 + }); 160 160 + 161 161 + const originalUpdatedAt = doc.updatedAt; 162 162 + 163 163 + // Wait a tiny bit to ensure timestamp changes 164 164 + setTimeout(() => { 165 165 + doc.updateContent('New content'); 166 166 + 167 167 + expect(doc.content.text).toBe('New content'); 168 168 + expect(doc.textContent).toBe('New content'); 169 169 + expect(doc.updatedAt.getTime()).toBeGreaterThan(originalUpdatedAt.getTime()); 170 170 + }, 10); 171 171 + }); 172 172 + 173 173 + // Test tag management 174 174 + test('should add tags', () => { 175 175 + const doc = new SiteStandardDocument({ 176 176 + site: 'https://example.com', 177 177 + path: '/post', 178 178 + title: 'Title', 179 179 + description: 'Description', 180 180 + markdownContent: 'Content', 181 181 + publishedAt: new Date(), 182 182 + updatedAt: new Date(), 183 183 + tags: ['existing'] 184 184 + }); 185 185 + 186 186 + doc.addTag('new-tag'); 187 187 + expect(doc.tags).toContain('new-tag'); 188 188 + expect(doc.tags).toContain('existing'); 189 189 + }); 190 190 + 191 191 + test('should not add duplicate tags', () => { 192 192 + const doc = new SiteStandardDocument({ 193 193 + site: 'https://example.com', 194 194 + path: '/post', 195 195 + title: 'Title', 196 196 + description: 'Description', 197 197 + markdownContent: 'Content', 198 198 + publishedAt: new Date(), 199 199 + updatedAt: new Date(), 200 200 + tags: ['existing'] 201 201 + }); 202 202 + 203 203 + doc.addTag('existing'); 204 204 + expect(doc.tags.filter(t => t === 'existing')).toHaveLength(1); 205 205 + }); 206 206 + 207 207 + test('should remove tags', () => { 208 208 + const doc = new SiteStandardDocument({ 209 209 + site: 'https://example.com', 210 210 + path: '/post', 211 211 + title: 'Title', 212 212 + description: 'Description', 213 213 + markdownContent: 'Content', 214 214 + publishedAt: new Date(), 215 215 + updatedAt: new Date(), 216 216 + tags: ['tag1', 'tag2', 'tag3'] 217 217 + }); 218 218 + 219 219 + doc.removeTag('tag2'); 220 220 + expect(doc.tags).toEqual(['tag1', 'tag3']); 221 221 + }); 222 222 + 223 223 + // Test exportForMarkdown 224 224 + test('should export minimal markdown metadata', () => { 225 225 + const doc = new SiteStandardDocument({ 226 226 + site: 'https://example.com', 227 227 + path: '/post', 228 228 + title: 'Title', 229 229 + description: 'Description', 230 230 + markdownContent: 'Content', 231 231 + publishedAt: new Date(), 232 232 + updatedAt: new Date(), 233 233 + tags: [] 234 234 + }); 235 235 + 236 236 + const exported = doc.exportForMarkdown(); 237 237 + expect(exported).toEqual({ 238 238 + site: 'https://example.com' 239 239 + }); 240 240 + }); 241 241 + 242 242 + test('should export markdown metadata with optional fields', () => { 243 243 + const doc = new SiteStandardDocument({ 244 244 + site: 'https://example.com', 245 245 + path: '/post', 246 246 + title: 'Title', 247 247 + description: 'Description', 248 248 + markdownContent: 'Content', 249 249 + publishedAt: new Date(), 250 250 + updatedAt: new Date(), 251 251 + tags: [], 252 252 + rkey: 'test-rkey', 253 253 + coverImage: { 254 254 + $type: 'blob' as const, 255 255 + ref: { $link: 'bafyrei...' }, 256 256 + mimeType: 'image/png' as any 257 257 + }, 258 258 + bskyPostRef: { 259 259 + $type: 'com.atproto.repo.strongRef' as const, 260 260 + uri: 'at://did:plc:abc123' as const, 261 261 + cid: 'bafyrei123' 262 262 + } 263 263 + }); 264 264 + 265 265 + const exported = doc.exportForMarkdown(); 266 266 + expect(exported.site).toBe('https://example.com'); 267 267 + expect(exported.rkey).toBe('test-rkey'); 268 268 + expect(exported.coverBlob).toBe('bafyrei...'); 269 269 + expect(exported.bskyPostRef).toBeDefined(); 270 270 + }); 271 271 + 272 272 + // Test exportToAtProto 273 273 + test('should export to AT Protocol format with required fields', () => { 274 274 + const publishedAt = new Date('2024-01-01T12:00:00Z'); 275 275 + const updatedAt = new Date('2024-01-02T12:00:00Z'); 276 276 + 277 277 + const doc = new SiteStandardDocument({ 278 278 + site: 'https://example.com', 279 279 + path: '/post', 280 280 + title: 'Title', 281 281 + description: 'Description', 282 282 + markdownContent: 'Content', 283 283 + publishedAt, 284 284 + updatedAt, 285 285 + tags: ['tag1', 'tag2'] 286 286 + }); 287 287 + 288 288 + const exported = doc.exportToAtProto(); 289 289 + 290 290 + expect(exported.$type).toBe('site.standard.document'); 291 291 + expect(exported.site).toBe('https://example.com'); 292 292 + expect(exported.path).toBe('/post'); 293 293 + expect(exported.title).toBe('Title'); 294 294 + expect(exported.description).toBe('Description'); 295 295 + expect(exported.publishedAt).toBe(publishedAt.toISOString()); 296 296 + expect(exported.updatedAt).toBe(updatedAt.toISOString()); 297 297 + expect(exported.tags).toEqual(['tag1', 'tag2']); 298 298 + }); 299 299 + 300 300 + test('should enforce title maxLength constraint', () => { 301 301 + const longTitle = 'a'.repeat(6000); 302 302 + const doc = new SiteStandardDocument({ 303 303 + site: 'https://example.com', 304 304 + path: '/post', 305 305 + title: longTitle, 306 306 + description: 'Description', 307 307 + markdownContent: 'Content', 308 308 + publishedAt: new Date(), 309 309 + updatedAt: new Date(), 310 310 + tags: [] 311 311 + }); 312 312 + 313 313 + const exported = doc.exportToAtProto(); 314 314 + expect(exported.title).toHaveLength(5000); 315 315 + }); 316 316 + 317 317 + test('should enforce description maxLength constraint', () => { 318 318 + const longDescription = 'a'.repeat(35000); 319 319 + const doc = new SiteStandardDocument({ 320 320 + site: 'https://example.com', 321 321 + path: '/post', 322 322 + title: 'Title', 323 323 + description: longDescription, 324 324 + markdownContent: 'Content', 325 325 + publishedAt: new Date(), 326 326 + updatedAt: new Date(), 327 327 + tags: [] 328 328 + }); 329 329 + 330 330 + const exported = doc.exportToAtProto(); 331 331 + expect(exported.description).toHaveLength(30000); 332 332 + }); 333 333 + 334 334 + test('should enforce tag maxLength constraint', () => { 335 335 + const longTag = 'a'.repeat(200); 336 336 + const doc = new SiteStandardDocument({ 337 337 + site: 'https://example.com', 338 338 + path: '/post', 339 339 + title: 'Title', 340 340 + description: 'Description', 341 341 + markdownContent: 'Content', 342 342 + publishedAt: new Date(), 343 343 + updatedAt: new Date(), 344 344 + tags: [longTag, 'normal-tag'] 345 345 + }); 346 346 + 347 347 + const exported = doc.exportToAtProto(); 348 348 + expect(exported.tags[0]).toHaveLength(128); 349 349 + expect(exported.tags[1]).toBe('normal-tag'); 350 350 + }); 351 351 + 352 352 + test('should include optional fields in AT Proto export', () => { 353 353 + const doc = new SiteStandardDocument({ 354 354 + site: 'https://example.com', 355 355 + path: '/post', 356 356 + title: 'Title', 357 357 + description: 'Description', 358 358 + markdownContent: 'Content', 359 359 + publishedAt: new Date(), 360 360 + updatedAt: new Date(), 361 361 + tags: [], 362 362 + coverImage: { 363 363 + $type: 'blob' as const, 364 364 + ref: { $link: 'bafyrei...' }, 365 365 + mimeType: 'image/jpeg' as any 366 366 + }, 367 367 + bskyPostRef: { 368 368 + $type: 'com.atproto.repo.strongRef' as const, 369 369 + uri: 'at://did:plc:abc123' as const, 370 370 + cid: 'bafyrei123' 371 371 + } 372 372 + }); 373 373 + 374 374 + const exported = doc.exportToAtProto(); 375 375 + expect(exported.coverImage).toBeDefined(); 376 376 + expect(exported.bskyPostRef).toBeDefined(); 377 377 + }); 378 378 + 379 379 + test('should not include updatedAt if not provided', () => { 380 380 + const doc = new SiteStandardDocument({ 381 381 + site: 'https://example.com', 382 382 + path: '/post', 383 383 + title: 'Title', 384 384 + description: 'Description', 385 385 + markdownContent: 'Content', 386 386 + publishedAt: new Date(), 387 387 + updatedAt: new Date(), 388 388 + tags: [] 389 389 + }); 390 390 + 391 391 + // Manually remove updatedAt 392 392 + doc.updatedAt = undefined as any; 393 393 + 394 394 + const exported = doc.exportToAtProto(); 395 395 + expect(exported.updatedAt).toBeUndefined(); 396 396 + }); 397 397 + 398 398 + test('should update the bskyPostRef correctly', () => { 399 399 + const doc = new SiteStandardDocument({ 400 400 + site: 'https://example.com', 401 401 + path: '/post', 402 402 + title: 'Title', 403 403 + description: 'Description', 404 404 + markdownContent: 'Content', 405 405 + publishedAt: new Date(), 406 406 + updatedAt: new Date(), 407 407 + bskyPostRef: { 408 408 + uri: 'at://did:plc:abc123', 409 409 + cid: 'bafyrei123' 410 410 + }, 411 411 + tags: [] 412 412 + }); 413 413 + expect(doc.bskyPostRef).toBeDefined(); 414 414 + expect(doc.bskyPostRef).toEqual({ 415 415 + "$type": "com.atproto.repo.strongRef", 416 416 + "uri": "at://did:plc:abc123", 417 417 + "cid": "bafyrei123" 418 418 + }); 419 419 + 420 420 + }); 421 421 + test('process markdown content to plain text', () => { 422 422 + const doc = new SiteStandardDocument({ 423 423 + site: 'https://example.com', 424 424 + path: '/post', 425 425 + title: 'Title', 426 426 + description: 'Description', 427 427 + markdownContent: markdownText, 428 428 + publishedAt: new Date(), 429 429 + updatedAt: new Date(), 430 430 + bskyPostRef: { 431 431 + uri: 'at://did:plc:abc123', 432 432 + cid: 'bafyrei123' 433 433 + }, 434 434 + tags: [] 435 435 + }); 436 436 + expect(doc.textContent).toBeDefined(); 437 437 + expect(doc.textContent).toEqual(plainText); 438 438 + }); 439 439 + });

+4

src/processMarkdown.ts

reviewed

··· 23 23 return { data, content, fileName, pathToFile }; 24 24 } 25 25 26 26 + export const markdownToDocument(data: any, content: string) { 27 27 + 28 28 + } 29 29 + 26 30 export const setMarkdownFile = (pathToFile: string, titleProp: string, contentProp: string, fileData: any, overwriteData: boolean = false) => { 27 31 const fileWriteResult = processObjectToMarkdown(titleProp, contentProp, pathToFile, fileData, false, overwriteData); 28 32 return fileWriteResult;

+109 -27

src/standardizeToAtProto.ts

reviewed

··· 16 16 "mimeType": "image/jpeg", 17 17 "size": 347901 18 18 }, 19 19 - "textContent": "When I envision the web, I picture an infinite expanse of empty space that stretches as far as the eye can see. It's full of fertile soil, but no seeds have taken root. That is, except for about an acre of it.\n\n - Molly White, We can have a different web\nThe future of the internet seems up in the air. Consumed by rotting behemoths. What we have now is failing, but it is also part of our every-day life, our politics, our society, our communities and our friendships. All of those are at risk, in part because the ways we communicate are under attack. The fate of the open web is inextricable from the other ways our world is in crisis.\n\nWe face down a seemingly endless series of political, social, economic, and medical disasters on a melting planet. There is no doubt that the internet is deeply integrated into everything that is going both right and wrong.\n\nAt this moment, we are actively discussing how to build a future for our shared global telecommunications network. This is good. We need this discussion. We also need to acknowledge that the wide open platform for infinite data, innovation and communication isn't a shelter from the disaster, but it is at the heart of everything we do. The day we could go back from that, even if we wanted to, is long gone. That means rewilding the web isn't just a matter of a better internet. Our way to a better web can't be purely technical. It's the matter of being part of the making of a better world.\n\nWith the problem so large, how do we approach a solution? If we want to plant a forest how do we grow the ground cover?\n\nDe-abstract. What is the internet really? It's wires, and waves (and occasionally, joking aside, pipes) all put to the purpose of connecting people to each other.\n\nThe internet then is the series of connections between people, at a huge scale. Not everyone is connected to everyone, but if you're on the internet, you're connecting to someone. So it isn't so much a world wide web as it is world wide webs.\n\nThe future core of a humane internet, if we are going to be able to use it to do the essential work of surviving this century, is a series of linked routes that stick us all together and supports our communities.\n\nI don't think there is only one solution, I couldn't possibly know them all, but I have one approach that has led me to launch this particular website (and some others).\n\nThe thing about webs is they are failure resistent, one strand falling won't take the whole thing down. You can punch a hole through them, but webs are also rebuilt regularly, they can fail and come back quickly.\n\nWe need to think of our connections and how we transmit information to each other in the same way. An important message should be able to travel edge to edge without interference and make it from web to web successfully. We need to think about links. Not just in the literal sense. We weave our communities together by establishing bedrocks of shared knowledge.\n\nI think what really put this into sharp focus for me is the long death of Twitter. Twitter is still a platform where, for professional reasons, I'm quite active, but its biggest utility--directing me to information that I'm interested in or that is important to me--that's gone.\n\nWe need to rethink how amplification works without requiring big tech or even a steady internet connection. Here are three big assumptions sitting inside this larger question of how best to build our webs.\n\nAmplification through cross-linking between individual entities and feeds is essentially rejuvenating for the web.\nWe can't take the life of those links for granted and an essential part of this rejuvenation process is archiving links that are important enough to amplify.\nIn the current world it is nearly impossible to build without touching or using the technology of a mega-corporation in some way so we should expect to build on things that will fail, enshittify, or rot, and need to be moved off from quickly.\nTo that end I've considered some principles for my own projects.\nThese (and the project of crafting them) are heavily inspired by the IndieWeb principles.\n\nPrinciples of Amplification for better webs\nAggregation is amplification.\nIn a post-Twitter world, reach requires coordination.\nIf it has to query a backend to load it will one day die.\nAmplification is power, use it responsibly.\nMassive replication of your own work is good. Massive aggregation between members of a community is even better.\nAmplification is a method to build community\nYou can always check out, copy, or scrape your work from other platforms and take ownership.\nPrinciples of Archiving for better webs\nUsing the web downloads it to your computer, you should get to keep it for your personal use.\nKill the app. They are black boxes filled with content and knowledge that will one day be lost forever.\nEveryone should support the Internet Archive, but we shouldn't assume it is invulnerable.\nOwn your own archive.\nSingle points of failure are dangerous and, as an expected operating tool, destroy innovation.\nAnything that can't fit on its own offline drive will die.\nPrinciples of Web Weaving\nConnecting to each other connects our audiences.\nCurated connections form bridges out of the walled gardens that hurt us and our society.\nAccept the rotting giant corporations that harvest human connections for profit, realize you must exist in their context to help others, and use their platforms and technology as a vaulting poll to bring yourself and others outside the walls.\nThe baseline has shifted away from an open web, but if we act like we are already in the future we want to build we can shift them again.\nThe Long Next\nWith these concepts in mind, I think of a lot of acronyms I've been considering as part of the future of the web:\n\nESC - End Surveillance Capitalism\nPOSSE - Publish (on your) Own Site, Syndicate Elsewhere\nPESOS - Publish Elsewhere, Syndicate (to your) Own Site\nI want to add one more to the list.\n\nWe need to think about how to ENTER the wild web.\n\nExtract\nFor now it's impossible to interact with others and move through this world without touching big tech. With this in mind, we are all going to end up using some of those platforms. So what we need are methods to extract the data off those platforms. There are blunt ways like scraping, and potentially captured ways like platform APIs. However, thanks to the explosion of privacy and interoperability-related legislation around the world, most sites have an option to check out all your data.\n\nOnce you have reclaimed your own works, you can do all sorts of things with it. Eventually you can even leave the platform, delete everything, and take canonical status for that work with your own sites. I'm just getting started, but I plan on writing more about how to do this.\n\nNullify\nRedirect or cross link your own sites and platform over the big tech ones whenever possible. Once you've linked off a enshittified platform and brought someone out of the walled garden and into your web, it's important to give them ways to interact with the internet that don't just push them right back into the hands of big tech. Whenever possible, push readers to open and wild web alternatives.\n\nCreate sites and networks that aren't just informative, but also fun to explore.\n\nTransform\nTake the posts you've checked out or otherwise extracted and turn them into something new and cool. Everything we post on social media platforms owned by big tech companies is limited by the business models and requirement to render ad targeting data. It shapes our participation in ways we aren't aware of. Once we free our own works from those platforms, take the time and effort to turn them into something cool and new, no longer limited by the constraints from inside the wall.\n\nOne of my main projects for this site is building tools for transforming data extractions from Letterboxd, Pocket, and Kindle.\n\nExchange\nBuild community though connecting. Link to other people, this is hardly a new idea, but it does seem to have fallen by the wayside at various periods of the social web.\n\nTruly fair exchanges between members of the (or a) web means more than that though, it means actively getting permission from others to remix their work and doing it. It means using and building on top of the work of others in all formats, not just code.\n\nIt also means: copyleft your work. The future of the web is not copyrighted. The future of the web is post-capitalism. Trying to play copyright games with big tech is a losing game, all we get is siloed culture that big tech steals anyway, either directly or in forms of aggregation or by bundling it into AI which spits it back out as plagiarism and bullshit. The future of the web isn't just open source code, it's open creativity. It's is share and share alike to build, remix and replicate the best things on the internet.\n\nReplicate\nWe should understand that there is no \"Long Now\". VCs, big clocks and big money are a fleeting phenomenon that is more likely to self-destruct or destroy us than it is to save us. The foundation of projects like The Long Now is built on the resources of people who think we need to maintain the present, scrolled out into an eternally status quo-maintaining future.\n\nLet me be blunt: The present isn't worth preserving.\n\nLet's think instead about The Long Next, a future where we build sustainably; live with the knowledge that we can't be always-online; and consider a world that we sustain in our projects, groups and communities. The Long Next isn't about preserving wealth and capital long enough to catapult the rich to Mars, it is about holistic rewilding of the web, the Earth, the future. A world we can live in, not one we are going to sacrifice on the alter of wealthy egos.\n\nCentral archives are incredibly important, but they're also central points of failure, vulnerable not just to physical attacks, but to legal ones as well. Worse, it is at the mercy of big donors who have proved time and again that they'd rather burn the present to build their personal vision of the future than sustain it. It's hard to believe such people will forever consider supporting projects like The Internet Archive in their best interest, especially as they sit above companies whose works are focused on essentially in-housing that work as a competitive advantage.\n\nWe should continue to support big important archival projects. I think their work is amazing and deserves all our support to keep them going as long as we can.\n\nA long next, a sustainable future, means that we need to do more than that, we need to think about how to keep the archives of what is important to us close to home, replicate them across our local networks and retain their information in case of the worst.\n\nThe parts of the web that are most important to you should be available to you, even if the internet isn't.\n\nSo use your newly wild websites to preserve the parts of the web that matter to you. Host site-local archives with indicators that point back to and 300s back to the originals. Keep the option to take them with you, or to drop redirects if they die so you'll have them just in case. Make sure to set up those local archives so they can be accessed by you, by others, and using your own search tools. Submit those links to The Internet Archive at the same time.\n\nMake sure what you build is easy for others to archive.\n\nThe Future Soon\nThis is a summary. I think I could explore any one of these points more deeply and maybe I will. This is a good start at trying to describe what I want to do with this site and the other sites I'm working on.\n\nI think that working in public and thinking in public keeps me honest, so that's what I've got here. If it's interesting to you, maybe it is something you can explore as well, try to follow through with on your own websites. Increasingly I wish for a zine-web filled with tiny one-off websites and fun projects, a million well-tended webs intersecting with each other, and archived by their peers.\n\nWe can join our own little webs together. One thing is certain, making the web a better place isn't a project to undertake alone or in private.\n\nReach out on Mastodon to talk.\n\n~~\n\nSelected Responses:\n\nDon Marti\nI really like the idea of bringing back Diskmags. I was not familiar with the concept at all, but ended up finding out about it when I researched the response and I like the idea of removable media used as a potential model for a sometimes-on web.\nThere is a really cool list of articles that talk about this topic of the future of a better web which you can check out.", 19 19 + "textContent": "When I envision the web, I picture an infinite expanse of empty space that stretches as far as the eye can see. It's full of fertile soil, but no seeds have taken root. That is, except for about an acre of it.\n\n - Molly White, We can have a different web\nThe future of the internet seems up in the air. Consumed by rotting behemoths. What we have now is failing, but it is also part of our every-day life, our politics, our society, our communities and our friendships. All of those are at risk, in part because the ways we communicate are under attack. The fate of the open web is inextricable from the other ways our world is in crisis.\n\nWe face down a seemingly endless series of political, social, economic, and medical disasters on a melting planet. There is no doubt that the internet is deeply integrated into everything that is going both right and wrong.\n\nAt this moment, we are actively discussing how to build a future for our shared global telecommunications network. This is good. We need this discussion. We also need to acknowledge that the wide open platform for infinite data, innovation and communication isn't a shelter from the disaster, but it is at the heart of everything we do. The day we could go back from that, even if we wanted to, is long gone. That means rewilding the web isn't just a matter of a better internet. Our way to a better web can't be purely technical. It's the matter of being part of the making of a better world.\n\nWith the problem so large, how do we approach a solution? If we want to plant a forest how do we grow the ground cover?\n\nDe-abstract. What is the internet really? It's wires, and waves (and occasionally, joking aside, pipes) all put to the purpose of connecting people to each other.\n\nThe internet then is the series of connections between people, at a huge scale. Not everyone is connected to everyone, but if you're on the internet, you're connecting to someone. So it isn't so much a world wide web as it is world wide webs.\n\nThe future core of a humane internet, if we are going to be able to use it to do the essential work of surviving this century, is a series of linked routes that stick us all together and supports our communities.\n\nI don't think there is only one solution, I couldn't possibly know them all, but I have one approach that has led me to launch this particular website (and some others).\n\nThe thing about webs is they are failure resistent, one strand falling won't take the whole thing down. You can punch a hole through them, but webs are also rebuilt regularly, they can fail and come back quickly.\n\nWe need to think of our connections and how we transmit information to each other in the same way. An important message should be able to travel edge to edge without interference and make it from web to web successfully. We need to think about links. Not just in the literal sense. We weave our communities together by establishing bedrocks of shared knowledge.\n\nI think what really put this into sharp focus for me is the long death of Twitter. Twitter is still a platform where, for professional reasons, I'm quite active, but its biggest utility--directing me to information that I'm interested in or that is important to me--that's gone.\n\nWe need to rethink how amplification works without requiring big tech or even a steady internet connection. Here are three big assumptions sitting inside this larger question of how best to build our webs.\n\nAmplification through cross-linking between individual entities and feeds is essentially rejuvenating for the web.\nWe can't take the life of those links for granted and an essential part of this rejuvenation process is archiving links that are important enough to amplify.\nIn the current world it is nearly impossible to build without touching or using the technology of a mega-corporation in some way so we should expect to build on things that will fail, enshittify, or rot, and need to be moved off from quickly.\nTo that end I've considered some principles for my own projects.\nThese (and the project of crafting them) are heavily inspired by the IndieWeb principles.\n\nPrinciples of Amplification for better webs\nAggregation is amplification.\nIn a post-Twitter world, reach requires coordination.\nIf it has to query a backend to load it will one day die.\nAmplification is power, use it responsibly.\nMassive replication of your own work is good. Massive aggregation between members of a community is even better.\nAmplification is a method to build community\nYou can always check out, copy, or scrape your work from other platforms and take ownership.\nPrinciples of Archiving for better webs\nUsing the web downloads it to your computer, you should get to keep it for your personal use.\nKill the app. They are black boxes filled with content and knowledge that will one day be lost forever.\nEveryone should support the Internet Archive, but we shouldn't assume it is invulnerable.\nOwn your own archive.\nSingle points of failure are dangerous and, as an expected operating tool, destroy innovation.\nAnything that can't fit on its own offline drive will die.\nPrinciples of Web Weaving\nConnecting to each other connects our audiences.\nCurated connections form bridges out of the walled gardens that hurt us and our society.\nAccept the rotting giant corporations that harvest human connections for profit, realize you must exist in their context to help others, and use their platforms and technology as a vaulting poll to bring yourself and others outside the walls.\nThe baseline has shifted away from an open web, but if we act like we are already in the future we want to build we can shift them again.\nThe Long Next\nWith these concepts in mind, I think of a lot of acronyms I've been considering as part of the future of the web:\n\nESC - End Surveillance Capitalism\nPOSSE - Publish (on your) Own Site, Syndicate Elsewhere\nPESOS - Publish Elsewhere, Syndicate (to your) Own Site\nI want to add one more to the list.\n\nWe need to think about how to ENTER the wild web.\n\nExtract\nFor now it's impossible to interact with others and move through this world without touching big tech. With this in mind, we are all going to end up using some of those platforms. So what we need are methods to extract the data off those platforms. There are blunt ways like scraping, and potentially captured ways like platform APIs. However, thanks to the explosion of privacy and interoperability-related legislation around the world, most sites have an option to check out all your data.\n\nOnce you have reclaimed your own works, you can do all sorts of things with it. Eventually you can even leave the platform, delete everything, and take canonical status for that work with your own sites. I'm just getting started, but I plan on writing more about how to do this.\n\nNullify\nRedirect or cross link your own sites and platform over the big tech ones whenever possible. Once you've linked off a enshittified platform and brought someone out of the walled garden and into your web, it's important to give them ways to interact with the internet that don't just push them right back into the hands of big tech. Whenever possible, push readers to open and wild web alternatives.\n\nCreate sites and networks that aren't just informative, but also fun to explore.\n\nTransform\nTake the posts you've checked out or otherwise extracted and turn them into something new and cool. Everything we post on social media platforms owned by big tech companies is limited by the business models and requirement to render ad targeting data. It shapes our participation in ways we aren't aware of. Once we free our own works from those platforms, take the time and effort to turn them into something cool and new, no longer limited by the constraints from inside the wall.\n\nOne of my main projects for this site is building tools for transforming data extractions from [Letterboxd](/lists/film-and-tv/), [Pocket](/amplify/), and [Kindle](/resources/quotes/).\n\nExchange\nBuild community though connecting. Link to other people, this is hardly a new idea, but it does seem to have fallen by the wayside at various periods of the social web.\n\nTruly fair exchanges between members of the (or a) web means more than that though, it means actively getting permission from others to remix their work and doing it. It means using and building on top of the work of others in all formats, not just code.\n\nIt also means: [copyleft](https://en.wikipedia.org/wiki/Copyleft) your work. The future of the web is not copyrighted. The future of the web is post-capitalism. Trying to play copyright games with big tech is a losing game, all we get is siloed culture that big tech steals anyway, either directly or in forms of aggregation or by bundling it into AI which spits it back out as plagiarism and bullshit. The future of the web isn't just open source code, it's [open creativity](https://adactio.com/articles/1522/). It's is [share and share alike](https://creativecommons.org/share-your-work/cclicenses/) to build, remix and replicate the best things on the internet. \n\n### Replicate \n\nWe should understand that there is no \"Long Now\". VCs, big clocks and big money are a fleeting phenomenon that is more likely to self-destruct or destroy us than it is to save us. The foundation of projects like The Long Now is built on the resources of people who think we need to maintain the present, scrolled out into an eternally status quo-maintaining future. \n\nLet me be blunt: The present isn't worth preserving. \n\nLet's think instead about **The Long Next**, a future where we build sustainably; live with the knowledge that [we can't be always-online](/resources/bookmarks/low-tech-magazine/); and consider a world that **we** sustain in our projects, groups and communities. The Long Next isn't about preserving wealth and capital long enough to catapult the rich to Mars, it is about holistic rewilding of the web, the Earth, the future. A world we can live in, not one we are going to sacrifice on the alter of wealthy egos. \n\nCentral archives are incredibly important, but they're also central points of failure, vulnerable not just to physical attacks, but to legal ones as well. Worse, it is at the mercy of big donors who have proved time and again that they'd rather burn the present to build their personal vision of the future than sustain it. It's hard to believe such people will forever consider supporting projects like The Internet Archive in their best interest, especially as they sit above companies whose works are focused on essentially in-housing that work *as a competitive advantage*. \n\nWe should continue to support big important archival projects. I think their work is amazing and deserves all our support to keep them going as long as we can. \n\nA long next, a sustainable future, means that we need to do more than that, we need to think about how to keep the archives of what is important to us close to home, replicate them across our local networks and retain their information in case of the worst. \n\nThe parts of the web that are most important to you should be available to you, even if the internet isn't.\n\nSo use your newly wild websites to preserve the parts of the web that matter to you. Host site-local archives with indicators that point back to and [300s](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection) back to the originals. Keep the option to take them with you, or to drop redirects if they die so you'll have them *just in case*. Make sure to set up those local archives so they can be accessed by you, by others, and using your own search tools. [Submit](https://web.archive.org/save) [those](https://chromewebstore.google.com/detail/wayback-machine/fpnmgdkabkmnadcjpehmlllkndpkmiak) [links](https://archive.org/details/introduction-to-the-warc) [to](https://help.archive.org/help/uploading-a-basic-guide/) [The Internet Archive](https://help.archive.org/help/save-pages-in-the-wayback-machine/) at the same time. \n\n[Make sure what you build is easy for others to archive](https://jeffhuang.com/designed_to_last/).\n\n## The Future Soon \n\nThis is a summary. I think I could explore any one of these points more deeply and maybe I will. This is a good start at trying to describe what I want to do with this site and the other sites I'm working on.\n\nI think that working in public and thinking in public keeps me honest, so that's what I've got here. If it's interesting to you, maybe it is something you can explore as well, try to follow through with on your own websites. Increasingly I wish for a **zine-web** filled with tiny one-off websites and fun projects, a million well-tended webs intersecting with each other, and archived by their peers. \n\n\nWe can join our own little webs together. One thing is certain, making the web a better place isn't a project to undertake alone or in private. \n\n[Reach out on Mastodon to talk](https://indieweb.social/@Chronotope). \n\n~~\n\nSelected Responses: \n\n- [Don Marti](https://federate.social/@dmarti/112587045311696156)\n- I really like [the idea of bringing back Diskmags](https://elk.zone/indieweb.social/@teajaygrey@rap.social/112606872290801809). I was not familiar with the concept at all, but ended up [finding out about it when I researched the response](https://diskmags.de/index.php?title=Introduction) and I like the idea of removable media used as a potential model for a sometimes-on web. \n- There is [a really cool list of articles that talk about this topic of the future of a better web which you can check out](https://projects.kwon.nyc/internet-is-fun/). \n\n~~\n\nCover Image by <a href=\"https://commons.wikimedia.org/wiki/User:Chen-Pan_Liao\" title=\"User:Chen-Pan Liao\">Chen-Pan Liao</a> - <span class=\"int-own-work\" lang=\"en\">Own work</span>, <a href=\"https://creativecommons.org/licenses/by-sa/3.0\" title=\"Creative Commons Attribution-Share Alike 3.0\">CC BY-SA 3.0</a>, <a href=\"https://commons.wikimedia.org/w/index.php?curid=15926393\">Link</a>", 20 20 "content": { 21 21 "$type": "site.standard.content.markdown", 22 22 "text": "> When I envision the web, I picture an infinite expanse of empty space that stretches as far as the eye can see. It's full of fertile soil, but no seeds have taken root. That is, except for about an acre of it.\n - Molly White, [We can have a different web](/noteworthy/we-can-have-a-different-web/)\n\nThe future of the internet seems up in the air. Consumed by rotting behemoths. What we have now is failing, but it is also part of our every-day life, our politics, our society, our communities and our friendships. All of those are at risk, in part because the ways we communicate are under attack. The fate of the open web is inextricable from the other ways our world is in crisis. \n\nWe face down a seemingly endless series of political, social, economic, and medical disasters on a melting planet. There is no doubt that the internet is deeply integrated into everything that is going both right and wrong.\n\nAt this moment, [we are actively discussing how to build a future for our shared global telecommunications network](https://aramzs.xyz/lists/a-future-for-the-web/). This is good. We need this discussion. We also need to acknowledge that the wide open platform for infinite data, innovation and communication isn't a shelter from the disaster, but it *is* at the heart of everything we do. The day we could go back from that, even if we wanted to, is long gone. That means [rewilding the web](https://www.noemamag.com/we-need-to-rewild-the-internet/) isn't **just** a matter of a better internet. Our way to a better web can't be purely technical. It's the matter of being part of the making of a better world. \n\nWith the problem so large, how do we approach a solution? If we want to plant a forest how do we grow the ground cover? \n\nDe-abstract. What is the internet really? It's wires, and waves (and occasionally, joking aside, [pipes](https://www.windsystemsmag.com/the-latest-advancements-in-submarine-cables-protection/)) all put to the purpose of connecting people to each other. \n\nThe internet then is the series of connections between [people](https://adactio.com/journal/18337), at a huge scale. Not everyone is connected to everyone, but if you're on the internet, you're connecting to *someone*. So it isn't so much a world wide web as it is world wide **webs**. \n\nThe future core of a [humane](https://humanewebmanifesto.com/) internet, if we are going to be able to use it to do the essential work of surviving this century, is a series of linked routes that stick us all together and supports our communities.\n\nI don't think there is only one solution, I couldn't possibly know them all, but I have one approach that has led me to launch this particular website (and [some others](/projects/)). \n\nThe thing about webs is [they are failure resistent, one strand falling won't take the whole thing down](https://news.mit.edu/2012/spider-web-strength-0202). You can punch a hole through them, but webs are also rebuilt regularly, they can fail and come back quickly. \n\nWe need to think of our connections and how we transmit information to each other in the same way. An important message should be able to travel edge to edge without interference and make it from web to web successfully. We need to think about *links*. Not just in the literal sense. We weave our communities together by establishing bedrocks of shared knowledge. \n\nI think what really put this into sharp focus for me is [the long death of Twitter](https://www.schizochronotopia.com/p/on-twitter-we-look-down). Twitter is still a platform where, for professional reasons, I'm quite active, but its biggest utility--directing me to information that I'm interested in or that is important to me--that's gone. \n\nWe need to rethink how amplification works without requiring big tech or even a steady internet connection. Here are three big assumptions sitting inside this larger question of how best to build our webs. \n\n1. Amplification through cross-linking between individual entities and feeds is essentially rejuvenating for the web. \n2. We [can't](https://ahrefs.com/blog/link-rot-study/) [take the life of those links for granted](https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/) and an essential part of this rejuvenation process is archiving links that are important enough to amplify.\n3. In the current world [it is nearly impossible](https://www.nytimes.com/2020/07/31/technology/blocking-the-tech-giants.html) to build without touching or using the technology of a mega-corporation in some way so we should expect to build on things that will fail, enshittify, or rot, and need to be moved off from quickly.\n\n## To that end I've considered some principles for my own projects.\n\nThese (and the project of crafting them) are heavily inspired by [the IndieWeb principles](https://indieweb.org/principles). \n\n### Principles of Amplification for better webs\n\n- Aggregation is amplification.\n- In a post-Twitter world, reach requires coordination. \n- If it has to query a backend to load it will one day die. \n- Amplification is power, use it responsibly. \n- Massive replication of your own work is good. Massive aggregation between members of a community is even better. \n- [Amplification](/thoughts/using-amplify-as-a-type/) is a method to build community\n- You can always [check out](https://dataprivacymanager.net/what-is-data-subject-access-request-dsar/), copy, or scrape your work from other platforms and take ownership.\n\n### Principles of Archiving for better webs\n\n- Using the web downloads it to your computer, you should get to keep it for your personal use.\n- [Kill the app](https://tweets.aramzs.com/1390071898202120192/). They are black boxes filled with content and knowledge that will one day be lost forever.\n- Everyone should support the Internet Archive, but we [shouldn't](https://www.forbes.com/sites/mattnovak/2023/05/28/internet-archive-suffers-major-global-outage/) assume it is invulnerable. \n- Own your own archive. \n- Single points of failure are dangerous and, as an expected operating tool, destroy innovation.\n- Anything that can't fit on its own offline drive will die.\n\n### Principles of Web Weaving\n\n- Connecting to each other connects our audiences.\n- Curated connections form bridges out of the walled gardens that hurt us and our society. \n- Accept the rotting giant corporations that harvest human connections for profit, realize you must exist in their context to help others, and use their platforms and technology as a vaulting poll to bring yourself and others outside the walls.\n- The [baseline has shifted](/glossary/shifting-baselines/) away from an open web, but if we act like we are already in the future we want to build we can shift them again.\n\n## The Long Next\n\nWith these concepts in mind, I think of a lot of acronyms I've been considering as part of the future of the web: \n\n- [ESC](https://esc.fyi/) - End Surveillance Capitalism\n- [POSSE](https://indieweb.org/POSSE) - Publish (on your) Own Site, Syndicate Elsewhere\n- [PESOS](https://indieweb.org/PESOS) - Publish Elsewhere, Syndicate (to your) Own Site\n\nI want to add one more to the list. \n\nWe need to think about how to [ENTER](/glossary/enter/) the wild web.\n\n### Extract\n\nFor now it's [impossible to interact with others and move through this world without touching big tech](https://www.nytimes.com/2020/07/31/technology/blocking-the-tech-giants.html). With this in mind, we are all going to end up using some of those platforms. So what we need are methods to extract the data off those platforms. There are blunt ways like scraping, and potentially captured ways like platform APIs. However, thanks to the explosion of privacy and interoperability-related legislation around the world, [most sites have an option to check out all your data](https://dataprivacymanager.net/what-is-data-subject-access-request-dsar/). \n\nOnce you have reclaimed your own works, you can do all sorts of things with it. Eventually you can even leave the platform, delete everything, and take canonical status for that work with your own sites. [I'm just getting started](https://aramzs.xyz/tutorials/how-to-export-your-links-from-pocket/), but I plan on writing more about how to do this.\n\n### Nullify \n\nRedirect or cross link your own sites and platform over the big tech ones whenever possible. Once you've linked off a enshittified platform and brought someone out of the walled garden and into your web, it's important to give them ways to interact with the internet that don't just push them right back into the hands of big tech. Whenever possible, push readers to open and wild web alternatives. \n\nCreate sites and networks that aren't just informative, but also fun to explore. \n\n### Transform \n\nTake the posts you've checked out or otherwise extracted and turn them into something new and cool. Everything we post on social media platforms owned by big tech companies is limited by the business models and requirement to render ad targeting data. It shapes our participation in ways we aren't aware of. Once we free our *own* works from those platforms, take the time and effort to turn them into something cool and new, no longer limited by the constraints from inside the wall. \n\nOne of my main projects for this site is building tools for transforming data extractions from [Letterboxd](/lists/film-and-tv/), [Pocket](/amplify/), and [Kindle](/resources/quotes/).\n\n### Exchange \n\nBuild community though connecting. Link to other people, this is hardly a new idea, but it does seem to have fallen by the wayside at various periods of the social web. \n\nTruly fair exchanges between members of the (or a) web means more than that though, it means actively getting permission from others to remix their work and doing it. It means using and building on top of the work of others in all formats, not just code.\n\nIt also means: [copyleft](https://en.wikipedia.org/wiki/Copyleft) your work. The future of the web is not copyrighted. The future of the web is post-capitalism. Trying to play copyright games with big tech is a losing game, all we get is siloed culture that big tech steals anyway, either directly or in forms of aggregation or by bundling it into AI which spits it back out as plagiarism and bullshit. The future of the web isn't just open source code, it's [open creativity](https://adactio.com/articles/1522/). It's is [share and share alike](https://creativecommons.org/share-your-work/cclicenses/) to build, remix and replicate the best things on the internet. \n\n### Replicate \n\nWe should understand that there is no \"Long Now\". VCs, big clocks and big money are a fleeting phenomenon that is more likely to self-destruct or destroy us than it is to save us. The foundation of projects like The Long Now is built on the resources of people who think we need to maintain the present, scrolled out into an eternally status quo-maintaining future. \n\nLet me be blunt: The present isn't worth preserving. \n\nLet's think instead about **The Long Next**, a future where we build sustainably; live with the knowledge that [we can't be always-online](/resources/bookmarks/low-tech-magazine/); and consider a world that **we** sustain in our projects, groups and communities. The Long Next isn't about preserving wealth and capital long enough to catapult the rich to Mars, it is about holistic rewilding of the web, the Earth, the future. A world we can live in, not one we are going to sacrifice on the alter of wealthy egos. \n\nCentral archives are incredibly important, but they're also central points of failure, vulnerable not just to physical attacks, but to legal ones as well. Worse, it is at the mercy of big donors who have proved time and again that they'd rather burn the present to build their personal vision of the future than sustain it. It's hard to believe such people will forever consider supporting projects like The Internet Archive in their best interest, especially as they sit above companies whose works are focused on essentially in-housing that work *as a competitive advantage*. \n\nWe should continue to support big important archival projects. I think their work is amazing and deserves all our support to keep them going as long as we can. \n\nA long next, a sustainable future, means that we need to do more than that, we need to think about how to keep the archives of what is important to us close to home, replicate them across our local networks and retain their information in case of the worst. \n\nThe parts of the web that are most important to you should be available to you, even if the internet isn't.\n\nSo use your newly wild websites to preserve the parts of the web that matter to you. Host site-local archives with indicators that point back to and [300s](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection) back to the originals. Keep the option to take them with you, or to drop redirects if they die so you'll have them *just in case*. Make sure to set up those local archives so they can be accessed by you, by others, and using your own search tools. [Submit](https://web.archive.org/save) [those](https://chromewebstore.google.com/detail/wayback-machine/fpnmgdkabkmnadcjpehmlllkndpkmiak) [links](https://archive.org/details/introduction-to-the-warc) [to](https://help.archive.org/help/uploading-a-basic-guide/) [The Internet Archive](https://help.archive.org/help/save-pages-in-the-wayback-machine/) at the same time. \n\n[Make sure what you build is easy for others to archive](https://jeffhuang.com/designed_to_last/).\n\n## The Future Soon \n\nThis is a summary. I think I could explore any one of these points more deeply and maybe I will. This is a good start at trying to describe what I want to do with this site and the other sites I'm working on.\n\nI think that working in public and thinking in public keeps me honest, so that's what I've got here. If it's interesting to you, maybe it is something you can explore as well, try to follow through with on your own websites. Increasingly I wish for a **zine-web** filled with tiny one-off websites and fun projects, a million well-tended webs intersecting with each other, and archived by their peers. \n\n\nWe can join our own little webs together. One thing is certain, making the web a better place isn't a project to undertake alone or in private. \n\n[Reach out on Mastodon to talk](https://indieweb.social/@Chronotope). \n\n~~\n\nSelected Responses: \n\n- [Don Marti](https://federate.social/@dmarti/112587045311696156)\n- I really like [the idea of bringing back Diskmags](https://elk.zone/indieweb.social/@teajaygrey@rap.social/112606872290801809). I was not familiar with the concept at all, but ended up [finding out about it when I researched the response](https://diskmags.de/index.php?title=Introduction) and I like the idea of removable media used as a potential model for a sometimes-on web. \n- There is [a really cool list of articles that talk about this topic of the future of a better web which you can check out](https://projects.kwon.nyc/internet-is-fun/). \n\n~~\n\nCover Image by <a href=\"https://commons.wikimedia.org/wiki/User:Chen-Pan_Liao\" title=\"User:Chen-Pan Liao\">Chen-Pan Liao</a> - <span class=\"int-own-work\" lang=\"en\">Own work</span>, <a href=\"https://creativecommons.org/licenses/by-sa/3.0\" title=\"Creative Commons Attribution-Share Alike 3.0\">CC BY-SA 3.0</a>, <a href=\"https://commons.wikimedia.org/w/index.php?curid=15926393\">Link</a>", ··· 48 48 AVIF = "image/avif" 49 49 } 50 50 51 51 + type BskyPostRef = { 52 52 + "$type": "com.atproto.repo.strongRef", 53 53 + "uri": `at://did:plc:${string}`, 54 54 + "cid": string 55 55 + } 56 56 + 51 57 type SiteStandardDocInitObj = { 52 58 site: `at://${string}` | `https://${string}`, 53 59 path: `/${string}`, ··· 62 68 coverImage?: { 63 69 $type: "blob", 64 70 ref: { 65 65 - $link: string 71 71 + $link: string 66 72 }, 67 73 mimeType: ImageMimeType 68 74 }, 69 75 bskyPostRef?: { 70 70 - "$type": "com.atproto.repo.strongRef", 71 71 - "uri": `at://did:plc:${string}`, 72 72 - "cid": string 73 73 - }, 76 76 + "$type"?: "com.atproto.repo.strongRef", 77 77 + "uri": `at://did:plc:${string}`, 78 78 + "cid": string 79 79 + }, 74 80 content?: { 75 81 "$type": string, 76 82 "text"?: string, ··· 80 86 81 87 // Define a class that can hold the properties and logic of a site.standard.document 82 88 83 83 - class SiteStandardDocument { 89 89 + export class SiteStandardDocument { 84 90 public $type: string = "site.standard.document" 85 91 public site: `at://${string}` | `https://${string}` 86 92 public path: `/${string}` ··· 105 111 }, 106 112 mimeType: ImageMimeType 107 113 } 108 108 - public bskyPostRef?: { 109 109 - "$type": "com.atproto.repo.strongRef", 110 110 - "uri": `at://did:plc:${string}`, 111 111 - "cid": string 112 112 - } 114 114 + public bskyPostRef?: BskyPostRef 113 115 constructor(init: SiteStandardDocInitObj) { 114 116 this.$type = "site.standard.document" 115 117 this.site = init.site; ··· 121 123 this.tags = init.tags; 122 124 this.rkey = init.rkey; 123 125 this.coverImage = init.coverImage; 124 124 - this.bskyPostRef = init.bskyPostRef; 125 126 this.markdownContent = init.markdownContent; 126 127 this.textContent = this.markdownToTextContent(init.markdownContent); 127 128 if (init.content){ 128 129 this.content = init.content; 129 130 } else { 130 131 this.content = this.markdownToContent(init.markdownContent); 132 132 + } 133 133 + if (init.bskyPostRef) { 134 134 + this.setBskyRef(init.bskyPostRef.uri, init.bskyPostRef.cid); 131 135 } 132 136 // this.textContent = init.markdownContent; 133 137 // this.content = init.markdownContent; ··· 143 147 } 144 148 145 149 markdownToTextContent(value: string){ 146 146 - return ""; 150 150 + // strip value of any HTML markup or markdown 151 151 + // replace line breaks with `\n` 152 152 + 153 153 + let newValue = value; 154 154 + 155 155 + // Remove HTML tags 156 156 + newValue = newValue.replace(/<[^>]*>/g, ''); 157 157 + 158 158 + // Remove markdown images:  -> alt (do this before links) 159 159 + newValue = newValue.replace(/!\[([^\]]*)\]\([^\)]+\)/g, '$1'); 160 160 + 161 161 + // Remove markdown links but keep the text: [text](url) -> text 162 162 + newValue = newValue.replace(/\[([^\]]+)\]\([^\)]+\)/g, '$1'); 163 163 + 164 164 + // Remove markdown headers: ## Header -> Header 165 165 + newValue = newValue.replace(/^#{1,6}\s+/gm, ''); 166 166 + 167 167 + // Remove bold/italic: **text** or *text* or __text__ or _text_ -> text 168 168 + newValue = newValue.replace(/\*\*([^*]+)\*\*/g, '$1'); 169 169 + newValue = newValue.replace(/__([^_]+)__/g, '$1'); 170 170 + newValue = newValue.replace(/\*([^*]+)\*/g, '$1'); 171 171 + newValue = newValue.replace(/_([^_]+)_/g, '$1'); 172 172 + 173 173 + // Remove code blocks: ```code``` -> (remove entirely) 174 174 + // newValue = newValue.replace(/```[\s\S]*?```/g, ''); 175 175 + 176 176 + // Remove inline code: `code` -> code 177 177 + // newValue = newValue.replace(/`([^`]+)`/g, '$1'); 178 178 + 179 179 + // Remove blockquotes but keep content: > quote -> quote (with leading space) 180 180 + newValue = newValue.replace(/^>\s*/gm, ''); 181 181 + 182 182 + // Remove horizontal rules: --- or *** or ___ 183 183 + newValue = newValue.replace(/^[-*_]{3,}\s*$/gm, ''); 184 184 + 185 185 + // Remove list markers: - item or * item or + item or 1. item -> item 186 186 + newValue = newValue.replace(/^[\s]*[-*+]\s+/gm, ''); 187 187 + newValue = newValue.replace(/^[\s]*\d+\.\s+/gm, ''); 188 188 + 189 189 + // Normalize line breaks to \n 190 190 + newValue = newValue.replace(/\r\n/g, '\n'); 191 191 + newValue = newValue.replace(/\r/g, '\n'); 192 192 + 193 193 + // Remove excessive whitespace but preserve double line breaks 194 194 + newValue = newValue.replace(/\n{3,}/g, '\n\n'); 195 195 + 196 196 + // Clean up any remaining extra spaces at line starts/ends 197 197 + newValue = newValue.split('\n').map(line => line.trim()).join('\n'); 198 198 + 199 199 + // Final trim 200 200 + newValue = newValue.trim(); 201 201 + 202 202 + return newValue; 147 203 } 148 204 205 205 + setBskyRef(uri: `at://did:plc:${string}`, cid: string){ 206 206 + this.bskyPostRef = { 207 207 + "$type": "com.atproto.repo.strongRef", 208 208 + "uri": uri, 209 209 + "cid": cid 210 210 + }; 211 211 + } 149 212 150 213 // Method to update the content of the document 151 214 updateContent(newContent: string) { 152 152 - this.content = { 153 153 - "$type": "site.standard.content.markdown", 154 154 - "text": newContent, 155 155 - "version": "1.0" 156 156 - }; 157 157 - this.textContent = newContent; 215 215 + this.markdownContent = newContent 216 216 + this.content = this.markdownToContent(newContent); 217 217 + this.textContent = this.markdownToTextContent(newContent); 158 218 this.updatedAt = new Date(); 159 219 } 160 220 ··· 170 230 this.tags = this.tags.filter(t => t !== tag); 171 231 } 172 232 233 233 + exportForMarkdown() { 234 234 + const markdownReadyObj: { 235 235 + site: string, 236 236 + coverBlob?: string | false, 237 237 + bskyPostRef?: { 238 238 + "uri": `at://did:plc:${string}`, 239 239 + "cid": string 240 240 + }, 241 241 + rkey?: string 242 242 + } = { 243 243 + site: this.site 244 244 + } 245 245 + if (this.coverImage) { 246 246 + markdownReadyObj.coverBlob = this.coverImage ? this.coverImage.ref.$link : false 247 247 + } 248 248 + if (this.bskyPostRef) { 249 249 + markdownReadyObj.bskyPostRef = this.bskyPostRef; 250 250 + } 251 251 + if (this.rkey) { 252 252 + markdownReadyObj.rkey = this.rkey; 253 253 + } 254 254 + return markdownReadyObj; 255 255 + } 256 256 + 173 257 exportToAtProto() { 174 258 175 259 // return an object that meets the site.standard.document object specifications for writing to a PDS. ··· 189 273 tags: string[], 190 274 publishedAt: string, 191 275 updatedAt?: string, 192 192 - rkey?: string, 193 276 coverImage?: { 194 277 $type: "blob", 195 278 ref: { ··· 197 280 }, 198 281 mimeType: ImageMimeType 199 282 }, 200 200 - bskyPostRef?: { 201 201 - "$type": "com.atproto.repo.strongRef", 202 202 - "uri": `at://did:plc:${string}`, 203 203 - "cid": string 204 204 - } 283 283 + bskyPostRef?: BskyPostRef 205 284 } = { 206 285 $type: "site.standard.document", 207 286 site: this.site, ··· 223 302 atProtoDoc.coverImage = this.coverImage; 224 303 } 225 304 226 226 - if (this.bskyPostRef) { 305 305 + if (this.bskyPostRef && this.bskyPostRef["$type"] && this.bskyPostRef["$type"] === "com.atproto.repo.strongRef") { 306 306 + atProtoDoc.bskyPostRef = this.bskyPostRef; 307 307 + } else if (this.bskyPostRef) { 308 308 + this.setBskyRef(this.bskyPostRef.uri, this.bskyPostRef.cid) 227 309 atProtoDoc.bskyPostRef = this.bskyPostRef; 228 310 } 229 311