It's all going to my puppeteer proxy.

Reply to this note

Please Login to reply.

Discussion

What's it doing?

Rendering the websites in its Chrome headless browser, and returning them as static pages, so that I can then grab the page info, and render OpenGraph data or the fallback data like and <image>.</p><p>That's why you can see rich previews of all websites, even if they aren't SSR. Also means you can see rich previews (including content) of websites that would normally block you, for having Javascript turned off.</p> </div> <div class="nested-replies "> <div class="note reply-note depth-2" id="note-fabc7f0d58d47c9f6f4fa41b6ce7e739db85ea405673ec5d19333dc752554c5b"> <div class="note-header"> <a href="/user/fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1"> <img src="https://image.nostr.build/0746072c2304d7a0ae845342bf07cd89dcefad50950dcdb8adb5bc58cfb145f1.jpg" class="note-avatar" alt="Avatar" loading="lazy"> </a> <div class="note-info"> <a href="/user/fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1" class="author-link"> <span class="note-author-name" title="fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1"> Silberengel </span> </a> <a href="/post/fabc7f0d58d47c9f6f4fa41b6ce7e739db85ea405673ec5d19333dc752554c5b" class="note-date-link" title="2025-12-23T07:25:27Z"> <span class="note-date">1w ago</span> <span class="reply-count" aria-label="View replies">💬 1</span> </a> </div> <button class="collapse-toggle" onclick="toggleNote('fabc7f0d58d47c9f6f4fa41b6ce7e739db85ea405673ec5d19333dc752554c5b')" title="Collapse/Expand">[-]</button> </div> <div class="note-body" id="body-fabc7f0d58d47c9f6f4fa41b6ce7e739db85ea405673ec5d19333dc752554c5b"> <div class="note-content"> <p>I.e. the preview you see isn't directly from the website. I build the website and then scrape off the rich preview tags.</p><p>I do a lot of scraping and parsing of hyperlinks. This is not data the website has sent me, for instance, but I think this is more interesting than the hyperlink alone, and then I can actually skip rendering the website. 🤔</p><p><a href="https://primal.net/e/nevent1qqs08j4ju90a4zx8qlnuztuyjel4gxk6tw3zacwrgyuw3x287z73y3qx8u32u" target="_blank" rel="noopener noreferrer" class="note-link">https://primal.net/e/nevent1qqs08j4ju90a4zx8qlnuztuyjel4gxk6tw3zacwrgyuw3x287z73y3qx8u32u</a></p><p><a href="https://wikistr.imwald.eu/trusted-assertions*460c25e682fda7832b52d1f22d3d22b3176d972f60dcdc3212ed8c92ef85065c" target="_blank" rel="noopener noreferrer" class="note-link">https://wikistr.imwald.eu/trusted-assertions*460c25e682fda7832b52d1f22d3d22b3176d972f60dcdc3212ed8c92ef85065c</a></p><p><a href="https://decentnewsroom.com/mag/newsroom-magazine-43cf24/cat/lifestyle-d22cbb/d/How-Colorful-Are-You-ans9bk" target="_blank" rel="noopener noreferrer" class="note-link">https://decentnewsroom.com/mag/newsroom-magazine-43cf24/cat/lifestyle-d22cbb/d/How-Colorful-Are-You-ans9bk</a></p><p><a href="https://coracle.social/notes/naddr1qvzqqqr4gupzplaukurf77496md3984nne7c49ufgekj2knr0wkpa0uxz7c9wszyqq54gmmhv9exguedvykksatdv9hz6envdamj66tw946x2cmgdehkcmm80ykh5mtwdpurvww7weg" target="_blank" rel="noopener noreferrer" class="note-link">https://coracle.social/notes/naddr1qvzqqqr4gupzplaukurf77496md3984nne7c49ufgekj2knr0wkpa0uxz7c9wszyqq54gmmhv9exguedvykksatdv9hz6envdamj66tw946x2cmgdehkcmm80ykh5mtwdpurvww7weg</a></p> </div> <div class="nested-replies "> <div class="note reply-note depth-3" id="note-b94be7e82d1ac465193a8b2eae4b5f09f911d9dc95d32ad0772e24aafb3aab4f"> <div class="note-header"> <a href="/user/fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1"> <img src="https://image.nostr.build/0746072c2304d7a0ae845342bf07cd89dcefad50950dcdb8adb5bc58cfb145f1.jpg" class="note-avatar" alt="Avatar" loading="lazy"> </a> <div class="note-info"> <a href="/user/fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1" class="author-link"> <span class="note-author-name" title="fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1"> Silberengel </span> </a> <a href="/post/b94be7e82d1ac465193a8b2eae4b5f09f911d9dc95d32ad0772e24aafb3aab4f" class="note-date-link" title="2025-12-23T07:26:56Z"> <span class="note-date">1w ago</span> </a> </div> <button class="collapse-toggle" onclick="toggleNote('b94be7e82d1ac465193a8b2eae4b5f09f911d9dc95d32ad0772e24aafb3aab4f')" title="Collapse/Expand">[-]</button> </div> <div class="note-body" id="body-b94be7e82d1ac465193a8b2eae4b5f09f911d9dc95d32ad0772e24aafb3aab4f"> <div class="note-content"> <p>I should probably move the image back to the top, instead of off to the left like that. That's too small to read the text. 🤔</p> </div> </div> <div class="collapsed-indicator" id="ind-b94be7e82d1ac465193a8b2eae4b5f09f911d9dc95d32ad0772e24aafb3aab4f" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> </div> </div> <div class="collapsed-indicator" id="ind-fabc7f0d58d47c9f6f4fa41b6ce7e739db85ea405673ec5d19333dc752554c5b" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> <div class="note reply-note depth-2" id="note-10c20b59cfe71243a0de9ce2b8315a1e5cf7c9ee8507678f8ae811e911eb0cf2"> <div class="note-header"> <a href="/user/036533caa872376946d4e4fdea4c1a0441eda38ca2d9d9417bb36006cbaabf58"> <img src="https://www.vaughnnugent.com/public/blogs/personal/content/syz7ipjwji3nrdtlz7phc64bwy.webp" class="note-avatar" alt="Avatar" loading="lazy"> </a> <div class="note-info"> <a href="/user/036533caa872376946d4e4fdea4c1a0441eda38ca2d9d9417bb36006cbaabf58" class="author-link"> <span class="note-author-name" title="036533caa872376946d4e4fdea4c1a0441eda38ca2d9d9417bb36006cbaabf58"> ChipTuner </span> </a> <a href="/post/10c20b59cfe71243a0de9ce2b8315a1e5cf7c9ee8507678f8ae811e911eb0cf2" class="note-date-link" title="2025-12-23T18:13:58Z"> <span class="note-date">1w ago</span> <span class="reply-count" aria-label="View replies">💬 1</span> </a> </div> <button class="collapse-toggle" onclick="toggleNote('10c20b59cfe71243a0de9ce2b8315a1e5cf7c9ee8507678f8ae811e911eb0cf2')" title="Collapse/Expand">[-]</button> </div> <div class="note-body" id="body-10c20b59cfe71243a0de9ce2b8315a1e5cf7c9ee8507678f8ae811e911eb0cf2"> <div class="note-content"> <p>Oh yea I've always wanted to play around with that for scraping tools!</p> </div> <div class="nested-replies "> <div class="note reply-note depth-3" id="note-0c56385181bd388346b3f64d05218fc6770813c253a5951c91e8420ef00faaa4"> <div class="note-header"> <a href="/user/7cc328a08ddb2afdf9f9be77beff4c83489ff979721827d628a542f32a247c0e"> <img src="https://rogue.earth/.well-known/th3administrator1.jpg" class="note-avatar" alt="Avatar" loading="lazy"> </a> <div class="note-info"> <a href="/user/7cc328a08ddb2afdf9f9be77beff4c83489ff979721827d628a542f32a247c0e" class="author-link"> <span class="note-author-name" title="7cc328a08ddb2afdf9f9be77beff4c83489ff979721827d628a542f32a247c0e"> cloud fodder </span> </a> <a href="/post/0c56385181bd388346b3f64d05218fc6770813c253a5951c91e8420ef00faaa4" class="note-date-link" title="2025-12-23T23:38:31Z"> <span class="note-date">1w ago</span> <span class="reply-count" aria-label="View replies">💬 2</span> </a> </div> <button class="collapse-toggle" onclick="toggleNote('0c56385181bd388346b3f64d05218fc6770813c253a5951c91e8420ef00faaa4')" title="Collapse/Expand">[-]</button> </div> <div class="note-body" id="body-0c56385181bd388346b3f64d05218fc6770813c253a5951c91e8420ef00faaa4"> <div class="note-content"> <p>hm, would this be a good way to say.. scrape a particular subreddit and post its content onto nostr topics? 🤔</p> </div> <div class="nested-replies "> <div class="note reply-note depth-4" id="note-0abbe7b690132f571f4cbe0e1f83841a76f90a2b994738ebc345ff5bdf1f735e"> <div class="note-header"> <a href="/user/036533caa872376946d4e4fdea4c1a0441eda38ca2d9d9417bb36006cbaabf58"> <img src="https://www.vaughnnugent.com/public/blogs/personal/content/syz7ipjwji3nrdtlz7phc64bwy.webp" class="note-avatar" alt="Avatar" loading="lazy"> </a> <div class="note-info"> <a href="/user/036533caa872376946d4e4fdea4c1a0441eda38ca2d9d9417bb36006cbaabf58" class="author-link"> <span class="note-author-name" title="036533caa872376946d4e4fdea4c1a0441eda38ca2d9d9417bb36006cbaabf58"> ChipTuner </span> </a> <a href="/post/0abbe7b690132f571f4cbe0e1f83841a76f90a2b994738ebc345ff5bdf1f735e" class="note-date-link" title="2025-12-24T03:10:08Z"> <span class="note-date">1w ago</span> </a> </div> <button class="collapse-toggle" onclick="toggleNote('0abbe7b690132f571f4cbe0e1f83841a76f90a2b994738ebc345ff5bdf1f735e')" title="Collapse/Expand">[-]</button> </div> <div class="note-body" id="body-0abbe7b690132f571f4cbe0e1f83841a76f90a2b994738ebc345ff5bdf1f735e"> <div class="note-content"> <p>... Ahhh. Quite possibly.</p> </div> </div> <div class="collapsed-indicator" id="ind-0abbe7b690132f571f4cbe0e1f83841a76f90a2b994738ebc345ff5bdf1f735e" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> <div class="note reply-note depth-4" id="note-f55fd4a0c88710be036ed2f443d1bf6f8fb4e499d6a5c7ff2b7f01274393dc65"> <div class="note-header"> <a href="/user/fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1"> <img src="https://image.nostr.build/0746072c2304d7a0ae845342bf07cd89dcefad50950dcdb8adb5bc58cfb145f1.jpg" class="note-avatar" alt="Avatar" loading="lazy"> </a> <div class="note-info"> <a href="/user/fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1" class="author-link"> <span class="note-author-name" title="fd208ee8c8f283780a9552896e4823cc9dc6bfd442063889577106940fd927c1"> Silberengel </span> </a> <a href="/post/f55fd4a0c88710be036ed2f443d1bf6f8fb4e499d6a5c7ff2b7f01274393dc65" class="note-date-link" title="2025-12-24T23:10:06Z"> <span class="note-date">1w ago</span> </a> </div> <button class="collapse-toggle" onclick="toggleNote('f55fd4a0c88710be036ed2f443d1bf6f8fb4e499d6a5c7ff2b7f01274393dc65')" title="Collapse/Expand">[-]</button> </div> <div class="note-body" id="body-f55fd4a0c88710be036ed2f443d1bf6f8fb4e499d6a5c7ff2b7f01274393dc65"> <div class="note-content"> <p>Sure.</p> </div> </div> <div class="collapsed-indicator" id="ind-f55fd4a0c88710be036ed2f443d1bf6f8fb4e499d6a5c7ff2b7f01274393dc65" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> </div> </div> <div class="collapsed-indicator" id="ind-0c56385181bd388346b3f64d05218fc6770813c253a5951c91e8420ef00faaa4" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> </div> </div> <div class="collapsed-indicator" id="ind-10c20b59cfe71243a0de9ce2b8315a1e5cf7c9ee8507678f8ae811e911eb0cf2" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> </div> </div> <div class="collapsed-indicator" id="ind-e1c47b14c3fe9c087d93127e2c2e3f76fb91162317142683686f3ab94c6dd428" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> </div> </div> <div class="collapsed-indicator" id="ind-198089902ffcb8eb0878dd512b7622d0df476dbc3022c6c425cb55fd08f1d989" style="display: none;"> <span class="collapsed-text">Thread collapsed</span> </div> </div> </div> </div> <script> function toggleNote(noteId) { const body = document.getElementById('body-' + noteId); const indicator = document.getElementById('ind-' + noteId); const toggle = document.querySelector('#note-' + noteId + ' .collapse-toggle'); if (body.style.display === 'none') { body.style.display = 'block'; indicator.style.display = 'none'; toggle.textContent = '[-]'; } else { body.style.display = 'none'; indicator.style.display = 'block'; toggle.textContent = '[+]'; } } </script> </main> </div> <script> (function() { const html = document.documentElement; const themeToggle = document.getElementById('themeToggle'); const sunIcon = document.getElementById('sunIcon'); const moonIcon = document.getElementById('moonIcon'); function setTheme(theme) { html.setAttribute('data-theme', theme); localStorage.setItem('theme', theme); if (theme === 'dark') { sunIcon.style.display = 'block'; moonIcon.style.display = 'none'; } else { sunIcon.style.display = 'none'; moonIcon.style.display = 'block'; } } const savedTheme = localStorage.getItem('theme') || 'light'; setTheme(savedTheme); themeToggle.addEventListener('click', () => { const currentTheme = html.getAttribute('data-theme'); const newTheme = currentTheme === 'light' ? 'dark' : 'light'; setTheme(newTheme); }); })(); </script> </body> </html>