I’m incredibly excited about it, but also in the camp of not being fully in the loop. I will bookmark this note and check back later
Discussion
I think we move slowly, like a glacier, working in the background on stuff like an AI for mass-converting PDF to Asciidoc, and performance improvements, and containerizing the server landscape and etc., so it seems like nothing is going on, even tho it's a small horde of active developers, admins, and testers scrambling around.
But it has to handle millions of publications and one publication could be the Encyclopedia Britannica. We have to revamp and reengineer a lot, under the hood, but it seems to be working. Better than we originally expected.
I guess it sorta seems like pie-in-the-sky stuff.
and why not?
The vision is too grand. Way out of scope of what Nostr is normally aimed at. We're just supposed to be tweeting and selling each other home-knitted sweaters and stuff.
This is a Big Data project that doesn't need any influencers or hype to be interesting, which is why it attracts little funding, but lots of developers.
We announces that we were going to move all out of copywrite texts we could find on the Internet onto Nostr relays and everyone was like *polite clap*.
No, but fr, tho. 😅
We're going to scrape books and publications off of everything and turn Nostr into a new InternetArchive.
But with zaps. 😅
Excellent.
I’m getting interest from academics in my life who are publishing work that they fear will be censored. They are not nostr natives—they probably don’t even know nostr exists yet, except for my short conversations with them. Where should I point them for uploading their writing and research?
They're actually going to be the focus of the next version (named "Euler"), so you can show them our first research paper test case. That's a real paper, we pulled off of open access.
You can see, that we still need to render embedded Asciidoc and LaTeX properly and fix the footnotes, and stuff, but you can get the general idea. Euler will also allow them to export to LaTeX files, which is nice.
I used my CLI for uploading, but if they wait a couple of weeks, they can upload over #Alexandria, which will be more comfy.
Easy to read on mobile, as well, and Euler will come with the e-paper capability. So, they can read Nostr-published papers on their Kindle or whatnot, and write annotations or comments, etc.
We actually have universities interested in this, so that's why we're doing scientific papers before book clubs.
If they're curious enough, please point them our way. We want to build a home for them & their work.
I will for sure
nostr:npub1c0qyae9ggdxmrs9gnpkrc5t0dzncfgypvmrx9rzygclzyld5q4nqe9ja8j don't know if you've seen these developments, seems up your alley (at least last I heard)
Hi nostr:npub1fhpw2ux9flhcxyl6xp84996qgnkkcy59zqzjvq9fhpxcx7upymus69ds8n, how are you?
I hadn’t seen these developments, thanks! About my previous project, unfortunately I haven’t been able to continue the project because I’ve been working two jobs to pay for my master’s degree. But I’ll try to take a look and see if I can contribute something to this repo
Best off talking to the PO about it. nostr:npub1wqfzz2p880wq0tumuae9lfwyhs8uz35xd0kr34zrvrwyh3kvrzuskcqsyn
This is absolutely an amazing mission. As an academic, I despise PDFs. They're just a terrible medium for communicating scientific work, not to mention the fact that malware can be stuffed into them. It's frustrating to see my published research essentially get handicapped/walled off of any kind of novel and innovative interaction with the content.
One of our project team members is a biologist and I also come from a bionformatics background (Human Genome Project). So, we understand the problem that needs to be solved.
Nice! As in, the Human Genome Project from the early 2000s?
Yeah, I worked on software used to plot the arrays and on a project to create a visual model of scientific journal articles.
Sequencing has come a long way since then. You can sequence the human genome with a USB-connected device that's no bigger than a raspberry pi.
Anyway, I'll be sure to follow the latest from the project.
And another one is a physicist
Very nice, yes.
And there's a mathematician, at least two electrical engineers (I think the mathematician is also an EE), a DevOps engineer, and a computer scientist. I think two comp sci, actually. 🤔 Losing track.
And real writers doing the testing, and two of the writers are professional software testers.
TOTAL DREAM TEAM, BABY! 😅
Is the biologist on here?
Yes as a scientists as well I always prefer a journal article that has a web-readable format, because then it will (most of the time) scale nicely to a mobile device...
But what is the alternative for a "portable" format? ePub?
I'm not sure I understand what you mean by alternative to a portable format
By "portable" I guess that means take the article offline... store on your phone, for a plane trip, etc. Right now PDF is basically the way to do that, unless you're cacheing the web page or something...
Okay I see what you mean. To me the gold standard really is just plaintext. For scientific reporting, you can mark up reports with simple tools like asciidoc. So now you have a format-agnostic presentation that can be consumed any way you like, transmitted through essentially anything you like, for example Nostr or whatever offline reader.
Quite the opposite from saying "this is the (probably DRM'd) format, it has text embedded, and you can only use X/Y/Z (probably locked/closed/restricted) app to consume it".
Yeah, I just download the publication events (they're in Asciidoc) to the #Citrine relay on my phone, or to the nostr-rs relay on my laptop, and then they can be read offline.
But I'm even more excited about the e-paper functionality because I read lots of books and publications on my Kindle or Tolino. So much less distraction and no glaring lights.
I mean, the cool thing about doing this all in Nostr events is gonna be sharing the annotations and highlights and stuff. And all of the AI navigation nostr:npub1m3xdppkd0njmrqe2ma8a6ys39zvgp5k8u22mev8xsnqp4nh80srqhqa5sf is building.
You have to remember, also, that the books you see on Alexandria are all a bunch of different events, rendered in order.
So, the Bible will be split down into books, chapters, and verses, and all 30k+ verses will be a different event, so that people can refer to them, or annotate/comment on them, individually.
So, when you're looking at this edition of Jane Eyre, for instance, that is a different for every chapter. It just renders as one document. That can be read online or offline.
Accessing content this way is just so much better.
Yeah, and the performance is about to get a major boost, with the new parser and some smoother rendering and caching.
The key is to make leaving it on-Nostr so useful, that people can't be bothered exporting to other formats. Because then you end up with one, static document stored where-the-heck-evah. Dead papers. Totally goes against the idea of a scientific paper, as those are supposed to fuel discourse and imitations and connections. Papers need to be alive.
Another thing Nostr brings is WORA (write once, read anywhere). You publish your paper to Nostr and it immediately shows up in every client that handles that event type.
That's completely different to a static document file, where you have to apply to this or that committee or website, and then send them the document or whatnot.
Everything automatically ripples out to the relays and clients, on Nostr. And, if you update the document, that update also ripples out. We have a relay that will do versioning of changes, too, which is cool.
You folks are working on a ton of great stuff. This is the future I want to see.
But it's a Nerd Future, not an Influencer Future, so it's hard to get any publicity, on here, but oh well.
It will get built and be used, even if only people who can #NameTenBooks can be bothered to look at it.
There are actually a lot of nerds, out there. Nerds are a huge market. 🤣
Agreed. Basically any academic I interact with has at one time or another complained about the issues with the current paradigm for disseminating scientific work. I'm not talking about politically charged research, just genuine scientists in apolitical fields trying to push the boundaries of knowledge.
Content through nostr is the kind of thing they subconsciously know they need but they just haven't seen the potential for nostr yet.
If we build it, they will come.
I really hope so. I'll be doing my part.
A big drag on this evolution will likely come from the parasitic science publishing industry, but probably also from funding agencies, and even the researchers themselves to an extent. The publishing monopoloy gets fat margins for doing essentially nothing aside from hosting. They do none of the research/validation/peer review/advocating. Funding agencies will complain that they don't have any other way to decide which research to fund because they rely so heavily on manufactured metrics by the publishing industry. Academics might be reluctant to change when their careers are on the line.
Surely you've thought about these issues before, so I'm curious to hear what you think.
Yeah, that's why this only works on Nostr. Every person can publish to the relays and they're done. They don't need to host anything or etc. since they're using the same infrastucture used to post cute cat pics and Bitcoin memes.
And you can publish from an anon npub over Tor, you know. Nobody can tell who did that.
Also, Nostr brings the potential to create other, more modern quality measures, through curation, annotation, remixing, AI, and WoT.
Also, Nostr adds the public discussion. This is really important.
Would you rather have some paper, buried 20 pages in, to a journal hardly anyone reads, or have your paper trending on social media and getting you invites to podcasts and etc.? There's a reason, why so many scientists are on social media. Be seen, engage, get mentioned, or your work is irrelevant to the wider discussion.
Trending/not trending is not really relevant for me since the impact of research can take years/decades to pop off. Yoshua Bengio is a great example of this. Nobody cared about all the fundamental ML algorithms he was coming up with in the 1990s/2000s until deep learning took off in early 2010s and those old publications were suddenly getting cited thousands of times a year.
The transparency and openess that comes with searchability and possibility for public discussion are much more interesting to me. They go hand-in-hand.
Well, academia was practically a village, back then.
I didnt know about Alexandria either. Sounds brilliant. Where can I find the code for Alexandria clients / relays and converters (assuming its all OSS)? And is there a guide on how to convert a document starting from, say, Markdown or AsciiDoc?
I convert with my own parser (which is too ugly to share 😅), but there's standard converters like Pandoc.
Thanks for the info! Now I have a good excuse to explore GitCitadel as well.
Just to make sure I understand the second sentence, by "there's Pandoc", you mean using it to generate the AsciiDoc contents for each individual kind 30041 content "section" correct? (If I got this right from this link.)
What I was trying to ask is whether there's a tool that, given an entire Markdown or AsciiDoc document (possibly with some metadata comments), it generates both the publication index and content events for me. Or is Alexandria's UI the best way to handle this for now? (Hopefully, my question makes more sense now,sorry if it was unclear!)
Also, one more question if you don’t mind. Is there a group where people are discussi Alexandria, especially its more technical aspects?
Well, we have a Slack for the project team but nothing on Nostr, yet. It hasn't really interested anyone else, so far.
Got it. I'm not sure if you wish to open it to other curious devs. If you do feel free to DM me an invite to the Slack Workspace. I'm interested for sure :).
It's an implementation-focused Slack, not generalized for all 30040 clients.
Best thing is to DM nostr:npub1wqfzz2p880wq0tumuae9lfwyhs8uz35xd0kr34zrvrwyh3kvrzuskcqsyn . He's the product owner, so that's up to him. He's always interested in more devs, if you want to work on the Alex project, specifically. 😊
Pandoc is a converter that can transpose a file from Markdown to Asciidoc.
This is my CLI for uploading test data, but the Alexandria "New Note" menu item should be opening, soon. Took it down because we're revamping the parser.
Almost any relay can handle the events. Our web client is community-based, and it displays the community relay only, unless you login (then you can see everything from your inboxes).
Oh IDK, I think this could hit a much broader audience... To me this fits in pretty well with projects like Bookdown, which is a way of creating open-source documents for publication using R notebooks. This is a way towards "reproducible research" in the sense that data analysis methods and calculations can be embedded directly (hidden) into the final document that gets published.
The "atomization" part of this is very similar to what's used in constructing both R and (the more popular) Jupyter notebook, where each "code chunk" can be separated in a similar way that separate verses / paragraphs are, as you previously mentioned.
But yeah as I write this... still Nerd stuff 🙃
First you think about the regular text search capabilities that already exist on nostr - that's already a massive plus over navigating papers. But then you add in vector embeddings for semantic search and we're in a new world of untapped potential...
This seems ideal for something like The Network State, which is published online in a similar kind of "chunked" way, but also as a live document that gets periodically updated...
Anyway I wasn't really aware of Alexandria, really cool project! You should put more on the "about" page, because it wasn't immediately clear from all of the features you're describing... particularly the "content atomization" aspect of it.
Yeah, the About page needs to be revamped. I'm waiting for the Table of Contents and search and LLM features to go live, so that I can make pretty, new screen shots. 😅
We can export easily to ePUB, too, but the e-paper native app will be cooler because then you don't lose all of the Nostr features.
But we have to start someplace and ePUB is easy because Asciidoctor already comes with that. ePUB, LaTeX, and Asciimath.
I think people wonder why we're going through all of this trouble, instead of just adding zaps to a PDF viewer, but... it's already got some awesome capabilities and we haven't even finished v0.1.0, yet.
I hear that. Hard to imagine something more than a PDF when we've had 500+ years of printing press bias and influence from incumbents on how users can/should interact with content. In my own field of work, the scienctific publishing monopoly is especially guilty of this.
Yeah. That's why the first edition of #Alexandria is called "Gutenberg". We want to reimagine the printing press.
A PDF is essentially just Postscript images of pages of paper, and publications haven't moved much beyond that. This is something fundamentally different, that can be displayed _as if it were_ a PDF.
But it's designed to be navigated and produced by human and machine curators. That's why NIP-62 is called "Curated Publications", not "books". It looks like a book, but it's high-tech under the hood.
Still amazing. If it's working on that end then I'm looking forward to seeing the final result.