fair critique!

i don't claim ground truths are perfect. but when they are combined it may work.

and i tried to simplify it, we are not machines :)

Llama 4 is out. 👀

Llama 4 Maverick (402B) and Scout (109B) - natively multimodal, multilingual and scaled to 10 MILLION context! BEATS DeepSeek v3🔥

Llama 4 Maverick:

> 17B active parameters, 128 experts, 400B total parameters > Beats GPT-4o & Gemini 2.0 Flash, competitive with DeepSeek v3 at half the active parameters > 1417 ELO on LMArena (chat performance). > Optimized for image understanding, reasoning, and multilingual tasks

Llama 4 Scout:

> 17B active parameters, 16 experts, 109B total parameters

> Best-in-class multimodal model for its size, fits on a single H100 GPU (with Int4 quantization)

> 10M token context window

> Outperforms Gemma 3, Gemini 2.0 Flash-Lite, Mistral 3.1 on benchmarks

Architecture & Innovations

> Mixture-of-Experts (MoE):

First natively multimodal Llama models with MoE

> Llama 4 Maverick: 128 experts, shared expert + routed experts for better efficiency.

Native Multimodality & Early Fusion:

> Jointly pre-trained on text, images, video (30T+ tokens, 2x Llama 3)

> MetaCLIP-based vision encoder, optimized for LLM integration

> Supports multi-image inputs (up to 8 tested, 48 pre-trained)

Long Context & iRoPE Architecture:

> 10M token support (Llama 4 Scout)

> Interleaved attention layers (no positional embeddings)

> Temperature-scaled attention for better length generalization

Training Efficiency:

> FP8 precision (390 TFLOPs/GPU on 32K GPUs for Behemoth)

> MetaP technique: Auto-tuning hyperparameters (learning rates, initialization)

Revamped Pipeline:

> Lightweight Supervised Fine-Tuning (SFT) → Online RL → Lightweight DPO

> Hard-prompt filtering (50%+ easy data removed) for better reasoning/coding

> Continuous Online RL: Adaptive filtering for medium/hard prompts

finishing a project is 10x harder than starting it. cleaning up the bugs is 10x more boring than writing that feature. maintaining a project is 10x less exciting than releasing it.

- do I have other open projects that require my attention instead?

- will I be able to finish this new project or will I only start it?

- should I drop a project so I can start a new one?

take responsibility for your ideas. let them go when it's time.

I just paid 9000 sats for a bottle of water. FML.

i love nostr and i remain incredibly bullish on nostr, however these are the apps that i use daily that cause me frustrations. when i said this morning that i go from bullish to frustrated and back, these are the reasons:

nostrudel - i have to refresh or reload the app often to get profile names and photos to display. sometimes new notes don't load and i have to refresh or reload to display them.

chachi - chats just don't load most of the time. i have to nuke local storage and reload and refresh constantly to get new chat messages to display. this is tedious.

olas - i haven't used olas since the last update. i can't upload images. the images upload blank or never finish publishing. even the feed is inconsistent showing me mostly my images instead of images from people i follow.

pokey - mute thread is not consistent. sometimes it works and sometimes it doesn't.

citrine - no new events get published. i've been at 207,513 events for months. i think this happened whenever i tried to import all of my events from all of my relays. whenever i open citrine there's a notification that it's exporting events, but that never finishes. i think i need to cut my losses and my local data and nuke and start from scratch, but then ill lose all of my local data :(

wavlake - a popup window for amber is displayed every single song that i listen, because it's trying to set a status for the song im listening to. this continues even if you attempt to remember or save the setting.

primal - my NWC somehow stops working. i can generate a new string and apply it and have zaps work for a few days, but eventually it will stop working and ill have to disconnect and regenerate new NWC strings.

fountain - i just don't boost as often as i'd like, because having a separate wallet balance makes boosting harder. NWC is coming to fountain though. that will resolve this.

---

honestly the most stable and consistent apps that i use daily are #0xchat, #Amethyst and #Notedeck (jumble.social is great too, i just don't use it that often.)

---

looking at the larger picture, these are not showstopping issues, but mere annoyances along a bumpy path forward. and that's where we are moving. forward.

keep building.

Is there a DVM that connects me to an AI via Nostr chat sessions?

GM☀️👻✍️ #art #artstr #nostr

what are you looking for?

Why you at ETH Denver? We sending out Bitcoin missionaries now to spread the good news? 😂