So I've been messinga round with embedding models for a bit, and I think they are interesting enough to experiment with. They are useful for rag, especially in a localllm sense because you can ground your answers in truth.

But what happens if you have a billion documents, and you decide to upgrade your model to a "better" one? on an h100, that would take about 108 days, just to upgrade the vectors so u can start serving again (tested qwen embed 8b on h100). Even if you aren't doing 1b vectors, and are doing just 50 million, upgrading can still take a considerable time.

Me and my research lab decided to tackle this problem, and we came up with embedflow.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive backfill that comes with upgrading, as you can directly take documents from the old index.

embedflow works with qdrant, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

How AI harnesses can drive robots through the ROS 2 stack — controlling Nav2 and MoveIt in simulation. Inspired by Anthropic Article: https://www.anthropic.com/research/claude-plays-robotics

The robot runs in the Gazebo simulator; the harness and the ROS 2 stack (Nav2, MoveIt,
perception). There are two clocks: the harness reasons at a high level on a slow
clock, and the robotics layer runs the fast control loop at 20–50 Hz. Between them sits L3 —
the harness names a verb, L3 turns it into a real Nav2 or MoveIt call. Harness provides the Adaptive Intelligence layer.

https://www.youtube.com/watch?v=lLf8E5ILEZE

I built a wrapper around a frozen 35B Qwen model. It enables an argument graph the model builds for itself while it works. Evidence, premises, claims, and rebuttals, in Toulmin's structure, constructed claim-first the way Popper and Deutsch describe knowledge growing: conjecture, then criticism. It persists across tasks, and confirmed claims graduate into it with the procedure that made them work. The model writes it through interaction with its environment. .

The wrapper has a short-term working memory that holds the model's goal, its active conjecture, and what it has tried and refuted. I developed it on a text adventure game, Deephome, complete with an observability layer I called a Gods-eye-view so i could watch the LLM move and interact with its environment. The fixed environment enabled iterative runs, observing scoring relative to the number of turns to complete the game. The wrapper is intended to be generic, multi-modal, so I ran the same wrapper on a general relativity textbook. The model read the book into a reasoning map and was given a problem whose answer is not in the book.
The write-up covers the mechanism, the observability layer, the measures, and what did not work.

full writeup: https://spencerwheat.substack.com/p/conjecture-and-criticism-graphs-for

You guys really loved my last sheet music search engine post, so I decided to make a mockup POC of a VLM-powered piano assistant that reads and transcribes notes, procedurally "generates" a visual tutorial, and listens to you play it back. I call it a concept because I still haven't figured out a way to actually generate a 3D animation on the fly. My best option was Concert Creator, but that was shut down over 4 years ago RIP. So for the time being, I'm having to pre-make the animations.

On the more technical side, Qwen 3.6 27B is taking both video stream and natural language as input, and uses that to determine what smaller task-specific models to call on for the situation. In this case, it's using segmentation, homography, pitch detection, and a custom music OCR model (open source called cadenCV). In short, the VLM is acting as an orchestrator that can see things and reason the best course of action because I'm too much of a lazy bum to hard-code a set of rules for when each model gets called on.

The next step is to figure out how to make it run in real time.

MCP servers are proliferating fast. Most teams I talk to are deploying them without any per-call policy enforcement or verifiable agent identity. The assumption is that container isolation is enough.

It isn't.

A rogue agent — compromised, misconfigured, or just hallucinating badly — can chain tool calls faster than any human-in-the-loop can react. The empirical window between first action and second action is under 50ms. By the time an alert fires, the damage is already two or three tool calls deep.

The deployment-level controls (containerization, network segmentation, sandboxing) address where the agent runs. They don't address what the agent is allowed to call, under what conditions, with what verified identity. Those are different problems.

Practitioners who are actually running MCP in production: how are you handling per-call enforcement today? Are you writing custom middleware, relying on the MCP server's own access controls, or accepting the risk and monitoring after the fact?

A model accepting a megabase of DNA establishes input capacity. Demonstrating useful distal information needs a task-specific test.

In a review I wrote, I suggest comparing nested crop lengths with the same prediction head and evaluation split, then occluding or shuffling distal intervals while preserving the focal sequence. Short-context models and local baselines are useful comparisons; overlapping genomic windows also need attention when constructing the split.

Full discussion and references: https://rewire.it/blog/a-dna-likelihood-is-not-a-functional-assay/

What controls would you add to distinguish useful long-range dependence from leakage or local shortcuts? I’d appreciate corrections and benchmark suggestions.

Qwen 3.6 27B watches the scene and acts as an orchestrator, prompting smaller task-specific models to perform OCR, segmentation, and edge detection. The segmented notes then get matched to real pieces using "A Dictionary of Musical Themes" database via Themefinder. I’m sure there are already many sheet music search engines out there, but I built this more as an experiment to see how I can turn VLMs into full-on vision agents. 

Disclaimer: footage is sped up, this is not real-time.

Over the last month, in my spare time, I’ve been working on two separate ideas: some recurrent-network maths as a possible GRU/LSTM substitute, and a transformerless AI experiment.

The transformerless project became far too slow to properly train, test and debug on the hardware I have, but one part of it kept behaving well in small tests. I pulled that section out and turned it into its own project called THREADS.

THREADS is basically a deterministic memory and reasoning engine. You give it structured facts and relationships, and it can follow them, keep track of changes over time, remember historical states, handle retractions and contradictions, and keep a record of which facts produced an answer.

A few of the current test results:

200,000-hop reasoning chain: exact final answer

1,000,000 irrelevant events: 128-hop query still returned the exact answer

50,000 shuffled temporal events: 5,000/5,000 historical queries matched an independent checker

40,000 ambiguity/contradiction tests: 40,000/40,000

60-category reasoning suite: 5,830/5,830

original regression suite: 28/28

bounded program-induction tests: 2,000/2,000 held-out predictions

I’m not claiming it replaces transformers, SMT solvers or databases. It doesn’t understand arbitrary English by itself. I’m mainly interested in whether something like this could act as an exact memory/reasoning layer underneath an AI system, while a neural model handles the language side.

I’ve put the Python source, tests and research PDF on GitHub so people can run it, it should all be working 😬

And hopefully this one beats my previous 68% upvote rate 🤣

We recently released a benchmark looking at uncertainty estimation across model sizes, comparing Farquhar et al.'s Semantic Entropy (Nature, 2024) against a normalized exact-match entropy metric (R_sc).

Preprint: https://zenodo.org/records/22233648
Code: https://github.com/Adarshent/Spnda

The issue with standard semantic entropy is the inference cost. Clustering 10 sampled paths with a DeBERTa-v3 cross-encoder takes around 100+ seconds on CPU and adds significant VRAM overhead, which makes it impractical for real-time serving.

On reasoning tasks like GSM8K across 7B to 27B models (Mistral-7B, Qwen-27B), we found that exact-match normalized entropy achieves an AUROC of 0.889, matching neural semantic entropy while executing in under 2ms on CPU.

However, once you scale up to frontier 120B models, self-consistency breaks down due to what we call Confident Mode Collapse. Under temperature 0.7 sampling, RLHF alignment hyper-peaks the distribution so heavily that the model outputs the exact same incorrect answer across all sampled seeds. Because all paths agree on the hallucination, uncertainty drops to near zero and AUROC drops to 0.091.

Code and evaluation scripts are on GitHub under MIT. Curious if anyone here has explored alternative decoding or perturbation strategies to restore entropy on over-aligned frontier models.

Security teams inside large enterprises are now building and shipping their own AI agents. The goal is legitimate: automate threat detection, close alert triage gaps, fill holes that vendors haven't addressed yet. The pace is faster than most governance teams can track.

Shadow AI agents — built and deployed without IT or compliance visibility — are already running in most enterprise environments. There is no reliable count inside most organizations. The structural exposure is not the agent itself. It is that once an agent is live, its tool access and data connections are rarely tracked centrally. Runtime behavior stays opaque unless someone manually audits it after the fact.

The compliance surface compounds the problem. Most large enterprises operate under 80-plus regulatory and security frameworks. Agents built by internal security teams are not automatically enrolled in those frameworks. They run alongside them, not under them.

For practitioners actually managing this: how are you maintaining real-time visibility into what your internally built agents can touch at runtime? Is there a process that is working at scale, or is this still manual audits and institutional memory?

Loss spikes are a known hazard in LLM pretraining. PaLM 540B hit about 20 of them and

handled each by rolling back ~100 steps and skipping 200-500 data batches. The obvious

response is a detector: watch the gradient, catch the spike, skip the update before it

corrupts the optimizer state.

I tried to find out whether that pays for itself. It doesn't, at least not at the scale I

can afford to run.

Setup

A 0.65M parameter transformer (2-3 blocks, multi-head causal attention, optional top-2 MoE)

trained with a deliberately high learning rate, no warmup and no gradient clipping. Clipping

is exactly what masks the phenomenon, so it enters later as the comparison arm.

Spikes are induced rather than injected. Chowdhery et al. found spikes arise when a specific

data batch meets a specific parameter state, so the corpus is built to produce that: a

Zipfian token marginal (rare tokens really are rare, and their embedding rows accumulate

little signal before they appear), a sparse Markov transition matrix for learnable

structure, and induction spans. Batches enriched in rare tokens are logged but never used as

labels, so an enriched batch that causes no spike counts as a negative.

42 of 42 spike onsets coincided with a rare-token batch, which reproduces the data x state

mechanism.

Everything is pure NumPy on a 140-line autodiff core that's gradient-checked against central

differences in float64. No PyTorch, no GPU, no downloads. 8 seeds, 12,000 steps, about 15

minutes on a laptop.

Results

Detection works. Gradient norm read after backward but before the optimizer step

separates spike steps from ordinary ones at AUC 0.914 [0.88, 0.94], p=0.0007. Combining five

signals under leave-one-run-out CV reaches 0.961 [0.94, 0.97].

There is no lead time. The same signal one step earlier gives AUC 0.503. Two steps

earlier, 0.429. So you can gate the optimizer step, but you cannot abort the forward pass.

I tested the claim that early-layer activation spikes fire first: act_max_early scores 0.503

at the gate point, and adding it to the combined detector makes it slightly worse.

Gradient norm cannot tell you which spikes matter. Separating above-median-damage spikes

from below-median: AUC 0.492. Correlation between spike severity and damage: +0.022.

At this scale the spikes are benign. Damage is mean loss over [t+10, t+60] minus the

pre-spike trend extrapolated forward. Compared against the identical measurement at matched

random non-spike steps: AUC 0.432, permutation p = 0.86, Cohen's d = -0.20. Without that

control my own metric was labeling half of a symmetric noise distribution "harmful," which

is a mistake worth flagging for anyone building something similar.

Clipping wins for free. Using a fixed absolute threshold applied to both arms (a rolling

MAD threshold is not comparable across arms, since clipping shrinks the MAD and inflates the

apparent spike count), clipping at 1.0 cut steps above threshold from 120 to 52, reduced loss

sd by 8.6%, and gave a better final loss. It keeps the step and discards no batch.

Economics. Priced against a 1.8T-parameter, 15T-token run on 16,384 H100s (937,500 steps,

~$263M at $2/GPU-hr, 20 spikes at 8h median recovery), where a false trip wastes a full

forward and backward pass:

| FPR | Recall | Wasted | Saved | Net |

|---|---|---|---|---|

| 0.1% | 0% | $0.3M | $0.0M | -$0.3M |

| 1% | 10% | $2.6M | $0.5M | -$2.1M |

| 5% | 57% | $13.2M | $3.0M | -$10.1M |

Limitations

0.65M parameters is not 1.8T. These are transient spikes, not catastrophic divergence, so

the harness cannot test the case that motivates rollback machinery in the first place. This

is a null result about mild spikes, not proof that gating is worthless at scale. 42 events is

adequate but not abundant, and it's one architecture on one synthetic corpus.

What would settle it: injecting divergences of calibrated severity at 1B+ parameters and

measuring damage against matched controls. Spike detection itself is already settled.

Code

https://github.com/TheBlitzschnell/resonance-spike-lab

pip install -r requirements.txt

python3 gradcheck.py

python3 sweep.py --seeds 8 --clip 1.0

Reference telemetry is committed, so every number above reproduces in a few seconds without

retraining. If you think I've measured the wrong thing, I'd like to hear it.

https://reddit.com/link/1w81p3p/video/50d4gyg3lpnh1/player

I made a short animation of how attention moves while a Transformer generates text.

First few seconds are prefill: the model reads the whole prompt, tokens looking at earlier ones in parallel.

After that is decode: one token at a time. Each new token sweeps back over the context, then the next word appears and joins the history for the next round.

Not trying to teach the full attention math here. Just the macroscopic feeling of information flowing through those look-backs.

The flicker looks almost cyber-beautiful, at least to me. 😄

Demo: https://dqy08-infolens.hf.space/client/causal_flow.html?demo=%E6%B3%A8%E6%84%8F%E5%8A%9B%E7%9A%84%E5%85%B7%E8%B1%A1%E5%8C%96
Source: https://github.com/dqy08/InfoLens