Thought-Provoking #2

Trajectory vs Output Evaluations

“Trajectory evaluations are equally important as output evaluations. An answer that seemed right, but was arrived via a bad trajectory is more dangerous than an answer that is obviously wrong.”

I think this is an overlooked area in AI evals… Many evals just look at the final output because it is cheaper and easier to do it that way. The OG evals like MMLU, HumanEval, SWE-Bench are all output-based.

By ignoring the agent’s trajectory, we 1) fail to capture why an agent failed to do something, 2) risk encouraging wrong trajectories when it produces correct outputs. The latter reminds me of “reward hacking,” where AI learns to exploit the task to generate the best score, but not actually completing the task at hand - a misalignment between the AI’s actual trajectory and what the human expected.

There are newer benchmarks that focus on trajectories too, like TRAJECT-Bench, or Google’s Trajectory evaluations - to name a few.

Stock Market ≠ Economy

The stock market is not the economy. The market is primarily a collection of large public companies, and its performance doesn’t necessarily reflect the financial health of ordinary people. Share prices are forward-looking signals that are based on expected future cash flows, rather than the actual well-being of households.

That’s why the S&P 500 can still hit near record highs, in spite of major ongoing conflicts, slower economic growth, fears of a “recession,” and lowered consumer spending. Record high stock prices doesn’t equate to record high living standards.

This divergence is exacerbated with AI - which has caused the market to be even more optimistic. Financial markets are pricing the promise of AI-led productivity and lower costs. The AI-led transition can displace workers and concentrate returns in the hands of capital owners - widening the wealth and social gap.

3D-printed batteries

This is an interesting piece that merges two technologies - additive manufacturing (3D printing) and battery storage.

“If we’re able to 3D print batteries, we could potentially put energy storage inside any device… batteries could fill any available space instead of taking a rigid shape - the entire airframe of a drone could be filed with energy storage for increased range.”

When I think of battery tech, I often think of the chemistry aspects - how to make it safer, or more energy dense. I realize I overlook the physical factors like shape and weight, which heavily affects its deployment and usage. There is a real design constraint in battery packaging - it has to fit into the product (in size and shape), and not weight it down.

These 3D-printed batteries - when made possible & stable - will help enable a wave of new applications that uses small or intricate shapes. Like smart textiles - we can design batteries that seamlessly integrate with the shirts we wear. Or batteries that occupy the frame of smart glasses so there isn’t a visible protruding battery pack at the side of our glasses. It will enable a wave of bespoke electronics.

Intelligence Curse

https://intelligence-curse.ai/

Intelligence curse: Countries that strike oil often end up worse governed, because once a state’s wealth comes from a resource in the ground rather than from taxing its citizens’ work, it stops needing those citizens and therefore stops investing in them. Governments will generate revenue from on-demand intelligence rather than from the people.

So, incentives are weakened. When governments no longer need to rely on citizens for economic growth and output, they lack incentives to educate and employ them.

AI can also extend that logic. As AI replaces human workers and governments increasingly draw revenue from AI firms, citizens can also become economically optional to the state.

But this doesn’t mean citizens automatically become politically optional. States still need legitimacy and stability to survive. It’ll be interesting to see if people still retain their bargaining power when human labor is no longer central to the economy.

New Drug Mechanisms

https://www.notboring.co/p/weekly-dose-of-optimism-197

Most of today’s cancer drugs are inhibitors - they disable an overactive protein that is pushing cancer cells to multiply.

Yet, one of the most common cancer drivers is the reverse. TP53 gene, which encodes the tumour-suppressor protein p53, is the most frequently mutated gene in human cancer, altered in roughly half of tumours (meaning TP53 contributes to the development or progression of roughly half of human tumors). Normally, p53 helps damaged cells pause division, repair itself, or self-destruct. When TP53 is mutated, p53 often loses this protective function, allowing damaged cells to keep growing and contributing to cancer.

And that’s the challenge. We can’t restore a lost function by inhibiting something. That’s why we still don’t have an FDA-approved therapy that directly targets mutated p53 - because the issue isn’t that the gene is doing too much of something (which is what today’s drugs can handle), but that it’s not doing anything at all.

Traditionally, CRISPR is used to edit genes, but this new system uses CRISPR-Cas12a2 as a programmable kill switch - it recognises a specific RNA sequence inside a cell and, when it finds a match, triggers DNA damage that kills that cell.

This is similar to CAR-T, where a patient’s immune cells are genetically modified to target and kill specific cancer cells. The difference is that CAR-T’s engineered receptors usually bind proteins displayed on the outside of a cell, while this CRISPR system can recognise a chosen RNA sequence inside it.

Like CAR-T, many cancer treatments can only act on what they can physically detect or bind on a cell’s surface. But many cancer-driving mutations are effectively hidden inside the cell.

So, if this CRISPR system can be safely delivered in humans, the promise is pretty huge - it can eliminate cells based on molecular signatures inaccessible to many existing cancer treatments, while leaving the healthy cells alone.