Base Mech Interp

Start writing awesome blog post from Notion

Use cases

Trading

• Data - News price up/down/no change

• Layer -16

• Probe and SAE - trained on above data and Layer

• Buy and sell when both results from probe/SAE agree.

• Use vectorbt and quantstats - for backtesting and reporting.

• Result:

Audit and agentic tracing

Agentic

• Setup - Get market data, get news data, get sector info, investment recommedation agent.

• Green, Amber or red based on the expected features activate. Work needs to be done

• Question : Value add on top of COT ? Ground truth ?

• Sample Results:

Audit

• For a prompt, look at the top “selected” features activated across layers

What happens during fine tuning

Hallucination

• SAE features classification

• In progress:
• Train linear probe and get the hallucination percentage for a sentence.
• Check whether they overlap

• Aspirational : https://arxiv.org/abs/2509.03531

Other supervised : doc parsing, sentiment, credit risk

• probe vs SAE based classification

Mech Interp : Domain specific model/tool prep

SAE Training

• Findings on SAE training with following items and SAE trained on LLama/BERT
• Reconstruction loss vs. sparsity.
• Importance of OOS
• #Latents for data/model size

Auto-interp

• Domain specific vs generic data - superposition

• F1 and other metrics

• Self-interp based cross validation of auto-interp labels

• Feature clustering - similar to below from anthropic

Supervised tool prep

• Probe - Increasing the accuracy, advantages, lasso

• SAE classifier - Increasing accuracy through Lasso, random forest etc. Using relevant features