Provably unmasking malicious behavior through execution traces

46•PaulHoule•2w ago

Comments

causalmodels•2w ago

Interesting direction but the 98.8% FPR in Table 1 seems like a dealbreaker. Anyone understand what's going on with the contradictory results between the text and tables?

dwattttt•2w ago

> Empirically, CTVP attains very good detection rates with reliable false positives

A novel use of the word "reliable"? Jokes aside, either they mean the FPR as the opposite of what you'd expect, the table is not representative of their approach, or they're just... really optimistic?

godelski•2w ago

  >  Anyone understand what's going on with the contradictory results between the text and tables?

Well Figure 1 would also disagree. It shows a FPR of 47.5%.

From Sec 3, end of second to last paragraph

  | The protocol is deterministic given fixed RNG seeds, caches model outputs

by program hash, and *bounds false positives via the chosen percentile and gap parameters.*

I believe this is a choice, though I think it is suspect that the FPR is pushed this high to get the TP results.

Disclaimer: I only gave this a very cursory skim so don't rely on me too much

thethirdone•2w ago

Based on Table 1: This method is actually worse than generating a random number (0-100% independent of the program) and testing if it is less than 98.8%. That would achieve a better detection rate without increasing the false positive rate.

It doesn't seem worth it to try to follow the math to see if there is something interesting.

Joel_Mckay•2w ago

"'Forbidden' AI Technique" (Computerphile)

https://www.youtube.com/watch?v=Xx4Tpsk_fnM

"The Hard Problem of Controlling Powerful AI Systems" (Computerphile)

https://www.youtube.com/watch?v=JAcwtV_bFp4

Attempting to guide statistical salience of LLM reasoning model procedures, usually just created an evasive interface facade in the output. =3

Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

Haskell for all: Beyond agentic coding

SectorC: A C Compiler in 512 bytes (2023)

Speed up responses with fast mode

Software factories and the agentic moment

Brookhaven Lab's RHIC concludes 25-year run with final collisions

Vouch

Do you have a mathematically attractive face?

Stories from 25 Years of Software Development

Hoot: Scheme on WebAssembly

IBM Beam Spring: The Ultimate Retro Keyboard

First Proof

FDA intends to take action against non-FDA-approved GLP-1 drugs

Vocal Guide – belt sing without killing yourself

Show HN: I saw this cool navigation reveal, so I made a simple HTML+CSS version

Al Lowe on model trains, funny deaths and working with Disney

Show HN: Axiomeer – An open marketplace for AI agents

LLMs as the new high level language

Show HN: A luma dependent chroma compression algorithm (image compression)

Start all of your commands with a comma (2009)

The AI boom is causing shortages everywhere else

Microsoft account bugs locked me out of Notepad – Are thin clients ruining PCs?

The F Word

I write games in C (yes, C) (2016)

Selection rather than prediction

Eigen: Building a Workspace

The silent death of good code

Reinforcement Learning from Human Feedback

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

Learning from context is harder than we thought