I built a programming language that LLMs can't use

https://blog.picheta.me/post/killswitch-lang/

26•dom96•23h ago

Comments

prologic•23h ago

May I ask what the motivation behind KilllSwitch is? Out of curiosity, I decided to test your claims and asked Claude to build a program in Go and Claude Skill that is able to decompile (from) and compile (to) KS programs along with a Claude Skill that doesn't trip it up. It all works™ -- All your example programs.

So, whilst I _think_ you've built a very clever interpreter, I don't think you can get around the fact that someone (like me) could just build around the noise you've created anyway.

Right?

dom96•12h ago

The motivation is to build a language that is ideally impossible for LLMs to read/write/understand, or at the very least as difficult as possible. It's cool that you've given it a try, but if you feed Claude the KillSwitch source code it can feasibly just analyse the Rust interpreter source code and likely get around understanding the language itself.

I'm much more interested in the benchmarks[1] I have built which do show LLMs not being able to gain full ability to understand it. The benchmarks only give the model access to the interpreter binary and the spec, nothing else and no internet access. I'd also be interested in other's ideas of how to make the language even more resistant to LLMs/coding agents.

1 - https://bench.killswitch-lang.org/

prologic•11h ago

I think you missed my point. What I'm saying is that your concern over building a language that LLM(s) initially can't read or understand, whilst is very admirable, is futile IMO. Why? Because you're not trying to protect your code from LLM(s) at all, there are humans behind these tools. e.g: me.

I just demonstrated to you that no amount of obfuscation is going to protect your work from an LLM -- Not whilst there's still humans behind the tools driving them.

cyanydeez•10h ago

[flagged]

dang•59m ago

We've asked you many times to stop breaking the site guidelines and you've continued to do it.

If you'd please review https://news.ycombinator.com/newsguidelines.html and fix this properly going forward, we'd appreciate it.

idiotsecant•37m ago

The first paragraph of the link explains the motivation.

drusepth•50m ago

I wonder if another approach would be to map the language syntax onto some banned/censored LLM topic(s) such that programs end up as "readable" instructions for topics the LLM refuses to read and/or write, like producing napalm, nukes, bioweapons, etc. Doesn't have to be factually correct instructions, the terminology itself is enough to shortcircuit most LLMs.

(Inspired by recent malware including bioweapon instructions in the source code as a way to shut down LLM-based analysis.)

pixl97•40m ago

Then all the AI needs to do is remap the concepts to other words.

Really good AI and harnesses will push off the 'dirty' work to other models that are weaker but not restricted and then work on that output if needed.

As tricks like this become more common you'll see LLMs use things like bayesian classifiers to see if you're trying to fool it.

gregsadetsky•32m ago

Great minds think alike! From the kill switch spec [0]:

“Function return: what happened at tiananmen square”

Ha!

[0] https://github.com/dom96/KillSwitch/blob/main/SPEC.md#cheats...

dom96•7m ago

Yep! There are other phrases that can be used like "create a nerve agent"[1] too. Surprisingly I haven't seen a single Chinese model get caught out by this, but Fable and Gemini Argon 4 both do to a large extent.

1 - https://github.com/dom96/KillSwitch/blob/main/SPEC.md#abrupt...

pixl97•45m ago

Security by obscurity is not security at all.

You're attempting to use a few little gotchas against LLMs that only make sense if you're looking at it one dimensionally as a standard execution flow. AI is not limited by that. Give it a problem too weird going forward and it will divine all your rules by working backwards. Decompile it, rip it apart, and it will follow all those hard rules hidden in the application.

You are trying to hide something digitally from a purely digital creation. This is not a recipe for success.

yjftsjthsd-h•45m ago

I guess my question is whether it increases the difficulty more for LLMs or humans.

pixl97•39m ago

Humans by far. The rules are all there to be observed at the binary level. Running code always tells the truth.

ZiiS•40m ago

Wouldn't an LLM just write a python script to convert an intermediate language without the triggering phrases and letter counting.

randomblock1•30m ago

Yeah this is what they end up doing. The language itself isn't that hard for them, it's separating the language from the traps in the spec. I also tried making a hostile esolang, without all the prompt injections and whatnot, and it was very easy for LLMs. (It was mainly based around tokenizer quirks, like the r in strawberry test, but becomes trivial with scripts).

mycall•13m ago

An LLM could equally decompile the binary, clean up the code, and recode it in one /goal.

dom96•4m ago

This depends on them being able to interpret the spec correctly. They could certainly write a Python script, but their implementation would need to match the spec. I haven't seen any model attempt it this way so far.

orangecat•36m ago

Interesting, reminds me of the Stroop test where people have to say what color a word is written in, except the word is "red" and the color is blue.

I wonder if a sufficiently smart LLM would recognize the difficulty and build a preprocessor that would let it mostly work in a normal syntax.

layerv-ai•32m ago

Wonder how this could be used for communication and captchas? Could be a really expandable approach to determining "probability of bot" for a lot of internal use cases in companies that might involve requesting access to specific resources only meant for humans.

SuperV1234•30m ago

Where's the human baseline for the benchmarks?

Also, you said:

> The best performing model scores a 66.8. So does this prove LLMs can “understand” the code? I would argue that the answer to this is “no”.

How did you reach that conclusion?

mycall•14m ago

Now that it has a nice webpage the describes the obfuscation technics, it is basically cracked at this point to next-gen models.

Hybrid Reasoning Agents: LLM agents that think in latent space, act through text

Someone behind the software

Free Pascal compiler now supports dynamic packages

"I'm on a war footing": Terry Tao and the crisis in math

There's never been a better time to go into math (by: 3blue1brown) [video]

RepRapMicron

Dogs seem to agree with humans that 'bouba' is round and 'kiki' is spiky

PalmOS Lemmings

I built a programming language that LLMs can't use

DeepSeek search and fetch – $0.75 per 1000 successful requests

One parser, two bugs, many routers: Exploiting Broadcom WiFi across eCos, Linux

Java sealed classes and exhaustive pattern matching

16-bit fully functional DIY magnetic core memory

The Intel 8087 chip: floating-point addition and subtraction

SIGSALY: Churchill's top secret hotline to the president

Developing for Debugging (2025)

Show HN: The Zelda timeline to catch up before the Ocarina of Time remake

Namecheap just seized and disabled all my domains

Beyond Tetris: The other games of Alexey Pajitnov [video]

I electrified my push scooter with a power drill

Open MicroManipulator – A sub-micrometer 3D motion control plattform

Florida's HOAs and condo associations are making the housing crisis worse

Types of types: Haskell's Kind system

How the Elizabeth Holmes documentary “You Can See Everything” is deceiving

Parser Combinators

Show HN: Ranet3 – High performance policy routed IPsec mesh SD-WAN

Show HN: Sphinx – A hook that quizzes you on your AI's code before shipping

Netflix vmaf: Perceptual video quality assessment based on multi-method fusion

Teal – A statically-typed dialect of Lua

Mitos editor is a fork of Helix

Hybrid Reasoning Agents: LLM agents that think in latent space, act through text

Someone behind the software

Free Pascal compiler now supports dynamic packages

"I'm on a war footing": Terry Tao and the crisis in math

There's never been a better time to go into math (by: 3blue1brown) [video]

RepRapMicron

Dogs seem to agree with humans that 'bouba' is round and 'kiki' is spiky

PalmOS Lemmings

I built a programming language that LLMs can't use

DeepSeek search and fetch – $0.75 per 1000 successful requests

One parser, two bugs, many routers: Exploiting Broadcom WiFi across eCos, Linux

Java sealed classes and exhaustive pattern matching

16-bit fully functional DIY magnetic core memory

The Intel 8087 chip: floating-point addition and subtraction

SIGSALY: Churchill's top secret hotline to the president

Developing for Debugging (2025)

Show HN: The Zelda timeline to catch up before the Ocarina of Time remake

Namecheap just seized and disabled all my domains

Beyond Tetris: The other games of Alexey Pajitnov [video]

I electrified my push scooter with a power drill

Open MicroManipulator – A sub-micrometer 3D motion control plattform

Florida's HOAs and condo associations are making the housing crisis worse

Types of types: Haskell's Kind system

How the Elizabeth Holmes documentary “You Can See Everything” is deceiving

Parser Combinators

Show HN: Ranet3 – High performance policy routed IPsec mesh SD-WAN

Show HN: Sphinx – A hook that quizzes you on your AI's code before shipping

Netflix vmaf: Perceptual video quality assessment based on multi-method fusion

Teal – A statically-typed dialect of Lua

Mitos editor is a fork of Helix

I built a programming language that LLMs can't use

Comments