I’m aware that all of these measures have limitations and that many are controversial or imperfect. My goal is discovery and understanding, not to defend or attack any particular framework.
I’d love to hear:
- What measures, benchmarks, or methodologies you think belong on this list
- What you see as their key strengths and failure modes
- How (or whether) you personally use them to interpret AI progress
johnnyfived•1mo ago
I think this would take the form of something more abstract instead of concrete with raw numbers, like a revised Turing Test.
kayo_20211030•1mo ago
johnnyfived•1mo ago
This might sound basic but I keep coming back to this idea again and again. Alex Garland really did have the right idea with Ex-Machina, where in the film Caleb claims that he purposefully designed Ava (the AI robot) to have all the internal mechanisms shown, so people would understand always they are interacting with a machine. The point of his Turing test was to show whether they could see past the machine and still empathize with it as a human.