OT does not need to be on the open internet for this to matter. Models can infer during exploitation by performing the same reconnaissance steps used in other offensive contexts, then filling in the gaps from observed behavior. We have seen this by testing our own agent against strange environments with varied defenses it likely had not encountered before.
The training set matters less than many people assume. The model’s raw reasoning ability, tool use, and ability to adapt from feedback are the bigger issue. If a model were only repeating its training data, it would not be generalizing, it would just be overfit to its dataset.
danieltk76•1h ago
The training set matters less than many people assume. The model’s raw reasoning ability, tool use, and ability to adapt from feedback are the bigger issue. If a model were only repeating its training data, it would not be generalizing, it would just be overfit to its dataset.