The standard view is that RLHF relies on explicit feedback (thumbs up/down), and polite tokens are just noise adding compute cost.

But could natural replies like "thanks!" or "no, that's wrong" be a richer, more frequent implicit feedback signal than button clicks? People likely give that sort of feedback more often (at least I do.) It also mirrors how we naturally provide feedback as humans.

Could model providers be mining these chat logs for genuine user sentiment to guide future RLHF, justifying the cost? And might this "socialization" be crucial for future agentic AI needing conversational nuance?

Questions for HN:

Do you know of anyone using this implicit sentiment as a core alignment signal?

How valuable is noisy text sentiment vs. clean button clicks for training?

Does potential training value offset the compute cost mentioned?

Are we underestimating the value of 'socializing' LLMs this way?

What do you think Altman meant by "well spent"? Is it purely about user experience, valuable training data, something else entirely?