this is a major risk factor that the LLM providers aren't contending with adequately. from chatbot addiction to the risk of manipulation. there are a lot of "AI Safety" issues that hardly have to do with potential superintelligence.

This is more or less the same issue we have faced and facing with social media. Engagement, stickiness, monetization - whatever it is these companies are looking for, they are not looking for is best for all of us.

Same thing with chatbots, LLMs, genai, xyzai etc. They even call a setting ‘syncophancy’ - https://openai.com/index/sycophancy-in-gpt-4o/ - that should tell us where these companies priorities lie.