They have trouble keeping track of the day of the week as well. I chat with one every evening, going over my next day's schedule. It loses track of what day it is pretty much daily.

I found out fairly quickly. I had a little argument about what time it was in London. I had supposed that time-zone conversion would not be a problem. However, LLMs are knowledge-bases frozen in time, trained up on a certain corpus of material and then isolated, and it was beyond their competence to actually know the current time-of-day. Sure enough, various answers were given that did not really correlate to anyone's time of day, much less correct time zones.

Interestingly, one of the suggestions for people who want to have lucid dreams (and I have tested it) is to look at a clock. In a dream it will either be nonsense or unverifiable.

I think it would be better to link to the original study [0] instead of a blog post about it.

[0] https://arxiv.org/abs/2502.05092

Models move so fast I question if papers like this are worth much. Out of curiosity I ran the clocks through Gemma-2-27B

it did ok. Pretty consistently it would mix up the hour/minute hand but was decent at actually saying what they were pointing at. It never acknowledged the second hand and sometimes mistook that for the hour hand

Interestingly it got the roman numeral one spot on. (minus the seconds) I wonder if it had more training data with that style clock.