Why are you worried GPT knows your location? If you use any Apple stuff, it knows your exact location. Android is the same. Windows knows the exact location also. So why would not GPT know that? Most webpages know your location, but since they also know your IP, they know in where you are, the city name at least, within a few KM. You are talking about the location. You should be worried about using Apple, Android or Windows because they know a lot more than that.

update: forgot to mention that if you any of antisocial networks like Facebook or Twitter, they know where you live. And if you are on your room, kitchen or living room. Or outside. Or anywhere you go.