Lately we've been noticing a non-trivial amount of traffic in our logs that doesn't look like typical bots.

Not the usual noisy crawlers or obvious scrapers. The behavior is different with fewer hits, more selective page access.

Some of the user agents suggest AI crawlers, but some do not. How can we track these visitors?