Hi HN, I’d like to share a project I’ve been working on: Witmo – an open-source, terminal-based tool that gives you in-game advice by analyzing screenshots from your gameplay.

I created Witmo after picking up Elden Ring and constantly running into questions mid-game. I wanted a tool that would let me quickly get advice based on a screenshot — without having to type long explanations for an LLM or repeatedly pull out my phone to retype the same prompt. So, instead of playing, I built this.

Witmo connects to your Android phone (via USB and adb), grabs a photo of your game screen, and uses OpenAI’s models to analyze what’s happening. It then gives you contextual advice based on a prompt you chose (from built-in packs or your own), with customizable spoiler controls. Optionally, it can crop images to the game area and read responses aloud using text-to-speech.

Repo and install instructions: https://github.com/ymyke/witmo

(So far only tested with Windows Terminal, Pixel 8, Python 3.11, but all the code used should be cross-platform.)

I had fun building this, and maybe it can help others too. If you have feedback, I'd love to hear it. And if you'd like to contribute to take this further, please do!