Being very familiar with Bedrock and its related APIs, in particular:

- running accuracy and related metrics

- building RAG solutions using Knowledge Bases, separating in the response and in the flows what is and what isn't AI-generated

- adding Bedrock Guardrails to catch the "bad stuff" and using Bedrock Reranker to improve accuracy

A giant portion of the job is just asking "What do you want?" over and over again. And also just asking "What do you want to happen when it cannot answer?" Gathering (or building) ground truth data to be able to do Accuracy evaluations.

Also doing a lot of mitigation around the limitations of the tools: adding rate-limiters, resilience4j kind of stuff.

What they build ranges from product features to internal tools. They make heavy use of LLM vendor inference APIs, vector databases, etc. They end up writing a lot of glue code and software to manage the context of the LLM, query for data, integrate with other systems and so on. They also develop front-end interfaces for their applications.

Only recently have they started to, lightly, explore the idea of training LLMs with managed services like AWS SageMaker.

All this is to say that, the “AI Engineer/SWE” title will probably represent vastly different things depending on the technical sophistication of the organization.

If someone told me they were an AI Engineer at OpenAI I’d be more inclined to expect their role to be more fundamental, elsewhere, not so much.

My day generally looks like this:

8am read email

9am standup

Usually some kind of meeting (planning, 1-on-1s, retro, something else)

Heads down time

Lunch, read at the park

Heads down time until 5pm

Fridays we alternate having a team symposium or a book club that I lead. Right now we happen to be reading the book referenced in [1].

Tasks are usually code based. Fixing/extending the agent code, tool writing/bug fixing, writing pipelines for data ingestion, etc.

Part of my job is technology recommendations, so staying on top of the fast moving field and being able to match problems to best-in-class/stable tech choices is a must. I have a long software engineering background and am an excellent debugger - I rarely get stuck on a bug, only slowed down. I can rapidly prototype an idea, and then take it all the way to development, qa, and deployment, given the right resources.

1. In line with Chip Huyen's AI Engineering book. ISBN 978-1098166304

For the last 7 years, I’ve specialized in “cloud native development” and the last five working more in strategy and leading implementations working for consulting companies full time.

Gen AI is just another tool in my tool belt. One of my specialties before Gen AI was working with Amazon Connect - a cloud hosted call center + Amazon Lex (the AWS version of Alexa) for voice, text and web and chatbots. I use Gen AI to understand customer’s utterances better with Bedrock. But at the end of the day, it’s really just another AWS SDK (Boto3) call.

But my day to day is more about working with clients gathering requirements, integrating call centers with their backends, and everything that goes into a modern implementation - regular development, infrastructure as code, “DevOps” [sic], training, solving XYProblems, etc.

I spend a lot of time telling clients “Gen AI isn’t what you want”. You don’t want Gen AI being responsible for output to customers, maybe a standard ML implementation might be better, etc.

Vendor LLM APIs + Software engineer = AI Engineer