I work at IBM Research in the Responsible Tech team.
We are working on an open-source project called the Responsible Prompting API. This is the Github - https://github.com/IBM/responsible-prompting-api
It is a lightweight system that provides recommendations to tweak the prompt to an LLM so that the output is more responsible (less harmful, more productive, more accurate, etc...) and all of this is done pre-inference. This separates the system from the existing techniques like alignment fine-tuning (training time) and guardrails (post-inference).
The team's vision is that it will be helpful for domain experts with little to no prompting knowledge. They know what they want to ask but maybe not how best to convey it to the LLM. So, this system can help them be more precise, include socially good values, remove any potential harms. Again, this is only a recommender system...so, the user can choose to use or ignore the recommendations.
This system will also help the user be more precise in their prompting. This will potentially reduce the number of iterations in tweaking the prompt to reach the desired outputs saving the time and effort.
On the safety side, it won't be a replacement for guardrails. But it definitely would reduce the amount of harmful outputs, potentially saving up on the inference costs/time on outputs that would end up being rejected by the guardrails.
This paper (https://arxiv.org/abs/2504.08757) talks about the technical details of this system if anyone's interested. And more importantly, this paper (https://dl.acm.org/doi/10.1145/3706598.3713365), presented at CHI'25, contains the results of a user study in a pool of users who use LLMs in the daily life for different types of workflows (technical, business consulting, etc...). We are working on improving the system further based on the feedback received.
At the core of this system is a values database, which we believe would benefit greatly from contributions from different parts of the world with different perspectives and values. We are working on growing a community around it!
So, I wanted to put this project out here to ask the community for feedback and support. Feel free to let us know what you all think about this system / project as a whole (be as critical as you want to be), suggest features you would like to see, point out things that are frustrating, identify other potential use-cases that we might have missed, etc...
Here is a demo (https://huggingface.co/spaces/responsible-prompting/responsible-prompting-demo) hosted on HuggingFace that you can try out this project in. Edit the prompt to start seeing recommendations. Click on the values recommended to accept/remove the suggestion in your prompt. (In case the inference limit is reached on this space because of multiple users, you can duplicate the space and add your HF_TOKEN to try this out.)
Feel free to comment regarding any questions, feedback or comment about this project. Hope you all find it valuable!