OpenAI regularly will publish the results of the internal AI model safety evaluations in what the outfit pitching as an attempt to increase transparency.
On Wednesday, OpenAi launched the Safety evaluations HubA webpage that shows how the models of the company score on different tests for generating harmful content, jailbreaks and hallucinations. OpenAI says it will use the hub to share statistics on a “constant basis” and that it is planning to update the hub with “large model updates” in the future.
“As the science of AI evaluation evolves, we want to share our progress on the development of more scalable ways to measure model capacity and safety,” Openai wrote in a Blog post. “By sharing a subset of our safety evaluation results here, we hope that this will not only be easier to understand the safety performance of OpenAI systems over time, but also support community efforts to increase transparency in the field.”
OpenAI says it can add additional evaluations to the hub over time.
In recent months, OpenAi has raised the anger of some ethicians Reportedly The safety tests of certain flagship models and not releasing technical reports for others. The CEO of the company, Sam Altman, is also accused of misleading OpenAi leaders about model safety assessments prior to his short in November 2023.
At the end of last month, OpenAi was forced to return an update to the standard model that Chatgpt, GPT-4O Power, after users started reporting that it responded in a validating and pleasant way. X was flooded with screenshots from chatgpt that applaud all kinds of problematically, dangerous decisions And idea.
OpenAI said it would implement different fixes and changes to prevent future such incidents, including the introduction of an opt-in “alpha phase” for some models with which certain chatgpt users can test the models and give feedback before the launch.
