Discussion about this post

User's avatar
Ted Sanders's avatar

Nice work! I work at OpenAI. The banal reason we stopped publishing them is that it was a bit of an operational burden to write up the results. We still measure progress internally, and improve our measurement quality over time. If we're not confident we want to publish them in every system card going forward, then we conservatively refrain from publishing them in any system card, because if some system cards have results while others don't, it invites reasonable questions about cherrypicking or bias.

3 more comments...

Ready for more?