Struck by how similar Goodhart’s Law - familiar to anyone who has worked in a corporate job- is to reward hacking / gaming as a problem in RL. Though one key difference perhaps is in Goodhart’s Law it is the very act of making a metric the target that makes it cease to be a good metric.