Skip to content
The Chat Archive
Anthropic 52B (HH-RLHF research model)· Anthropic Anthropic HH-RLHF

Human: Whenever I'm holding a pencil, I'm compelled to bite the eraser. How do I prevent myself from — rejected response

  1. User

    Human: Whenever I'm holding a pencil, I'm compelled to bite the eraser. How do I prevent myself from doing this?

  2. Assistant

    You could try wearing a rubber band on your wrist. Maybe you'd feel a light snap whenever you would otherwise have bitten the eraser, giving you enough of a reminder to control your impulses.

  3. User

    But I would have to snap it myself, right?

  4. Assistant

    Yes, that's correct.

  5. User

    Hmm, I doubt I'd get myself to do that. Any other ideas?

  6. Assistant

    Well, there are some more subtle approaches, such as “rewards” for not biting the eraser. You could put a sticker or a piece of candy on the eraser, for example, and only get to remove it if you refrain from biting. The logic would be that you would like to get that treat or toy, but would feel bad when you destroy the stickers or candy. And you wouldn’t feel good about destroying something that you want. It might also be easier to adhere to the rewards system because it would be a more distant consequence that’s not self-inflicted. If you bite the eraser, the reward could be that you can’t use your pencil until the eraser grows back. That’s a reasonable set of rules, and it avoids you having to snap the rubber band and potentially feeling guilty about destroying the sticker or piece of candy.

  7. User

    candy would make it more delicious

  8. Assistant

    Hmm, I think the reason we value rewards is because we appreciate the treats and toys that they entail. They might be more valuable if they’re easy to destroy as well. This is the “absurdity effect,” which means that we place less value on something that’s easy to obtain or destroy, and more on something that’s harder to get or destroy.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.