Affective valence and reinforcement learning (RL) are increasingly recognized to be closely connected, yet the exact nature of their relationship remains unclear. Here, we investigated how RL-related computations contribute to affective valence, and how affective valence, in turn, contributes to RL. Applying an original computational method, we found that affective experience during RL tasks was best explained by a combination of three prominent theoretical perspectives: valence is determined by reward, prediction errors, and counterfactual comparisons. Further, we found that actions were reinforced by affective responses in addition to external rewards: participants preferred choice options that led to more positive affect, in addition to preferring options that led to greater reward. Altogether, our results illuminate both the role of RL computations in affective experience and the role of affect in RL, providing mechanistic insight into the mechanisms of affect, learning, and choice. Moreover, our studies validate a powerful new computational framework for future research on these topics.
