In this analysis, seven participants (including six of the explorers identified by the primary model) were best LY2157299 ic50 fit with positive ε, and the remaining eight were fit with negative ε. Analysis of relative uncertainty in the explore subjects identified from this model yielded reliable effects in ventral RLPFC (XYZ = 30 56 −12; p < 0.05 [FWE cluster level]) and IPS (XYZ = 36 −46 56; p < 0.005 [FWE cluster level]). Participants with negative ε from this model did not yield positive or negative correlations of relative uncertainty with activation in RLPFC. Another reason ε could attain negative values is due to participants' tendencies to repeatedly

select the same option as previous trials click here (independent from their values; Lau and Glimcher, 2005 and Schönberg et al., 2007), where again this repeated option would have greater certainty. To factor out this perseveration or “sticky choice” component, we not only allowed the immediately preceding trial’s RT to influence the current trial, but also allowed multiple previous trials to

do so with exponential decay. This analysis allowed ε to be estimated as positive or negative across all trials. Here, six of the original eight explorers were best fit with positive ε, and the remaining participants had negative ε. This model with unconstrained ε and sticky choice provided a reliably better fit than the model without either sticky choice or uncertainty, even penalizing for the additional model complexity (improvement in ΔAIC = 31.0 [9.2]), or compared to a model that does include sticky choice but no uncertainty (ΔAIC = 3.3 [1.8]). Furthermore, as in the RT swing model, the fitted ε parameter value correlated with this improvement in fit (r = 0.51,

p = 0.05; and r = 0.53, p = 0.04 for the two model comparisons), suggesting that more positive uncertainty-driven exploration parameters are contributing to better fits rather than the negative ones. Analysis of the fMRI data restricted to the six subjects estimated to be explorers by this model still yielded reliable relative uncertainty effects in dorsal RLPFC Oxymatrine (XYZ = 26 52 16; p < 0.001 [FWE cluster level]) along with SPL (XYZ = −6 −60 60; p < 0.001 [FWE cluster level]; Table S2). Participants estimated to have a negative ε again did not show positive or negative correlations of relative uncertainty with activation in RLPFC. Finally, we constructed a model that fit categorical rather than continuous RT distributions. As already noted, a feature of the primary model is that it predicts continuous RT distributions consistent with the continuous nature of RT in this task. However, reward statistics are tracked based on two modes of responding, fast or slow.