Our decisions are biased by previous experience. One well-known bias is repulsive bias. Suppose that you should classify whether the size of a medium-sized dog falls into the small or large category. If you saw a large dog previously, you may classify the medium dog as small, and vice versa. Therefore, your choice is biased to be repelled from the previous stimulus size, which is called the repulsive bias. We investigated the source of the repulsive bias. There would be two potential sources. One is the bias of the classification boundary, and the other is the bias of stimulus perception. To address this question, we recorded the whole brain signal of human subjects by using functional magnetic resonance imaging (fMRI) while they classified a ring-shaped stimulus. Then, we investigated whether the brain signal of boundary or that of stimulus perception is biased to induce the repulsive bias. We found that it was not the perception signal but the boundary signal that was associated with the trial-to-trial choice repulsion from the previous stimulus. This boundary signal was observed in the superior temporal cortex and inferior parietal lobe. Our study indicates that the origin of the repulsive bias is not a low-level sensory signal but a high-level memory signal.