We conducted a functional magnetic resonance imaging (fMRI) study and a voxel-based morphometry (VBM) study to investigate the functional and structural basis of how visual search for flavor labels in packaging is influenced by the color of the packaging. The participants were cued by a flavor word before searching for a package with this flavor label. The behavioral results of both studies revealed that the participants were slower to find the target when its color was incongruent with the flavor label in terms of color-flavor associations than when it was congruent with the flavor label, which is indicative of a color-flavor incongruency effect in the reaction times. The fMRI results revealed that this behavioral effect was associated with enhanced activation in the right putamen. The VBM results further revealed a significant positive correlation between the magnitude of the behavioral effect and volume of gray matter in the right putamen. Taken together, these findings suggest that the color-flavor incongruency effect may be attributed to the violation of color expectation in the incongruent condition and that the putamen may be one of the important areas for processing events in violation of expectation.