There is much debate as to the evolutionary origins of color perception in humans. A commonly held theory is that color vision allowed humans to find fruit, and improving nutrition always has a strong evolutionary pressure. (Most think the subtle reliance on judging skins tones for signs of sexual health/availability as a later development, unlike some birds that rely on colored plumage) Why this matters in photography is that it gives some idea what part of the brain lights up when we perceive a bright color. If this perception does trigger an instinctive pleasure response, then it would distract from the more cerebral mental response to a B&W.
Steve (thoroughly enjoying complicating the issue.)