In my experience with a large archives colletion, it *does* make a difference.
Greyscale is typically 256 shades of grey. Basic color at 8 bits per channel is
almost 17 million shades. Where this becomes apparent is in wide areas of
gradation, like a photo with the sky showing, or an expanse of wall. With 256
shades of grey, "banding" can become visible in the sky where one shade changes
abruptly to the next shade - I'm sure you've seen this happen. Also, sometimes
the color of the paper, ink, stains and other things can become important at an
unknown future date, so maybe you want to keep that information.
Once you toss the color information and detail, you can never get it back.
When trying to save hard drive space, consider using 8 bits per channel rather
than 16 bits per channel in color images. Makes a huge difference in file size
and I personally cannot detect a visible difference, even on very high-end
reproductions.
-frank