>Less perf on what ? Runtime performance. -Os disables code and data alignment (-falign-*), that saves space, but decreases speed on CPUs without efficient unaligned access. According to google and some testing not long ago on Phoronix and linux.org.ru, it is a compromise between ~10% saved space or 10% of performance.
A small suggestion - try playing with LTO and l1/l2 cache size parameteres.
