Removing bar numbers from empty, hiden staves

[Jogchum Reitsma via LilyPond user discussion]

Hi,

I use to print bar numbers for every bar. But if I remove empty staves 
with "context { \Score   \RemoveAllEmptyStaves }" bar numbers are still 
being printed, hovering in the empty space.

Is there a way to remove (only) those bar numbers?

Thanks

cheers, Jogchum Reitsma