Product count per category

On the category page we display the subcategories (one level down in the hierarchy) of the current category. But I also need to show the product count (of visible products) of these subcategories. What would be a good way to find the number of visible products but not always calculate in real time?