The text used in a chart like the chart title, axis titles, and legend, all have their own default font size. If you change the font size of a chart, the chart title is resized to that size. After refreshing the chart, the rest of the text is also resized, maintaining the same ratio.
Select the chart, and in the Home ribbon tab, in Font Size, select a new font size. The chart title resizes.
It is not possible to resize the numbers in the bullets in front of legend items.