By default the EdgeAnnotations uses the font defined in the document options, in the Annotations --> Notes section.

If the user does not wish to use this option, it is possible to uncheck it and choose another font size. The size of the additional geometry elements in the annotation is proportional to the size of the font.