I have found that it is not so much how many custom controls so much as what each custom control is doing that negatively impacts performance.

 

In addition I have also found that browsers cannot parse/load a very large number of nodes without either being slowed or simply not working at all.

 

So...a lot of controls on a page seems to be just fine if they are only mapped to fields.  However, if you have a custom control that has ten fields on it and half of those fields only display when the other half of those fields are selected and you are using SSJS to display those fields dynamically (which means you are using the Dojo XPage extensions vs. doing all of this client-side) and you display your custom control 500 times using a repeat control...well then you may run into serious functionality/performance issues.

 

That being said I have XPages with 10-20 non repeating custom controls (maybe more) and I have never had any problem.