GOT IT!!!
I had to open the source for the com.ibm.xsp.extlib.component.layout.impl.BasicApplicationConfigurationImpl.java to find the answer...
I was calling the wrong method...
<xp:this.afterPageLoad><![CDATA[#{javascript:if(compositeData.actions){
var oneui = getComponent("oneUILayout1");
var uiConfig = oneui.getConfiguration();
uiConfig.addPlaceBarAction(compositeData.actions);
}}]]></xp:this.afterPageLoad>
the compositeData.actions is of type com.ibm.xsp.extlib.tree.complex.ComplexContainerTreeNode and when it is used in an xpage you just have to addyour child nodes and set the container to transparent...
Now I just have to get the onItemClick events to bubble up from the custom control so they can be handled at the page level...