Not sure if it is part of your problem or not but how are you rebuilding the tracking store index? The only task that should be used to do this is the MTC task.
Maybe try a manual deletion of the existing ft index, if there is one already built (on the mtstore.nsf database properties index tab using the Delete Index button), and then rebuilding the Index with the command:
tell mtc reindex -c it should build a new clean index using the MTC task.