There are several ways to influence the performance of viewpanels but first you should find out why the view you are using takes that long to load.
How many documents does the view display?
How many columns does the view contain?
Maybe it would be an good idea to create a seperate xpage view displaying subset of documents / columns. Depending on the wishes ofcourse.
Properties you could take a look at :
All properties > Data > dataCache