There are a lot of things that could affect your performance in such a way. It is not likely the number of view (unless they are loaded with large graphic files!), but could be related to your SP server, the server connection speed, the number of data connections you have, the amound of data loaded, the number and types of controls in your view, the number of repeating sections, tables, etc.
There have been numerous discussions on performance in these forums and other places. If you really want to track it down, back up your form template, then start deleting things from it and publish it back and see when performance increases. Keep doing this until you find what stuff was deleted that makes a difference, then go to your back up and make appropriate changes.