Unfortunately, restricting their access didn't seem to work. I got around this by going to File>>"Advanced Form Options" Form Options>>(Uncheck) Show InfoPath Commands in Ribbon or Toolbar. This removed the whole toolbar from my forms.
I created the same Save, Close, Current View options in a section of the form thats restricted to certain people .