Is this in the browser or in filler? In filler, in the control properties, you can create a restriction. In the browser, you'd need to use code or
qRules to get the file extension of the attachment and then you could remove the file if it is in the wrong format, or display an error, or whatever you'd like for your UI.