Thanks for the suggestion. This behaviour is present on a number of systems. I have followed the microsoft support documents
How to configure Internet Explorer to open Office documents in the appropriate Office program instead of in Internet Explorer
A new window opens when you try to view a 2007 Microsoft Office program document in Windows Internet Explorer 7 or Internet Explorer 8
Modifying the file associations to not allow opening in same window has no effect.
We also use liferay but this always prompts to either open or download.
I notice the end of the document url (shown below) has additional arguments which I presume is what is allowing direct access (as opposed to open/download dialog box.
I will investigate other documents and in the meantime suggest users use the download link.