Is it possible to restrict user for a particular serial port?

We have created a user in digi and we have configured one serial port to access only by that user. However the user can access the other serial ports too. how can we restrict that?