In the old 7-mode days, you could create a list of IP addresses on a filer that would restrict administration to someone coming from one of those IP's.
In cluster mode, you can do the same with the SP:
However, I've not found anything to indicate this can be done for regular SSH access or System Manager access. Does anyone know if this type of restriction exists?