Access denied for all local users except root

I am encountering an Access denied issue while connecting to my 7 Mode filer through SSH and system manager. I have 7mode cluster running on 8.1.4. Users are member of local Administrator group of the filer. I can access the one node of the cluster without any issue, but when I try to access the other node, I can only access it through root. All other users get access denied error.