oracle password file in Oracle 11gr2 on rac cluster

Since Oracle 11gr2 there is very cool new enhancement regarding the password file while running a rac cluster. If you add a user to the password file on node 1 is also automaticaly propagated to node 2 password file.

SQL> select * from gv$pwfile_users order by inst_id;
INST_ID     USERNAME            SYSDBA          SYSOPER         SYSASM
---------- ------------------------ --------------- --------------- ---------------
         1          SYS                               TRUE            TRUE            FALSE
         2          SYS                               TRUE            TRUE            FALSE
  
There is the user sys on node 1 and 2. The password file is for both node's on a local filesystem $ORACLE_HOME/dbs

now we create the user asmsnmp.

SQL> create user asmsnmp identified by whatever;
SQL> grant sysdba to asmsnmp;
SQL> select * from gv$pwfile_users order by inst_id;
   INST_ID  USERNAME   SYSDBA          SYSOPER         SYSASM
----------   ---------- --------------- --------------- ---------------
         1         SYS                TRUE            TRUE            FALSE
         1        ASMSNMP    TRUE            FALSE           FALSE
         2        SYS                TRUE            TRUE             FALSE
         2        ASMSNMP    TRUE            FALSE           FALSE

And there you have it. The user asnmsnmp is on both node's.
Normally the user asmsnmp already exsist. Only when you removed the password file then you have to created again.

In all version before 11gr2 the user was created on the node where you created the user.

SQL> select * from v$version;
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit Production
PL/SQL Release 11.1.0.6.0 - Production
CORE    11.1.0.6.0      Production
TNS for Linux: Version 11.1.0.6.0 - Production
NLSRTL Version 11.1.0.6.0 - Production
SQL> select * from gv$pwfile_users order by inst_id;
INST_ID     USERNAME            SYSDBA          SYSOPER         SYSASM
---------- ------------------------ --------------- --------------- ---------------
         1          SYS                               TRUE            TRUE            FALSE
         2          SYS                               TRUE            TRUE            FALSE

Now create the user asmsnmp.

SQL> create user asmsnmp identified by whatever;
SQL> grant sysdba to asmsnmp;
SQL> select * from gv$pwfile_users order by inst_id;
   INST_ID  USERNAME   SYSDBA          SYSOPER         SYSASM
----------   ---------- --------------- --------------- ---------------
         1         SYS                TRUE            TRUE            FALSE
         1        ASMSNMP    TRUE            FALSE           FALSE
         2        SYS                TRUE            TRUE             FALSE
       
As you see the user asmsnmp is only created on the first node. You have now an inconsistentie.
So before 11gr2 you never should forget to run the create command or grant command also on the second node.