HCL Domino 12 – Active Directory Password Sync (part 2)

During my tests with HCL Domino 12 Beta2 I ran into a couple of issues where the Active Directory Password Sync did not work as expected. The problems are not caused by the beta code. However, I am writing down my observations here, since the errors may also appear later in a productive environment. Maybe my lines will save one or the other a long search.

Request Creator replicates with multiple servers

In the console log of a request creator you see replcation events with multiple servers.

There are 2 databases that are replicated on the request creator, Domino Directory (names.nsf) and Directory Assistance (da.nsf). Both databases are replicated with their admin servers to start every config update cycle. Important to know is that the admin server for each database is determined seperately separately. In my test environment, the admin server for da.nsf was different than the one for names.nsf resulting in multiple replication events.

Wrong configuration document

If you have more than 1 Domain Controller that syncs users passwords with your Domino environment, you must install and configure a request creator on each Domain Controller. In addition, you need a configuration document for each of your request creators. Each request creator should have the same configuration settings. Best practice would therefore be a single configuration document that applies to a group of servers.

During my tests I could observe that instead of the assigned configuration document the default configuration document was used.

There was no Statistics output in the console log and I could also not see any DEBUG log output. Also the configuration was not refreshed in the configured interval. I discussed this with HCL Development and it turnd out that the configuration document was not used due to a problem with the view index in names.nsf on the request creator.

I needed to stop the sync process on the request creator, open names.nsf in the Notes Client. After restarting the AD Domain Controller machine, the correct configuration document was used.

Most likely, this issue occurs when you use a configuration document for a single request creator and later switch to a group based configuration document. I would suggest to use a group based configuration document from the start on.

Is Active Directory Password Sync cluster ready?

You can designate as many request processors as you want, and it’s a good way to provide failover.

The decision as to where the AD Domain Controller sends a request is admittedly not overly sophisticated. It works sequentially through an in-memory list it holds of all servers which are designated as request processors, until it finds an available one. Have a look at the ($PWSProcessors) view in the directory. (Mark De Lellis, HCL Development)

This is where the list is built from, using all servers with Role “2” (Request Processor), in the order they appear in the view.


HCL Domino 12 – Active Directory Password Sync

Active Directory Password Sync applies the Windows passwords of users registered in an Active Directory domain to their Domino HTTP and/or Notes ID passwords.

Password synchronization is supported for

  • Registered HCL Notes, HCL Nomad, HCL Verse, and HCL iNotes users accessing Domino servers with HTTP passwords or Notes IDs.
  • HCL Traveler users accessing their mail through the web browser on their mobile devices.
  • Web users who are not registered in Domino but who have Person documents in the Domino directory accessing Domino web applications with HTTP passwords.

To setup and configure Active Directory Password Sync, you need at least 2 Domino v12 servers. One server acts as Request Creator and the other one has the role Request Processor.

On the “Active Directory Password Sync” tab of the configuration document for each of the servers you can configure which password changes should be processed. The refresh interval specifes the amount of times in minutes after a request is created to allow the request to be processed.

To sync passwords to Notes IDs, the Request Processor servers require password reset authority to the ID vault. I you miss this configuration step, you’ll see an error message on the console of the Request Processor:

[1DFC:0059-1694] 07.04.2021 10:13:23,00 AD Password Sync> PWSyncProcessStoredRequests: Failed to update password in Notes ID for CN=Herbert Feuerstein/O=singultus: Agent containing ResetUserPassword method must be signed by a designated Password Resetter.
[1DFC:0059-1694] 07.04.2021 10:13:23   Password Sync: Active Directory Password Sync failed for objectGUID 6630b191119c8b45b78b77865a37cc70: Agent containing ResetUserPassword method must be signed by a designated Password Resetter.

Request Creator must be installed on the AD Domain Controller. The install type is “Utility Server”. The server must be configured but will never run as a service.

After you have successfully configured the Request Creator, The installer will remove some of the Domino server executables

[0C14:0002-1284] Deleted Executable File 'C:\Domino\nserver.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nhttp.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nldap.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nsmtp.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\npop3.exe' for security reasons.
[0C14:0002-1284] Deleted Executable File 'C:\Domino\nimap.exe' for security reasons.

The installer finally adds the Domino password library “npwsync.dll” to the Windows registry. The entry can be found here

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Notification Packages

For security reason, the server id that is used for the Request Creator is encrypted during the installation and configuration process. Make sure that the id does not have a password on it. The secured id cannot be used on any other machine than the Request Creator.

A new application is created on the Request Processor. The application stores the requests that are created by the Request Creator. After a request has been successfully processed, the document is removed from the database.

Most of the items in the document are encrypted.

After you have setup and configured Request Creator and Request Processor, you should restart both machines (OS level).

When a user now updates his / her password, LSA processed the change thru the Domino password library, and a new document is created in the Active Directory Password Sync application. The Request Processor reads new documents from the application and processes them according to its configuration.

Here is an example of what you see on the Request Processor console

[0BD4:0002-1C70] Authenticate {19EC003F}: CN=serv01/O=singultus
[0BD4:0002-1C70]             T:RC2:128 E:0:  P:c:  S:RC2:0 A:2:0 L:N:N:N FS: 
[0BD4:0002-1C70] 07.04.2021 10:22:28   Starting replication with server serv01/singultus
[0BD4:0002-1C70] Authenticate {19EC0040}: CN=serv01/O=singultus
[0BD4:0002-1C70]             T:RC2:128 E:0:  P:c:  S:RC2:0 A:2:0 L:N:N:N FS: 
[0BD4:0002-1C70] 07.04.2021 10:22:28   Pulling names.nsf from serv01/singultus names.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Replicator added 1 document(s) to names.nsf from serv01/singultus names.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Replicator updated 2 document(s) in names.nsf from serv01/singultus names.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Pulling ddm.nsf from serv01/singultus ddm.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Pushing ddm.nsf to serv01/singultus ddm.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Pulling products\traveler\traveler-rules.nsf from serv01/singultus products\traveler\traveler-rules.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Replicator updated 10 document(s) in products\traveler\traveler-rules.nsf from serv01/singultus products\traveler\traveler-rules.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Pushing products\traveler\traveler-rules.nsf to serv01/singultus products\traveler\traveler-rules.nsf
[0BD4:0002-1C70] 07.04.2021 10:22:28   Finished replication with server serv01/singultus
[1DFC:0059-1694] Authenticate {19EC0041}: CN=serv04/O=singultus
[1DFC:0059-1694]             T:AES:128 E:1:  P:c:e S:AES-GCM:128 A:2:1 L:N:N:N FS:DHE-2048+X25519
[1DFC:0061-1234] Authenticate {19EC003E}: CN=serv04/O=singultus
[1DFC:0061-1234]             T:AES:128 E:1:  P:t:e S:AES-GCM:128 A:2:1 L:N:N:N FS:DHE-2048+X25519
[1DFC:0061-1234] 07.04.2021 10:23:23 IDVAULT: idvdb.c IDVFindIDInVault: IDVOpenVault: No error
[1DFC:0061-1234] 07.04.2021 10:23:23 IDVAULT: idvdb.c IDVFindIDInVault: IDVOpenNoteFromView: No error
[1DFC:0061-1234] 07.04.2021 10:23:23 IDVAULT: idvdb.c IDVFindIDInVault: Exiting: No error
[1DFC:0059-1694] ----- Trust cert lookup Subject: CN=serv03/O=singultus; Issuer: CN=Herbert Feuerstein/O=singultus, OrgCombo: O=singultus:PR:O=singultus
[1DFC:0059-1694] >> Matched Trust Certificate IssuedBy CN=Ulrich Krause/O=singultus, IssuedTo O=singultus, NoteId 8010
[1DFC:0059-1694] >> Matched Trust Certificate IssuedBy CN=serv04/O=singultus, IssuedTo O=singultus, NoteId 17150
[1DFC:0059-1694] >> Matched Trust Certificate IssuedBy CN=serv03/O=singultus, IssuedTo O=singultus, NoteId 19490
[1DFC:0059-1694] >> Matched Trust Certificate IssuedBy CN=serv02/O=singultus, IssuedTo O=singultus, NoteId 17146
[1DFC:0059-1694] >> Matched Trust Certificate IssuedBy CN=serv01/O=singultus, IssuedTo O=singultus, NoteId 3742
[1DFC:0061-1BE8] Authenticate {19EC0042}: CN=serv04/O=singultus
[1DFC:0061-1BE8]             T:AES:128 E:1:  P:t:e S:AES-GCM:128 A:2:1 L:N:N:N FS:DHE-2048+X25519
[1DFC:0059-1694] Authenticate {19EC0043}: CN=serv04/O=singultus
[1DFC:0059-1694]             T:AES:128 E:1:  P:c:e S:AES-GCM:128 A:2:1 L:N:N:N FS:DHE-2048+X25519
[1DFC:0061-11A0] 07.04.2021 10:23:23 IDVAULT: idvdb.c IDVFindIDInVault: IDVOpenVault: No error
[1DFC:0061-11A0] 07.04.2021 10:23:23 IDVAULT: idvdb.c IDVFindIDInVault: IDVOpenNoteFromView: No error
[1DFC:0061-11A0] 07.04.2021 10:23:23 IDVAULT: idvdb.c IDVFindIDInVault: Exiting: No error
[1DFC:0061-11A0] ----- Trust cert lookup Subject: CN=serv04/O=singultus; Issuer: CN=Herbert Feuerstein/O=singultus, OrgCombo: O=singultus:PR:O=singultus
[1DFC:0061-11A0] >> Matched Trust Certificate IssuedBy CN=Ulrich Krause/O=singultus, IssuedTo O=singultus, NoteId 8010
[1DFC:0061-11A0] >> Matched Trust Certificate IssuedBy CN=serv04/O=singultus, IssuedTo O=singultus, NoteId 17150
[1DFC:0061-11A0] >> Matched Trust Certificate IssuedBy CN=serv03/O=singultus, IssuedTo O=singultus, NoteId 19490
[1DFC:0061-11A0] >> Matched Trust Certificate IssuedBy CN=serv02/O=singultus, IssuedTo O=singultus, NoteId 17146
[1DFC:0061-11A0] >> Matched Trust Certificate IssuedBy CN=serv01/O=singultus, IssuedTo O=singultus, NoteId 3742
[1DFC:0061-11A0] ----- Trust cert lookup Subject: O=IDVAULT; Issuer: CN=Herbert Feuerstein/O=singultus, OrgCombo: O=singultus:VT:O=IDVAULT
[1DFC:0061-11A0] >> Matched Trust Certificate IssuedBy O=IDVAULT, IssuedTo O=singultus, NoteId 17142
[1DFC:0061-11A0] -------- IDVValidateVTIssuer: PWD Resetter CN=serv04/O=singultus
[1DFC:0061-11A0] -------- PWD Resetter CN=serv04/O=singultus, Vault trust issuer O=singultus, need cross cert? 0
[1DFC:0061-11A0] ------- vault trust xcert PubKeySize	294
[1DFC:0061-11A0] ------- vault trust xcert SubjectNameDesc O=IDVAULT
[1DFC:0061-11A0] ------- vault trust xcert IssuerNameDesc	O=singultus
[1DFC:0061-11A0] ------- vault trust xcert ExpTimeDesc 13.03.2046 06:54:57
[1DFC:0061-11A0] ------- Vault VO cert PubKeySize	294
[1DFC:0061-11A0] ------- Vault VO cert SubjectNameDesc CN=VO-lrxr-oidv/OU=serv01/O=IDVAULT
[1DFC:0061-11A0] ------- Vault VO cert IssuerNameDesc	CN=serv01/O=singultus
[1DFC:0061-11A0] ------- Vault VO cert ExpTimeDesc 
[1DFC:0061-11A0] ------- Vault chain cert PubKeySize	294
[1DFC:0061-11A0] ------- Vault chain cert SubjectNameDesc O=IDVAULT
[1DFC:0061-11A0] ------- Vault chain cert IssuerNameDesc	O=IDVAULT
[1DFC:0061-11A0] ------- Vault chain cert ExpTimeDesc 13.03.2101 06:54:29
[1DFC:0061-11A0] ------- Vault chain cert PubKeySize	162
[1DFC:0061-11A0] ------- Vault chain cert SubjectNameDesc CN=serv01/O=singultus
[1DFC:0061-11A0] ------- Vault chain cert IssuerNameDesc	O=IDVAULT
[1DFC:0061-11A0] ------- Vault chain cert ExpTimeDesc 13.03.2121 06:54:29
[1DFC:0061-11A0] ------ Cert subject CN=serv01/O=singultus (=CN=serv01/O=singultus),	VO Cert	issuer CN=serv01/O=singultus
[1DFC:0059-1694] ------- vault trust xcert PubKeySize	294
[1DFC:0059-1694] ------- vault trust xcert SubjectNameDesc O=IDVAULT
[1DFC:0059-1694] ------- vault trust xcert IssuerNameDesc	O=singultus
[1DFC:0059-1694] ------- vault trust xcert ExpTimeDesc 13.03.2046 06:54:57
[1DFC:0059-1694] ------- Vault VO cert PubKeySize	294
[1DFC:0059-1694] ------- Vault VO cert SubjectNameDesc CN=VO-lrxr-oidv/OU=serv01/O=IDVAULT
[1DFC:0059-1694] ------- Vault VO cert IssuerNameDesc	CN=serv01/O=singultus
[1DFC:0059-1694] ------- Vault VO cert ExpTimeDesc 
[1DFC:0059-1694] ------- Vault chain cert PubKeySize	294
[1DFC:0059-1694] ------- Vault chain cert SubjectNameDesc O=IDVAULT
[1DFC:0059-1694] ------- Vault chain cert IssuerNameDesc	O=IDVAULT
[1DFC:0059-1694] ------- Vault chain cert ExpTimeDesc 13.03.2101 06:54:29
[1DFC:0059-1694] ------- Vault chain cert PubKeySize	162
[1DFC:0059-1694] ------- Vault chain cert SubjectNameDesc CN=serv01/O=singultus
[1DFC:0059-1694] ------- Vault chain cert IssuerNameDesc	O=IDVAULT
[1DFC:0059-1694] ------- Vault chain cert ExpTimeDesc 13.03.2121 06:54:29
[1DFC:0059-1694] ------ Cert subject CN=serv01/O=singultus (=CN=serv01/O=singultus),	VO Cert	issuer CN=serv01/O=singultus
[1DFC:0059-1694] -------- IDVValidateVTIssuer: PWD Resetter CN=serv04/O=singultus
[1DFC:0059-1694] -------- PWD Resetter CN=serv04/O=singultus, Vault trust issuer O=singultus, need cross cert? 0
[1DFC:0062-0BCC] Authenticate {19EC0045}: CN=serv01/O=singultus
[1DFC:0062-0BCC]             T:RC2:128 E:0:  P:c:  S:RC2:0 A:2:0 L:N:N:N FS: 
[1DFC:0062-0BCC] 07.04.2021 10:23:26   Starting replication with server serv01/singultus
[1DFC:0062-0BCC] 07.04.2021 10:23:26   Pushing C:\Domino\data\IBM_ID_VAULT\IDVAULT.nsf to serv01/singultus IBM_ID_VAULT\IDVAULT.nsf
[1DFC:0062-0BCC] Authenticate {19EC0046}: CN=serv02/O=singultus
[1DFC:0062-0BCC]             T:RC2:128 E:0:  P:c:  S:RC2:0 A:2:0 L:N:N:N FS: 
[1DFC:0062-0BCC] 07.04.2021 10:23:26   Replicator updated 1 document(s) in serv01/singultus IBM_ID_VAULT\IDVAULT.nsf from C:\Domino\data\IBM_ID_VAULT\IDVAULT.nsf
[1DFC:0062-0BCC] 07.04.2021 10:23:26   Finished replication with server serv01/singultus
[1DFC:0061-11A0] Authenticate {19EC0044}: CN=serv04/O=singultus
[1DFC:0061-11A0]             T:AES:128 E:0:  P:t:  S:RC2:0 A:2:0 L:N:N:N FS: 
[1DFC:0062-0BCC] Authenticate {19EC0047}: CN=serv04/O=singultus
[1DFC:0062-0BCC]             T:AES:128 E:0:  P:c:  S:RC2:0 A:2:0 L:N:N:N FS: 

Detailed information about installation and configuration can be found here https://help.hcltechsw.com/domino/12.0.0/admin/conf_adsync_password_sync.html