HCL Domino V11 – Directory Synchronzation – Part 5

Debugging & Monitoring

To monitor DirSync output, you can use the server console or the server log.nsf.
This in general lets you identify possible errors during DirSync processing.

If you need a more verbose output, or you want to dig deeper into DirSync functionallity, use the following notes.ini variable to create some kind of trace mode.

set con DIRSYNC_DEFAULT_ARGS=-v

You do not need to restart the DirSync task to switch verbose logging on / off.

Instead of just the summary line

[1730:0004-0BE0] 20.01.2020 12:13:56   DIRSYNC From Active Directory (AD) - Summary (0.003 sec, Start=234796, Adds=0, Modifies=0, Deletes=0, Skips=0, Errors=0, End=234795)

by setting the parameter, you will get a more verbose output.

[1730:0004-0BE0] DirSync  ResyncAll by CheckBox: 0
[1730:0004-0BE0] DirSync  Preview:        0
[1730:0004-0BE0] DirSync  Level:          16
[1730:0004-0BE0] DirSync  SyncFlows:      2
[1730:0004-0BE0] DirSync  OnPremCookie:   
[1730:0004-0BE0] DirSync  UserDirCookie:  234795
[1730:0004-0BE0] DirSync  CSyncFromAD::SyncSpan (NAMEldap_search_ext_s call) : (&(&(|(objectClass=Group)(objectClass=Person))(|(mail=@brightside.)(mail=@darkside.)(mail=@msdn.)))(uSNChanged>=234796)) took 1 msec
[1730:0004-0BE0] 20.01.2020 12:14:47   DIRSYNC From Active Directory (AD) - Summary (0.001 sec, Start=234796, Adds=0, Modifies=0, Deletes=0, Skips=0, Errors=0, End=234795)

With this parameter in place, you will also be able to monitor which attributes have been changed in Active Directory.

[1428:0005-17F8] DirSync  Processing ldap entry (SyncSpan) #1 from page #1, total entries #1: 'CN=James Kirk,CN=Sync,DC=ad,DC=fritz,DC=box'
[1428:0005-17F8] DirSync  Entry with mail address 'j.kirk@brightside.org' - NoteID 10082 was found in the target directory.
[1428:0005-17F8] DirSync  
 DirSync  CSyncFromAD::DoModify(dn = 'CN=James Kirk,CN=Sync,DC=ad,DC=fritz,DC=box', newentry=0)
[1428:0005-17F8] DirSync  Modified MiddleInitial from '' to 'T'
[1428:0005-17F8] DirSync  Modified uSNChanged from '234775' to '234796'
[1428:0005-17F8] DirSync  'person' Document updated, UTF8 Name = 'CN=James Kirk,CN=Sync,DC=ad,DC=fritz,DC=box' 
[1428:0005-17F8] DirSync  CSyncFromAD::DoModify - Modified existing Note for 'CN=James Kirk,CN=Sync,DC=ad,DC=fritz,DC=box'
[1428:0005-17F8] DirSync  
[1428:0005-17F8] 20.01.2020 12:19:37   DIRSYNC From Active Directory (AD) - Summary (0.037 sec, Start=234796, Adds=0, Modifies=1, Deletes=0, Skips=0, Errors=0, End=234796)

The above output also indicates a possible bug in the DirSync task setting values on the Status tab of the configuration document.

While the LDAP change number corresponds to the End=234796 value in the summary record, the Last sync time (20.01.2020 12:19:37) entry is not updated.

Not sure, at what time this value is being set. I will keep an eye on it.

DirSync also collects several stats. You can access them either from the Admin Client.

or issuing a show stat DirSync command on the server console

show stat dirsync
[1CDC:0009-1EC8]   DirSync.AD.FromAD.Adds = 6
[1CDC:0009-1EC8]   DirSync.AD.FromAD.Deletes = 6
[1CDC:0009-1EC8]   DirSync.AD.FromAD.Millis = 18086
[1CDC:0009-1EC8]   DirSync.AD.FromAD.Modifies = 2
[1CDC:0009-1EC8]   DirSync.AD.FromAD.Skips = 9
[1CDC:0009-1EC8]   DirSync.MIDPOINTS.FromAD.Millis = 108
[1CDC:0009-1EC8]   DirSync.Totals.LatestAllDirSyncDocsMillis = 3
[1CDC:0009-1EC8]   DirSync.Totals.LongestAllDirSyncDocsMillis = 191
[1CDC:0009-1EC8]   DirSync.Totals.LongestSyncTimeMillis = 190
[1CDC:0009-1EC8]   DirSync.Totals.LongestSyncTimeNABName = names.nsf
[1CDC:0009-1EC8]   DirSync.Totals.NumADChangeQueriesSyncSpan = 1221
[1CDC:0009-1EC8]   DirSync.Totals.TimeADChangeQueriesSyncSpan = 15769
[1CDC:0009-1EC8]   DirSync.Totals.TotalDirSyncDocs = 2
[1CDC:0009-1EC8]   DirSync.Totals.TotalRequestDocs = 0
[1CDC:0009-1EC8]   DirSync.Totals.TotalSuccessfulNABSyncs = 30
[1CDC:0009-1EC8]   15 statistics found

In addition to the notes.ini parameter mentioned above, DirSync comes with a lot more notes.ini parameters. They are not documented and should only be used when support advices you to do so.

You can find a list of those parameters using the following command on the server console.

te dirsync show
[1730:0004-0BE0] DirSync> DIRSYNC_DEFAULT_ARGS=-v
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_STRICT_ASSERT=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_THREAD_TRACE_LEVEL=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_DISABLE_BG_THREADS=0
[1730:0004-0BE0] DirSync> DIRSYNC_THREADS=10
[1730:0004-0BE0] DirSync> DIRSYNC_BG_THREADS=10
[1730:0004-0BE0] DirSync> DIRSYNC_BG_THREADS_CUSTOMER_ALLOWED=3
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_STARTUP_DELAY=60000
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_STARTUP_THREAD_ABORT=10
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_DELAY=60000
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_FORCE_RUNMODE=1
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_REPLICATE=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_YELLOWZONE=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_MARK_INVALID_INTERVAL_MINUTES=60
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_CHANGELOG_TIMELIMIT=300
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_CHANGELOG_SIZELIMIT=10000
[1730:0004-0BE0] DirSync> DEBUG_IDRSYNC_CHANGELOG_CHANGETIME_LIMIT=10000
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_SKIPLOG_LIMIT=50
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_SERVERROLE_CHECK=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_REPLICATION_SERVER_CHECK=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_CONFLICT_CLEANUP=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_FIRSTSYNC_OPTIMIZATION=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_ACL_CHECK=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_ECL_CHECK=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_MAILHUBS_CHECK=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_NO_DOMAINNAME_CHECK=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_ELSEWHERE_CHECK_INTERVAL_MINS=60
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_TRIANGULATION_INTERVAL_MINS=10
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_TRACE_NOTESDN=
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_RESYNC_ONCE_NOTESDN=
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_FLUSH_CONFIG_FREQUENTLY=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_CERTIFIER_BLACKLIST=
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_CDIR_UPDATE_VIEWNAMES=
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_HYGIENICRESTART_THRESHOLD=100000
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_EXCLUDE_CUSTOMERID=
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_EXCLUDE_USERID=
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_INCLUDE_ONLY_CUSTOMERID=
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_DISABLE_AUTO_SYNC_ALL=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_LEAKS=0
[1730:0004-0BE0] DirSync> DEBUG_MAX_CHANGELOG_RENAME_SEARCH_ITERATIONS=20
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_DONT_PANIC_ON_MAX_MEM_SEGMENTS=0
[1730:0004-0BE0] DirSync> DEBUG_INVALID_BBLOCK_IN_SYNCTHREAD=0
[1730:0004-0BE0] DirSync> DEBUG_SKIP_INVALID_BBLOCK_IN_SYNCTHREAD=0
[1730:0004-0BE0] DirSync> DEBUG_USE_OLD_RESYNCONCE_PROCESSING=0
[1730:0004-0BE0] DirSync> DEBUG_USE_DIRSYNC_RESPONSE_DOCS=0
[1730:0004-0BE0] DirSync> DEBUG_MULTI_BG_THREADS_TO_A_CUSTOMER=0
[1730:0004-0BE0] DirSync> DEBUG_DIRSYNC_MAX_REQUEST_ERRORS=0

Use / change this parameters only when told by HCL support.


Notes 11 – Splash Screen Redesign

With version 11 of HCL Notes & Domino we also got a new splash screen. To be honest, I think the new screen is butt ugly. The gradient isn’t a proper gradient, and the logos are low quality.

Here is a redesigned splash screen. I have used Krita do create the gradient. In fact there are two gradient on 2 layers. The rest is plain text and some transparent images.

To activate the screen, download the image from here and unzip the archive to a destination of your choice.

Next add the following lines to your client notes.ini file.

HasNotesOverlay=1 
SESPlashPath=<path-to-bmp>

Now restart your client. You should now see the new design.


Debug AutoPop process

The auto-populated groups feature in Domino 8.5.x is great, but when you run into problems an error message like:

“Search for auto-populated group members failed for group TEST.: Error looking up name on LDAP Server; See server log for further details.”

is not very helpful, because there is no more info in the server log.

Add debug_autopop=2 to the server’s notes.ini and the next time, the autopop process will be more verbose.

restart task update
01/26/2010 08:32:54.59 [0AA8:0005-0A04] AutoPop>  …Shutting down
01/26/2010 08:32:54   Index update process shutdown
01/26/2010 08:32:55.73 [0E40:0002-06F0] AutoPop>  Admin server is CN=MyServer/O=foo/C=de
01/26/2010 08:32:55.73 [0E40:0002-06F0] AutoPop>  Current server is CN=MyServer/O=foo/C=de
01/26/2010 08:32:55.73 [0E40:0002-06F0] AutoPop>  Starting the Auto Populated Groups thread…
01/26/2010 08:32:55.73 [0E40:0002-06F0] AutoPop>  Started the Auto Populated Groups thread.
01/26/2010 08:32:55.74 [0E40:0005-0218] AutoPop>  Starting up…
01/26/2010 08:32:55.82 [0E40:0005-0218] AutoPop>  Found [1] Name and Address Book(s), opening first [names.nsf].
01/26/2010 08:32:55.82 [0E40:0005-0218] AutoPop>  Member update interval 30
01/26/2010 08:32:55.82 [0E40:0005-0218] AutoPop>  Domain = foo
01/26/2010 08:32:55   Index update process started:
01/26/2010 08:32:56.82 [0E40:0005-0218] AutoPop>  Time to perform Member updates for all groups: 01/26/2010 08:32:55
01/26/2010 08:32:56.82 [0E40:0005-0218] AutoPop>  Next Member update time: 01/26/2010 09:02:56
01/26/2010 08:32:56.82 [0E40:0005-0218] AutoPop>  Looking for autopopulated groups
01/26/2010 08:32:56.96 [0E40:0005-0218] AutoPop>  Found 1 auto-populated groups to process.
01/26/2010 08:32:57.96 [0E40:0005-0218] AutoPop>  Starting to work on group = TEST
01/26/2010 08:32:57.96 [0E40:0005-0218] AutoPop>  Selection Criteria is= [(&(ObjectClass=person)(mailserver=CN=MyServer/O=foo/C=de))], size = 56
01/26/2010 08:32:58.10 [0E40:0005-0218] AutoPop>  Failed applying selection criteria = Error looking up name on LDAP Server; See server log for further details., (&(ObjectClass=person)(mailserver=CN=MyServer/O=foo/C=de))
01/26/2010 08:32:58.10 [0E40:0005-0218] AutoPop>  Finished work on group = TEST
01/26/2010 08:32:58.10 [0E40:0005-0218] AutoPop>  Finished Member updates for all auto-populated groups
01/26/2010 08:32:58   Search for auto-populated group members failed for group TEST.: Error looking up name on LDAP Server; See server log for further details.


DISABLE_LOG_DBGROUP

Found this in my inbox today:

“Thanks to help from the IBM development team and support the issue some customers have been experiencing when running custom server tasks and Extension Manager on Domino 8.5.1. has been resolved.

The root cause of this issue was that IBM has introduced some I/O performance improvements for logging with 8.5.1, which unfortunately had the sideeffect that servers were haulting when running custom server tasks … .

But the good news is that IBM introduced a way of turning this new logging off and reverting to the way logging was done in 8.5.0.

So all you have to do is put in an extra line in the notes.ini of the server.

DISABLE_LOG_DBGROUP=1

After that everything works again.This has already been validated. The only adverse effect of this change is that logging will not be faster than it was in 8.5.0. We expect the issue to be fully resolved in the next service release of Domino.”


Notes.ini WiKi

A few days ago Vladislavs Tatarincevs sent me this email

Hi, Ulrich,
in your blog, you have posted some posts that relates to notes.ini,
Maybe we could join forces to create a set of Notes.ini for server and client, and post it,
I think many young administrators will benefit from this.  😉
I have opened discussion on my blog, but only few people responded and shared their notes.ini settings.

What do you think?

vlad

I think, this is a good idea. There are already some sources, where you can find notes.ini related information.

Notes.ini Entries” and even IBM’s “Lotus Notes/Domino Notes.ini settings” database are only two of these sources. All have one thing in common. You have to know the parameter or a part of it to find information about this parameter. None of the sources are able to answer questions like: “How can I force my client to create ODS48 databases?”. This is only a simple sample and can be easily answered with “Add CREATE_R8_DATABASES=1 to your notes.ini and run compact -c”. But there might be also more difficult questions to be answered and sometimes adding a certain parameter can have side effects.

So we thougt that having a WiKi for this would be great and we would like to ask our Notes Community for some feedback. So please feel free to post your thoughs as a comment here or if you do not want to give your feedback in public, write me an email.

Looking forward to reading your feedback.


Look Into Your Notes.ini, Please

If you recently have installed a new Domino server and enabled the compression of the TCPIP Port during install, the server notes.ini parameter has a wrong value set.

The line added to the notes.ini file is as follows: TCPIP=TCP,0,15,0,,32
The correct value is TCPIP=TCP,0,15,0,,12320

To correct the problem, take the following steps to set the correct value:

  • Launch the Domino server.
  • Disable the TCPIP port compression.  To do so in the Domino Administrator client, go to Configurations -> Tools -> Server -> Ports Setup.
  • Save the document.
  • Enable the compression again, and Save.

EVENT_CORRELATION_POOL_SIZE

Have you ever seen this error message on the Domino server console?

Error: “11/24/2006 10:51:05   Event: Error updating a Domino Domain Monitoring event document (NoteID 0xF2AA): Event correlation cache is full. You  can increase its size via the NOTES.INI setting  EVENT_CORRELATION_POOL_SIZE”

If so, then I assume you know that you can increase the cache size by adding the parameter

EVENT_CORRELATION_POOL_SIZE=104800000

to the server’s notes.ini for a maximum cache size of 100MB.

But, surprise, surprise, the error will occur again. I opened a PMR today because I could not find any useful information on the web. Mateja Kozamernik from IBM support Slovenia pushed me into the right direction and sent me the information I needed.

“Even with the maximum configured pool size, you may still see the above error. This occurs because Domino creates cached documents that are 2MB in size and the server allows a maximum of 200 cached documents by default.

Since 2MB x 200 is greater than the maximum allowable pool size, the error can occur unless additional parameters are enacted to control the document size and/or the maximum number of cached documents. The following notes.ini parameters are available for these purposes.

DDM_MAX_NOTE_CACHE (number of documents in the cache)
DDM_MAX_DOCSIZE_MB (size in megabytes of documents in the cache)

To ensure that the Event Correlation cache will not be exhausted when configured to the maximum 100MB size, use these parameters so that the following formula is true:

DDM_MAX_NOTE_CACHE x DDM_MAX_DOCSIZE_MB <= 100″


Why Is the Last Line in the NOTES.INI Not Taking Effect?

Although all experienced Notes administrators and developers (might) know why, I would like to commemorate the anwer to the question.

Any parameter that is added as the last line in the NOTES.INI file requires that there be a hard carriage return (the ENTER key) after the last line to ensure that it will take effect.  Without a hard carriage return, the last line will not be recognized.

May this help a Notes newbee to save a few minutes time …


Suppressing The Prompts On Failover

If a failover condition occurs , Notes will close the current tab and prompt you to switch to a different replica of the database, allowing you to attempt the action again.

A notes.ini variable has been introduced in Lotus Notes® 8.0.1 to prevent the failover prompt when the server holding the current application is no longer responding, To suppress these prompts, use the client notes.ini variable:

HidePromptFailoverInc=1

Beginning with Lotus Notes® 8.0.1 the following scenarios have added failover functionality

  • Opening a view
  • Scrolling a view
  • Pressing F9 to refresh a view

Use mail.box When Multiple mail.boxes Are Enabled

When multiple mail boxes are enabled, they are titled mail1.box, mail2.box, and so on and the router uses these mail boxes for routing messages.
To force the router to use “mail.box”, add the parameter to the notes.ini file:

Mail_Enable_Mailbox_Compatibility=1

The parameter keeps the “mail.box” file and still will create mail2.box, mail3.box, and so on.
This is useful if you have a third party applications that requires the existence of a “mail.box”.


Notes.ini Preferences Plug-In For Lotus Notes 8

Notes.ini preferences plug-in (sreenshot) is the Eclipse 3.2 compatible plug-in integrated into Lotus Notes 8 client’s File/Preferences area, which allows you perform such tasks as:

– view all settings of your local Lotus Notes 8 client’s notes.ini file in sorted way
– quick lookup of each setting via the type ahead control
– add/edit/remove any setting
– request each setting description from configured description repositories via WebServices (one of such WebServices enabled repositories is provided by company called Kalechi Design, USA, which hosts Notes.ini database prepared by Rudi Knegt from RKJ-Soft)

[via Infosana]