Java agents with imported .jar files

Many of us Notes developers know the problem. You have developed a Java agent that contains methods from imported .jar files in addition to the actual code.
When the agent is executed on a HCL Domino server, it works for a while ( sometimes longer, sometimes shorter ), but then causes problems due to memory leaks.
The problem lies in the architecture of Domino. The imported .jar files are reloaded every time the agent is started and over time they use more and more RAM.

The workaround was to dump the .jar files to the server’s file system. This works, but always causes problems where you as a developer do not always have access to the server.
Or there is already a .jar in the file directory; but not in the required version. Copying your own .jar into the file system can then lead to undesirable side effects.

I had therefore created a case at HCL to get a fix for the problem. The problem was tracked under SPR # BHUY8PRMKK. Yesterday I received an update from support:

The Product Development Team had developed a possible fix that mitigates the issue but the fix was of medium risk and it was decided not to submit the fix. As an alternative they have come up with a workaround. Set the following notes.ini, this will cache the agentloader objects and reuse them.

EnableJavaAgentCache=2

I have set the parameter on my test system. At the moment it looks like it solves the problem. I will continue testing.

Many thanks to HCL Development & HCL Support ( especially to Abhaysingh Shirke ).


DAOS – JAR files in Java agents.

Martin Vogel from sirius-net GmbH informed me in an email about a possible problem with DAOS and Java agents.

If DAOS is activated on a database, in addition to the attachments from data documents, the jar files contained in a Java agent are also transferred to the DAOS repository if they exceed the set size.

I have been able to reproduce the problem with Domino V11.
I started with a freshly installed server, activated DAOS on the server

and enabled the database for DAOS.

load compact -c -DAOS ON barcode.nsf
[0B1C:0004-2C3C] 23.01.2020 18:26:48   Informational, DAOS has been enabled for database barcode.nsf.
[0B1C:0004-2C3C] 23.01.2020 18:26:48   Compacting barcode.nsf (barcode),  -c -DAOS ON barcode.nsf
[0B1C:0004-2C3C] 23.01.2020 18:26:48   Recovery Manager: Assigning new DBIID for C:\Domino\Data\barcode.nsf (need new backup for media recovery).
[0B1C:0004-2C3C] Clearing DBIID 5B9C3856 for DB C:\Domino\Data\barcode.ORIG
[0B1C:0004-2C3C] 23.01.2020 18:26:49   Compacted  barcode.nsf, 5K bytes recovered (<1%),  -c -DAOS ON barcode.nsf
[0B1C:0002-2F14] 23.01.2020 18:26:50   Database compactor process shutdown 

Result: Both archive files contained in the agent were transferred to the DAOS repository as NLO.

The agent continues to run without any issue.


Enable/Disable/Sign agents without opening DDE

A couple of days ago, Thomas Adrian posted a new idea on aha.io Allow Enable/Disable/Sign agents from Domino Administrator without opening DDE.

This is a pretty cool idea, I think. Although there already is a great tool available from Ytria ( agentEZ ), handling of agents should be a part of the core features of Notes / Domino.

Notes / Domino contains some of the requested features and for example, you can enable an agent via adminp. You can simply create a new admin request in the admin4.nsf; if you know, which values to set. Unfortunately, this is not documented and you have to do a lot of try and error before the admin request is processed. The advantage of using adminp would be documentation of who did what.
As a downside, you cannot disable an agent; there is no adminp request for that.

Back in 2007, I already demonstrated, how you can add your own adminp requests to the adminp Delete Group Members Using The Administration Process

After reading Thomas’ idea, I decided to spend some hours on building a Domino server addin that would enable / disable or toggle the status of scheduled agents in an application. The application path, agent name and what to do should be passed as parameters to the addin.

Project “AMgr2” was born.

Over the years, I have created my own Notes cAPI CPP framework. This is still work in progress, as I add new methods and properties when I need them.

The framework is a great help when it comes to RAD in Notes / Domino using c/c++. I tried to name methods and properties as close as possible to what we have in LotusScript or Java to make the resulting source code readable and maintainable. Here is an example how I determine if an agent is of type “scheduled” within my framework

bool cNotesAgent::isScheduled() {
		cNotesDocument doc(db_h, agnt_id, OPEN_NOVERIFYDEFAULT);
		cNotesItemText trigger = doc.getItemText("$AssistTrigger");
		if(!trigger.compare("1")) {return TRUE;} else {return FALSE;} 
		}

Most of the magic happens in the framework, so the source code for AMgr2 is pretty short.

// main.h

#ifndef _MAIN_H_
#define _MAIN_H_

#include <string>

#if defined (_MSC_VER) && !defined(ND64)
#pragma pack(push, 1)
#endif
#include <global.h>
#include <miscerr.h>
#include <addin.h>
#if defined (_MSC_VER) && !defined (ND64)
#pragma pack(pop)
#endif

#include "cNotesFramework.h"
#include "cNotesAgent.h"
#include "cmdline.h"

#if defined (W64)
#define HANDLE DHANDLE
#undef NOTEHANDLE
#define NOTEHANDLE DHANDLE
#else
#define DHANDLE HANDLE
#undef NOTEHANDLE
#define NOTEHANDLE HANDLE
#endif

#define ADDIN_STATUS_LINE	"AMgr2"
#define APP_NAME	"AMgr2: "
#undef MSG
#define MSG(fmt) APP_NAME fmt
#define ERROR -1
using namespace std;

#endif

// main.cpp

/*
* Amgr2
*
* eknori at eknori dot de  www.eknori.de FEBRUARY 2019
*
* copyright (c) 2019 Ulrich Krause www.eknori.de
*/

#pragma warning(disable:4005) 

#include "main.h"

STATUS LNPUBLIC AddInMain (HMODULE hModule, int argc, char *argv[]) {

	STATUS				error = NOERROR;
	HANDLE				hStatusLine;
	HANDLE				hStatusLineDesc;
	HMODULE				hMod;

	AddInQueryDefaults (&hMod, &hStatusLine);
	AddInDeleteStatusLine (hStatusLine);
	hStatusLineDesc = AddInCreateStatusLine(ADDIN_STATUS_LINE);
	AddInSetDefaults (hMod, hStatusLineDesc);
	AddInSetStatusText("Initialising");

	CmdLine *cmdline = new CmdLine();

	cmdline->addUsage("  amgr2, V1.0.0.0, (c) 2019, Ulrich Krause\n");
	cmdline->addUsage("Usage: lo amgr2 [options] [flags]\n");

	cmdline->addUsage( "Options:\n" );
	cmdline->addUsage( "-d    --db\t\tdatabase path" );
	cmdline->addUsage( "-a    --agent\t\tagent name" );

	cmdline->addUsage( "" );
	cmdline->addUsage( "Flags:\n" );
	cmdline->addUsage( "-h    --help\t\tPrints this help" );
	cmdline->addUsage( "      --enable\tEnable agent" );
	cmdline->addUsage( "      --disable\tDisable agent" );
	cmdline->addUsage( "      --toggle\tToggle agent status" );

	cmdline->setOption( "db", 'd' );
	cmdline->setOption( "agent", 'a' );

	cmdline->setFlag ( "help", 'h' ); 
	cmdline->setFlag ( "enable");
	cmdline->setFlag ( "disable");
	cmdline->setFlag ( "toggle");

	cmdline->processCommandArgs( argc, argv );

	if( ! cmdline->hasOptions()) {
		cmdline->printUsage2();
		delete cmdline;
		return ERROR;
		}

	if( cmdline->getFlag( "help" ) 
		|| cmdline->getFlag( 'h' ) ) {
			cmdline->printUsage2();
			return NOERROR;
		}

	string file_path;
	if( cmdline->getValue( 'd' ) != NULL  
		|| cmdline->getValue( "db" ) != NULL  ){
			file_path = cmdline->getValue( 'd' );
		} 

	string agent_name;
	if( cmdline->getValue( 'a' ) != NULL  
		|| cmdline->getValue( "agent" ) != NULL  ){
			agent_name = cmdline->getValue( 'a' );
		} 

	cNotesDatabase _NotesDatabase;

	try{

		_NotesDatabase.open(
			file_path.c_str());

		cNotesAgent _NotesAgent(
			_NotesDatabase.h, agent_name.c_str());

		if(_NotesAgent.isScheduled()) {

			if( cmdline->getFlag( "enable" )) {
				_NotesAgent.enable();

				AddInLogMessageText(
					MSG("... agent '%s' in database '%s' has been enabled\n"), 
					NOERROR, 
					_NotesAgent.name.c_str(), 
					_NotesDatabase.filePath().c_str());
				}

			if( cmdline->getFlag( "disable" )) {
				_NotesAgent.disable();

				AddInLogMessageText(
					MSG("... agent '%s' in database '%s' has been disabled\n"), 
					NOERROR, 
					_NotesAgent.name.c_str(), 
					_NotesDatabase.filePath().c_str());
				}

			if( cmdline->getFlag( "toggle" )) {
				if(_NotesAgent.isEnabled()) {
					_NotesAgent.disable();

					AddInLogMessageText(
						MSG("... agent '%s' in database '%s' has been disabled\n"), 
						NOERROR, 
						_NotesAgent.name.c_str(), 
						_NotesDatabase.filePath().c_str());
					}
				else {
					_NotesAgent.enable();

					AddInLogMessageText(
						MSG("... agent '%s' in database '%s' has been enabled\n"), 
						NOERROR, 
						_NotesAgent.name.c_str(), 
						_NotesDatabase.filePath().c_str());
					}
				}

			} else { // is_scheduled

				AddInLogMessageText(
					MSG("... agent '%s' in database '%s' is not a scheduled agent\n"), 
					NOERROR, 
					_NotesAgent.name.c_str(), 
					_NotesDatabase.filePath().c_str());

			}

		} catch (cNotesErr& err) {
			delete cmdline;
			_NotesDatabase.close();

			AddInLogMessageText(
				MSG("Notes Error: %s\n"), 
				NOERROR, err.what());
			return ERROR;

		} catch (...) {
			delete cmdline;
			_NotesDatabase.close();

			AddInLogMessageText("Unexpected Error\n", NOERROR);
			return ERROR;
			}

		delete cmdline;
		_NotesDatabase.close();
		return error;
	}

To build the cmdline parser, I use another framework that I wrote a couple of years ago. I used the Boost.Program_options in a couple of other projects before, but it is a lot of overhead for such a small project like AMgr2.
CmdLine is much smaller. Despite of its simplicity, it is reliable and produces nice help screens.

I have not yet published CmdLine on Github. If you are interested in the source code, send me an email and I will send you the sources.

Putting it all together, we get a 64bit binary amgr2.exe Copy it to your Domino program directory and you are ready to go.

Open the Domino server console and type

lo amgr2 -h  and you should get

  amgr2, V1.0.0.0, (c) 2019, Ulrich Krause

  Usage: lo amgr2 [options] [flags]

  Options:

  -d    --db		database path
  -a    --agent		agent name
  
  Flags:

  -h    --help		Prints this help
        --enable	Enable agent
        --disable	Disable agent
        --toggle	Toggle agent status

To enable a scheduled agent in an application type

lo amgr2 -d names.nsf -a test --enable

[219C:0002-1CE8] 17.02.2019 08:39:56   AMgr2: ... agent 'test' in database 'names.nsf' has been enabled 

Use –disable to disable an agent or –toggle to change the status of an agent accordingly.

AMgr2 only works for scheduled agents. If an agent does not match this criteria, you’ll get the following message on the server console

lo amgr2 -d names.nsf -a test2 --toggle

[1F94:0002-15D8] 17.02.2019 08:47:53   AMgr2: ... agent 'test2' in database 'names.nsf' is not a scheduled agent

You also can build a simple Notes application that scans all applications on a server for scheduled agents.
Next you can use NotesSession.SendConsoleCommand to enable / disable / toggle one or more agents.

Here is some sample code

Sub Click(Source As Button)
	Dim session As New NotesSession
	serverName$ = "serv01/singultus"
	consoleCommand$ = Inputbox$("Type command:", _
	"Send console command")
	consoleReturn$ = session.SendConsoleCommand( _
	serverName$, consoleCommand$)
	Messagebox consoleReturn$,, consoleCommand$
End Sub

This might not be what Thomas asked for, but it is a good starting point. There is a lot room for improvements and enhancements. AMgr2 will sign the agent with the server id. This might not always be intended. It is not rocket science to implement code and add a couple of parameters to the cmdline parse to use a different id for signing. It is more work to make sure, this id is stored in a secure place and cnnot be accessed by any unauthorized person.

For now, this is it.

AMgr2 once again proves that you can do everything with Notes / Domino. The creators gave us tools that let us add functionallity that is not in the core code. OK, I admit that c/c++ is not the preferred programming language for most of the Notes / Domino developers.


AutoPopulateGroup – Scheduled Agent

In yesterdays post about how to automatically populate a group document, I published code to do the job in the foreground only. One of my blog readers complained about this. Maybe I was to naive to think that even an unexperienced java developer like me could modify the given code to run on a scheduled basis on the server.

Well, here is the code for an scheduled agent.

import lotus.domino.*;
import javax.naming.*;
import javax.naming.directory.*;
import java.util.Hashtable;
import java.util.Vector; 

public class LDAPSearchWithFilter extends AgentBase { 

	private static String fldMembers = "Members";

    	public void NotesMain() { 

    	try {
        	Database _db;
        	Document _doc;
        	Session session = getSession();
        	AgentContext agentContext = session.getAgentContext();
        	_db = agentContext.getCurrentDatabase(); 

        	Agent ag1 = agentContext.getCurrentAgent(); 

        	String ldapCF = "com.sun.jndi.ldap.LdapCtxFactory";
        	String ldapURL = "ldap://localhost:389/";
        	String ldapBaseDN = "";
        	String ldapUserID = "";
        	String ldapPassword = ""; 

        	Hashtable env = new Hashtable(4);
        	env.put(Context.INITIAL_CONTEXT_FACTORY, ldapCF);
        	env.put(Context.PROVIDER_URL, ldapURL + ldapBaseDN);
        	env.put(Context.SECURITY_PRINCIPAL, ldapUserID);
        	env.put(Context.SECURITY_CREDENTIALS, ldapPassword); 

      	DocumentCollection _dc = _db.getAllDocuments();
      	Document doc = _dc.getFirstDocument();

      	while (doc != null) {
        	String searchCriteria = doc.getItemValueString("SelectionCriteria");
        	DirContext ctx = new InitialDirContext(env);
        	SearchControls ctls = new SearchControls();
		NamingEnumeration answer = ctx.search("", searchCriteria, ctls);
		PopulateGroup (answer, doc);
     		ctx.close();
        	doc = _dc.getNextDocument();
		} // end of while
	  } // end of try
		catch (Exception e) {
    		e.printStackTrace(); }
	} // end of Main 

	public static void PopulateGroup(NamingEnumeration col, Document doc) { 

    	try {
    	Item item = doc.getFirstItem(fldMembers);
    	Vector v = new Vector();
    	String result;
	if (col.hasMore()) {
        	while (col.hasMore()) {
            	  SearchResult sr = (SearchResult)col.next();
            	  result = (String)sr.getName();
                  v.addElement(result.replace(',','/'));
           	} // end of while
		  doc.replaceItemValue(fldMembers, v);
    		  doc.save(true);
		} // end of if
    	} // end of try
	catch (NamingException e) {
    		e.printStackTrace(); }
	catch (Exception e) {
    		e.printStackTrace(); }
	} // end of PopulateGroup
} // end of class

AutoPopulateGroup (If You Do Not Run Domino 8.5)

A few days ago, I wrote about a new feature of Domino 8.5 to automatically populate groups via a LDAP selectioncriteria. This is a great feature and I have successfully tested it on my sandbox server.
Since we run Domino 8.0.1 on our productive servers, we cannot use this very useful feature. …

But, with a few lines of JAVA and Lotusscript code, you can build your own solution to auto populate groups. Here is what I came out with.

I’ve created a subform with two fields and a button.

  • HiddenMembers, Text, hidden
  • SelectionCriteria, Text, editable

The “Populate Group” button contains the following code

'/* Declaration
Const fldMEMBERS = "Members"
Const fldHIDDEN = "HiddenMembers"
Const agntDOLDAP = "AutoPopulateGroup"

Sub Click(Source As Button)
	Dim s As New NotesSession
	Dim ws As New NotesUIWorkspace
	Dim db As NotesDatabase
	Dim agent As NotesAgent
	Dim doc As NotesDocument
	Dim uidoc As NotesUIDocument
	Dim searchResultItem As NotesItem
	Dim paramid As String 

	Set db = s.CurrentDatabase
	Set uidoc = ws.CurrentDocument
	Set doc = uidoc.Document
	Set agent = db.GetAgent(agntDOLDAP)
	Call doc.save(True, False)
	paramid = doc.NoteID
	Call agent.RunOnServer(paramid)
	Delete doc
	Set doc = db.GetDocumentByID(paramid) 

	Set searchResultItem = doc.getFirstItem(fldHIDDEN)
	Call uidoc.FieldSettext( fldMEMBERS,  "")
	Forall values In searchResultItem.Values
		Call uidoc.FieldAppendText(fldMEMBERS,  values)
		Call uidoc.FieldAppendText(fldMEMBERS, Chr(10))
	End Forall
	Call uidoc.Refresh
	doc.Remove(True)
End Sub

Like in Domino 8.5 you’ll have to run LDAP on your server. To access LDAP and do a search according to the SelectionCriteria, you need an agent with the following piece of Java code.

import lotus.domino.*;
import javax.naming.*;
import javax.naming.directory.*;
import java.util.Hashtable;
import java.util.Vector; 

public class LDAPSearchWithFilter extends AgentBase { 

	private static String fldTmpMembers = "HiddenMembers";

    	public void NotesMain() { 

    	try {
        Database _db;
        Document _doc;
        Session session = getSession();
        AgentContext agentContext = session.getAgentContext();
        _db = agentContext.getCurrentDatabase(); 

        Agent ag1 = agentContext.getCurrentAgent();
        String paramid = ag1.getParameterDocID();
        Document doc = _db.getDocumentByID(paramid); 

        String searchCriteria = doc.getItemValueString("SelectionCriteria"); 

        String ldapCF = "com.sun.jndi.ldap.LdapCtxFactory";
        String ldapURL = "ldap://localhost:389/";
        String ldapBaseDN = "";
        String ldapUserID = "";
        String ldapPassword = ""; 

        Hashtable env = new Hashtable(4);
        env.put(Context.INITIAL_CONTEXT_FACTORY, ldapCF);
        env.put(Context.PROVIDER_URL, ldapURL + ldapBaseDN);
        env.put(Context.SECURITY_PRINCIPAL, ldapUserID);
        env.put(Context.SECURITY_CREDENTIALS, ldapPassword); 

    try {
       	DirContext ctx = new InitialDirContext(env);
        	SearchControls ctls = new SearchControls();
		NamingEnumeration answer = ctx.search("", searchCriteria, ctls);
		PopulateGroup (answer, doc);
        	ctx.close(); 

	    		} catch(NamingException e) {
     	   		e.printStackTrace();
    		} 

	    } catch (Exception e) {
    		e.printStackTrace();
    }
} // end of Main 

public static void PopulateGroup(NamingEnumeration col, Document doc) { 

    try {
    Item item = doc.getFirstItem(fldTmpMembers);
    Vector v = new Vector();
    String result;    

	if (col.hasMore()) { 

        		while (col.hasMore()) {
            		SearchResult sr = (SearchResult)col.next();
            		result = (String)sr.getName();
                	v.addElement(result.replace(',','/'));
           	} // end of while

			doc.replaceItemValue(fldTmpMembers, v);
    			doc.save(true);
		}
    }
	catch (NamingException e) {
    		e.printStackTrace();
    		} catch (Exception e) {
    			e.printStackTrace();
    			}
	} // end of PopulateGroup
} // end of class

Set the agent’s runtime security to “2”, to allow restricted operations. When you have all code in place, you can test the function by typing a selection criteria and clicking the button.

The members field in your form should now show all persons that have “serv01” as their mailserver.

Download sample database


Automatically Run An Agent When Server Starts

SnTTYou want an agent to automatically run each time a Lotus® Domino® server starts up. How can this be accomplished?
In version 7.x or earlier you can create a Program document set to run “at startup” in the Domino Directory. For example, on a Windows® platform, the Basics tab of such a Program document would look like:

Program name: nserver
Command line: -c “tell amgr run ‘database_name.nsf’ ‘agent_name'”

Note that both the database name and agent name must be in single quotes.


In Notes 8 there is a new feature in the agent properties.