Building a commandline interface with boost::program_options

July 26, 2015 – 8:08 am

My ONTF project “DomBackUp” can be configured and run via the command-line without having the configuration database installed on the Domino server.
DomBackUp is a fork of nKBackup by Tom Lyne et al. I found the command-line interface code a bit hard to maintain when it comes to enhancements.
So I was looking for an alternative way to create the command-line interface in C++ platform-independent.

boost::program_options is a library that makes it easy to parse command-line options, for example, for console applications.

Here is the code snippet from the DomBackUp project

try {
#include<boost/program_options/options_description.hpp>
#include<boost/program_options/parsers.hpp>
#include<boost/program_options/variables_map.hpp>
 
...
 
namespace po = boost::program_options;
 
...
 
 
po::options_description description("DomBackUp usage",PO_LINE_LENGTH);
description.add_options()
	("help,h", "Display this help message")
 
	("source,s", po::value( &szSource ), "source file/folder")
 
	("dest,d", po::value( &szDestination ), "destination file/folder")
 
	("include-sub-dirs,inc", po::bool_switch( &bIncludeSubDirs )->default_value(false), 
	"Include subdirectories, applies to folder backup only (optional, default = false)")
 
	("throttle,t", po::bool_switch( &bThrottle )->default_value(false), 
	"Wait short time between file writes (optional, default = false)")
 
	("zip,z", po::bool_switch( &bZipAfterBackup )->default_value(false),
	"Move file to .zip archive after backup (optional, default = false)")
 
	("unique-filename,u", po::bool_switch( &bUniqueFileName )->default_value(false),
	"appends a unix timestamp to the archive filename (optional, default = false)")
 
	("application-type,a", po::value( &szAppType )->default_value("nsf"),
	"nsf,ntf,both = application type (optional, default = nsf only)")
 
	("input-file,f", po::value( &szInputFile ), 
	"Backup all the files specified in an .ind file created in the Domino data folder to <dest>")
 
	("version,v", "Display the version number");
 
po::variables_map vm;
po::store(po::command_line_parser(argc, argv).options(description).run(), vm);
po::notify(vm);
 
if(vm.count("help")) {
	std::cout << description << "\n";
	return FALSE;
}
 
if(vm.count("version")) {
	AddInLogMessageText("V %s on %s", NOERROR, szVersionNumber.c_str(), szServer);
	AddInLogMessageText("Termination complete.\n", NOERROR);
	return FALSE;
}
}
 
catch(po::error& e) { 
AddInLogMessageText((char*)e.what(), NOERROR);
return (ERR(error));
}

First of all, you have to include some header files and (optional) set the namespace to be used. The only downside of using boost::program_options is the fact that you have to build the static .lib. While most parts of the boost libraries are header only, boost::program_options is not. But once you have the library at hand, it is easy to write the code.

The first line of code creates your interface and sets the Title as well as the line-length. The line-length is 80 by default.

po::options_description description("DomBackUp usage",PO_LINE_LENGTH);

The next thing to do is to add all the options you like to control via the command-line

description.add_options()
	("help,h", "Display this help message")
 
	("source,s", po::value( &szSource ), "source file/folder")

Each option can have a description and the command, both verbose (–help) and abbreviated (-h).
In addition, you can bind the command to a variable. The variable can be of any type, string, int or boolean.

For boolean variables, you can implement some kind of toggle mechanism.

description.add_options()
	("zip,z", po::bool_switch( &bZipAfterBackup )->default_value(false),
	"Move file to .zip archive after backup (optional, default = false)")

In this example, &bZipAfterBackup by default is set to false. To enable the feature, simply add –zip or -z to the command-line. This will toggle from false tu true.

The –help (-h) command does not have any value binding. So we have to add a parser and some ther lines of code that are processed, when the –help (-h) command is being used.

        po::variables_map vm;
	po::store(po::command_line_parser(argc, argv).options(description).run(), vm);
	po::notify(vm);
 
	if(vm.count("help")) {
		std::cout << "\n" << description << "\n";
		return FALSE;
	}

This is what you get when you type “load nbackup -h”

commandlineCool, isn’t it?. Just a couple of lines of code and you get a professional command-line interface.

As said before, the downside is the overhead of having the boost libraries included in your project.

But the footprint is minimal, although the boost libraries are many GB of code.

Building ONTF DomBackUp with TeamCity

July 26, 2015 – 7:00 am

Building binaries for multiple platforms from source is time consuming. For my ONTF project “DomBackUp” I have to build binaries for AIX (not sure, if I can support AIX in future builds ) , LINUX and Windows, both on 32Bit and 64Bit.

Aside from Atlassian JIRA and STASH, a couple of virtual machines are involved. I also started using TEAMCITY to automatically build the binaries without going to each of the build systems and invoke the build process manually.

TeamCity is a Java-based CI server package. The TeamCity installation and configuration is quick and easy. The fact that it is Java-based should not be an impediment to Windows development shops.
The TeamCity server is a main component, but the browser-hosted interface serves as the primary way to administer TeamCity users, agents, projects, and build configurations.

The main advantages are

  • Easy to setup, use, and configure
  • Widely-used and well documented
  • Integration with a wide variety of tools and technologies
  • Professional Server is free for up to 3 agents and 20 build configurations.

teamcitybuild

You can create the build steps manually or let TeamCity search your GIT repository for existing .sln files and propose build steps automatically. All you have to do is to review the steps and select the correct build configuration ( x64 or Win32 ). For the Linux builds, I use boost build scripts. So you only have to tell TeamCity to invoke the script when the build agent runs.

TeamCity will automatically grab the output from the build script. This makes it very easy to identify errors in the script itself or even compile errors.

teamcitybuild1

teamcitybuild2

Using a CI solution makes it very easy to create nightly builds. Due to the flexibility in configuration, you are able to create the binaries aside from any needed template, have CI update and harmonize the version and build numbers across all parts of your project, create release notes from JIRA and put all parts together to build a shippable package.

The process can be triggered manually or scheduled to create nightly builds.

You are also able to rebuild any previous version by simply checkout the correct commit from your repository.

[CMake] How to turn off incremental linking

July 17, 2015 – 9:47 am

I recently had to build a static library using CMake. In the current CMake version, apparently something has changed in the linker settings.

The build always ended with an error:

LINK : fatal error LNK1123: failure during conversion to COFF: file invalid
or corrupt [D:\0.GIT\libarchive-3.1.2\vc10.32\CMakeFiles\CMakeTmp\cmTC_bbd0c.vcxproj]

Looking at the linker settings, I found that CMake seems to enable incremental linking by default and this is the root cause for the trouble; at least with Visual Studio 2010

Link:

C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\link.exe /ERRORREPORT:QUEUE
/OUT:”D:\0.GIT\libarchive-3.1.2\vc10.32\CMakeFiles\CMakeTmp\Debug\cmTC_bbd0c.exe”
/INCREMENTAL
/NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib
/MANIFEST /ManifestFile:”cmTC_bbd0c.dir\Debug\cmTC_bbd0c.exe.intermediate.manifest”
/MANIFESTUAC:”level=’asInvoker’ uiAccess=’false'”
/DEBUG /PDB:”D:/0.GIT/libarchive-3.1.2/vc10.32/CMakeFiles/CMakeTmp/Debug/cmTC_bbd0c.pdb”
/SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT
/IMPLIB:”D:/0.GIT/libarchive-3.1.2/vc10.32/CMakeFiles/CMakeTmp/Debug/cmTC_bbd0c.lib”
/MACHINE:X86 cmTC_bbd0c.dir\Debug\cmTC_bbd0c.exe.embed.manifest.res
cmTC_bbd0c.dir\Debug\testCCompiler.obj /machine:X86 /debug

I searched the web for a solution, but could not find anything that really works. So here is what I did to fix the issue.

In in [CMakePrgmDir]\share\cmake-3.3\Modules\Platform\windows.cmake

modify the line 242 ( might be different in other versions )


# add /debug and /INCREMENTAL:YES to DEBUG and RELWITHDEBINFO also add pdbtype
# on versions that support it
set( MSVC_INCREMENTAL_YES_FLAG "")
if(NOT WINDOWS_PHONE AND NOT WINDOWS_STORE)
if(NOT MSVC_INCREMENTAL_DEFAULT)
set( MSVC_INCREMENTAL_YES_FLAG "/INCREMENTAL:YES")
else()
#set( MSVC_INCREMENTAL_YES_FLAG "/INCREMENTAL" ) ' comment out here
set( MSVC_INCREMENTAL_YES_FLAG "/INCREMENTAL:NO" )
endif()
endif()

With this modification in place, I was now able to let CMake create the project configuration files.

Code Quest

July 16, 2015 – 7:37 am

A couple of days ago, I had to investigate on an issue in a rather old template. The template had been delivered to a customer ages ago. After downloading the template from the repository ( no, not from a GIT or SVN repository ) and after Domino Designer had opened the template, I saw a lot of errors in the Script Libraries. No worries at this time, because all of those issues can be solved.

ErrorInLS

I then I opened one of the libs that contains the specific function. The contents of that lib looked different to the curent version; no surprise is that.
Then I searched the template for the function. Nothing. The function was not there, althoug I could see from the code, that it IS in the template.

Calls to that function did not show an error, so it had to be somewhere inside the code. But I could not find it.

I then opened the template in YTRIA ScanEz and navigated to the lib that was supposed to contain the function. Aside from the usual $ScriptLib items, the design also had $ScriptLib_error items.

ErrorInLS1In the end, I found my function in one of the $ScriptLib items. This template had been build in Designer 7 and the current Domino Designer 9.0.1 seems to have a problem with this mix of $ScriptLib and $ScriptLib_error items when opening such a design element in the editor.

As an aside, I found that the “Recompile All LotusScript” apparently does not have any problems finding the correct code; all code compiled without any complaint.

Next, I deleted the $ScriptLib_error items from the design

ErrorInLS2

I omitted the warning ( don’t do that, if you do not know, what you’re doing !!)

ErrorInLS3

and after another recompile and reopen of the template in Domino Designer , the template was fully functional again.

TeamCity Start/Stop Script for Linux

June 11, 2015 – 6:13 pm

I am using Teamcity on Ubuntu 14.04. Here is a simple script to start / stop TeamCity server.

#!/bin/sh
# /etc/init.d/teamcity – startup script for teamcity
export TEAMCITY_DATA_PATH=”/home/teamcity/.BuildServer”
export TEAMCITY_SERVER_OPTS=-Djava.awt.headless=true # Configure TeamCity for use on a headless OS.

case $1 in
start)
start-stop-daemon –start -c teamcity –exec /home/teamcity/TeamCity/bin/runAll.sh start
;;

stop)
start-stop-daemon –start -c teamcity –exec /home/teamcity/TeamCity/bin/runAll.sh stop
;;

esac

exit 0

Copy the script (teamcity) to etc/init.d.

Register the file as a startup script

sudo update-rc.d teamcity defaults

Make the file executable

sudo chmod +x /etc/init.d/teamcity

Now you can start teamcity with sudo service teamcity start or stop ist with sudo service teamcity stop

install git on SLES 11 SP 3

June 6, 2015 – 1:16 pm
sudo zypper addrepo http://download.opensuse.org/repositories/devel:/tools:/scm/SLE_11_SP3/devel:tools:scm.repo  
sudo zypper addrepo http://download.opensuse.org/repositories/devel:/languages:/perl/SLE_11_SP3/devel:languages:perl.repo  
 
sudo zypper install git-core

Execute A Program From A C / C++ Program

May 22, 2015 – 5:21 pm

To start an external program from a C / C++ program, I used the following code.

GetStartupInfo( &start_info ); // You must fill in this structure
ret = CreateProcess( NULL,  // specify the executable program
(LPSTR)buffer,   // the command line arguments
NULL,
NULL,
FALSE,
NORMAL_PRIORITY_CLASS | CREATE_NO_WINDOW,
NULL,
NULL,
&start_info,
&process_info);
 
// Successfully created the process.  Wait for it to finish.
WaitForSingleObject( process_info.hProcess, INFINITE );
 
// Get the exit code.
result = GetExitCodeProcess(process_info.hProcess, &exitCode);
 
// Close the handles.
CloseHandle( process_info.hProcess );
CloseHandle( process_info.hThread );

The code starts an external program and waits, until this program has finished before processing the next lines of code.

After using this code for a couple of years now, I found a shorter version today …

ret = system((char *)buffer);

I found only one limitation. It will print any of your fprint statements to the Domino console; using CREATE_NO_WINDOW with CreateProcess, you can suppress the output.

Slides And Sample Application from My Presentation ( ENGAGE & ICS.UG)

April 3, 2015 – 1:24 pm

 

The presentation and sample can be downloaded here

A smallish problem installing JVMPatch_SR16FP3_W64_901.3_Server

March 24, 2015 – 10:17 am

I recently ran into an issue, where my Java SQL and LDAP connectors stopped working after applying FP3 to Domino 9.0.1 / 64Bit.

Thanks to the help of the community, I was pointed to an already existing fix for this issue. IBM had released an JVM patch . I downloaded the patch and tried to install it by simply double-clicking the downloaded file.

On the machine ( a testing environment ), where I tried to install the patch, also the Notes Client is installed.

This obviously confuses the installer and so it tried to apply the patch to the Notes client instead of the Domino server.

jvmpatch

There is no way to change the install path in the GUI. I looked into the install log and found an interesting entry

3 Command Line Args:
0:  LAX_VM
1:  D:\Notes\jvm\bin\java.exe
2:  -D$USER_INSTALL_DIR$=D:\Notes

So, I gave it a try and added the path to my Domino installation as parameters to the .exe file

D:\temp\jvmpatch>9.0.1.3_Server_w64_JVM_Patch_20150303_142312.exe LAX_VM d:\Domino\jvm\bin\java.exe -D$USER_INSTALL_DIR$=D:\Domino

Now the patch was successfully applied to the Domino server JVM …

I have not tried the 32Bit installation. Here the file states to be “ClientServer”. So, maybe, you can make a selection in the 32Bit version and the 64Bit version is simply missing this option.

If the command line is documented somewhere, blame me. If not, maybe this post can help saving some time.

A quick tipp for using the ONTF Dependency Documenter

February 12, 2015 – 7:20 pm

A quick tipp for using the ONTF Dependency Documenter from Andrew Magerman.
If you have a lot of libraries in your database, the resulting image might be too large to fit into the page and the result seems to be truncated.

I have talked too Andrew already, and it seems that there is some kind of limitation on the image size for a page design element.

You can work around this issue by resizing the image in the image properties.

We also discussed some options and I am sure that Andrew will provide an updated version of the plugin soon.

 

 

Serious issue when signing a database in the background on the server

January 31, 2015 – 2:01 pm

While working with the cAPI, I ran into a nasty problem with signing design elemnts in an application on the server with a different ID than the server ID.
This seems only to be a problem with XPages related design elements.
When you try to open an XPage from that database in the browser, you will get a 403 Error message.
When I sign the same database from the client with the same ID using Ytria SignEZ, everything works as expected.

Other elements are not affected. So forms, views, agents and script libraries are signed correctly. Well, don’t get me wrong, even XPages are signed correctly, but they just do not work any longer after signing.

So, how do I sign a database in the background on the server with a different ID?

Here is, what I do. The ID is stored as an attachment in a configuration document. The file is detached at runtime to a temporary directory outside the Domino installation path. The code opens the file and returns a handle to the ID

if ( NOERROR != SECKFMOpen(&hKFC, 
                (char*)detachedFile.c_str(), 
                (char*)if_file_pwd.c_str(), 
                SECKFM_open_All, 0, NULL))
{
...
}

An XPage is defined by NOTE_CLASS_FORM and DESIGN_FLAG_XSPPAGE, so later in the code, we have to open the note EXPANDED

ret = tnote.open1(tdb.hDb, nid, OPEN_EXPAND);

Finally, the design note is signed using NSFNoteSignExt3

ret = NSFNoteSignExt3(
      tnote.h, hKFC, NULL, MAXWORD, NULLHANDLE, 
      SIGN_NOTES_IF_MIME_PRESENT, 0, NULL);

and contracted afterwards

ret = NSFNoteContract( tnote.h );

To verify, if anything is wrong with the signature, I use

ret = NSFNoteVerifySignature (
      tnote.h, 0, 0, retSigner, retCertifier);

So far, so good. As I said before, the code runs without any issues and the verification also does not report any errors.

At ConnectED, I went to the “Meet the Developers Lab” and asked for advice. The problem seems not to be in my code, but the Devs had a vague idea, what might be the problem.
I did some tests today and here is what I found out.

When you sign a design element using the client, The last entry in the $UpdatedBy field is the name of the signer.
When you do the same on the server, then the server’s name is the last name in the list.
Hence it is different from the name that is stored in the $Signature field, the error occurs.
This also explains, why I can open XPages that have been signed using my code and the server ID instead of the signer ID. In this case, names in $UpdatedBy and $Signature are identical.

As far as I can see, there is no workaround for that. So the only way to get this solved is to open a PMR with IBM …

How to determine, if a shared library is loaded on AIX

January 7, 2015 – 2:21 pm

Recently, I had to figure out, if a shared library on an AIX system was loaded at Domino server startup. I am not an expert on UNIX, so I asked GOOGLE for some help.

I came across the UNIX command genkld, whic did exactly what I wanted to do

genkld will create a list that is printed to the console that shows all loaded shared libraries

genkld

You can find more useful information about “Shared library memory footprints on AIXhere

 

Remove Windows SP1 Backups To Free Up Disk Space

April 23, 2014 – 4:59 pm

I recently installed SP1 on a Windows 2008 R2 Server. The partition for the OS only had 10GB space available. After SP1 had been successfully installed, only 7GB of free space was left.

You can delete the backup files if SP1 has been installed without any issues.

Use the following command to free up disk space after the service pack installation:

dism /online /cleanup-image /spsuperseded

freeSpace

The process takes a few minutes to complete, it ends with the sentences “Service Pack Cleanup operation completed. The operation completed successfully”.

Please remember that you cannot uninstall the service pack after you have cleaned up the disk space.

Add ProFTPD to ESXi 5.1

April 21, 2014 – 2:41 pm

Uploading files to an ESXi datastore ist terrible slow using the build-in upload mechanism. However if you add FTP to your ESXi server, and use an ftp client to upload/download files, the upload/download will be much faster.

ProFTPD runs good in ESXi 5.x and offers high transfer rate over normal transfers.

To add ProFTPD Service, do the following.

Enable SSH in ESXi.

  • Connect to your ESXi host
  • Goto Configuration
  • Goto Security Profiles
  • Click on Properties
  • Select SSH from the list of services
  • Click Options end start the service

Connect to your ESXi host using any ssh client like putty.
Navigate to your datastore folder  cd /vmfs/volumes/datastore/
[Replace “datastore” to match your datastore name]

Create any folder where your FTP Service files can be downloaded from the internet or any other source.

mkdir install
cd install

Now download ProFTPD using wget

wget http://esxi-customizer.googlecode.com/files/ProFTPD-1.3.3-8-offline_bundle.zip

install ProFTPD using esxcli

esxcli software vib install --no-sig-check 
-d /vmfs/volumes/datastore/install/ProFTPD-1.3.3-8-offline_bundle.zip

I needed to restart the ESXi system to activate ProFTPD in the Security Profiles.

Forcing a localized vSphere Client or vSphere Web Client installation to launch in English

April 21, 2014 – 10:15 am

My vSphere Client starts with a german User Interface. Obviously, the language depends on the OS language. In my cas, it’s German …

To be able to provide screenshots with english labels and error messages in a way, that I can find results on Google more easily. I was looking for a way to switch the language of my vSphere Client to English.

 

vSphereClientLang

There is no setting in the UI itself, but here is a little trick, how it can be done without pushing language files around like in the pre WW1 days.

Simply add one of the following parameters to the start parameter of the vSphere Client desktop icon

  • English: en_US
  • German: de_DE
  • Japanese: ja_JP
  • Simplified Chinese: zh_CN
  • French: fr_FR
  • Korean: ko_KR

"C:\Program Files\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient.exe" -locale en_US

You can do the same with your web based client.

Th switch the language to French, simply add the language parameter to the URL

https://client-hostname:9443/vsphere-client/?locale=fr_FR

 

consoleEZ – new tool from YTRIA

January 21, 2014 – 8:08 am

Ytria’s consoleEZ lets you improve the way you work by giving you access to one or several IBM Domino server consoles in one dashboard.

In fact with consoleEZ, you can:

  • Get a complete picture of your domain activities without having to switch from one server to another.
  • Open several sessions for as many servers as you need, apply filters, and select/copy anything you see in the consoles.
  • View past events with the Log Analyzer, filter and search them, and consolidate your data with various options.

consoleEz

You can download a time limited beta version here.

 

 

OpenNTF: TriggerHappy 2.0 (Update)

January 19, 2014 – 2:06 pm

TriggerHappy 2.0 for Domino on Linux has been released on OpenNTF. It is still BETA and not intended to be used in production.
For now, I only provide the 32Bit version, because despite the 64Bit Version compiles and links, it freezes my server on startup.

The 32Bit version starts without issues, but invokes NSD on server quit and restart. Will look into that, but with Connect 2014 just around the corner, I do not have much spare time to do so.

TriggerHappy201

If anyone is willing to offer help and could review the sourcecode ( Ben? Nathan?, Daniel? ) … Much appreciated.

 

OpenNTF: TriggerHappy 2.0

January 18, 2014 – 2:23 pm

18.01.2014 14:07:11 TriggerHappy initially started on Domino 9.0.1 LINUX/64Bit

TriggerHappy20
Still having a smallish issue when the agent is started. Seems to be a problem with Thread Local Storage implementation. Looking for assistance.

If anyone is willing to look into the code, I will send it. Ping me an email if you want to help.

AUTO_SORT_DATE

January 5, 2014 – 11:12 am

As of Notes 9.0.1, you can now control the date column in the mailfile preferences to automatically sort ascending or descending.  (

autosort

The option can also be set via the notes.ini

AUTO_SORT_DATE=11 or 12

A value of 11 displays the most recent message at the end of the view (the default) and 12 displays it at the beginning of the view.

Build cAPI Programs On Linux/64

December 16, 2013 – 3:23 pm

Today I built a compile and build environment for Domino 9.0.1 on Linux 64Bit. I had done this for 32-Bit Linux a while ago and all the steps for 32Bit also work in a 64Bit environment.

There is only one symlink you have to add in addition to the symlinks you have to create on 32-Bit

ln -s $Notes_ExecDirectory/libgsk8iccs_64.so /usr/lib64/libgsk8iccs_64.so

where $Notes_ExecDirectory=/opt/ibm/domino64/domino/notes/latest/linux in my environment