[IBM Traveler 9.0.1.20] – Error encountered while saving database configuration.

December 6, 2017 – 9:16 am

I ran into an issue when trying to add a new IBM Traveler server to an existing HA pool.

The pool already contains 1 IBM Traveler server, so there already is an existing DB2 database instance. ALso, I made sure that the new server can access the DB2 database.

To add a server to the pool, you have to prepare the existing LotusTraveler.nsf. This is done be executing the following command from inside the /local/notesdata/traveler/util/ directory in the shell or in an command prompt if you are on a Windows machine.

./travelerUtil db set url=jdbc:db2://server.example.tld:50000/TRAVELER user=db2user pw=db2password

The command will add a new view “(TravelerDb)” to the LotusTraveler.nsf and then creates a new document to store the credentials to access the Db2 database.

But the command did not ran es expected; it threw an NotesException:

Using JDBC jar: /opt/ibm/domino/notes/90010/linux/Traveler/lib/db2jcc4.jar [25990:00006-00007F47D0C01700] 06.12.2017 09:26:59 03:3E [25990:00006-00007F47D0C01700] 06.12.2017 09:26:59 03:3E Checking database connection to: jdbc:db2://castor.midpoints.net:50000/TRAVELER Connection successful. Error encountered while saving database configuration. NotesException: 0xFCC at lotus.domino.local.View.NcreateColumn(Native Method) at lotus.domino.local.View.createColumn(Unknown Source) at com.lotus.sync.util.ConfigurationBackendDomino.setPasswordFields(ConfigurationBackendDomino.java:1048) at com.lotus.sync.util.ConfigurationBackendDomino.setPW(ConfigurationBackendDomino.java:1132) at com.lotus.sync.util.OfflineUtilities.handleDB(OfflineUtilities.java:1487) at com.lotus.sync.util.OfflineUtilities.execute(OfflineUtilities.java:357) at com.lotus.sync.util.OfflineUtilities.main(OfflineUtilities.java:2677)

Apparently, it failed to create a column in the view. I opened the LotusTraveler in DDE and saw the following in the view section.

The view has been created, but not with the correct name and apparently 1 or more columns are missing. Good news. The document holding the credentials has been created.

I opened the LotusTraveler.nsf from my existing HA server. Here is what the view looks like in DDE

As a workaround, you can delete the (untitled) view and replace it with the (TravelerDb) view from your existing LotusTraveler.nsf

Then you can restart LotusTraveler and the server will be added to the existung pool.

I am not sure, if this is 9.0.1.20 specific. Also I could not find anything about this error on the web. I will open a PMR with IBM.

Extreme slow RDP performance on Windows 2012 R2 server running on VMware ESXi

September 8, 2017 – 7:16 am

I am running a  Windows 2012 R2 servers on a VMware ESXi environment ( 6.5.0 Update 1 (Build 5969303 ) . I experience an extreme poor performance on the Windows 2012R2 server when connection with any RDP-client ( Windows and Mac )

The hardware shouldn’t be an issue.

  • the server does not have a high overall load
  • there is no high CPU load
  • there is enough RAM
  • there is no high I/O

This is what I did to solve the issue and get back to fast RDP-performance.

1. Finetune “Remote Desktop Services” in Group Policy

Open Group Policy Editor ( Start -> Run -> gpedit.msc )

Goto Computer Config > Windows Settings > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Connections > Select RDP transport protocol = Use only TCP

You can also set this on the client side by specifying:

Computer Config > Windows Settings > Admin Templates > Windows Components > Remote Desktop Services > Remote Desktop Connection Client > Turn off UDP on Client = Enabled

2. Disable “DisableTaskOffload” in the Registry

I also added below registry setting to improve performance.

A little explanation of TCP Offloading:

“TCP offload engine is a function used in network interface cards (NIC) to offload processing of the entire TCP/IP stack to the network controller. By moving some or all of the processing to dedicated hardware, a TCP offload engine frees the system’s main CPU for other tasks. However, TCP offloading has been known to cause some issues, and disabling it can help avoid these issues.”

  • Open RegEdit on the Windows Server machine.
  • Navigate to this registry key in the tree on the left: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
  • Right-click on the right side, and add a new DWORD (32-bit) Value
  • Set the value name to DisableTaskOffload and the value data to 1

Now Reconnect to the Server via RDP (to a new session) and your performance should be normal.

SSH2_MSG_UNIMPLEMENTED packet error with PuTTY and RHEL 7.2

September 1, 2017 – 10:19 am

I recently updated one of my RHEL 7.2 servers with yum update -y. After a reboot, I tried to connect to the machine using PuTTY client, but the connection faild with the following error:

To solve the issue, open the configuration for the connection and navigate to SSH -> Kex. In the “Algorithm selection policy”, move the “Diffie-Hellman group 14” entry to the first position in the list.

Now click open and continue working …

 

IBM Verse on Premises 1.0.2 available an Passport Advantage

August 30, 2017 – 6:13 am

IBM Verse on Premises 1.0.2 is available an Passport Advantage. PartNo. CNL9UML

Whats new: https://www.ibm.com/support/knowledgecenter/SS4RQV_1.0.2/whats_new/wn_102.html

[FREE] – midpoints Let’s Encrypt 4 Domino (LE4D)

August 23, 2017 – 5:41 pm

To enable HTTPS on your website, you need to get a certificate from a Certificate Authority. These certificates can be rather expensive, especially if you have several domains or domains that use subject alternate names (SAN).

Let’s Encrypt is a CA that offers certificates for FREE. The only limit is that the certificates expire after 90 days. But you can renew them as often as you like.

In order to get a certificate for your website’s domain from Let’s Encrypt, you have to demonstrate control over the domain. With Let’s Encrypt, you do this using software that uses the ACME protocol, which typically runs on your web host.

Let’s Encrypt has a long list of clients that can be used for certificate creation and renewal. There are clients for Windows or Linux; none of the clients runs on both OS. You could use scripts; but you would have to install Perl, Python or other script interpreters on your Domino server, which is not always possible due to security policies.

And, there is no client for IBM Domino.

midpoints Let’s Encrypt for Domino ( midpoints LE4D ) closes this gap.

  • midpoints LE4D provides all parts of the certificate creation / renewal process in a single Domino application.
  • midpoints LE4D lets you fully automate the process, including renewal of certificates in the keyring file and HTTP task restart.
  • midpoints LE4D has been tested on Domino 9.0.1 FP7 and FP9, but due to it’s Java compliance, midpoints LE4D should also work on Domino versions prior to the tested versions.
  • midpoints LE4D runs on Windows and Linux.
  • midpoints LE4D does not need any 3rd party software ( except for IBM Kyrtool )

Create a new application from the template, create a configuration for your domain, install kyrtool on the server and start an agent ( the agent can later be started on a scheduled basis using a program document ).

midpoints LE4D will register a new account for your domain, creates a private user and domain key. It will then create the certificate signing request and sends it to Let’s Encrypt. Then it receives a challenge token and puts it on your server.

After Let’s Encrypt has validated the token, your certificates are being downloaded and moved to the keyring file on your server. Additionally midpoints LE4D can restart the HTTP task for you.

Interested? Then get your copy of midpoints LE4D today for FREE.

What is the SIZE of a Docker container?

August 21, 2017 – 9:45 am

I recently was  asked, if it is possible to tell the size of a container and, speaking of disk-space, what are the costs when running multiple instances of a container.

Let’s take the IBM Domino server from my previous post as an example.

You can get the SIZE of a container with the following command:

# docker ps -as -f “name=901FP9”
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
5f37c4d6a826 eknori/domino:domino_9_0_1_FP_9 “/docker-entrypoint.s” 2 hours ago Exited (137) 6 seconds ago 901FP9 0 B (virtual 3.296 GB)

We get a SIZE of 0 B (virtual 3.296 GB) as a result. Virtual size? What is that?

Let me try and explain:
When starting a container, the image that the container is started from is mounted read-only. On top of that, a writable layer is mounted, in which any changes made to the container are written.
The read-only layers of an image can be shared between any container that is started from the same image, whereas the “writable” layer is unique per container (because: you don’t want changes made in container “a” to appear in container “b” )
Back to the docker ps -s output;

  • The “size” information shows the amount of data (on disk) that is used for the writable layer of each container
  • The “virtual size” is the amount of disk-space used for the read-only image data used by the container.

So, with a 0 B container size, it does not make any difference, if we start 1 or 100 containers.

Be aware that the size shown does not include all disk space used for a container. Things that are not included currently are;

  1. volumes used by the container
  2. disk space used for the container’s configuration files (hostconfig.json, config.v2.json, hosts, hostname, resolv.conf) – although these files are small
  3. memory written to disk (if swapping is enabled)
  4. checkpoints (if you’re using the experimental checkpoint/restore feature)
  5. disk space used for log-files (if you use the json-file logging driver) – which can be quite a bit if your container generates a lot of logs, and log-rotation (max-file / max-size logging options) is not configured

So, let’s see what we have to add to the 0 B to get the overall size of our container.

We are using a volume “domino_data” for our Domino server . To get some information about this volume (1) type

# docker volume inspect domino_data
[
{
“Name”: “domino_data”,
“Driver”: “local”,
“Mountpoint”: “/var/lib/docker/volumes/domino_data/_data”,
“Labels”: {},
“Scope”: “local”
}
]

This gives us the physical location of that volume. Now we can get the size of the volume, summing up the size of all files in the volume.

# du -hs /var/lib/docker/volumes/domino_data/_data
1.1G /var/lib/docker/volumes/domino_data/_data

To get the size of the container configuration (2), we need to find the location for our container.

# ls /var/lib/docker/containers/
5f37c4d6a8267246bbaff668b3437f121b0fe375d8319364bf7eb10f50d72c69

Now we have the long Id for our CONTAINER ID. Next type

# du -hs 5f37c4d6a8267246bbaff668b3437f121b0fe375d8319364bf7eb10f50d72c69/
160K 5f37c4d6a8267246bbaff668b3437f121b0fe375d8319364bf7eb10f50d72c69/

Now do the math yourself.  x = (0B + 1.1GB + 160kB ) * n .

I leave it up to you to find out the other sizes ( 3 – 4 ) .

Sizes may vary and will change during runtime; but I assume that you got the idea.  Important to know is that all containers that are using the same image in the FROM command in a Dockerfile share this (readonly) image, so there is only one copy of it on disk.

 

Domino on Docker

August 20, 2017 – 9:23 am

IBM recently announced Docker support for Domino. It is supposed to come with FP10 at the end of this year.

Domino IMHO is not a microservic, but Domino on Docker has other advantages.

Think about a support person maintaining a product. All he needs to investigate a customer’s issue is the data from the customer and a Domino environment that is known to run the application stable. He can the create a new container from a Docker image, copy the files from the customer into the container, start Domino and then he can try to reproduce the issue.

You can also do this with VMs, but Docker images are more flexible. Our supporter might expect that the customer uses a specific version of Linux for the Domino server installation. But it turns out, that he uses the lastest build of the Linux OS.  You would need to setup a new VM with the Linux version that is needed, install and configure Domino etc … Waste of time and resources. Using Docker, this is just one change in a Dockerfile.

I will be speaking about Docker at AdminCamp 2017 in September. I will talk about Docker in general and also about Domino on Docker. In this blogpost, I want to show, how easy it is to create a Domino image ( optional with FP), and then build and run a Docker container from the image.

I assume that you already have Docker installed on a host. I am using RHEL 7 as the host OS for Docker.

Let us start with the basic Domino 9.0.1 image. I am using the excellent start scripts for Domino by Daniel Nashed. If you run Domino on Linux and you do not already have the scripts, get and use them.

First of all, create a new directory on your host. This directory will be used to store the needed Dockerfiles. You can also download the files and use them.

All Domino installation files should be accessible from a web server. replace the YOUR_HOST placeholder with your webserver

Here is the Dockerfile for the Domino 9.0.1 basic installation.

FROM centos
 
ENV DOM_SCR=resources/initscripts 
ENV DOM_CONF=resources/serverconfig 
ENV NUI_NOTESDIR /opt/ibm/domino/
 
RUN yum update -y && \
    yum install -y which && \
    yum install -y wget && \
	yum install -y perl && \
    useradd -ms /bin/bash notes && \
    usermod -aG notes notes && \
    usermod -d /local/notesdata notes && \
    sed -i '$d' /etc/security/limits.conf && \
    echo 'notes soft nofile 60000' >> /etc/security/limits.conf && \
    echo 'notes hard nofile 80000' >> /etc/security/limits.conf && \
    echo '# End of file' >> /etc/security/limits.conf
 
COPY ${DOM_CONF}/ /tmp/sw-repo/serverconfig
 
RUN mkdir -p /tmp/sw-repo/ && \
    cd /tmp/sw-repo/ && \
    wget -q http://YOUR_HOST/DOMINO_9.0.1_64_BIT_LIN_XS_EN.tar && \
    tar -xf DOMINO_9.0.1_64_BIT_LIN_XS_EN.tar &&\
    cd /tmp/sw-repo/linux64/domino && \
    /bin/bash -c "./install -silent -options /tmp/sw-repo/serverconfig/domino901_response.dat" && \
    cd / && \
    rm /tmp/* -R
 
RUN mkdir -p /etc/sysconfig/
COPY ${DOM_SCR}/rc_domino /etc/init.d/
RUN chmod u+x /etc/init.d/rc_domino && \
    chown root.root /etc/init.d/rc_domino
COPY ${DOM_SCR}/rc_domino_script /opt/ibm/domino/
RUN chmod u+x /opt/ibm/domino/rc_domino_script && \
    chown notes.notes /opt/ibm/domino/rc_domino_script
COPY ${DOM_SCR}/rc_domino_config_notes /etc/sysconfig/

We install Domino on the latest CentOS build; if you want to use a specific CentOS build, change the first line in the Dockerfile and add the buildnumber.

You can see a lot of commands that have been combined into one RUN statement. Doing it this way, you can keep the image size a bit smaller. Each RUN command would create an extra layer and this will increase the size of your image.

So, in the first part, we update the CentOS image from the Docker repository with the latest fixes and also install additional packages that we need for the Domino installation.
Next, we copy our response.dat file and the start scripts to our image.
Now we download the Domino 9.0.1 installation package, unpack it and do a silent installation using our response.dat file for configuration.
Last part is installation of the start script files, assigning user and group and granting permissions

All temporary files are also deleted.

Now we can create an image from the Dockerfile.

docker build -t eknori/domino:9_0_1 -f Dockerfile .

This will take about 10 – 15 minutes. When the build is completed, we can list our image

# docker images

eknori/domino 9_0_1 96b6220d177c 14 hours ago 1.883 GB

Next we will use this image and install FP9. If you need some other FP, tweak the Dockerfile to your own needs. Once you get familiar to Docker, this is easy.

FROM eknori/domino:9_0_1
 
ENV DOM_CONF=resources/serverconfig
ENV NUI_NOTESDIR /opt/ibm/domino/
 
COPY ${DOM_CONF}/ /tmp/sw-repo/serverconfig
 
RUN mkdir -p /tmp/sw-repo/ && \
cd /tmp/sw-repo/ && \
wget -q http://YOUR_HOST/domino901FP9_linux64_x86.tar && \
tar -xf domino901FP9_linux64_x86.tar &&\
cd /tmp/sw-repo/linux64/domino && \
/bin/bash -c "./install -script /tmp/sw-repo/serverconfig/domino901_fp9_response.dat" && \
cd / && \
rm /tmp/* -R && \
rm /opt/ibm/domino/notes/90010/linux/90010/* -R

A much shorter Dockerfile, as we already have installed Domino and now can reuse the 9_0_1 image as the base image for our 9_0_1_FP_9 image.

The last line in the RUN command removes the uninstall information. Maybe this can be done in the response.dat file also, but you should do this anyway, as we do not need the backup files.

Again, build the new image.

docker build -t eknori/domino:9_0_1_FP_9 -f Dockerfile .

# docker images
eknori/domino 9_0_1_FP_9 ed0276f21d73 14 hours ago 3.296 GB

Now we can build or final Domino 9.0.1FP9 from the 9_0_1_FP_9.

Our Dockerfile looks like this

FROM eknori/domino:9_0_1_FP_9
 
EXPOSE 25 80 443 1352
 
COPY resources/docker-entrypoint.sh /
RUN chmod 775 /docker-entrypoint.sh
 
USER notes
WORKDIR /local/notesdata
ENV LOGNAME=notes
ENV PATH=$PATH:/opt/ibm/domino/
 
ENTRYPOINT ["/docker-entrypoint.sh"]

and the file used in the ENTRYPOINT command contains the following

#!/bin/bash
 
serverID=/local/notesdata/server.id
 
if [ ! -f "$serverID" ]; then
/opt/ibm/domino/bin/server -listen 1352
else
/opt/ibm/domino/rc_domino_script start
/bin/bash
fi

The ENTRYPOINT is executed when the container starts. The script just checks, if the server is already configured. If not, it starts the server in LISTEN mode. If it finds a server.id, it starts the server.

Let us build or final image.

docker build -t eknori/domino:domino_9_0_1_FP_9 -f Dockerfile .

# docker images
eknori/domino domino_9_0_1_FP_9 1fae2fe73df4 2 hours ago 3.296 GB

Now we are ready to create a container from the image. But we need one additional step. All changes that we make in a container will be lost, once the container is stopped. So we need to create a persitent data store and attach it to the container.

To create a persitent volume, type

docker volume create –name=domino_data

And then type

docker run -it -p 1352:1352 -p 8888:80 -p 8443:443 –name 901FP7 -v domino_data:/local/notesdata eknori/domino:domino_9_0_1_FP_9

to build and run the container. I have used port 1352 instead of 8585 to avoid to open another port on the host system.

After the build, the ENTRYPOINT will start the Domino server in LISTEN mode. You can now setup your server using the remote setup tool.

After you have setup your server, close the remote setup and stop Domino. This will also stop your container.

You can start and get access to the container with

docker start 901FP9
docker attach 901FP9

This gives you great flexibility. Once FP10 is in the wild, create a new image from the 9_0_1 image and install FP10. Then create a new image for your final Domino installation. Run this image and attach the persitent volume.

[Docker CLI] – Delete Containers

May 31, 2017 – 4:35 pm

If you want to delete ALL containers , running or exited, you can do this with a single command.

docker stop $(docker ps -a -q)

docker rm $(docker ps -a -q)

If you only want to delete containers that have ‘exited’, then use:

docker ps -a | grep Exited | cut -d ‘ ‘ -f 1 | xargs  docker rm

IBM Spectrum Conductor for Containers

May 30, 2017 – 1:13 pm

IBM® Spectrum Conductor for Containers is a server platform for developing and managing on-premises, containerized applications. It is an integrated environment for managing containers that includes the container orchestrator Kubernetes, a private image repository, a management console, and monitoring frameworks.

For my upcoming AdminCamp 2017 session later this year, I wanted to put together a nice session about Docker in general as well as Kubernetes and how to orchestrate containers without creating .yaml files.
IBM® Spectrum Conductor for Containers is installed as part of orient.me and is the foundation for all the new and upcoming contaierized stuff in Connections 6.

But it is also available as a standalone component.  There are also other graphical tools that work on top of Docker ( and Kubernetes ) but I thought that it is a good idea to use IBM® Spectrum Conductor for Containers.
The installation is more or less just running a Docker container and sit and wait. That is, what I thought.

After reading through the documentation, I decided to use RHEL 7.2 in a VM on ESXi 6.5. I wanted to document the installation process and all the configurations to give attendees a step-by-step instruction how to setup and configure the OS, install additional software like Docker and finally prepare the configuration for the CfC installer. It is all in the installation guide provided by IBM, but I like to have it in one textfile where I just need to copy the commands into the Linux console instead of jumping back and forth in the HTML document.

After configuring the system and tweaking here and there, I tried the install with 1 CPU / 4 GB resulting in a hang in the middle of the installation process.
The installer does not give you any hint, what went wrong. And also the logs are not very helpful.

Next attempt was 2 CPU / 8 GB. It went a bit further in the installation process, but hung at a different point then. Also here, no hint from the installer or in the logs

Final try was 4 CPU / 8 GB. Now the installation finished and I could open the dashboard.

This stuff is the foundation for Connections Next and I can live with the requirements regarding CPU / RAM.

If you just want to use Docker with Kubernetes plus one of the other UI tools, then you are good with a “normal” sized VM ( 1 CPU / 4 GB ). This will also be part of my Docker session at AdminCamp 2017.

Notes FP8 (IF1) might stop your custom sidebar plugins from working

May 11, 2017 – 11:39 am

We got a call from one of our customers reporting a defect in our midpoints doc.Store sidebar plugin. It worked in Notes 9.0.1FP7 but stopped working after the upgrade to FP8.

I was able to reproduce in our development environment. In the error log, we saw the following error message

at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.ViewEntry.getDocument(Unknown Source)
at de.midpoints.docstore.notes.model.DocStoreDocumentCollectionBuilder.calculateDocumentCollection(Unknown Source)
at de.midpoints.docstore.notes.views.DocStoreView$32.run(Unknown Source)
at org.eclipse.core.internal.jobs.Worker.run(Unknown Source)

I was able to find a fix for this particular issue. But there is also an entry in the German Notes Forum reporting similar defects after the upgrade.

I opened a PMR with IBM. IBM is already aware of the issue. According to IBM support, a fix is supposed to be shipped with FP9.

IBM support also proposed a workaround:

The issue does not occur when using the Notes.jar of the 901FP7 with the 901FP8 installation.

Some error messages from the lower levels of the JavaStack:

java.lang.ClassCastException: lotus.domino.local.View
incompatible with lotus.domino.local.Session
at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.Database.getView(Unknown Source)
—–

java.lang.ClassCastException: lotus.domino.local.Document
incompatible with lotus.domino.local.Session
at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.Database.getDocumentByUNID(Unknown
Source)
—————

java.lang.ClassCastException: lotus.domino.local.Item
incompatible with lotus.domino.local.Session
at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.Session.FindOrCreate(Unknown Source)
at lotus.domino.local.Document.getItems(Unknown Source)

The issue is tracked at IBM under SPR # RGAUALXF5R / APAR LO92228

Install Docker On RHEL 7 in 3 Easy Steps

March 31, 2017 – 12:42 pm

You are just 3 steps away from running docker containers on a RHEL7.

  1. curl -sSL https://get.docker.com/ | sh
  2. systemctl enable docker
  3. systemctl start docker

Now you can test your installation running

  • docker run hello-world

That’s it.

 

C / c++ for Notes & Domino Developers

March 28, 2017 – 7:41 am

C/ C++ for Notes & Domino Developers von Ulrich Krause

Fun with IBM Traveler and Java

February 5, 2017 – 9:57 am

Today I stumbled upon a very strange behaviour of some Java code, and I do not have any clue about the why.

I am parsing the response (text) file from the “tell traveler show user” command.
The response file is written to the system temp directory and contains all information that you would also see when you invoke the command on the server console. No problem so far.

The response file contains a section that lists all mail file replicas for the user.

IBM Traveler has validated that it can access the database mail/ukrause.nsf.
Monitoring of the database for changes is enabled.
Encrypting, decrypting and signing messages are enabled because the Notes ID is in the mail file or the ID vault.

Canonical Name: CN=Ulrich Krause/O=singultus
Internet Address: ulrich.krause@eknori.de
Home Mail Server: CN=serv01/O=singultus
Home Mail File: mail/ukrause.nsf
Current Monitor Server: CN=serv01/O=singultus Release 9.0.1FP8
Current Monitor File: mail/ukrause.nsf
Mail File Replicas:
[CN=serv02/O=singultus, mail/ukrause.nsf] is reachable.
ACL for Ulrich Krause/singultus: Access=Manager Capabilities=create,update,read,delete,copy Missing Capabilities=none
ACL for serv01/singultus: Access=Manager Capabilities=create,update,read,delete,copy Missing Capabilities=none
[CN=serv01/O=singultus, mail/ukrause.nsf] is reachable.
ACL for Ulrich Krause/singultus: Access=Manager Capabilities=create,update,read,delete,copy Missing Capabilities=none
ACL for serv01/singultus: Access=Manager Capabilities=create,update,read,delete,copy Missing Capabilities=none

Notes ID: Mail File contains the Notes ID which was last updated by CN=serv01/O=singultus on Tuesday, June 16, 2015 1:09:16 PM CEST.

If a server for a replica is down or not reachable the output looks like this

IBM Traveler has validated that it can access the database mail/ukrause.nsf.
Monitoring of the database for changes is enabled.
Encrypting, decrypting and signing messages are enabled because the Notes ID is in the mail file or the ID vault.

Canonical Name: CN=Ulrich Krause/O=singultus
Internet Address: ulrich.krause@eknori.de
Home Mail Server: CN=serv01/O=singultus
Home Mail File: mail/ukrause.nsf
Current Monitor Server: CN=serv01/O=singultus Release 9.0.1FP8
Current Monitor File: mail/ukrause.nsf
Mail File Replicas:
[CN=serv01/O=singultus, mail/ukrause.nsf] is reachable.
ACL for Ulrich Krause/singultus: Access=Manager Capabilities=create,update,read,delete,copy Missing Capabilities=none
ACL for serv01/singultus: Access=Manager Capabilities=create,update,read,delete,copy Missing Capabilities=none
[CN=serv02/O=singultus, mail/ukrause.nsf] is not reachable, status(0x807) “The server is not responding. The server may be down or you may be experiencing network problems. Contact your system administrator if this problem persists.”.

Notes ID: Mail File contains the Notes ID which was last updated by CN=serv01/O=singultus on Tuesday, June 16, 2015 1:09:16 PM CEST.

Here is the code fragment that I use to parse the response file. I am using a LineIterator.

import org.apache.commons.io.FileUtils;
import org.apache.commons.io.LineIterator;
 
import com.google.common.base.Joiner;
import com.google.common.collect.Lists;
 
public class UserFileParser {
 
	private String			filename;
	private LineIterator		lineIterator;
 
	public void process() {
		try {
			lineIterator = FileUtils.lineIterator(new File(filename));
 
			while (lineIterator.hasNext()) {
				String line = lineIterator.nextLine().trim();
				System.out.println(line);

The expected behaviour is that the code will print every line inside the response file to the server console. So far for the theory.

BUT … the code behaves different if the response file contains information about not reachable replicas or not.
I have tested the code in Eclipse on a Windows 10 client without any issues. The problem only exists on the server when the code is executed from within a DOTS task.

If the response file lists all replicas as reachable, the code works as expected. I can see all lines printed to the console.
If the response file contains information about a replica that is not reachable, the code stops after reading

Current Monitor File: mail/ukrause.nsf

It does not get to

Mail File Replicas:

By the way, it does not make any difference, if I use any other kind of reader.

I have changed my code to

	public void process() {
		String line = "";
		try {
			br = new BufferedReader(new FileReader(new File(filename)));
			while ((line = br.readLine().trim()) != null) { 
				System.out.println(line);
				lines.add(line);

Now I get a NullPointerException, but also the code stops at exact the same line in the response file. I all replicas are reachable, no NPE.

java.lang.NullPointerException
at de.eknori.dots.provider.parser.UserFileParser.process(UserFileParser.java:65)

I have already investigated the 2 response files for hidden characters and stuff, but cannot see anything that would explain this behaviour.

From the data in the response file you can see that I have FP8 (Beta) installed; I have not yet checked with FP7, but I expect the same weirdness.

U P D A T E:

FP7 shows the same behaviour.

I have tried reading the file char by char

Reader reader = new InputStreamReader(new FileInputStream(filename), "UTF-8");
Integer i;
 
while ((i = reader.read()) != -1) {
	System.out.println(i);
}

and, indeed, there is a -1 value for i in the middle of the file.

105
99
97
115
58
32
13
10
-1

So, no surprise, that all readers stop to read past this char.

Verse on premises – first impression

December 31, 2016 – 7:07 am

IBM promised to deliver Verse On Premises ( VoP ) on 30-DEC-2016. And they did.

I tried to download it; all I got was

24 hours later, the download worked.

The “installation” is a mixture of click and install and copy files from here to there. I expected something like IBM Notes Traveler installer.

I had VoP installed as a Beta user, so installation was not a big deal. IBM changes the target location and you have to uninstall the beta installation first; no surprise is that.

There is an online documentation on how to install VoP. See https://www.ibm.com/support/knowledgecenter/SS4RQV_1.0.0/admin/topics/vop_configuring_server.html

After “installing” VoP, I was eager to see, how it looks and feels. As I already said, I had BETA 3 of VoP installed.

BETA 3 worked OK so far; not all features were in place and also the design of the calender and what else still was iNotes.

So I expected something new in this area.

This is, what I saw, when I opened VoP

So, fact is: Apart from the issue with showing the content of my mailfile ( we have had this in Beta 2 ), the calendar is still iNotes.

Needless to say that this is a bit of a disappointment.

Also, you will never get a real VoP in the future. For files and contacts, you need a Connections installation. OK, this can be done on premises, but is a huge overhead. For some features, Watson is needed. And there is no WoP available. Will it ever be? I doubt.

VoP is a big disappointment. I already uninstalled it from my Domino. iNotes as it is works for me.

At least, IBM delivered something on 30-Dec-2016 as promised.

IBM Notes and Domino V9.0.1 extends support and enhances its collaboration toolset with social capabilities from IBM Connections V5.5

September 13, 2016 – 8:02 am

read this for detailled information

Domino JNA vs. IBM Domino Java API

September 11, 2016 – 12:02 pm

Today, I finally found the time to take a closer look into the Domino JNA project. The project has been created by Karsten Lehmann, mindoo.

The goal of the project is to provide low level API methods that can be used in Java to speed-up the retrival of data; especially, if you have to deal with lots of data, this can be a significant performance impact.

My szenarion is the following:

Read entries from a Notes Application into a Java bean and add data from another document in another application to the bean.The number of documents in the source application can be from 1 – n. I do not “own” the design of the source database, so I cannot modify it.

In this article, I will concentrate on reading the source application in the fastest way possible. I will show, how I do it right now and also, how you can use Domino JNA.

Here is a small piece of Java to measure the duration of data retrival.

public class StopWatch {
 
	private long	startTime	= 0;
	private long	stopTime	= 0;
	private boolean	running		= false;
 
	public void start() {
		this.startTime = System.nanoTime();
		this.running = true;
	}
 
	public void stop() {
		this.stopTime = System.nanoTime();
		this.running = false;
	}
 
	// elaspsed time in milliseconds
	public long getElapsedTime() {
		long elapsed;
		if (running) {
			elapsed = (System.nanoTime() - startTime);
		} else {
			elapsed = (stopTime - startTime);
		}
		return elapsed;
	}
 
	// elaspsed time in seconds
	public long getElapsedTimeSecs() {
		long elapsed;
		if (running) {
			elapsed = ((System.nanoTime() - startTime) / 1000);
		} else {
			elapsed = ((stopTime - startTime) / 1000);
		}
		return elapsed;
	}
}

I am using “fakenames.nsf” in my sample code. You can download the two sample databases fakenames.nsf and fakenames-views.nsf from this URL:

ftp://domino_jna:domino_jna@www2.mindoo.de

Next, place them in the data folder of your IBM Notes Client.

here is the code, that I use in my application. It uses a NotesNavigator to traverse the view. It then opens the underlying document for each entry found using entry.getDocument() and prints values for some items to the console.

I need to do it this way, because I need the contents of some items to identify the document in the other database. Unfortunately not all needed values are in the view. So just reading the column values is not an option.

import lotus.domino.Database;
import lotus.domino.Document;
import lotus.domino.NotesFactory;
import lotus.domino.NotesThread;
import lotus.domino.Session;
import lotus.domino.View;
import lotus.domino.ViewEntry;
import lotus.domino.ViewNavigator;
 
public class Domino {
 
	public static void main(String[] args) {
		try {
			StopWatch stopWatch = new StopWatch();
			NotesThread.sinitThread();
			Session session = NotesFactory.createSession();
			stopWatch.start();
			Database dbData = session.getDatabase("", "fakenames.nsf");
			View view = dbData.getView("People");
			ViewNavigator navUsers = null;
			ViewEntry vweUser = null;
			ViewEntry vweTemp = null;
			Document docUser = null;
 
			view.setAutoUpdate(false);
			navUsers = view.createViewNav();
			navUsers.setEntryOptions(ViewNavigator.VN_ENTRYOPT_NOCOUNTDATA + ViewNavigator.VN_ENTRYOPT_NOCOLUMNVALUES);
 
			vweUser = navUsers.getFirst();
 
			navUsers.setCacheGuidance(Integer.MAX_VALUE, ViewNavigator.VN_CACHEGUIDANCE_READSELECTIVE);
 
			while (vweUser != null) {
				docUser = vweUser.getDocument();
				System.out.println(
				        docUser.getItemValueString("lastname") + ", " + docUser.getItemValueString("firstname"));
				vweTemp = navUsers.getNext(vweUser);
				docUser.recycle();
				vweUser.recycle();
				vweUser = vweTemp;
			}
			stopWatch.stop();
			System.out.println(stopWatch.getElapsedTimeSecs());
 
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			NotesThread.stermThread();
		}
	}
}

Now to Domino JNA. Here is the code. First, I get all IDs from the documents in the view and then I take the result to get the underlying documents and the data.

import java.util.EnumSet;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.concurrent.Callable;
 
import com.mindoo.domino.jna.NotesCollection;
import com.mindoo.domino.jna.NotesCollection.EntriesAsListCallback;
import com.mindoo.domino.jna.NotesDatabase;
import com.mindoo.domino.jna.NotesIDTable;
import com.mindoo.domino.jna.NotesNote;
import com.mindoo.domino.jna.NotesViewEntryData;
import com.mindoo.domino.jna.constants.Navigate;
import com.mindoo.domino.jna.constants.OpenNote;
import com.mindoo.domino.jna.constants.ReadMask;
import com.mindoo.domino.jna.gc.NotesGC;
 
import lotus.domino.NotesException;
import lotus.domino.NotesFactory;
import lotus.domino.NotesThread;
import lotus.domino.Session;
 
public class DominoApi {
 
	public static void main(String[] args) throws NotesException {
		try {
			NotesGC.runWithAutoGC(new Callable() {
 
				@Override
				public Object call() throws Exception {
					StopWatch stopWatch = new StopWatch();
					NotesThread.sinitThread();
					Session session = NotesFactory.createSession();
					stopWatch.start();
					NotesDatabase dbData = new NotesDatabase(session, "", "fakenames.nsf");
 
					NotesCollection colFromDbData = dbData.openCollectionByName("People");
 
					boolean includeCategoryIds = false;
					LinkedHashSet allIds = colFromDbData.getAllIds(includeCategoryIds);
					NotesIDTable selectedList = colFromDbData.getSelectedList();
					selectedList.clear();
					selectedList.addNotes(allIds);
					String startPos = "0";
					int entriesToSkip = 1;
					int entriesToReturn = Integer.MAX_VALUE;
					EnumSet returnNavigator = EnumSet.of(Navigate.NEXT_SELECTED);
					int bufferSize = Integer.MAX_VALUE;
					EnumSet returnData = EnumSet.of(ReadMask.NOTEID, ReadMask.SUMMARY);
 
					List selectedEntries = colFromDbData.getAllEntries(startPos, entriesToSkip,
			                returnNavigator, bufferSize, returnData, new EntriesAsListCallback(entriesToReturn));
 
					for (NotesViewEntryData currEntry : selectedEntries) {
						NotesNote note = dbData.openNoteById(currEntry.getNoteId(), EnumSet.noneOf(OpenNote.class));
 
						System.out.println(
			                    note.getItemValueString("lastname") + ", " + note.getItemValueString("firstname"));
						note.recycle();
					}
					stopWatch.stop();
					System.out.println(stopWatch.getElapsedTimeSecs());
					return null;
				}
			});
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			NotesThread.stermThread();
		}
 
	}
 
}

Now, what do you think? Which code is faster? Domino JNA? Well, not really in my szenario.

I have done a couple of tests on a local machine for both code sample.

The average time ( from 100 runs each ) for my code to get 40.000 documents from the “fakenames.nsf” and to print the values from the firstname and lastName item is 9.70 seconds; the average for Domino JNA is 10.65 seconds.

This does not mean that Domino JNA does not have any advantage over the standard IBM Domino Java API; It depends on the szenario. And in my szenario, there is no advantage in using Domino JNA. It would only result in an advanced complexity and platform dependancies.

If you have read this far, here is an extra for you.  I played with the options and found that the

navUsers.<strong>setCacheGuidance</strong>(Integer.MAX_VALUE, ViewNavigator.VN_CACHEGUIDANCE_READSELECTIVE);

is a significant performance boost.

 

Without setting the cache guidance, the average time to get the data out of the application was 11.40 seconds. I could not see any difference in using VN_CACHEGUIDANCE_READALL instead of  VN_CACHEGUIDANCE_READSELECTIVE.

ONTF DomBackup 1.0.0.5 released

April 29, 2016 – 4:27 pm

I have uploaded a new version of ONTF DomBackup. Version 1.0.0.5 contains updated binaries for Windows 32/64 Bit.

It adresses an issue that leads to data losses when compressing files larger 2GB. The issue is described here. The issue is Windows related only!

The fixed version uses 7Zip as an external compression tool.

 

DAOS – Find a note for a missing NLO file

March 8, 2016 – 5:35 pm

Today, I saw the following message on my Domino console

[0E88:005A-0FF4] 08.03.2016 17:19:01   The database d:\Domino\data\mail\ukrause.nsf was unable to open or read the file d:\DAOS\0002\97FC43BEED143800A6608E557BE888498DB9BC5100015B7C.nlo: File truncated – file may have been damaged

If you see such a message in your production environment, you should immediately find out

  • what causes the damage?
  • which note does the .nlo belong to?

In my case, the answer for the first one is: Anti Virus Software.

And here is, how I found the answer to the second on

If not already in place, set

DEBUG_DAOS_DIAGNOSTICS=1

Next trigger a console command

tell daosmgr LISTNLO MAP -V mail/ukrause.nsf

This will create a file listNLO.txt in you Domino data directory

Open the file and search for the .nlo file in question

68258,0X10AA2,69422,0X10F2E,88956,88956,DAOS,

97FC43BEED143800A6608E557BE888498DB9BC5100015B7C,

0X8,VALID,ö1,97FC43BEED143800A6608E557BE888498DB9BC5100015B7C,[0:1],97FC43BEED143800A6608E557BE888498DB9BC5100015B7C,[0:1],Shared,1

You now have the noteId of the document that has a ticket to the damaged .nlo.

Use your favorite tool to find and open the document.

missingFile1

In my case, it was just a SPAM mail. So no worries. But if this happens in production, you should now go and find the ( hopefully intact ) nlo in your backup and restore it.

Windows Fixpack Update Idiocy

February 28, 2016 – 9:12 am

Today I decided to install some recommended and optional fixes on my “productive” Windows 2008 R2 /64 Server.

In general, this has been a straight-forward task in the past. Select all fixes and click Install. Grab a new cup of coffee, restart the machine after upgrade and carry on with daily business.

As always, I looked for the available free space and found it to be sufficient.

disk163

The overall size of all fixes to be installed was ~80MB, so 1.63GB should be more than enough.

10updates

8 of 10 fixes installed without any issues, but the remainin 2 reported errors. As always, the MS error messages are completely useless.

So I asked Google for advice and found at least one hint how to install the DotNet 4.5.2 update. “Download the 4.5.2 offline installer”

I did and ran it locally. A message box popped up and I could not believe what I could see.

dotnet452

So, the successful install consumed 900MB for ~70MB of fixes.

MS should really re-think their upgrade strategy.

And yes, I know. This is ALL so much better on Linux and MAC.

 

UPDATE:

After another fixpack install ( 1.1 MB )

nextupdate

So, where has my free space gone??

Even removal of features needs free disk space. Insane !!

remove

By the way. Free disk space is now 0Bytes …

[Vaadin] – Create a simple twin column modal multi-select dialog

January 30, 2016 – 10:29 am

Vaadin is an open source Web application framework for rich Internet applications. In contrast to JavaScript libraries and browser-plugin based solutions, it features a server-side architecture, which means that the majority of the logic runs on the servers. Ajax technology is used at the browser-side to ensure a rich and interactive user experience. On the client-side Vaadin is built on top of and can be extended with Google Web Toolkit.

To let the user interact and enter or change data, you can use simple text fields. But, to keep data consitent, sometimes you want to make only specific values available and let the user select one ore more of the values.

Here is a small sample of a twin column modal select dialog box. It uses only Vaadins basic features, so no 3rd party plugins are needed.

Here is, how it looks

TwinColl

And here is the source code

package org.bluemix.challenge;
 
import javax.servlet.annotation.WebServlet;
 
import com.vaadin.annotations.Theme;
import com.vaadin.annotations.VaadinServletConfiguration;
import com.vaadin.annotations.Widgetset;
import com.vaadin.data.Property.ValueChangeEvent;
import com.vaadin.data.Property.ValueChangeListener;
import com.vaadin.event.ShortcutAction;
import com.vaadin.server.VaadinRequest;
import com.vaadin.server.VaadinServlet;
import com.vaadin.ui.Button;
import com.vaadin.ui.Button.ClickEvent;
import com.vaadin.ui.FormLayout;
import com.vaadin.ui.TwinColSelect;
import com.vaadin.ui.UI;
import com.vaadin.ui.VerticalLayout;
import com.vaadin.ui.Window;
import com.vaadin.ui.Window.CloseEvent;
import com.vaadin.ui.Window.CloseListener;
 
@Theme("mytheme")
@Widgetset("org.bluemix.challenge.MyAppWidgetset")
@SuppressWarnings("serial")
public class MyUI extends UI {
	private static final int OPTION_COUNT = 6;
 
	public String selectedOptions = "";
 
	@Override
	protected void init(VaadinRequest vaadinRequest) {
 
		final VerticalLayout layout = new VerticalLayout();
		layout.setMargin(true);
		setContent(layout);
 
		Button button = new Button("Click Me");
		button.addClickListener(new Button.ClickListener() {
 
			@Override
			public void buttonClick(ClickEvent event) {
 
				final Window dialog = new Window("Select one or more ...");
				dialog.setWidth(400.0f, Unit.PIXELS);
				dialog.setModal(true);
				dialog.setClosable(false);
				dialog.setResizable(false);
 
				// register esc key as close shortcut
				dialog.setCloseShortcut(ShortcutAction.KeyCode.ESCAPE);
 
				TwinColSelect twinColl = new TwinColSelect();
 
				for (int i = 0; i &lt; OPTION_COUNT; i++) {
					twinColl.addItem(i);
					twinColl.setItemCaption(i, "Option " + i);
				}
 
				twinColl.setRows(OPTION_COUNT);
				twinColl.setNullSelectionAllowed(true);
				twinColl.setMultiSelect(true);
				twinColl.setImmediate(true);
				twinColl.setLeftColumnCaption("Available options");
				twinColl.setRightColumnCaption("Selected options");
				twinColl.setWidth(95.0f, Unit.PERCENTAGE);
 
				twinColl.addValueChangeListener(new ValueChangeListener() {
 
					@Override
					public void valueChange(ValueChangeEvent event) {
						selectedOptions = String.valueOf(event.getProperty().getValue());
 
					}
				});
 
				final FormLayout dialogContent = new FormLayout();
 
				dialogContent.addComponents(twinColl);
				dialogContent.addComponent(new Button("Done", new Button.ClickListener() {
 
					public void buttonClick(ClickEvent event) {
						UI.getCurrent().removeWindow(dialog);
					}
				}));
 
				dialog.addCloseListener(new CloseListener() {
					@Override
					public void windowClose(CloseEvent e) {
						System.out.println(selectedOptions);
					}
				});
 
				dialog.setContent(dialogContent);
				UI.getCurrent().addWindow(dialog);
			}
		});
		layout.addComponent(button);
	}
 
	@WebServlet(urlPatterns = "/*", name = "MyUIServlet", asyncSupported = true)
	@VaadinServletConfiguration(ui = MyUI.class, productionMode = false)
	public static class MyUIServlet extends VaadinServlet {
		private static final long serialVersionUID = 452468769467758600L;
	}
}

When the dialog is closed, it prints the selected options to the console.

The dialog can be closed by hitting the ESC key. Also here the code returns the selected options ( if any )

dialog.setCloseShortcut(ShortcutAction.KeyCode.ESCAPE);

This is just a basic sample; you can create a custom component and also pass the available options using a bean or whatever suites best for your need.