Migrating OSGi Plugins to standalone applications

13. Januar 2020 Posted by Sven Hasselbach

I have created an example project for migrating an OSGI-based Spring Boot application into an standalone application:

https://github.com/hasselbach/domino-springboot/blob/maven/domino-springboot.plugin

The application runs in the JVM of the Notes Client, only a user id is required, not a complete server installation (and ID). The benefit is that you can run as much JVM instances you want (which allows a better resource management and deployment) and it is easier to integrate the applications into existing CD environments. Also, Non-Domino developers can use their preferred IDE and are able to use existing / standard knowledge when developing Java applications.

When using the Java NAPI, you need to remove the spring-boot-devtools dependency because of problems with the class loader and DLLs.

Schulungsangebot – Spring Boot & Domino: REST APIs

14. Februar 2019 Posted by Sven Hasselbach

Kurs „Spring Boot & Domino: REST APIs“

Für große Anwendungen ist eine REST Schnittstelle heutzutage unverzichtbar. Doch mit den Bordmitteln sind schnell Grenzen erreicht und auch die Einbeziehung von Entwicklern ohne Domino-Hintergrundwissen gestaltet sich schwierig

Spring Boot ist der Defacto-Standard für Unternehmensanwendungen im Java-Umfeld und eine robuste wie auch flexible Alternative zu existierenden Frameworks des Domino-Umfelds.

Der Kurs vermittelt die nötigen Kenntnisse, um eine Spring Boot-basierte REST API zu entwickeln und diese als OSGi Plugin zu deployen. Ergänzt wird der Kurs durch langjährige Praxiserfahrung im täglichen Einsatz. Themen wie Token-Authentifizierung, Caching-Strategien für Hochperformante Anwendungen, Multi-Threading-Support, automatisierte Dokumentation dank Swagger uvw. werden im Kurs ausführlich behandelt.

Als Startpunkt für eigene Entwicklungen ist der verwendete Source Code Lizenzfrei enthalten.

Dauer: 3 Tage / Preis 3.500 € zzgl. MwSt. (Unabhängig von der Teilnehmerzahl)

Zielgruppe: Anwendungsentwickler Lotus Notes / Domino. Webentwicklungs- und Java- kenntnisse vorausgesetzt.

Bei Interesse oder Fragen bitte eine Mail an mich (freelancer AT hasselba.ch) oder das Kontaktformular nutzen.

Schulungsangebot – Office 365 & Domino: Anwendungsentwicklung

14. Februar 2019 Posted by Sven Hasselbach

Kurs „Office 365 & Domino: Anwendungsentwicklung“

Der Wechsel auf Office 365 / Exchange ist beschlossene Sache, doch was ist mit den bestehenden Lotus Notes / Domino-Anwendungen?

Im Kurs werden mögliche Anbindungen der Domino-Umgebung an Exchange aufgezeigt und die Entwicklung eigener Office Add-ins erlernt. Mögliche Strategien und Lösungsansätze werden besprochen und durch umfangreiche Erfahrungen aus der Praxis ergänzt.

Als Startpunkt für eigene Entwicklungen ist der verwendete Source Code Lizenzfrei enthalten.

Dauer: 3 Tage / Preis 3.500 € zzgl. MwSt. (Unabhängig von der Teilnehmerzahl)

Zielgruppe: Anwendungsentwickler Lotus Notes / Domino. Webentwicklungs- und Java kenntnisse vorausgesetzt.

Bei Interesse oder Fragen bitte eine Mail an mich (freelancer AT hasselba.ch) oder das Kontaktformular nutzen.

Re: Make sure that the “Names.nsf” cannot be accessed via Internet!

11. Februar 2019 Posted by Sven Hasselbach

Because my comments are still awaiting moderation (tried two times hours ago, but no luck), I have decided to answer to this post from Milan in my blog:

„Yes, it is not good that these passwords are reachable from „outside“, but keep in mind that under normal circumstances the access to the names.nsf is restricted by default, because “Anonymous” is set to “No access”. So, the “exposed data” to the web are the informations which would be accessible for all users when using the Notes client.

The exposed data itself are hashed and salted, so even if the password hashes are reachable this does not mean that this is directly a security nightmare – you should not sleep well, but it is enough time to fix this in the next days. I don’t want to play it down, but the CVE is from 2016 and does not contain an exploit to extract the passwords from the hashes.

For more details, Ben has written a very informational post on passwords and hashes: Deep Dive into IBM Domino Security Part 1: Password Hashes

Also, you can use xACL for protecting these sensitiv data. Here is what IBM wrote about this topic: Securing Internet passwords

Exchange API for Java: Allow *all* type of certificates

29. Januar 2019 Posted by Sven Hasselbach

I had troubles accessing internal Exchange servers using the EWS Java API because of self-signed SSL certificates (and not matching host names), that’s why I created a patch which overrides the existing certificate check and allows all type of SSL certificates.

The code can be found here: https://github.com/hasselbach/ews-java-api

Be aware that this is lowers the security, so you should know what you are doing before using it in productive environments!

#dominoforever?

11. Dezember 2018 Posted by Sven Hasselbach

 https://www.dictionary.com/browse/forever


node.js, domino-db & Docker (13): Logging gRPC requests

29. November 2018 Posted by Sven Hasselbach

For logging the gRPC requests you can use nginx as a reverse proxy. gRPC support was added in version 1.13.10.

To install it locally on your Domino server (RHEL), open a terminal and do the following:

1. Identify which version you are running

cat /etc/redhat-release

The resulting output tells you which major version you are using:

[notes@redhat-dev-hq /]$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.6 (Maipo)

2. Add the nginx repository

sudo vi /etc/yum.repos.d/nginx.repo

3. Add the repo data

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/rhel/7/$basearch/
gpgcheck=0
enabled=1

The „7“ in the base url is the major version from step 1.

4. Install nginx

sudo yum install nginx

5. Create a new config for gRPC

sudo vi /etc/nginx/conf.d/grpc.conf

6. Add the debug log configuration

server {
     listen 81 http2;

     access_log /var/log/nginx/grpc-access.log main;
     error_log /var/log/nginx/grpc-debug.log debug;

     location / {
        grpc_pass grpc://localhost:3002;
     }
}

The reverse proxy is now listening on port 81. You have to change the port in the application configuration. The „error_log“ is our debug log, the „access_log“ logs just the requests itself.

If you are new to vim: First you have to press „i“ to insert text. After pasting the configuration above, press <ESC>, then :w<ENTER>, then :q<ENTER>

7. Test the configuration

[notes@redhat-dev-hq /]$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

8. Start the nginx debug task & enjoy the results

sudo service nginx stop
sudo service nginx-debug start

Now the log file is full with the complete debug information about the incoming gRPC request.

node node.js, domino-db & Docker (12): DominoDB and a big NO-NO?

15. November 2018 Posted by Sven Hasselbach

Disclaimer: This is a response on Heiko’s post about his security considerations with the domino-db module. It is good to have such a discussion, and hopefully this discussion will go on. This is my personal view on this topic. If you have another opinion, feel free to add a comment.

What is gRPC?

gRPC was designed for inter-system communication, and uses HTTP/2 instead of HTTP.

Is it cool?

Yes. And super fast. Millenials will love it. It’s lightweight. Did I mention that it is super fast?

Can I use it in my node.js application for accessing Domino?

gRPC was designed exactly for this purpose. You can also directly use it for connections from a desktop or mobile app, if you want. Or for data access from IOT devices. It may be used directly within the browser in the future (if IBM/HCL gives us access to it.)

Is it safe?

Google developed it for its microservices architecture. If you are not trusting Google’s technical experience, you should shutdown your computer right now. And don’t power it on again.

Should others systems be allowed to access the Proton task directly?

Why not? This is inter-system communication. The traffic is encrypted when using certificates. If you need an additional security layer for limiting access, use a firewall. Or tunnel the traffic with VPN/SSH. This is the typical setup for cloud applications.

The Proton port shouldn’t be reachable from outside

Why not? NRPC is also open. And HTTP, HTTPS, LDAP, SMTP, IMAP, POP3, …

gRPC is bad and voodoo!

Really? What do you think does NRPC stands for? You are using RPC for decades… By the way, which encryption algorithms are you using on your Domino servers for NRPC?

What are theses client certificates?

The certificates are the same as a username / password. Nothing else. And nothing more. This has nothing to do with a Notes ID.

Isn’t it insecure to use client cerificates?

No, because it is the same as when you giving access to your system with username/password. Ever created a webservice provider or a REST API for a 3rd Party system? How do you give these systems access?

But I have to trust a external system…

Sure you need to. Same thing must other systems do when you are connecting to them from Domino. This is the reason why you have to fill out 500 pages and get a sign-off by a long list of involved persons before this is allowed (especially in the financial sector).

I am running a local node.js server on my domino…

Fine. This is still as insecure as running the system somewhere in the cloud. If you are doing the user authentication in your node.js application, you are still making a „insecure“ request to Domino, and Domino has to „trust“ the incoming request.

The client key is stored without password!

If the „other“ system is compromised, it doesn’t matter which kind of authentication was used. Where do you think are they storing username / password for accessing your WebService / REST API?

How to handle user authentication?

This is an open topic. domino-db is still a beta. But this must be solved by IBM/HCL. At least we need a way to run queries „in the name of“ a user.

But you also encrypt the keys…

Yes. But for other reasons: For preventing accidential check-ins in code repositories. And to prevent to store them in backups or direct access from „outside“ by a bug.

I have created a REST API with node.js as a wrapper for gRPC/domino-db

So why did you use the domino-db module? Write it directly on top of Domino, as Servlet or XPages REST API. Then you don’t have any limitations and the authentication problem is solved too.

But this is a secure approach to use it in production!

No. It’s a beta. Don’t use it in production. Period.

You were sceptical about node.js & Domino

Yes, and I am still thinking there is a lot of work to do to use it. But please read again what I have written in my post:

"So far I am still open for a big surprise and hopefully HCL can convince me of the contrary."

node.js, domino-db & Docker (11): Usefull Docker Commands

14. November 2018 Posted by Sven Hasselbach

Building Images

docker build -t domino-express .

Creates a new docker container named domino-express, using the current folder („.“).

Starting / Stopping

docker run --name dominoexpress -p 3000:3000 -d -it domino-express

Starts the image domino-express, allows access to port 3000 and gives the container the static name „dominoexpress„. If option –name is not available, a random name is generated.

Additional parameter -e allows setting of environment variables.

docker stop dominoexpress

Stops the container with the name „dominoexpress

List instances

docker ps

Lists the current running containers.

docker ps -a

Lists all containers, even if they have stoped working / not running.

Delete container

docker rm dominoexpress

Deletes the container with the name „dominoexpress„.

Logs

docker logs dominoexpress

Get the logs of the container „dominoexpress„. Usefull to check why a container crashed.

Getting „inside“

docker exec -it dominoexpress bash

Starts a process (in this case a bash) in the running container „dominoexpress„.

node.js, domino-db & Docker (10): Protecting Proton Keys

12. November 2018 Posted by Sven Hasselbach

Before we are looking into the details how to setup a non-anynomous connection to Domino’s Proton server, I have an advice for protecting the key files required for the connection.

The keys are not password protected, and this is a high risk which should be avoided: First, if you make a mistake or if there is a bug in the software, the keys could be accessed from „outside“. And second, if you check in your code in a repository, the IT security should roast you. It’s clear that the keys are still unprotected if someone hacks the application, but this is a different topic.

First thing to to is to encrypt the keys with AES. To achive this, we are using openssl:

openssl enc -aes-256-cbc -k "0123456789" -in domino-express.key -out domino-express.key.enc

The parameter -k is the key to use for encryption. The path to the unencrypted file is passed with -in, and -out is the path of the encrypted file.

I also encryted the .crt files, which is not required, but it does not hurt either.

The config.js file must now be changed to the follwing (a new method for reading and decrypting the files with openssl is added):

const exec = require('child_process').execSync;

const path = require('path');

/**
 * reads an aes-256-cbc encrypted file & decrypts it
 * 
 * @param {*} fileName 
 *  the encrpyted file 
 * @param {*} key 
 *  the decryption key
 */
const readFileEncrypted = (fileName, key) => {
  try {
    // resolve the filepath
    const filePath = path.resolve(fileName);

    // use openssl to decrypt the keys
    return exec(`openssl enc -d -aes-256-cbc -k "${key}" -in "${filePath}"`, 
    (error, stdout) => {
      if (error) {
        console.error(`exec error: ${error}`);
      }
      return stdout;
    });
  } catch (error) {
    console.error(error);
    return undefined;
  }
};

// the password (stored in environment)
const password = process.env.PASSWORD_KEYFILES;
if (!password) {
  console.error('Environment Variable "PASSWORD_KEYFILES" is not set.');
  process.exit(1);
}

// load the keys & certificates
const rootCertificate = readFileEncrypted('./app/certs/ca.crt.enc', password);
const clientCertificate = readFileEncrypted('./app/certs/domino-express.crt.enc', password);
const clientKey = readFileEncrypted('./app/certs/domino-express.key.enc', password);

// check if the keys are ok 
if (!rootCertificate || !clientCertificate || !clientKey) {
  console.error('Unable to load certificates and/or key.');
  process.exit(1);
}

const Config = {
    serverConfig: {
        hostName: process.env.NODE_ENV === 'development' ? 'dev.example.com' : 'example.com', // Host name of your server
        connection: {
          port: '3002', // Proton port on your server
          secure: true,
        }, 
        credentials: {
            rootCertificate,
            clientCertificate,
            clientKey,
          },
      },
      databaseConfig: {
        filePath: 'testnode.nsf', // The database file name
      }
};

module.exports = Config;

But where to store the key? Nowhere, the key is stored in an evironmental variable.

This line reads the value from the variable PASSWORD_KEYFILES.

const password = process.env.PASSWORD_KEYFILES;

When running your application in a docker container, just start the container with the following command including the variable:

docker run -e "PASSWORD_KEYFILES=0123456789" --name dominoexpress -p 3000:3000 -d -it shasselba/domino-express

To use it in a IDE like Visual Studio Code, you have to add it to the debugging configuration:

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "node",
            "request": "launch",
            "name": "Programm starten",
            "program": "${workspaceFolder}/app/bin/www",
            "env": {"PASSWORD_KEYFILES": "0123456789"}
        },
    ]
}

Now you can add the .enc files to your application in folder /app/certs without worrying about it. If the variable is not set, the application terminates with error code 1.

P.S. Don’t forget to remove the original key files from your project!

node.js, domino-db & Docker (9): Global Configurations

9. November 2018 Posted by Sven Hasselbach

The database configuration should not be changing during the different requests, that’s why it is a good idea to store the configuration in a central place of our express application. There are multiple ways of doing this, e.g. you can use app.locals for this. Or you can use the app.set method.

But the best option in my eyes is to have a global configuration file, because using the methods above requires access to the app object (see below how this works).

Using a global configuration file

1. Create a new file in the /app folder named config.js

2. Add a global configuration object

const Config = {};

module.exports = Config;

3. Add the configuration for our database

const Config = {
    serverConfig: {
        hostName: process.env.NODE_ENV === 'development' ? 'dev.example.com' : 'www.example.com', // Host name of your server
        connection: {
          port: '3002', // Proton port on your server
        },
      },
      databaseConfig: {
        filePath: 'node-demo.nsf', // The database file name
      }
};

module.exports = Config;

The configuration above checks the environment variables. In development mode, the dev.example.com server is used, otherwise www.example.com.

4. Import the file in your code

const config = require('../config');

5. Use it by deconstructing the values

const { serverConfig, databaseConfig } = config;

This can be used everywhere in your application.

Using app.set

1. Open the file app.js in the /app folder

2. Add the following code before the module.export statement:

app.set('db', {
  serverConfig: {
    hostName: 'your.server.com', // Host name of your server
    connection: {
      port: '3002', // Proton port on your server
    },
  },
  databaseConfig: {
    filePath: 'node-demo.nsf', // The database file name
  }
});

3. If you want to have different configurations depending of development / productive environment, you can compute the property like this:

...
hostName: app.get('env') === 'development' ? 'your.devserver.com' : 'your.server.com',
...

4. To use the configuration in the router, just use the app.get method (provided by the request object) and deconstruct the values:

router.get('/', (req, res) => {
  const { serverConfig, databaseConfig } = req.app.get('db');
  ...

node.js, domino-db & Docker (8): Security

9. November 2018 Posted by Sven Hasselbach

Security is a big topic when developing node.js applications. A simple helper for writing secure code is the plugin. It checks for common mistakes during writing code, for example using the eval statement with external input, or unsafe RegEx expressions…

To install the plugin, just save it to the project with

npm install --save-dev eslint-plugin-security

To enable it, you need to change the .eslintrc configuration file:

{
  "plugins": ["security"],
  "extends": [
    "plugin:security/recommended",
    "rallycoding"
  ]
}

node.js, domino-db & Docker (7): The ValueHolder

7. November 2018 Posted by Sven Hasselbach

I am using this for years in Java, so I thought it would be great to use this approach also in the JavaScript world: The ValueHolder. The class allows to easily define „cachable“ code and it’s result, without having to handle the memcached part and – maybe in the future – background processing stuff.

To give you an idea what it is for here is a small example:

const allDummyDocs = new ValueHolder('allDummyDocs', 60, async () => {
  // get all documents with the Form 'dummy'
  return useServer(serverConfig).then(
    async server => {
      const db = await server.useDatabase(databaseConfig);
      const response = await db.bulkReadDocuments({
        query: "Form = 'dummy'"
      });
      return JSON.stringify(response);
    }).catch(err => {
      console.log(err);
      return err;
    });
});

The first parameter is the key used to store/retreive the value from memcached. The second one is the time how long the value should be cached. And the third parameter is the code to execute.

To use the definition in the application, you now have to use the get method of the value holder:

router.get('/showAllDummyDocs', (req, res) => {
  allDummyDocs.get(
      (error, result) => {
        if (error) {
          res.render('error', { title: 'Error', error });
        } else {
          res.render('index', { title: 'Express', result: `Result: ${result}` });
        }
      }
    );
  }
);

The ValueHolder checks now automatically, if the result is stored in the cache. If not, the code is executed and stored in the cache.

Here is the ValueHolder.js file (which has to be created in the /app/classes folder):

const mf = require('../classes/MemcachedFactory');

const nullHelper = '###NULL###';
/**
 * Helper class for cached values
 * 
 * @author Sven Hasselbach
 * @version 0.1
 */
class ValueHolder {

    /**
     * 
     * @param {string} key 
     *  unique identifier
     * @param {number} ttl
     *  time-to-live in seconds 
     * @param {function} code 
     *  code to execute to calculate the value
     */
    constructor(key, ttl, code) {
        this.key = key;
        this.code = code;
        this.ttl = ttl;
    }

    /**
     * loads the value from cache or 
     * computes it and stores it in the cache
     * 
     * @param {function} callback 
     * @returns Promise
     */
    async get(callback) {
        const { code, ttl, key } = this;
    
        // check if value is in cache...
        mf.getInstance().get(key, (error, value) => {
            if (error) {
                callback(error);
                return;
            }
            if (value != null) {
                console.debug(`Found '${key}' in cache.`);
                if (value === nullHelper) {
                    // result is "special", so let's return null
                    callback(error, null);
                } else {
                    console.log(value);
                    callback(error, JSON.parse(value));
                }
            } else {
                console.debug(`Computing '${key}' and adding to cache with ttl ${ttl}.`);

                // execute the computation
                code().then((result) => {
                    // check if result must be stored "special" or not
                    if (result === null) {
                        mf.getInstance().set(key, nullHelper, ttl);
                    } else {
                        mf.getInstance().set(key, JSON.stringify(result), ttl);
                    }   
                    callback(error, result);
                });
            }
        });
     }

}

module.exports = ValueHolder;

node.js, domino-db & Docker (6): Using memcached

7. November 2018 Posted by Sven Hasselbach

mem.js

I am using mem.js as client library for accessing memcached. To use it, the first thing to do is to add the requirement to your package.json:

npm install memjs --save

MemcachedFactory

Then we can create a simple helper class to have an abstraction layer between our code and the library itself.

1. Create a folder in /app named classes

2. Create a new file with the name MemcachedFactory.js

3. Add the following code:

const memjs = require('memjs');
/**
 * Helper class for using Memcache
 * 
 * @author Sven Hasselbach
 * @version 0.1
 */
class MemcachedFactory {

    constructor() {
      this.client = memjs.Client.create('127.0.0.1:11211');
    }

    /**
     * returns a single instance of the class
     */
    static getInstance() {
      if (this.instance == null) {
        this.instance = new MemcachedFactory();
      }
      return this.instance;
    }

    /**
     * stores a value in memcache
     * @param {string} key 
     *  the key used
     * @param {*} value
     *  the value to store
     * @param {number} ttl 
     *  time-to-live in seconds
     */
    set(key, value, ttl) {
      this.client.set(key, value, { expires: ttl }, err => {
        if (err) {
          console.log(err);
          throw err;
        }
      });
    }
    /**
     * gets a value from memcache
     * 
     * @param {string} key 
     *  the key used
     * @param {function} callback
     *  the callback containing the value
     */
    get(key, callback) {
      this.client.get(key, (err, value) => {
        if (err) {
          console.error(err);
          callback(err);
        }
        if (value == null) {
          callback(err, null);
        } else {
          callback(err, value.toString());
        }
      });
  }
}
module.exports = MemcachedFactory;

5. To use the class in our code, we have to add the requirement first:

const mf = require('../classes/MemcachedFactory');

6. Here is a small example how the class is used:

mf.getInstance().get(key, (error, value) => {
  if (error) {
    // handle error here
  }else{
    // we have a value
    console.log(`The value is ${value}`); 
  }
});

The getInstance method returns an instance of the class. Then the get method is used with the key to retreive, and a callback method which is called when the request to memcached is completed.

The set method allows to put a key to memcached, and with ttl we can define how long the key is valid and stored.

node.js, domino-db & Docker (5): memcached

5. November 2018 Posted by Sven Hasselbach

To use memcached in our Docker container, we have to modify the existing Dockerfile a little bit. First it is required to install memcached in the container itself, and then it is required to change the CMD command to start the service and our express application.

1. Create a folder /conf in the project folder

2. Create a file memcached.conf in this folder with the following content:

# Memory: 256 MB
-m 256

# Port to use
-p 11211

# User for the service
-u memcache

# Listening IP Adress 
-l 127.0.0.1

3. Create a script startup.sh in the newly created folder:

#!/bin/bash
service memcached start
npm start

4. Modify the Dockerfile to install memcached

# install memcached
RUN apt-get update
RUN apt-get install -y memcached

5. Copy the startup.sh script and the memcached.conf

COPY ./conf/memcached.conf /etc
COPY ./conf/startup.sh .
RUN chmod +x /usr/src/app/startup.sh

6. Change the CMD to use the startup.sh script

CMD [ "./startup.sh" ]

The reason for the startup script is that in Docker containers the services are disabled by default, so we need to start the service by our own. Also, only one CMD command is allowed, so we have to use a script.