Advanced Configuration (Optional)

This section covers the steps to configure a few optional components as required.

HTTPS mode
Ab Initio ETL Enablement to deploy Ab Initio component
Workload Assessment Performance Tuning for bulky workloads
Onboarding of a new tenant in Governance application
Configuring LDAP authentication mechanism
Steps for upgrading the LeapLogic build
Enable automated service monitoring and rectification

HTTPS Mode

By default, LeapLogic services run in HTTP mode. LeapLogic also supports HTTPS mode. This section covers the process of configuring HTTPS for LeapLogic services. You may like to either install your SSL certificate or create a new one.

Follow the below steps to enable a valid SSL certificate.

Prerequisites: The application must be running on the domain for which the certificate has been issued and the value of the IDW_HOST env variable will be the domain name itself.

Configure the below properties in the idw-env.sh file.

PROPERTY NAME	DESCRIPTION
WEB_PROTOCOL	By default, the property is set to “http”. Update the value to: “https”
IS_HTTPS_ENABLED_WM	By default, the property is set to “false”. Update the value to: “true”
IS_WM_HTTP_PORT_ENABLED	By default, the property is set to “true”. Update the value to: “false”

Configure the below property in the idw-default-env.sh file.

PROPERTY NAME	DESCRIPTION
KEYSTORE_DEFAULT_ALIAS	Add the alias name of the certificate. By default, the property is set to “hostname -f”. Update it as per your hostname or IP with which you will login.

Import the certificate into the Java Keystore
Rename the certificates to wm-tomcat.cer and wm-tomcat.jks, if required.
Place both the files (.cer and .jks) in the directory: IDW_HOME/conf/

Note:

If the SSL certificates are created using the automated script, then skip the above step.

Set the password in idw-default-env.sh KEYSTORE_DEFAULT_PASSWORD=”password”. By default, it is “password” only.

Note:

If you have generated the SSL certificate on your own, please configure the password which you have given while generating the certificate.

Proceed with the normal deployment process by executing configure.sh, init.sh

Note:

In case of self-signed certificate, follow the steps to create the self-signed certificate given below. However, if you already have the required certificates directly execute enable-https.sh and start-all.sh scripts.

Following are the steps to create a self-signed HTTPS certificate for LeapLogic services.

The IDW_HOME/bin folder for IDW directory contains enable-https.sh script file. Execute the enable-https.sh file to configure HTTPS for LeapLogic services.

This script does the following:

Updates the LeapLogic services as per the HTTPS configuration.
Configures LeapLogic to use the Keystore file created in the previous step.

${IDW_HOME}/bin/enable-https.sh

Note:

If you are deploying with the property export IS_HADOOP_CLUSTER_PRESENT=”true”;

Then after executing ./enable-https.sh, execute the below two commands.

chmod +x idw-deployment.sh
./idw-deployment.sh

Lastly, start all the services.

${IDW_HOME}/bin/start-all.sh

AbInitio ETL Enablement

This section provides information for the deployment of the AbInitio ETL suite. Before deployment, certain prerequisites need to be taken care for a hassle-free experience.

Prerequisites

dos2unix should be installed on the edge node.
The edge node should be accessible using SSH.
wget command must be available on the edge node.

Deployment

Open and edit the AbinitioUDFRegister.hive file from the path IDW_HOME/application/workload-migration/lib/abinitio-external-libs/hive_udf. Replace ${HDFS_LIB_DIRECTORY} with hdfs:///tmp. Following this, create your Hive schema and use it to execute the updated script to register the UDFs.

Steps to register the UDFs in the Hive database:

Create a new database in Hive
Navigate to the following location IDW_HOME/application/workload-migration/lib/abinitio-external-libs/hive_udf
Update the required variables and schema name in registerUdf.sh file and execute it

Workload Assessment for Large Files

Workload Assessment engine needs some configurational changes for efficient performance.

Following are the lists of properties that should be modified if the size of query execution logs is too large.

Modify IDW_HOME/application/assessment/bin/workload-assessment-execution.sh
1. -Xms{file size * 1/4} for example -Xms3g for a file of size 10 GB
2. -Xmx{file size * 1/2}
3. -XX:MetaspaceSize = {file size * 1/15}
4. -XX:MaxMetaspaceSize = {file size * 1/10}
Modify IDW_HOME/application/assessment/conf/assessment-site.properties
1. spark.executor.instances = {file size * 1/4}
2. spark.executor.memory = {file size * 1/4}
3. spark.driver.memory={file size – (file size * 1/4)}
4. spark.sql.shuffle.partitions = {file size * 5}
5. assessment.cleanser.querylog.partition = {file size * 20}
6. assessment.analyzer.querylog.partition = {file size * 20}
7. assessment.reporting.partition = {file size * 14}
8. spark.addtional.properties = spark.yarn.executor.memoryOverhead ={file size * 1/5}

If the value is in decimal, round it off to the nearest whole number.

New Tenant Onboarding

Onboarding a new tenant in the LeapLogic application is easy and configurable. Follow the below given steps to do the same.

export TENANT_NAME=””;
export TENANT_DESCRIPTION=””;

LDAP Authentication

While LeapLogic is deployed, the authentication mechanism is typically configured only once, where you have two options: Authentication through a database or through LDAP.

Authentication through Database

This section provides the steps to configure user authentication through the database on the customer’s premises. This is the default authentication type. By default, a user namely “idwadmin” is already created.

Authentication through LDAP (Optional)

This section provides the steps to configure user authentication through LDAP within an enterprise.

Configure the below property for enabling user authentication through LDAP

export IS_LDAP_ENABLED=true

Note:

This feature offers a variety of configurations as given below, however, it has limited capabilities and will work in certain scenarios only such as where administrator access can be provisioned, etc.

The properties that need to be configured in the idw-env.sh file for LDAP are as follows.

Property Name	Description
export LDAP_CONNECTION_URL=VALUE_LDAP_CONNECTION_URL	e.g. ='ldap:// < LDAP server host > :,< LDAP server port >'
export LDAP_ROOT_BASE=VALUE_LDAP_ROOT_BASE	e.g. CN=Users, DC = < LDAP Directory Distinct Name Value > DC=com
export LDAP_MANAGER_DN=VALUE_LDAP_MANAGER_DN	e.g. cn= < Manager User Distinct Name >, CN= Users, DC=< LDAP Directory Distinct Name Value > DC=com
export LDAP_MANAGER_PASSWORD=VALUE_LDAP_MANAGER_PASSWORD	Manager User Password
export LDAP_USER_BASE_DN=VALUE_LDAP_USER_BASE_DN	e.g. cn={0},CN=Users, DC= < LDAP Directory Distinct Name Value > DC=com
export LDAP_USER_DEFAULT_COMPANY=VALUE_LDAP_DEFUALT_COMPANY	Organization name
export DEFAULT_ADMIN_USERNAME=${IDW_DEFAULT_USER}	This property needs to be set only when LDAP properties are configured. The value for this property needs to be set as 'Administrator' if the LDAP flag is true

Build Upgrade

Following are the steps to upgrade the LeapLogic build from one version to another.

Navigate to the <Deployment folder>/bin/upgrade
Set the permission

chmod -R 777 upgrade

Update the path of old and new deployment in the upgrade-env.sh file
1. export OLD_DEPLOYMENT_PATH=?
2. export NEW_DEPLOYMENT_PATH=?

Execute the following command to start the upgrade process

./upgrade.sh

Service Monitoring, Alerts and Restart

This feature allows any service which is down to get restarted on its own. It also sends out a notification on the configured email id.

PROPERTY NAME	DESCRIPTION
MONIT_ENABLED	Enables monitoring and automated rectification when configured as “true”. Can also be set as “false”
MONIT_PORT	Port on which this runs e.g. 8686
MONIT_EMAIL_TO_NOTIFY	The email where an automated notification will be sent in case any service is down e.g. info@leaplogic.io