Step 4. Configure the Recommendation Settings

Recommendation settings provide configuration options to baseline the recommendations for transformation and target platform-specific recommendations.

Based on the threshold complexity values ideal migration candidates are identified. By default, Queries’ complexity value above 1 and files’ complexity value above 2 are recommended for migration. You can customize these values if necessary.

The files and queries are recommended for migration if the complexity values are above the threshold values. The minimum and maximum complexity values vary from 0 to 10.

Steps:

Prerequisites
Step 1. Provide Primary Inputs
Step 2. Add the Additional Inputs
Step 3. Update the Source Configuration
Step 4. Configure the Recommendation Settings

Note:

Files’ threshold complexity filter values: A combination of filter values is available for different parameters such as CPU utilization, I/O utilization, query complexity, and execution time for OLAP queries.

Queries’ threshold complexity filter values: A filter for generating recommendations based on the query complexity.

Follow the below steps to configure the Recommendation Settings:

To transform the DML queries into the target equivalent queries, turn on Code Transformation toggle.

In Orchestration Optimization, choose the split size from the sliding bar. The split size defines the maximum number of queries that can be executed in parallel for orchestration optimization.
In Presets, choose Default or Custom to configure the thresholds that filter the baselines for the recommendation of ideal migration candidates from the set of uploaded artifacts.
- Default: To apply a combination of default filter values for queries and files.
- Custom : To customize the filter values for queries and files such as CPU utilization, I/O utilization, query/ file complexity and execution time for OLAP queries.

Additional Information:

If the selected Target Data Source is Databricks, you can configure the Schema Optimization Settings. The Schema Optimization Settings allow you to set the data volume thresholds based on which the optimizations are recommended for the delta tables.
To configure the Schema Optimization Settings, choose the unit (KB, MB, GB, and TB) of threshold values from the respective threshold criteria (MB is the recommended threshold value unit) and then adjust the threshold values from the sliding bar. The Schema Optimization Settings threshold criteria include:
- Tune Based on Workload: If a Delta table experiences frequent MERGE, UPDATE, or DELETE operation, then this helps to tune file sizes for rewrites by setting the table property delta.tuneFileSizesForRewrites.
- Others:
  - Tune Based on Table Sizes: Helps to configure the target file size based on source table size, for example, for 1 TB table size, the size of the target file should be 256 MB; for 10 TB, it should be 1 GB and likewise.
  - Compaction (bin-packing): Delta Lake on Databricks improves the speed of read queries from a table by coalescing small files into larger ones.
If the selected Target Data Source is AWS, you can configure the Schema Optimization Settings. The Schema Optimization Settings allow you to set the data volume thresholds based on which the optimizations are recommended for the delta tables. To configure the Schema Optimization Settings, choose the unit (KB, MB, GB, or TB) of threshold values from the respective threshold criteria and then adjust the threshold values from the sliding bar. The Schema Optimization Settings threshold criteria includes:
- Sortkey: Specifies the data storage method for a 1MB disc block. Sorting enables efficient handling of range-restricted predicates. The min and max values for each block are stored in the metadata.
- Distkey: DIST Keys (Redshift Distribution Keys) determine where data is stored in Redshift cluster nodes.
- Compression Encodings: If there are any compression on source column, you need to add compression encoding on Redshift tables
- Analyze Command: Run ANALYZE command to collect statistics for tables that are used in the following types of queries:
  - Table used together most frequently
  - Tables on which WHERE predicates are applied
  - Tables with low cardinality columns (duplicate rows)

Click Execute to get comprehensive insights about your workloads excluding end-to-end data and process lineage. To include end-to-end data and process lineage, click on dropup list on the Execute button and click Comprehensive.

If the Automation Coverage toggle is enabled an alert pop-up message appears. This message displays that enabling Automation Coverage requires more time for assessment execution. If you want to proceed, click Yes or else No.

Execute

Status	Description
Running	A request to perform the task has been received and it is in processing.
Failed	The assessment has failed.
Success	The assessment has been executed successfully.
Queued	Waiting in line and making continuous attempts to execute the task.
Saved	A reserved task that is ready for execution.

Note:

ETL workload assessment gives the privilege to execute the assessment in 3 different ways based on your requirements.
- Comprehensive: Provides comprehensive insights about your workloads. It includes highlights of the input source scripts and assesses the existing inventory and provides comprehensive complexity analysis and end-to-end data and process lineage.
- Inventory (Dashboard): Provides quick assessment of the source scripts. It showcases the high-level summary of the existing inventory without performing query analysis.
- Exclude Lineage: To get comprehensive insights about your workloads excluding end-to-end data and process lineage.

Click your assessment to see the details of the analyzed workloads and to download reports.

Note:

Use Google Chrome, or Mozilla Firefox, or Safari web browsers for the best experience of all LeapLogic features and functionalities. Other than these three, the product has not been thoroughly tested on any other browser.
When you license LeapLogic, it comes with a certain quota limit, for example upload size quota, or number of queries which can be assessed or transformed, and likewise. The system alarms the user when the limit exceeds and highlights quota exhaustion message.
Databricks Delta liquid clustering is supported in the following Databricks Runtime versions:
- Databricks Runtime 15.2 and above: Full support for all tables with liquid clustering enabled.
- Databricks Runtime 13.3 LTS and above: Public preview support with limitations.