Configuring Talend – Leaplogic

Configuring Talend

This topic provides steps to configure the Talend ETL conversion stage.

Select the ETL Type as Talend.
In Input Artifacts, upload the source data via:
- Browse Files: To select the source files from the local system.
- Select From Data Source: To select the source files from the data source. To do so, follow the steps below:
  - Click Select From Data Source.
  - Choose repository.
  - Select data source.
  - Select the entities.
  - Click to save the source data source.

Select Target Type as AWS Glue Job, Databricks Lakehouse or Spark to which you need to transform the source scripts.
Click Data Configuration.

The Input column in the table below provides the input requirements based on the Target Type selection.

Target Type	Input
AWS Glue Job	In Source Database Connection, select the required source database to load the data such as Oracle, SQL Server, Teradata, Netezza, DB2, Vertica. In Attainable Automation, select the way you want the system to calculate achievable automation for the transformation of the source scripts. Assessment-Based: Calculates the level of automation based on assessment logic. The conversion-config.json file contains a pre-defined automation percentage for each component and you can also modify it as required. Transformation-Based: Calculates the level of automation based on actual conversion. In this method, automation percentage is calculated for each component based on the used, supported, and unsupported properties. In S3 Bucket Base Path, provide the S3 storage repository path where you need to store the source and target files. In Artifacts Location, specify the location from where you need to call external files such as parameter files, orchestration scripts.
Databricks Lakehouse	Choose the Output Type as Python 3. In Source Database Connection, select the required source database to load the data such as Oracle, SQL Server, Teradata, Netezza, etc. In Attainable Automation, select the way you want the system to calculate achievable automation for the transformation of the source scripts. Assessment-Based: Calculates the level of automation based on assessment logic. The conversion-config.json file contains a pre-defined automation percentage for each component and you can also modify it as required. Transformation-Based: Calculates the level of automation based on actual conversion. In this method, automation percentage is calculated for each component based on the used, supported, and unsupported properties. In DBFS File Base Path, specify the DBFS (Databricks File System) location where you need to fetch the input files and store the transformed data. In other words, it is a base path for input files and output data.
Spark	Choose the Output Type as: Python 2 Python 3 In Source Database Connection, select the required source database to load the data such as Oracle, SQL Server, Teradata, Netezza, DB2, Vertica. In Attainable Automation, select the way you want the system to calculate achievable automation for the transformation of the source scripts. Assessment-Based: Calculates the level of automation based on assessment logic. The conversion-config.json file contains a pre-defined automation percentage for each component and you can also modify it as required. Transformation-Based: Calculates the level of automation based on actual conversion. In this method, automation percentage is calculated for each component based on the used, supported, and unsupported properties. In File Base Path, specify the base path for input files and output data. In Artifacts Location, specify the location from where you need to call external files such as parameter files, orchestration scripts. Choose the Validation Type - None or Cluster. If the Validation type is Cluster, upload the Data source.

Click Save to update the changes.
An alert pop-up message appears. This message prompts you to refer your respective assessment to determine the anticipated quota deduction required when converting your scripts to target. Then click Ok.

Click to provide a preferred pipeline name.
Click to execute the pipeline. Clicking (Execute) navigates you to the listing page which shows your pipeline status as Running state. It changes its state to Success when it is completed successfully.
Click pipeline card to see report.

To view the Talend conversion report, visit Talend Transformation Report.