Configuring SnapLogic

This topic provides steps to configure SnapLogic conversion stage.

Select Target Type as Spark to which you need to transform the source scripts.
Click Data Configuration.

In Data Interaction Technique, select your data interaction method. Following are the options:

Spark-Native: Select Spark-Native to fetch, process, and store data in Hive.
Spark-Native: External: Select this data interaction technique to fetch input data from an external source such as Oracle, Netezza, Teradata, etc., and process that data in Spark, and then move the processed or output data to an external target. For instance, if the source input file contains data from any external source like Oracle, you need to select Oracle as the Databases to establish the database connection and load the input data. Then data is processed in Spark, and finally the processed or output data gets stored at an external target (Oracle). However, if you select Oracle as the Databases but the source input file contains data from an external source other than Oracle, such as Teradata, then by default, it will run on Spark.

In Source Database Connection, select the database you want to connect to. This establishes the database connection to load data from external sources like Oracle, Teradata, etc. If the database is selected, the converted code will have connection parameters (in the output artifacts) related to the database. If the database is not selected, you need to add the database connection details manually to the parameter file to execute the dataset; otherwise, by default, it executes on Spark.

In Output Type, select Python 2 or Python 3 as the output type format for the generated artifacts.
Click Save to update the changes.

An alert pop-up message appears. This message prompts you to refer your respective assessment to determine the anticipated quota deduction required when converting your scripts to target. Then click Ok.

Click to provide a preferred pipeline name.
Click to execute the pipeline. Clicking (Execute) navigates you to the listing page which shows your pipeline status as Running state. It changes its state to Success when it is completed successfully.
Click pipeline card to see report.

To view the SnapLogic conversion report, visit SnapLogic Conversion Report.