Talend Assessment Report

This topic contains information about the Talend assessment report. The assessment assesses workloads and produces in-depth insights that help to plan the migration. The input format for the Talend assessment is ITEM files.

In This Topic:

Highlights
- Summary
- Complexity
Analysis
- Source Analysis
Lineage
Downloadable Reports

Highlights

The highlights section gives you a high-level overview of your assessment summary of the analytics performed on the selected workloads. It includes a graphical depiction of the complexity of files as well as the summary of the files used.

Summary

This section illustrates the input Talend files that were analyzed across the various components. Here you can see the number of files, components, and connections as well as queries from the input workload.

Complexity

This section provides a summarized graphical representation of the complexity of files that helps in making different decisions, including budget estimation.

Analysis

This topic provides a detailed examination of source files and artifacts.

Source Analysis

This section provides a comprehensive report of the source files including information about each file’s components, complexity, connections, and so on, as well as the total number of files.

File Name: Name of the file.
Source Databases: Provides the type of source databases.
Components: Used to carry out certain tasks or jobs such as, read data component, get data component, write data component, etc.
Connections: Interface to connect between components.
Expressions: Used to accomplish custom tasks or jobs. The custom expressions can be written in the Expression editor of the tMap component.
Commands: To perform specified functionalities.
Queries: Provides the number of queries in the file.
Complexity: Provides the complexity of the file.

Lineage

End-to-end data and process lineage identify the complete dependency structure through interactive and drill-down options to the last level.

Typically, even within one line of business, multiple data sources, entry points, ETL tools, and orchestration mechanisms exist. Decoding this complex data web and translating it into a simple visual flow can be extremely challenging during large-scale modernization programs. The visual lineage graph adds tremendous value and helps define the roadmap to the modern data architecture. It deep dives into all the existing flows, like Autosys jobs, applications, ETL scripts, BTEQ/Shell (KSH) scripts, procedures, input and output tables, and provides integrated insights. These insights help data teams make strategic decisions with greater accuracy and completeness. Enterprises can proactively leverage integrated analysis to mitigate the risks associated with migration and avoid business disruption.

Now, let’s see how you can efficiently manage lineage.

To view the required lineage:

Select either the Process or Data tab to enable process or data lineage respectively.
Enter the keywords in the Search Keywords field you want to search. Else, simply select the entities, jobs, databases, etc., from the Search dropdown.
To generate column-level lineage, turn on the Include Columns toggle (available only in Data Lineage).

Click the Search icon to generate the lineage.

Lineage helps you understand dependencies across processes, tables, and columns:

Process lineage illustrates the dependencies between two or more processes such as entities, jobs, etc.
Data lineage depicts the table-level dependencies and column-level dependencies–input tables, output tables, and reference tables.
Column-Level lineage shows granular transformations and updates at the minutest level.

In addition, the filter search icon allows you to include or exclude particular nodes to obtain the required dependency structure. It helps you quickly narrow down complex dependency graphs to focus on the exact nodes, relationships, or data flows you want to analyze. Instead of navigating through an entire lineage network, you can apply filters based on object names, types, or relationship categories—making it easier to isolate specific tables, workflows, or transformation paths. This feature enhances traceability, reduces visual clutter, and enables you to precisely identify upstream or downstream dependencies that are most relevant to your analysis or migration scope.

You can also choose the direction of the lineage. By default, the Dependency Direction is Left to Right Hierarchy. This insight is useful to identify and analyze upstream data connections and relationships. You can also choose Right to Left Hierarchy – most useful for how data is getting consumed through the downstream route and which are the participating tables and columns, or Bidirectional dependency direction to visualize both kinds in a single view.

Moreover, you can also increase the Hierarchy Levels to nth level. This again allows great freedom to visualize the end-to-end hierarchy in one go.

Lineage facilitates you visualize how your selected nodes are connected and depend on each other. The nodes and their connecting edges (relationships) help you to understand the overall structure and dependencies.

Nodes	Edges
Tables	Call
File	Read
Job	Execute
Autosys Box	Write
Flat File	OTHERS
DB_TYPE

Manage Lineage

This feature enables you to view and manage your lineage. You can add, modify, or delete nodes and their relationships to generate an accurate representation of the required dependency structure. There are two ways to update the lineage: either using Complete Lineage report or Lineage Template.

Using Complete Lineage report

Follow the below steps to modify the lineage:

Click the Manage Graph icon.

Click Download Complete Lineage to update, add, or delete the nodes and their relationships in the current lineage.

Once the complete lineage report is downloaded, you can make necessary updates such as updating, deleting or adding the nodes and its relationships.
After making the required changes, upload the updated lineage report in Upload to Modify Lineage.
Click Apply to incorporate the updates into the dependency structure.
Generate the required process or data lineage.

Using Lineage Template

Follow the below steps to add new nodes and their relationships to the current lineage report:

Click the Manage Graph icon.

Click Download Lineage Template.

Once the lineage template is downloaded, you can add new nodes and relationships in the template.
After making the required changes, upload the template in Upload to Modify Lineage.
Click Apply to incorporate the updates into the complete dependency structure.
Generate the required process or data lineage.

Important

To effectively manage lineage, you must adhere to the following rules:

Do not modify the column headers or their order in the Complete Lineage report or the Lineage template. The following are the column headers: source_name, source_type, target_name, target_type, relation_type, and database_id.
When deleting a row, retain the value in the database_id column and clear all the other column values.
When inserting a row, add all the column values except in the database_id column.
When updating a row, ensure that all the columns have relevant data.

You can also apply:

Feature	Icon	Use
Filter		Used to filter the lineage.
Reload Graph		Assists in reloading graphs.
Manage Graph		To view and manage lineage by adding, modifying, or deleting nodes and their relationships to accurately reflect dependency structures.
Save		Used to save the lineage.
Download		Used to download the file.
Expand		Used to enlarge the screen.

Downloadable Reports

Downloadable reports allow you to export detailed assessment reports of your source data which enables you to gain in-depth insights with ease. To access these assessment reports, click Reports.

Types of Reports

In the Reports section, you can see various types of reports such as Insights and Recommendations, Source Inventory Analysis, and Lineage Analysis reports. Each report type offers detailed information allowing you to explore your assessment results.

Insights and Recommendations

This report provides an in-depth insight into the source input files. It contains the final output including the details of queries, complexity, jobs, and so on.

Here, you can see the talend folder along with Talend Report.xlsx.

Talend Report.xlsx: This report provides insights about the source inventory. It helps you plan the next frontier of a modern data platform methodically. It includes a report summary, aggregated inventory, file summary, job summary, and more.

This report contains the following information:

Report Summary: Provides information about all the generated artifacts.
Volumetric Info: Presents a summary of the aggregated inventory after analyzing the source files. For instance, it provides volumetric info about the total number of jobs, components, queries, and likewise. It also provides job and query-level complexity.
Job Summary: Lists all the jobs associated with the input files. It also provides information about the conversion coverage, count of queries, components, length of expressions, complexity, and a lot more.
Component Summary: Lists all the components in the jobs along with component types, expressions, length of expressions, tables, and more.
Connection Summary: Connections are interfaces to connect components. Lists all the connections along with information about files, jobs, meta names, source components, and target components.
Context EVN Summary: Lists all the jobs and the associated environments in which the jobs are expected to run.
Component Pattern Summary: Provides details about components with similar patterns along with the number of occurrences.
Job Pattern Summary: Provides details about jobs with similar patterns along with the number of occurrences.
Complexity Summary: Provides the job complexity details.

To access the Talend Queries.csv file, browse through talend > query.

Talend Queries.csv: This report provides information about queries including the used and impacted tables, analyzed status, complexity, and more. If the analyzed status is Analyzed, it indicates that the query is analyzed successfully. Conversely, a Not Analyzed status indicates that the query is not analyzed.

Source Inventory Analysis

It is an intermediate report which helps to debug failures or calculate the final report. It includes all the generated CSV reports along with unparsed ETL files, invalid query, and procedures reports.

keywordSearch.csv: Provides detailed insights into keyword occurrences— specifically jobs—identified across all uploaded source files. It captures information such as the search type, the file where the keyword was found, the line number of occurrence, and the actual line content where the keyword appears.

license_quota_info.csv: This report provides information about the anticipated license quota deduction when executing a transformation pipeline or notebook using the same source input file that was used during the assessment. It includes details about the expected quota consumption for units, blocks, and scripts.

Lineage_Raw.xlsx: This report provides complete dependency details for all nodes. It provides an end-to-end data and process lineage that helps to identify the complete dependency structure and the data flow.

This report contains the following information:

Volumetric Info: Provides volumetric information about the artifact types such as tables, files, talend jobs, and more.
Nodes: Lists all the source and target nodes along with their type. Each node represents a data object in the lineage—such as a table, file, job, etc.—making it easier to trace how data is consumed, transformed, and processed across the data flow.
Dependency (Process): Provides information about the process lineage. It offers detailed visibility into interdependencies between processes—such as tables, jobs, etc.,—helping you understand how they are connected within the data flow.
Dependency (Data): Provides information about the data lineage. It captures detailed table-level—including input tables, output tables, and reference tables—offering end-to-end visibility into how data flows and transforms across the data flow.
Dependency (Data Model): Provides dependency details about the data models. It highlights the end-to-end relationships and dependencies between model elements, helping you understand structure and trace linkages.

To access various outputs such as Assessment_unparsed_etl_files.csv, external_procedure_detail.csv, invalid_query.csv, External Executable File.csv, and more, navigate through the etl > talend folders.

Assessment_unparsed_etl_files.csv: This report lists all the unparsed Talend files along with its type and the reason for parsing failure.

external_procedure_detail.csv: Provides information about external procedure files. It includes information about components, procedures, and its availability.

invalid_query.csv: This report lists all the invalid queries.

Browse through the csv folder to access External Executable File.csv and External Procedures.csv reports.

External Executable File.csv: Provides information about external executable files including jobs, executable file paths, commands, and more.

External Procedures.csv: Provides information about external procedures files. It includes information about files and components.

Lineage Analysis

This section provides lineage-related reports, including entity_link.csv, entity_report.csv, entity_summary.csv, link.csv, script_report.csv reports.

entity_links.csv: This report provides information about how views are connected to entities or tables and how these links extend across multiple levels. Level 1 shows the immediate table to which a view is linked. If that table is further connected to another entity, the next connection appears in Level 2, and so on.

entity_report.csv: This report provides detailed lineage information for each entity within the uploaded source files. It provides a comprehensive list of all entities along with their respective types, identifies the processes or scripts that read from or write to each entity, and includes other dependency details.

entity_summary.csv: This report provides information about each entity along with the type of relation in which it appears—such as database type, Talend job—and the operation performed in that context, including Read, Write, or ReadWrite.

link.csv: This report provides information about entities linked to each view.

script_report.csv: This report provides detailed lineage information for each script. It lists all scripts along with their types, reads data from or writes data to a process or script, and includes other dependency details.