AutoSys Assessment Report
This topic contains information about the AutoSys assessments report. AutoSys is an automated job scheduling tool that controls scheduling, monitoring, and executing tasks. The input formats for AutoSys assessment are TXT or JIL.
In This Topic:
Highlights
The highlights section gives you a high-level overview of your assessment summary based on the analysis of the selected scheduler scripts. It includes a graphical depiction of the type of jobs as well as orchestration details.
Summary
This section summarizes the input AutoSys scripts, including the number of files, boxes, jobs, and so on.
- Files: Number of input source files.
- Boxes: A box job is a container for multiple jobs. It controls scheduling, monitoring, and executing tasks.
- Jobs: Controls the system for scheduling, monitoring, and executing tasks.
- Unique Commands: Displays the number of unique commands.
- Servers: Displays the number of database servers where the AutoSys job details are stored.
- Total Job Triggers: Displays the total number of triggers.
Job Type
This section provides comprehensive information about Autosys job types along with its counts.
- Command: Used to execute workloads.
- File Watcher: Used to keep track of the presence and size of files.
- Box: A box job is a container for multiple jobs.
- HTTP: Used to invoke HTTP services.
- DBPROC: Used to execute database stored procedures.
Complexity
This section provides a summarized graphical representation of the complexity of jobs and boxes that helps in making different decisions, including budget estimation and assessing the effort required to transform the workloads.
Job Trigger Frequency
This section provides information about triggers, including the total number of triggers, frequency (such as daily, weekly, and others), and the count of scheduled triggers.
- Total Job Triggers: Displays the total number of triggers.
- Frequency: Displays the frequency of jobs such as daily, weekly, twice a week, etc., along with the number of triggers.
Watch Files
This section provides comprehensive information about watch files, including the total number of watch files, types, and its count.
- Total Watch Files: Displays the total number of watch files.
- Name: Displays top 5 watch file types such as Zip, TXT, CSV, etc., along with its count.
Analysis
This topic provides a detailed examination of jobs existing in the AutoSys scripts.
Orchestration Details
This section provides a complete analysis of the jobs. Jobs are scheduler scripts (such as AutoSys or Control-M) that run in a certain order to perform a set of activities. It lists all the jobs along with the information about jobs such as types, status, start time, and more.
- Jobs Name: Displays the name of the jobs.
- Type: Displays the type of jobs such as box, file watcher, CMD, etc.
- Associated File: Displays the name of the source scripts that are associated with the jobs.
- Start Time: Displays the job execution time.
- Conditions: Displays the conditions that must be met for a job to run.
- Commands: Displays the commands used in the jobs.
- Availability: Displays the availability of jobs.
- Available: Jobs that are available in both scripts and execution logs.
- Unavailable: Jobs that are available in scripts but not in execution logs.
Artifacts
This section lists all the artifacts that could not be parsed completely due to some error.
- File Name: Displays name of the artifact.
- Unparsed Message: Displays the reason for not parsing the file.
Lineage
End-to-end data and process lineage identify the complete dependency structure through interactive and drill-down options to the last level.
Typically, even within one line of business, multiple data sources, entry points, ETL tools, and orchestration mechanisms exist. Decoding this complex data web and translating it into a simple visual flow can be extremely challenging during large-scale modernization programs. The visual lineage graph adds tremendous value and helps define the roadmap to the modern data architecture. It deep dives into all the existing flows, like Autosys jobs, applications, ETL scripts, BTEQ/Shell (KSH) scripts, procedures, input and output tables, and provides integrated insights. These insights help data teams make strategic decisions with greater accuracy and completeness. Enterprises can proactively leverage integrated analysis to mitigate the risks associated with migration and avoid business disruption.
Now, let’s see how you can efficiently manage lineage.
To view the required lineage:
- Select either the Process or Data tab.
- Enter the keywords in the Search Keywords field.
- Click the Search icon to generate the lineage.
Process lineage illustrates the dependencies between two or more processes such as files, jobs, box, etc., whereas data lineage depicts data flow between two or more data-holding components such as entities, flat files, etc.
In addition, the filter search icon allows you to include or exclude particular nodes to obtain the required dependency structure. You can also choose the direction of the lineage. By default, the Dependency Direction is Left to Right Hierarchy. You can also choose Right to Left Hierarchy or Bidirectional dependency directions as required. Moreover, you can also increase the Hierarchy Levels to nth level.
Lineage facilitates you visualize how your selected nodes are connected and depend on each other. The nodes and their connecting edges (relationships) help you to understand the overall structure and dependencies.
|
Nodes |
Edges |
 File |
 Contains |
 Job |
 OTHERS |
 Autosys Box |
|
Manage Lineage
This feature enables you to view and manage your lineage. You can add, modify, or delete nodes and their relationships to generate an accurate representation of the required dependency structure. There are two ways to update the lineage: either using Complete Lineage report or Lineage Template.
Using Complete Lineage report
Follow the below steps to modify the lineage:
- Click the Manage Graph icon.
- Click Download Complete Lineage to update, add, or delete the nodes and their relationships in the current lineage.
- Once the complete lineage report is downloaded, you can make necessary updates such as updating, deleting or adding the nodes and its relationships.
- After making the required changes, upload the updated lineage report in Upload to Modify Lineage.
- Click Apply to incorporate the updates into the dependency structure.
- Generate the required process or data lineage.
Using Lineage Template
Follow the below steps to add new nodes and their relationships to the current lineage report:
- Click the Manage Graph icon.
- Click Download Lineage Template.
- Once the lineage template is downloaded, you can add new nodes and relationships in the template.
- After making the required changes, upload the template in Upload to Modify Lineage.
- Click Apply to incorporate the updates into the complete dependency structure.
- Generate the required process or data lineage.
Downloadable Reports
Downloadable reports allow you to export detailed Autosys assessment reports of your source data which enables you to gain in-depth insights with ease. To access these assessment reports, click Reports.
Types of Reports
In the Reports section, you can see various types of reports such as Insights and Recommendations, Source Inventory Analysis, and Lineage Report. Each report type offers detailed information allowing you to explore your assessment results.
Insights and Recommendations
This report provides an in-depth insight into the source input files. It contains the final output including information about jobs, triggers, file watchers, files, and so on.
Here, you can see AutoSys_Assessment_Report.xlsx and Lineage Dependency Report.xlsx reports
AutoSys_ Assessment_Report.xlsx: This report provides insights about the source inventory. It helps you plan the next frontier of a modern data platform methodically. It includes information about jobs, triggers, file watchers, files, and more.
This report contains the following information:
- Report Summary: Provides information about all the generated artifacts.
- Volumetric Info: Presents a summary of the aggregated inventory after analyzing the source files. For instance, it provides volumetric info about jobs, unique commands, stored procedures, servers, and likewise.
- Job Analysis: Provides comprehensive information about the AutoSys job types along with its count.
- File Watcher Analysis: Provides summarized information about file watcher types along with its count.
- Trigger Analysis: Provides summarized information about frequency and count of scheduled jobs, such as daily, weekly, and others.
- Calendar Analysis: Provides summarized information about unique calendars along with its count.
- External Files: Lists all the external files associated with the jobs.
- Unparsed Files: Provides details about the AutoSys unparsed files.
- Stored Procedure Details: Provides a summary of the stored procedures including dependent jobs, conditions, and more.
- Box Summary: Lists all the boxes along with information about jobs, job and box complexity factors, complexity and more.
- Job Summary: Lists all the jobs along with information such as whether the box and job are scheduled differently, its complexity, and more.
Lineage Dependency Report.xlsx: This report contains information about views, job level and file level lineages. It includes information about used and impacted tables, views, files, direct dependencies, dependency hierarchy and more.
This report contains the following information:
- view_report: Provides information about the views.
- script_report: Provides information about job and file level lineages.
Source Inventory Analysis
It is an intermediate report which helps to debug failures or calculate the final report. It includes all the generated CSV reports
autosys_job_complexity.csv: This report provides information about jobs along with its complexity, and type.
CommandJobReport.csv: This report provides information about jobs including dependent jobs, types, commands, and more.
JobInfo.csv: This report provides detailed information about jobs including associated jobs, files, and more.
Missing Dependent Job.csv: This report provides detailed information about missing jobs along with its parent jobs.
unparsed-autosys-attribute.csv: This report lists all the unparsed AutoSys attributes along with the reason for parsing failure.
unparsed-autosys-job.csv: This report lists all the unparsed jobs along with the reason for parsing failure.
Lineage Report
This report provides complete dependency details for all nodes. It provides an end-to-end data and process lineage that helps to identify the complete dependency structure and the data flow.
This report contains the following information:
- Volumetric Info: Provides volumetric information about the artifact types such as orchestration jobs and files.
- Nodes: Lists all the source and target nodes along with its type.
- Dependency (Process): Provides information about the process lineage.
- Dependency (Data): Provides information about the data lineage.
- Dependency (Data Model): Provides dependency details about the data models.
Leaplogic