Note:
If the selected target is Databricks Lakehouse, enable Unity Catalog to transform the constraints such as primary key, foreign key, etc., present in the source data to the Databricks-native equivalent. Otherwise, the default Hive meta store will be used, and Hive or Spark tables will be generated without any constraints. In Databricks, there are two types of catalogs:
- Hive Meta Store: Hive meta store serves as a central repository to create, store, and manage large datasets or tables. However, it does not support constraints like primary key, foreign key, etc.
- Unity Catalog: Unity Catalog in Databricks serves as a centralized metadata repository with advanced metadata management features that support constraints including primary key, foreign key, etc. It serves as a comprehensive repository for storing metadata information within Databricks.
