Silver Layer

Run manually after a deployment to (re)create the Silver lakehouse's shortcuts and views for the environment. Notebooks live in the Deployment folder; all GUIDs come from the Variable_Library.

#	Activity	Notebook	What it does
1	Create Shortcuts	01_create_shortcuts	Creates a table shortcut in LANDING for every table in Bronze raw. Skips tables that already exist, so the materialised Identity_Pivoted is untouched.
2	Create One Lake Shortcuts	02_create_one_lake_shortcuts_from_config	Creates the D365Finance and D365EUFinance table shortcuts from the config file Files/Configs/One_Lake_Shortcuts.csv.
3	Create CRL Views	03_create_crl_views	Creates the CRL schema and its views (GetPersonIDs, GetProjectIDs) on the SQL endpoint.
4	Run Orchestration	—	Invokes the Orchestration pipeline (waits for completion), so a deployment ends with a full data refresh.

The recurring data refresh. Scheduled daily at 00:00 (GMT Standard Time), and also invoked as the last step of Deployment. Notebooks live in the Orchestration folder.

#	Activity	Notebook	What it does
1	Load Identity Pivoted	01_load_identity_pivoted	Rebuilds LANDING.Identity_Pivoted from scratch: joins LANDING.Identity to IntegratedSystem and pivots one column per source system.
2	Get CastCollate View Definitions	02_get_castcollate_view_definitions	For every table in LANDING, generates a CREATE OR ALTER VIEW CASTCOLLATE.{table} script casting string columns to their measured max length with a case-insensitive UTF8 collation. Returns the scripts as a JSON array.
3	fe_CastAndCollateView	—	ForEach over the returned scripts; each is executed by a Script activity against the MasterData_Silver_DWH SQL endpoint. The connection is selected per environment by an expression on the workspace ID.

From High-level IDM Entity Type by Master System (see _context\). Each built entity is a Load Master Data {Entity} notebook creating MASTERDATA.{entity}. The load notebooks are not yet orchestrated; they are run manually.

Entity Type	Master System	Rule	Status
Office	D365 Finance UK and EU	Join IdentityCanonicalValue with Identity Canonical, join to IntegratedSystem. Master in D365Finance if a record exists, else D365EUFinance	Built
Department	D365 Finance UK and EU	Same rule as Office. Department names can exist in both UK and EU	Built
Division	D365 Finance UK and EU	Same rule as Office	Built
LegalEntity	D365 Finance UK and EU	Reference IntegratedSystemId for master	Built
ProjectCategory	D365 Finance UK	When IdentityGroup is NULL	To build
OfficeDepartment	D365 Finance UK and EU	Same rule as Office	To build
PropertySector	D365Sales	—	To build
Customer	—	Not in the IDM document. A Load Master Data Customer notebook exists but its source rule is unconfirmed	To confirm