Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 402 Part III Beyond Relational FIGURE 17-1 Jean Trenary and her crew are the able Information Service Department at AdventureWorks. 1 Ken Sánchez Chief Executive Officer 267 Karen Berg Application Specialist 265 Ashvini Sharma Network Administrator 268 Ramesh Meyyappan Application Specialist 266 Peter Connelly Network Administrator 269 Dan Bacon Application Specialist 272 Janaina Bueno Application Specialist 264 Stephanie Conroy Network Manager 263 Jean Trenary Information Services Manager 270 François Ajenstat Database Administrator 271 Dan Wilson Database Administrator Adventure Works 2008 Information Service Department The next query result shows the data from the Information Service Department: BusinessEntityID ManagerID Name JobTitle Manager 263 Jean Trenary Information Services Manager 1 Ken S ´ anchez 264 Stephanie Conroy Network Manager 263 Jean Trenary 267 Karen Berg Application Specialist 263 Jean Trenary 268 Ramesh Meyyappan Application Specialist 263 Jean Trenary 269 Dan Bacon Application Specialist 263 Jean Trenary 270 Fran ¸ cois Ajenstat Database Administrator 263 Jean Trenary 271 Dan Wilson Database Administrator 263 Jean Trenary 272 Janaina Bueno Application Specialist 263 Jean Trenary 265 Ashvini Sharma Network Administrator 264 Stephanie Conroy 266 Peter Connelly Network Administrator 264 Stephanie Conroy 402 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 403 Traversing Hierarchies 17 To examine one row from the employee perspective, Karen Berg’s BusinessEntityID (PK) is 264. Her ManagerID is 263. That’s the BusinessEntityID for Jean Trenary, so Karen reports to Jean. From the supervisor’s point of view, his BusinessEntityID is stored in each of his direct report’s rows in the ManagerID column. For example, Jean Trenary’s BusinessEntityID is 263, so 263 is stored in the ManagerID column of everyone who reports directly to her. To maintain referential integrity, the ManagerID column has a foreign key constraint that refers to the BusinessEntityID column (the primary key) in the same table, as shown in Figure 17-2. The ManagerID column allows nulls so that the top person of the organization chart can report to no one. FIGURE 17-2 The Employee table has a foreign key reference from the ManagerID column to the same table’s primary key. That’s why this pattern is often called the self-join pattern . 403 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 404 Part III Beyond Relational Restoring AdventureWorks2008’s Adjacency List W ith the advent of the HierarchyID data type in SQL Server 2008, Microsoft removed the adjacency list pattern from AdventureWorks2008 and replaced it with HierarchyID. The following script repairs AdventureWorks2008 and readies the database for experimentation with the adjacency list pattern: ALTER TABLE HumanResources.Employee ADD ManagerID INT CONSTRAINT FK_AdjPairs FOREIGN KEY(ManagerID) REFERENCES HumanResources.Employee(BusinessEntityID); GO Update new ManagerID Column using join condition to match UPDATE E SET ManagerID = M.BusinessEntityID FROM HumanResources.Employee AS E JOIN HumanResources.Employee AS M ON M.OrganizationNode = E.OrganizationNode.GetAncestor(1); CREATE INDEX IxParentID ON HumanResources.Employee (ManagerID) AdventureWorks2008 is now ready for some adjacency list fun with Ken S´anchez, the CEO; Jean Trenary, the IT Manager; and Franc¸ois Ajenstat, the DBA. Single-level queries The simplest hierarchical task is to match every node with a direct connection one level up the hierar- chy. In this case, it means listing all AdventureWorks2008 employees and their supervisor. The crux of the query is the self-join between two instances of the employee table represented in the LEFT OUTER JOIN between the employee’s (E) ManagerID column and the manager’s (M) BusinessEntityID column. There’s still one physical employee table, but the query contains two references to the table and joins the two table references together, i.e., the query is joining the table back to itself. Notice that because it’s an outer join, an employee without a manager (i.e., the CEO) will still show up in the result list: basic self-join query SELECT E.BusinessEntityID as Employee, M.BusinessEntityID as Manager FROM HumanResources.Employee as E LEFT OUTER JOIN HumanResources.Employee as M ON E.ManagerID = M.BusinessEntityID; 404 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 405 Traversing Hierarchies 17 Result (abbreviated): Employee Manager 1 Null 21 16 1 25 1 234 1 273 1 32 43 Of course, listing integers isn’t very useful. The next query adds some meat to the bones and fleshes out the data into a readable result set: Employees and their Direct Supervisors SELECT E.BusinessEntityID AS EmpID, EP.FirstName + ‘ ‘ + EP.LastName AS EmpName, E.JobTitle AS EmpTitle, E.ManagerID AS [MgrID], MP.FirstName + ‘ ‘ + MP.LastName AS MgrName, M.JobTitle AS MgrTitle FROM HumanResources.Employee AS E the employee JOIN Person.Person AS EP the employee’s contact info ON E.BusinessEntityID = EP.BusinessEntityID LEFT OUTER JOIN HumanResources.Employee AS M the mgr(if there is one) ON E.ManagerID = M.BusinessEntityID LEFT JOIN Person.Person AS MP the manager’s contact info ON M.BusinessEntityID = MP.BusinessEntityID ORDER BY E.ManagerID , E.BusinessEntityID; The abbreviated result is shown in Figure 17-3. FIGURE 17-3 Results from the query showing every employee and his manager. 405 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 406 Part III Beyond Relational The reverse of ‘‘find all managers,’’ which searches one level up the hierarchy query, is searching down the hierarchy. This query uses a downward looking join to locate every employee who has direct reports. The key to understanding this query is the self-join between the Employee table as M, representing the managers, and the Employee as E, representing the employees: Every Manager and their direct Reports SELECT M.BusinessEntityID AS ‘MgrID’, MP.FirstName + ‘ ‘ + MP.LastName AS ‘MgrName’, M.JobTitle AS ‘MgrTitle’, E.BusinessEntityID AS ‘EmpID’, EP.FirstName + ‘ ‘ + EP.LastName AS ‘EmpName’, E.JobTitle AS EmpTitle FROM HumanResources.Employee AS M the manager JOIN Person.Person AS MP the manager’s contact info ON M.BusinessEntityID = MP.BusinessEntityID JOIN HumanResources.Employee AS E the direct report ON E.ManagerID = M.BusinessEntityID JOIN Person.Person AS EP the manager’s contact info ON E.BusinessEntityID = EP.BusinessEntityID ORDER BY M.BusinessEntityID The result is shown in Figure 17-4. FIGURE 17-4 Results of the query showing every manager and his direct reports. The last adjacency list single-level query performs a count of each manager’s direct reports. Here, the work is being done in the subquery, which groups by the ManagerID column and counts the number or rows for each ManagerID: Everyone who has Direct Reports w/count SELECT M.BusinessEntityID as ID, MP.FirstName + ‘ ‘ + MP.LastName AS Manager, M.JobTitle, C.DirectReports FROM HumanResources.Employee AS M the manager JOIN Person.Person AS MP the manager’s contact info ON M.BusinessEntityID = MP.BusinessEntityID JOIN (SELECT ManagerID, COUNT(*) AS DirectReports FROM HumanResources.Employee 406 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 407 Traversing Hierarchies 17 WHERE ManagerID IS NOT NULL GROUP BY ManagerID)ASC ON M.BusinessEntityID = C.ManagerID; ORDER BY Manager Result (abbreviated): ID Manager JobTitle DirectReports 222 A. Scott Wright Master Scheduler 4 287 Amy Alberts European Sales Manager 3 47 Andrew Hill Production Supervisor - WC10 7 192 Brenda Diaz Production Supervisor - WC40 12 273 Brian Welcker Vice President of Sales 3 228 Christian Kleinerman Maintenance Supervisor 4 Subtree queries The preceding queries work well with one level. If you need to return all the nodes for two levels, then you could add another self-join. As the number of levels needed increases, the queries become complex and the query is fixed to a certain number of levels. A continuing parade of self-joins is not the answer. SQL queries should be able to handle any amount of data, so an adjacency list subtree query should be able to handle any number of levels. There are two solutions to the subtree problem when using the adjacency model: a recursive CTE or a looping user-defined function. Recursive CTE down the hierarchy The most direct solution to the subtree problem is the recursive common table expression, introduced in SQL Server 2005. This variant of the CTE (the basic CTE is covered in Chapter 11, ‘‘Including Data with Subqueries and CTEs’’) uses a UNION ALL and two SELECT statements. The first SELECT statement defines the anchor node from which the recursion will start, i.e., the top of the subtree. When the recursion starts, the row(s) returned by this query are added to the CTE. The second SELECT defines how rows are recursively added to the result set. This SELECT statement joins with the CTE itself — similar to the earlier self-join — and uses the UNION ALL to add the rows to the output of the CTE. The key is that the second SELECT will continue to self-join through the lay- ers of the hierarchy until no more rows are found. The loop stops executing when no more data is returned from the recursive step of the CTE. The first example focuses on the recursive nature of the CTE. The first SELECT in the CTE locates the employee with a ManagerID of null, which should be the top of the hierarchy. The name of the CTE is OrgPath, so when the second select joins with OrgPath it can find all the related nodes. The CTE continues to execute the second SELECT for every level of the hierarchy until the hierarchy ends: Find all - Recursive CTE down the OrgChart All employees who report to the CEO simple query 407 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 408 Part III Beyond Relational WITH OrgPath (BusinessEntityID, ManagerID, lv) AS ( Anchor SELECT BusinessEntityID, ManagerID, 1 FROM HumanResources.Employee WHERE ManagerID IS NULL should only be EmployeeID 1 Recursive Call UNION ALL SELECT E.BusinessEntityID, E.ManagerID, lv + 1 FROM HumanResources.Employee AS E JOIN OrgPath ON E.ManagerID = OrgPath.BusinessEntityID ) SELECT BusinessEntityID, ManagerID, lv FROM OrgPath ORDER BY Lv, BusinessEntityID OPTION (MAXRECURSION 20); Result (abbreviated): BusinessEntityID ManagerID lv 1 NULL 1 212 16 1 2 25 1 2 234 1 2 273 1 2 Because you join to the CTE in the recursive section of the CTE, you can use columns from this iteration to calculate the level in the hierarchy. In the previous query, the Lv column is calculated using the lv column from the previous iteration + 1. Thus, each iteration is given a sequential level number. If the adjacency list data had a cyclic error (A reports to B, who reports to C, who reports to A), then the recursive query could run forever, in which case the maxrecursion option limits the levels of recursion. If the recursive CTE hits the maxrecursion limit and there’s still data yet unprocessed, it will raise a 530, level 16 error, so even if the requirement is to return five levels of the hierarchy, option(maxrecursion 5) will return an error. Horrors! A solution is to use a WHERE clause in the second SELECT: WHERE Lv <= 3 This will safely limit the second query to return the first four levels of the hierarchy without throwing an error. 408 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 409 Traversing Hierarchies 17 The next query extends the previous recursive CTE by adding joins to flesh out the data. It also uses a different anchor node — Jean Trenary, Adventure Works’ IT Manager: Find Subtree - Recursive CTE All employees who report to IT Manager full query with joins WITH OrgPath (BusinessEntityID, ManagerID, lv) AS ( Anchor SELECT BusinessEntityID, ManagerID, 1 FROM HumanResources.Employee WHERE BusinessEntityID = 263 Jean Trenary - IS Manager Recursive Call UNION ALL SELECT E.BusinessEntityID, E.ManagerID, lv + 1 FROM HumanResources.Employee AS E JOIN OrgPath ON E.ManagerID = OrgPath.BusinessEntityID ) SELECT Lv, Emp.BusinessEntityID, C.FirstName + ‘ ‘ + C.LastName AS [Name], Emp.JobTitle, OrgPath.ManagerID, M.FirstName + ‘ ‘ + M.LastName AS [Manager] FROM HumanResources.Employee AS Emp JOIN OrgPath ON Emp.BusinessEntityID = OrgPath.BusinessEntityID JOIN Person.Person AS C ON C.BusinessEntityID = Emp.BusinessEntityID LEFT JOIN Person.Person AS M ON Emp.ManagerID = M.BusinessEntityID ORDER BY Lv, BusinessEntityID OPTION (MAXRECURSION 20); Result (abbreviated): BusinessEntityID ManagerID Lv Name JobTitle Manager 1 263 Jean Trenary Information Services Manager 1 Ken S ´ anchez 2 264 Stephanie Conroy Network Manager 263 Jean Trenary 2 267 Karen Berg Application Specialist 263 Jean Trenary 2 268 Ramesh Meyyappan Application Specialist 263 Jean Trenary 2 269 Dan Bacon Application Specialist 263 Jean Trenary 2 270 Fran ¸ cois Ajenstat Database Administrator 263 Jean Trenary 2 271 Dan Wilson Database Administrator 263 Jean Trenary 2 272 Janaina Bueno Application Specialist 263 Jean Trenary 3 265 Ashvini Sharma Network Administrator 264 Stephanie Conroy 3 266 Peter Connelly Network Administrator 264 Stephanie Conroy 409 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 410 Part III Beyond Relational User-defined function down the hierarchy SQL Server 2005’s recursive CTE is not the only way to skin a hierarchy. Before 2005, hierarchies were handled with stored procedures or user-defined functions that looped through the hierarchy layers adding each layer as a set to a temp table or table variable — effectively doing the same thing as a CTE but with T-SQL. The following multiple-statement table-valued user-defined function accepts an EmployeeID as a parameter. The function returns this employee and all the employees below them in the organization tree. The function definition includes the structure for the returned table variable, called @Tree.The function populates @Tree, first inserting the anchor node, the employee for the EmployeeID passed to the function. This method jumps ahead a bit, using an advanced programming feature of SQL Server to solve the hierarchical problem. For more information about one of my personal favorite features of SQL Server, see Chapter 25, ‘‘Building User-Defined Functions.’’ The WHILE loop continues to INSERT INTO @tree the next hierarchy level until no more rows are found ( @@rowcount > 0). Even though this is a loop, it’s still performing nice set-based inserts, one for every hierarchy level. If the hierarchy has a million nodes and is nine levels deep, then nine set-based inserts complete the entire task from top to bottom: User-Defined Functions for Navigating Adjacency list CREATE FUNCTION dbo.OrgTree (@BusinessEntityID INT) RETURNS @Tree TABLE (BusinessEntityID INT, ManagerID INT, Lv INT) AS BEGIN DECLARE @LC INT = 1 insert the top level (anchor node) INSERT @Tree (BusinessEntityID, ManagerID, Lv) SELECT BusinessEntityID, ManagerID, @LC FROM HumanResources.Employee AS E the employee WHERE BusinessEntityID = @BusinessEntityID Loop through each lower levels WHILE @@RowCount > 0 BEGIN SET @LC = @LC + 1 insert the Next level of employees INSERT @Tree (BusinessEntityID, ManagerID, Lv) SELECT NextLevel.BusinessEntityID, NextLevel.ManagerID, @LC FROM HumanResources.Employee AS NextLevel JOIN @Tree AS CurrentLevel ON CurrentLevel.BusinessEntityID = NextLevel.ManagerID WHERE CurrentLevel.Lv = @LC - 1 410 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 411 Traversing Hierarchies 17 END RETURN END; end of function A table-valued function returns a result set so it’s called in the FROM clause. The following query returns AdventureWorks2008’s entire organizational chart: test UDF find all simple query SELECT * FROM dbo.OrgTree(1); Result (abbreviated): BusinessEntityID ManagerID Lv 1 NULL 1 212 16 1 2 25 1 2 234 1 2 263 1 2 Best Practice T he recursive CTE in this case is about twice as fast as the user-defined function. However, the function can include more variations of code, so it’s useful to know both methods. Just as with the recursive CTE, it’s easy to expand the query and include the data necessary for a human-readable result set. Just like the previous recursive CTE example, this query finds only the subtree of those who work for Jean Trenary in the IT dept: find subtree all who work in IT full query with joins SELECT Lv, Emp.BusinessEntityID, Emp.JobTitle, C.FirstName + ‘ ‘ + C.LastName AS [Name], OrgTree.ManagerID, M.FirstName + ‘ ‘ + M.LastName AS [Manager] FROM HumanResources.Employee AS Emp JOIN dbo.OrgTree (263) Jean Trenary, IT Manager ON Emp.BusinessEntityID = OrgTree.BusinessEntityID JOIN Person.Person AS C 411 www.getcoolebook.com . 07/21/2009 12:57pm Page 404 Part III Beyond Relational Restoring AdventureWorks2008’s Adjacency List W ith the advent of the HierarchyID data type in SQL Server 2008, Microsoft removed the adjacency list. using an advanced programming feature of SQL Server to solve the hierarchical problem. For more information about one of my personal favorite features of SQL Server, see Chapter 25, ‘‘Building User-Defined. 409 www.getcoolebook.com Nielsen c17.tex V4 - 07/21/2009 12:57pm Page 410 Part III Beyond Relational User-defined function down the hierarchy SQL Server 2005’s recursive CTE is not the only way to skin a hierarchy.