|| + + | |+ + | | +- + + | | | | + + +-+ + + Quantum | | 11510NYBaldwinSteve Heller| Data | + + +- +- Quantum | 333333333222222222211111111110000000000 Offset | 876543210987654321098765432109876543210 +- Note that we set the offset value of the deleted item equal to the offset of the item before it so that its length can be calculated as zero. Now let us add a new street address of "123 Main Street". After this change, our data structures look like those in Figure itemref3. Sample IRA, item index, and data, after field change (Figure itemref3) +- | Quantum Item | Index Number Number | + + Item | 0 | 3 1 + + Reference | 1 | 3 2 + +-+ Array | 2 | 3 3 + +-+ + (IRA) | 3 | 3 4 + +-+ +-+ | 4 | 3 5 + +-+ +-+-+ | + + | | | | | +- | | | | | | | | | | +- | | | | | |Item # Offset Type Index | | | | | | + + | | | | | | 1 | 12 -+ VARSTRING 0 |± + | | | | | 2 | 27 +| VARSTRING 1 |± + | | | Item | 3 | +-34 || VARSTRING 2 |± + | | Index | 4 | ++-36 || VARSTRING 3 |± + | for | 5 +-+-++-41 || VARSTRING 4 |± + Quantum | 6 | | || 38 || UNUSED 0 | 3 | 7 | | || 38 || UNUSED 0 | | 8 | | || 38 || UNUSED 0 | | 9 | | || 38 || UNUSED 0 | | 10 | | || 38 || UNUSED 0 | | | +-++ ++ + +- | || |+ + | || + + | | |+-+ | | +- | ++ | | | | ++ +-+ + + + Quantum | |11510NYBaldwin123 Main StreetSteve Heller| Data | + + +- +- Quantum | 443333333333222222222211111111110000000000 Offset | 109876543210987654321098765432109876543210 +- This is the actual mechanism used to change the value of an item: we delete the old item and add the new one. Of course, an item that increases in size may no longer fit in the same quantum, in which case we have to move it to a different quantum. Keeping track of such changes is not difficult as long as we always use the standard method of finding the address of an item: look it up in the IRA to get its quantum number and item number, then use its quantum number and item number to look it up in the item index. This allows us to move the item to a new quantum and change only its entries in the IRA and affected item indexes. For example, if an item was originally in quantum 3 but now no longer fits there, we might move it to quantum 6. In order to do this, we first delete it from quantum 3 and then add it to quantum 6. As before, all the other IRA entries for items in quantum 3 still refer to the same data items as before, after we have adjusted the item index to account for the removed data. Of course, the deleted item still takes up a slot in the item index of the old quantum; in our implementation this requires 10 bytes of storage. 9 We need a method of reclaiming that space, so that we do not gradually fill our quanta with indexes pointing to nothing. We have already seen part of the solution to this problem: whenever we are adding an item to a quantum, we search for an unused item index entry rather than simply adding a new entry at the end. This mechanism alone would only slow the growth of the item indexes; actually reducing the size of an index requires us to check for empty entries at the end of the item index after deleting an item. If such excess entries exist, we reclaim the space they occupy by reducing the entry count in the item index header to the number actually in use. This reduces wasted space in the item index without affecting entries in the IRA. Since we remove unused item index entries only when they are at the end of the index, no IRA entries can refer to items later in the same quantum. A Large Array Our records are stored in the quanta pointed to by our IRA. The IRA will be stored in its own quanta, which allows us to make it as large as needed. However, this means that we can't allocate the IRA in one piece. Since a quantum has a fixed size, we will have to break up our IRA into segments that will fit in a quantum; these segments will be collectively referred to as the little pointer array. We'll also need some way to find the segment that we need for a particular element in the IRA, which will be the responsibility of the big pointer array. Figure bigpointer shows the relationship between the big pointer array and the IRA segments. From the big pointer array to the IRA (Figure bigpointer) + + | +- | | Big | element_count = 8000 | | Pointer | last_quantum_added_to = 3 | | Array | | | Header | | | +- | | +- | | | Quantum | Quantum | | Index Number | 5 | | + + | | Big | 0 | 2 + + + | Pointer | 1 | 10 + + +-+ | Array | 2 | 12 + + +-+ + | | 3 | 14 + + +-+ ++ | | 4 | 11 + + +-+ +++ | | + + | | | ||| | +- | | | ||| | | | | ||| + + | | ||| + + | | ||| | |± + | ||| | +- | Quantum | ||| | | Quantum Item | 2 | ||| | | Index Number Number | | ||| | Item | + + | | ||| | Reference | 0 | 3 2 | | | ||| | Array | 1 | 3 4 | | | ||| | (IRA) | 2 | 3 5 | | | ||| | segment | 3 | 3 3 | | | ||| | 0 | 4 | 3 1 | | | ||| | | + + | | ||| | +- | | ||| + + | ||| + + | ||| | |± + ||| | +- | Quantum ||| | | Quantum Item | 10 ||| | | Index Number Number | ||| | Item | + + | ||| | Reference | 0 | 3 12 | | ||| | Array | 1 | 4 6 | | ||| | (IRA) | 2 | 4 1 | | ||| | segment | 3 | 4 2 | | ||| | 1 | 4 | 4 3 | | ²²² | | + + | | +- | + + Each IRA segment in the real program, of course, holds more than five entries: with a 16 KB quantum, about 4000 entries will fit in one quantum. However, the segmentation of the IRA works in the same way as this example. The segment number can be determined by dividing the element number by the number of elements in each IRA segment; the index into the IRA segment is the remainder of that division. The big pointer array sets a limit on the number of elements referenced by an IRA, since that array has to fit in one quantum. However, this is not a very restrictive limit, since one big pointer array can contain about 4000 elements, using a 16 KB quantum. Each of these elements points to a little pointer array, which contains pointers to about 4000 entries, as mentioned above. Therefore, we could theoretically create an array with about 16,000,000 items in it. Because our ArrayIndex type is defined as an unsigned long, which is usually 4 bytes, we can actually create arrays of that size. 10 Many Large Arrays There is one more generalization of the quantum file access method that will make it more generally useful: the ability to create more than one array of variable-length items. The mechanism by which we will accomplish this goal is called the "main object index", which contains the quantum number of the big pointer array for each object; the number of an object is set when the object is created, by a directory facility which we will examine later. Figure mainindex shows the path from the main object index through the big pointer array, the IRA segment, and the item index, to finally arrive at the data. From the main object index to the data (Figure mainindex) +- | Index Quantum Number | + + Main | 0 |NO_QUANTUM| Object| 1 | 5 + + Index | 2 | 9 | | | 3 | 1 | | | 4 | 7 | | +- + + | + + | |Big +- |± + |Pointer | element_count = 8000 | |Array | last_quantum_added_to = 3 | |Header +- | | +- | . to quantum 6. As before, all the other IRA entries for items in quantum 3 still refer to the same data items as before, after we have adjusted the item index to account for the removed data indexes pointing to nothing. We have already seen part of the solution to this problem: whenever we are adding an item to a quantum, we search for an unused item index entry rather than simply. as the little pointer array. We'll also need some way to find the segment that we need for a particular element in the IRA, which will be the responsibility of the big pointer array. Figure