Two common ways of storing a quantity in a class are either to let the quan- tity be an attribute itself or to insert the quantity in a dictionary and have the dictionary as an attribute. If you have many quantities and these fall into nat- ural categories, the dictionary approach has many attractive features. Some of these will be high-lighted in this section.
Suppose we have a class for solving a computational science problem. In this class we have a lot of physical parameters, a lot of numerical parameters, and perhaps a lot of visualization parameters. In addition we may allow future users of the class to insert new types of data that can be processed by future software tools without demanding us to update the class code.
Outline of the Class Structure. The problem setting and the sketched flexi- bility may be common to several applications so we split our class in a general part, implemented as a base class, and a problem-specific part, implemented as a subclass.
In the subclass we store parameters in dictionaries namedself.*_prm. As a start, we may think of having physical parameters in self.physical_prm and numerical parameters inself.numerical_prm. These dictionaries are sup- posed to be initialized with a fixed set of legal keys during the instance’s construction. A special base class attribute self._prm_list holds a list of the parameter dictionaries. General code can then process self._prm_list without needing to know anything about problem-specific ways of catego- rizing data. To enable users to store meta data in the class, we introduce a self.user_prm dictionary whose keys are completely flexible. These user- defined meta data can be processed by other classes.
Type-checking can sometimes be attractive to avoid erroneous use. We introduce in the base class a dictionary self._type_check where subclasses can register the parameter names to be type checked. Say we have two pa- rameters for which type checking is wanted:dtshould be a float, andqshould have its type determined by the initial value. Then we define
self._type_check[’dt’] = (float,) self._type_check[’q’] = True
When a parameter’s type is fixed by the constructor, the type possibilities are listed in a tuple. If the initial value determines the type, the value is true (a boolean or integer variable). A third option is to assign a user-supplied function, taking the value as argument and returning true if the value is acceptable, e.g.,
self._type_check[’v’] = lambda v: v in _legal_v
Here _legal_v is a list of legal values of v. A parameter whose name is not registered in the list self._type_check, or registered with a false value, will never be subject to type checking.
The base class might be outlined as follows:
class PrmDictBase(object):
def __init__(self):
self._prm_list = [] # fill in subclass self.user_prm = None # user’s meta data self._type_check = {} # fill in subclass
A subclass should fill the dictionaries with legal keys (parameter names):
class SomeSolver(PrmDictBase):
def __init__(self, **kwargs):
# register parameters:
PrmDictBase.__init__(self)
self.physical_prm = {’density’: 1.0, ’Cp’: 1.0,
’k’: 1.0, ’L’: 1.0}
self.numerical_prm = {’n’: 10, ’dt’: 0.1, ’tstop’: 3}
self._prm_list = [self.physical_prm, self.numerical_prm]
self._type_check.update({’n’: True, ’dt’: (float,)}) self.user_prm = None # no extra user parameters self.set(**kwargs)
Here we specify type checking of two parameters, and user-provided meta data cannot be registered. The convention is thatself.user_prm is a dictio- nary if meta data are allowed andNoneotherwise.
Assigning Parameter Values. Theself.set method takes an arbitrary set of keyword arguments and fills the dictionaries. The idea is that parameters, sayCpanddt, are set like
solver.set(Cp=0.1, dt=0.05)
Theset method goes through the dictionaries with fixed key sets first and sets the corresponding keys, here typically
self.physical_prm[’Cp’] = 0.1 self.numerical_prm[’dt’] = 0.05
Since the dt parameter is marked to be type checked, set must perform a test that the value is indeed a float.
If we call solver.set(color=’blue’) and color is not registered in the dictionaries with fixed key sets,self.user_prm[’color’] can be set to’blue’
ifself.user_prmis a dictionary and not None.
The set method must run a loop over the received keyword arguments (parameter names) with an inner loop over the relevant dictionaries. For each pass in the loop, a methodset_in_dict(prm, value, d)is called for storing the (prm,value) pair in a dictionaryd. Before we can execute d[prm]=value we need to test ifprmis registered as a parameter name, perform type checks if that is specified, etc. A parameter whose name is not registered may still be stored in theself.user_prmdictionary. All this functionality can be coded independent of any problem-specific application and placed in the base class PrmDictBase:
8.6. Classes 405 def set(self, **kwargs):
"""Set kwargs data in parameter dictionaries."""
for prm in kwargs:
_set = False
for d in self._prm_list: # for dicts with fixed keys try:
if self.set_in_dict(prm, kwargs[prm], d):
_set = True break
except TypeError, exception:
print exception break
if not _set: # maybe set prm as meta data?
if isinstance(self.user_prm, dict):
self.user_prm[prm] = kwargs[prm]
else:
raise NameError, \
’parameter "%s" not registered’ % prm self._update()
def set_in_dict(self, prm, value, d):
"""
Set d[prm]=value, but check if prm is registered in class dictionaries, if the type is acceptable, etc.
"""
can_set = False
# check that prm is a registered key if prm in d:
if prm in self._type_check:
# prm should be type-checked
if isinstance(self._type_check[prm], int):
# (bool is subclass of int) if self._type_check[prm]:
# type check against prev. value or None:
if isinstance(value, (type(d[prm]), None)):
can_set = True
# allow mixing int, float, complex:
elif operator.isNumberType(value) and\
operator.isNumberType(d[prm]):
can_set = True
elif isinstance(self._type_check[prm], (tuple,list,type)):
if isinstance(value, self._type_check[prm]):
can_set = True else:
raise TypeError, ...
elif callable(self._type_check[prm]):
can_set = self.type_check[prm](value) else:
can_set = True if can_set:
d[prm] = value return True return False
The set method calls self._update at the end. This is supposed to be a method in the subclass that performs consitency checks of all class data after parameters are updated. For example, if we change a parameter n, arrays may need redimensioning.
The setand set_in_dict methods can work with any set of dictionaries holding any sets of parameters. We have both parameter name checks and the possibility to store unregistered parameters. Instead of specifying the type as Python class types, one could use functions from theoperator mod- ule: isSequenceType, isNumberType, etc. (see Chapter 3.2.11), for controlling the types (typically we setself._type_check[’dt’]tooperator.isNumberType instead of (float,)).
The alternative way of storing data in a class is to let each parameter be an attribute. In that case we have all parameters, together with all other class data and methods, in a single dictionaryself.__dict__. The features in thesetmethod are much easier to implement when not all data are merged as attributes in one dictionary but instead classified in different categories.
Each category is represented by a dictionary, and we can write quite gen- eral methods for processing such dictionaries. More examples on this appear below.
Automatic Generation of Properties. Accessing a parameter in the class may involve a comprehensive syntax, e.g.,
dx = self.numerical_prm[’L’]/self.numerical_prm[’n’]
It would be simpler ifLandnwere attributes:
dx = self.L/self.n
This is easy to achieve. The safest approach is to generate properties at run time. Given some parameter namepin (say)self.physical_prm, we execute
X.p = property(fget=lambda self: self.physical_prm[p], doc=’read-only attribute’)
whereXis the class in which we want the property. Since all parameters are stored in dictionaries, the task is to run through the dictionaries, generate code segments, and bring the code into play by runningexec:
def properties(self, global_namespace):
"""Make properties out of local dictionaries."""
for ds in self._prm_dict_names():
d = eval(’self.’ + ds) for prm in d:
# properties cannot have whitespace:
prm = prm.replace(’ ’, ’_’) cmd = ’%s.%s = property(fget=’\
’lambda self: self.%s["%s"], %s)’ % \ (self.__class__.__name__, prm, ds, prm,
’ doc="read-only property"’) print cmd
exec cmd in global_namespace, locals()
8.6. Classes 407 The names of the self.*_prm dictionaries are constructed by the following function, which applies a very compact list comprehension:
def _prm_dict_names(self):
"""Return the name of all self.*_prm dictionaries."""
return [attr for attr in self.__dict__ if \ re.search(r’^[^_].*_prm$’, attr)]
Generating Attributes. Instead of making properties we could make standard attributes out of the parameters stored in the self.*_prm dictionaries. This is just a matter of looping over the keys in these dictionaries and register the (key,value) pair inself.__dict__. Such movement of data from a set of dictionaries to another dictionary can be coded as
def dicts2namespace(self, namespace, dicts, overwrite=True):
"""Make namespace variables out of dict items."""
# can be tuned in subclasses for d in dicts:
if overwrite:
namespace.update(d) else:
for key in d:
if key in namespace and not overwrite:
print ’cannot overwrite %s’ % key else:
namespace[key] = d[key]
The overwrite argument controls whether we allow to overwrite a key in namespace if it already exists. The call
self.dicts2namespace(self.__dict__, self._prm_list)
creates attributes in the class instance out of all the keys in the dictionaries with fixed key sets. If we also want to convert keys inself._user_prm, we can call
self.dicts2namespace(self.__dict__, self._prm_list+self._user_prm) Automatic Generation of Short Forms. As already mentioned, a parameter like
self.numerical_prm[’n’]
requires much writing and may in mathematical expressions yield less read- able code than a plain local variablen. Technically, we could manipulate the dictionary of local variables, locals(), in-place and thereby generate local variables from the keys in dictionaries:
self.dicts2namespace(locals(), self._prm_list)
This does not work. The dictionary of local variables is updated, but the vari- ables are not accessible as local variables. According to the Python Library Reference, one should not manipulatelocals() this way.
An alternative could be to pollute the global namespace with new vari- ables,
self.dicts2namespace(globals(), self._prm_list)
Now we canread self.numerical_prm[’n’] as (a global variable)n. Assign- ments tonare not reflected in the underlyingself.numerical_prm dictionary.
The approach may sound attractive, since we can translate dictionary con- tents to plain variables, which allows us to write
dx = L/n instead of
dx = self.numerical_prm[’L’]/self.numerical_prm[’n’]
It is against most programming recommendations to pollute the global names- pace the way we indicate here. The only excuse could be to perform this at the beginning of an algorithm, delete the generated global variables at the end, and carfully check that existing global variables are not affected (i.e., setting overwrite=False in thedicts2namespace call). A clean-up can be carried out by
def namespace2dicts(self, namespace, dicts):
"""Update dicts from variables in a namespace."""
keys = [] # all keys in namespace that are keys in dicts for key in namespace:
for d in dicts:
if key in d:
d[key] = namespace[key] # update value keys.append(key) # mark for delete
# clean up what we made in self.dicts2namespace:
for key in keys:
del namespace[key]
Running namespace2dicts(globals(), self._prm_list) at the end of an al- gorithm copies global data back to the dictionaries and removes the global data.
The ideas outlined here must be used with care. The flexibility is great, and very convenient tools can be made, but strange errors from polluting the global namespace may arise. These can be hard to track down.
A Safe Way of Generating Local Variables. Turning a dictionary entry, say self._physical_prm[’L’], into a plain variable L can of course be done manually. A simple technique is to define a function that returns a list of the particular variables we would like to have in short form when implementing an algorithm. Such functionality must be coded in the subclass.
8.6. Classes 409 def short_form1(self):
return self._physical_prm[’L’], self._numerical_prm[’dt’], self._numerical_prm[’n’]
We may use this function as follows:
def some_algorithm(self):
L, dt, n = self.short_form1() dx = L/float(n)
...
If we need to convert many parameters this way, it becomes tedious to write the code, but this more comprehensive solution is also much safer than the generic approaches in the previous paragraphs.
The tools outlined in this section are available through classPrmDictBase in the module scitools.PrmDictBase. Examples on applications appear in Chapter 12.3.3.