HappyDoc Generated Documentation | HappyDoc3-r3_1/happydoclib/docstring/StructuredText/regressions/ExtensionClass.stx |
---|---|
/ HappyDoc3-r3_1 / happydoclib / docstring / StructuredText / regressions / ExtensionClass.stx Extension Classes, Python Extension Types Become ClassesJim Fulton, Digital Creations, Inc. jim@digicool.com Copyright (C) 1996-1998, Digital Creations. AbstractA lightweight mechanism has been developed for making Python extension types more class-like. Classes can be developed in an extension language, such as C or C++, and these classes can be treated like other python classes:
An example class shows how extension classes are implemented and how they differ from extension types. Extension classes provide additional extensions to class and instance semantics, including:
Extension classes illustrate how the Python class mechanism can be extended and may provide a basis for improved or specialized class models. ReleasesTo find out what's changed in this release, see the release notes. ProblemCurrently, Python provides two ways of defining new kinds of objects:
Each approach has it's strengths. Extension types provide much greater control to the programmer and, generally, better performance. Because extension types are written in C, the programmer has greater access to external resources. (Note that Python's use of the term type has little to do with the notion of type as a formal specification.) Classes provide a higher level of abstraction and are generally much easier to develop. Classes provide full inheritance support, while support for inheritance when developing extension types is very limited. Classes provide run-time meta-data, such as method documentation strings, that are useful for documentation and discovery. Classes act as factories for creating instances, while separate functions must be provided to create instances of types. It would be useful to combine the features of the two approaches. It would be useful to be able to have better support for inheritance for types, or to be able to subclass from types in Python. It would be useful to be able to have class-like meta-data support for types and the ability to construct instances directly from types. Our software is developed in Python. When necessary, we convert debugged Python routines and classes to C for improved performance. In most cases, a small number of methods in a class is responsible for most of the computation. It should be possible to convert only these methods to C, while leaving the other method in Python. A natural way to approach this is to create a base class in C that contains only the performance-critical aspects of a class' implementation and mix this base class into a Python class. We have need, in a number of projects, for semantics that are
slightly different than the usual class and instance semantics,
yet we don't want to do most of our development in C. For
example, we have developed a persistence mechanism [1] that
redefines As another example, we would like to have greater control over how methods are bound. Currently, when accessing a class instance attribute, the attribute value is bound together with the instance in a method object if and only if the attribute value is a python function. For some applications, we might also want to be able to bind extension functions, or other types of callable objects, such as HTML document templates [2]. Furthermore, we might want to have greater control over how objects are bound. For example, we might want to bind instances and callable objects with special method objects that assure that no more than one thread accesses the object or method at one time. We can provide these special semantics in extension types, but we wish to provide them for classes developed in Python. BackgroundAt the first Python Workshop, Don Beaudry presented work [3] done at V.I. Corp to integrate Python with C++ frameworks. This system provided a number of important features, including:
This work was not released, initially. Shortly after the workshop, changes were made to Python to support the sub-classing features described in [3]. These changes were not documented until the fourth Python Workshop [4]. At the third Python workshop, I presented some work I had done on generating module documentation for extension types. Based on the discussion at this workshop, I developed a meta-type proposal [5]. This meta-type proposal was for an object that simply stored meta-information for a type, for the purpose of generating module documentation. In the summer of 1996, Don Beaudry released the system described in [3] under the name MESS [6]. MESS addresses a number of needs but has a few drawbacks:
As MESS matures, we expect most of these problems to be addressed. Extension ClassesTo meet short term needs for a C-based persistence mechanism [1], an extension class module was developed using the mechanism described in [4] and building on ideas from MESS [6]. The extension class module recasts extension types as "extension classes" by seeking to eliminate, or at least reduce semantic differences between types and classes. The module was designed to meet the following goal:
Note: I use non-standard terminology here. By standard python terminology, only standard python classes can be called classes. ExtensionClass "classes" are technically just "types" that happen to swim, walk and quack like python classes. Base extension classes and extension subclassesBase extension classes are implemented in C. Extension subclasses are implemented in Python and inherit, directly or indirectly from one or more base extension classes. An extension subclass may inherit from base extension classes, extension subclasses, and ordinary python classes. The usual inheritance order rules apply. Currently, extension subclasses must conform to the following two rules:
Meta InformationLike standard python classes, extension classes have the following attributes containing meta-data:
The class dictionary provides access to unbound methods and their documentation strings, including extension methods and special methods, such as methods that implement sequence and numeric protocols. Unbound methods can be called with instance first arguments. Subclass instance dataExtension subclass instances have instance dictionaries, just like Python class instances do. When fetching attribute values, extension class instances will first try to obtain data from the base extension class data structure, then from the instance dictionary, then from the class dictionary, and finally from base classes. When setting attributes, extension classes first attempt to use extension base class attribute setting operations, and if these fail, then data are placed in the instance dictionary. Implementing base extension classesA base extension class is implemented in much the same way that an extension type is implemented, except:
where Attribute lookup Attribute lookup is performed by calling the base extension class
v = Py_FindAttrString(self,name); where In addition, a macro is provided that replaces If an attribute name is contained in a Python string object,
rather than a C string object, then the macro Linking The extension class mechanism was designed to be useful with
dynamically linked extension modules. Modules that implement
extension classes do not have to be linked against an extension
class library. The macro Example: MultiMapping objectsAn example, is provided that illustrates the changes needed to convert an existing type to an ExtensionClass. Implementing base extension class constructors Some care should be taken when implementing or overriding base
class constructors. When a Python class overrides a base class
constructor and fails to call the base class constructor, a
program using the class may fail, but it will not crash the
interpreter. On the other hand, an extension subclass that
overrides a constructor in an extension base class must call the
extension base class constructor or risk crashing the interpreter.
This is because the base class constructor may set C pointers that,
if not set properly, will cause the interpreter to crash when
accessed. This is the case with the If no base class constructor is provided, extension class instance memory will be initialized to 0. It is a good idea to design extension base classes so that instance methods check for uninitialized memory and perform initialialization if necessary. This was not done above to simplify the example. Overriding methods inherited from Python base classesA problem occurs when trying to overide methods inherited from Python base classes. Consider the following example: from ExtensionClass import Base class Spam: def __init__(self, name): self.name=name class ECSpam(Base, Spam): def __init__(self, name, favorite_color): Spam.__init__(self,name) self.favorite_color=favorite_color This implementation will fail when an To overcome this problem, extension classes provide a class method
from ExtensionClass import Base class Spam: def __init__(self, name): self.name=name class ECSpam(Base, Spam): def __init__(self, name, favorite_color): ECSpam.inheritedAttribute('__init__')(self,name) self.favorite_color=favorite_color This isn't as pretty but does provide the desired result. New class and instance semanticsContext WrappingIt is sometimes useful to be able to wrap up an object together with a containing object. I call this "context wrapping" because an object is accessed in the context of the object it is accessed through. We have found many applications for this, including:
User-defined method objectsPython classes wrap Python function attributes into methods. When a class has a function attribute that is accessed as an instance attribute, a method object is created and returned that contains references to the original function and instance. When the method is called, the original function is called with the instance as the first argument followed by any arguments passed to the method. Extension classes provide a similar mechanism for attributes that
are Python functions or inherited extension functions. In
addition, if an extension class attribute is an instance of an
extension class that defines an Consider the following example: import ExtensionClass class CustomMethod(ExtensionClass.Base): def __call__(self,ob): print 'a %s was called' % ob.__class__.__name__ class wrapper: def __init__(self,m,o): self.meth, self.ob=m,o def __call__(self): self.meth(self.ob) def __of__(self,o): return self.wrapper(self,o) class bar(ExtensionClass.Base): hi=CustomMethod() x=bar() hi=x.hi() Note that When run, this program outputs: Computed Attributes It is not uncommon to wish to expose information via the
attribute interface without affecting implementation data
structures. One can use a custom The import ExtensionClass def __init__(self, func): self.func=func def __of__(self, parent): return self.func(parent) Then we can use this class to create computed attributes. In the example below, we create a computed attribute, 'radius': from math import sqrt def __init__(self, x, y): self.x, self.y = x, y radius=ComputedAttribute(lambda self: sqrt(self.x2+self.y2)) p=Point(2,2) print p.radius Overriding method calls Normally, when a method is called, the function wrapped by the
method is called directly by the method. In some cases, it is
useful for user-defined logic to participate in the actual
function call. Extension classes introduce a new protocol that
provides extension classes greater control over how their
methods are called. If an extension class defines a special
method, For example, consider the expression: To see how this feature may be used, see the Python module,
An interesting application of this mechanism would be to implement interface checking on method calls. Method attributesMethods of ExtensionClass instances can have user-defined attributes, which are stored in their associated instances. For example: class C(ExtensionClass.Base): def get_secret(self): "Get a secret" .... c=C() c.f.__roles__=['Trusted People'] print c.f.__roles__ # outputs ['Trusted People'] print c.f__roles__ # outputs ['Trusted People'] print C.f.__roles__ # fails, unbound method A bound method attribute is set by setting an attribute in it's
instance with a name consisting of the concatination of the
method's Class initialization Normal Python class initialization is similar to but subtley
different from instance initialization. An instance Extension classes provide a mechanism for specifying code to be
run after a class has been created. If a class or one of it's
base classes defines a Useful macros defined in ExtensionClass.h A number of useful macros are defined in ExtensionClass.h.
These are documented in Pickleability Classes created with ExtensionClass, including extension base
classes are automatically pickleable. The usual gymnastics
necessary to pickle StatusThe current release of the extension class module is 1.1. The core implementation has less than four thousand lines of code, including comments. This release requires Python 1.4 or higher. To find out what's changed in this release, see the release notes. Installation instructions, are provided. IssuesThere are a number of issues that came up in the course of this work and that deserve mention.
ApplicationsAside from test and demonstration applications, the extension class mechanism has been used to provide an extension-based implementation of the persistence mechanism described in [1]. We have developed this further to provide features such as automatic deactivation of objects not used after some period of time and to provide more efficient persistent-object cache management. Acquisition has been heavily used in our recent products. Synchonized classes have also been used in recent products. SummaryThe extension-class mechanism described here provides a way to add class services to extension types. It allows:
In addition, the extension class module provides a relatively concise example of the use of mechanisms that were added to Python to support MESS [6], and that were described at the fourth Python Workshop [4]. It is hoped that this will spur research in improved and specialized models for class implementation in Python. References [1] Fulton, J., Providing Persistence for World-Wide-Web Applications, Proceedings of the 5th Python Workshop. [2] Page, R. and Cropper, S., Document Template, Proceedings of the 5th Python Workshop. [3] Beaudry, D., Deriving Built-In Classes in Python, Proceedings of the First International Python Workshop. [4] Van Rossum, G., Don Beaudry Hack - MESS, presented in the Developer's Future Enhancements session of the 4th Python Workshop. [5] Fulton, J., Meta-Type Object, This is a small proposal, the text of which is contained in a sample implementation source file, [6] Beaudry, D., and Ascher, D., The Meta-Extension Set. |