You have a small number of C functions that have been compiled into a shared libraryor DLL. You would like to call these functions purely from Python without having towrite additional C code or using a third-party extension tool.
For small problems involving C code, it is often easy enough to use the ctypes modulethat is part of Python’s standard library. In order to use ctypes, you must first makesure the C code you want to access has been compiled into a shared library that iscompatible with the Python interpreter (e.g., same architecture, word size, compiler,etc.). For the purposes of this recipe, assume that a shared library, libsample.so, hasbeen created and that it contains nothing more than the code shown in the chapterintroduction. Further assume that the libsample.so file has been placed in the samedirectory as the sample.py file shown next.To access the resulting library, you make a Python module that wraps around it, suchas the following:# sample.pyimport ctypesimport os
def divide(x, y):
rem = ctypes.c_int()quot = _divide(x, y, rem)
return quot,rem.value
def fromparam(self, param):> typename = type(param).nameif hasattr(self, ‘[from](#)‘ + typename):
return getattr(self, ‘from_‘ + typename)(param)
elif isinstance(param, ctypes.Array):return paramelse:raise TypeError(“Can't convert %s” % typename)> # Cast from array.array objectsdef from_array(self, param):
if param.typecode != ‘d':raise TypeError(‘must be an array of doubles')> > ptr, _ = param.buffer_info()return ctypes.cast(ptr, ctypes.POINTER(ctypes.c_double))
Cast from lists/tuplesdef from_list(self, param):
val = ((ctypes.c_double)len(param))([](#)param)return val
from_tuple = from_list
Cast from a numpy arraydef from_ndarray(self, param):
return param.ctypes.data_as(ctypes.POINTER(ctypes.c_double))
DoubleArray = DoubleArrayType()_avg = _mod.avg_avg.argtypes = (DoubleArray, ctypes.c_int)_avg.restype = ctypes.c_double
def avg(values):return _avg(values, len(values))
fields = [(‘x', ctypes.c_double),(‘y', ctypes.c_double)]
If all goes well, you should be able to load the module and use the resulting C functions.For example:
>>> import sample
>>> sample.gcd(35,42)
7
>>> sample.in_mandel(0,0,500)
1
>>> sample.in_mandel(2.0,1.0,500)
0
>>> sample.divide(42,8)
(5, 2)
>>> sample.avg([1,2,3])
2.0
>>> p1 = sample.Point(1,2)
>>> p2 = sample.Point(4,5)
>>> sample.distance(p1,p2)
4.242640687119285
>>>There are several aspects of this recipe that warrant some discussion. The first issueconcerns the overall packaging of C and Python code together. If you are using ctypesto access C code that you have compiled yourself, you will need to make sure that theshared library gets placed in a location where the sample.py module can find it. Onepossibility is to put the resulting .so file in the same directory as the supporting Pythoncode. This is what’s shown at the first part of this recipe—sample.py looks at the filevariable to see where it has been installed, and then constructs a path that points to alibsample.so file in the same directory.If the C library is going to be installed elsewhere, then you’ll have to adjust the pathaccordingly. If the C library is installed as a standard library on your machine, you mightbe able to use the ctypes.util.find_library() function. For example:
>>> from ctypes.util import find_library
>>> find_library('m')
'/usr/lib/libm.dylib'
>>> find_library('pthread')
'/usr/lib/libpthread.dylib'
>>> find_library('sample')
'/usr/local/lib/libsample.so'
>>>Again, ctypes won’t work at all if it can’t locate the library with the C code. Thus, you’llneed to spend a few minutes thinking about how you want to install things.Once you know where the C library is located, you use ctypes.cdll.LoadLibrary()to load it. The following statement in the solution does this where _path is the fullpathname to the shared library:
_mod = ctypes.cdll.LoadLibrary(_path)
Once a library has been loaded, you need to write statements that extract specific sym‐bols and put type signatures on them. This is what’s happening in code fragments suchas this:
In this code, the .argtypes attribute is a tuple containing the input arguments to afunction, and .restype is the return type. ctypes defines a variety of type objects (e.g.,c_double, c_int, c_short, c_float, etc.) that represent common C data types. Attach‐ing the type signatures is critical if you want to make Python pass the right kinds ofarguments and convert data correctly (if you don’t do this, not only will the code notwork, but you might cause the entire interpreter process to crash).A somewhat tricky part of using ctypes is that the original C code may use idioms thatdon’t map cleanly to Python. The divide() function is a good example because it returnsa value through one of its arguments. Although that’s a common C technique, it’s oftennot clear how it’s supposed to work in Python. For example, you can’t do anythingstraightforward like this:
>>> divide = _mod.divide
>>> divide.argtypes = (ctypes.c_int, ctypes.c_int, ctypes.POINTER(ctypes.c_int))
>>> x = 0
>>> divide(10, 3, x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ctypes.ArgumentError: argument 3: <class 'TypeError'>: expected LP_c_int
instance instead of int
>>>Even if this did work, it would violate Python’s immutability of integers and probablycause the entire interpreter to be sucked into a black hole. For arguments involvingpointers, you usually have to construct a compatible ctypes object and pass it in likethis:
>>> x = ctypes.c_int()
>>> divide(10, 3, x)
3
>>> x.value
1
>>>Here an instance of a ctypes.c_int is created and passed in as the pointer object. Unlikea normal Python integer, a c_int object can be mutated. The .value attribute can beused to either retrieve or change the value as desired.
For cases where the C calling convention is “un-Pythonic,” it is common to write a smallwrapper function. In the solution, this code makes the divide() function return thetwo results using a tuple instead:# int divide(int, int, int *)_divide = _mod.divide_divide.argtypes = (ctypes.c_int, ctypes.c_int, ctypes.POINTER(ctypes.c_int))_divide.restype = ctypes.c_int
def divide(x, y):rem = ctypes.c_int()quot = _divide(x,y,rem)return quot, rem.value
The avg() function presents a new kind of challenge. The underlying C code expectsto receive a pointer and a length representing an array. However, from the Python side,we must consider the following questions: What is an array? Is it a list? A tuple? Anarray from the array module? A numpy array? Is it all of these? In practice, a Python“array” could take many different forms, and maybe you would like to support multiplepossibilities.The DoubleArrayType class shows how to handle this situation. In this class, a singlemethod from_param() is defined. The role of this method is to take a single parameterand narrow it down to a compatible ctypes object (a pointer to a ctypes.c_double, inthe example). Within from_param(), you are free to do anything that you wish. In thesolution, the typename of the parameter is extracted and used to dispatch to a morespecialized method. For example, if a list is passed, the typename is list and a methodfrom_list() is invoked.For lists and tuples, the from_list() method performs a conversion to a ctypes arrayobject. This looks a little weird, but here is an interactive example of converting a list toa ctypes array:
>>> nums = [1, 2, 3]
>>> a = (ctypes.c_double * len(nums))(*nums)
>>> a
<__main__.c_double_Array_3 object at 0x10069cd40>
>>> a[0]
1.0
>>> a[1]
2.0
>>> a[2]
3.0
>>>For array objects, the from_array() method extracts the underlying memory pointerand casts it to a ctypes pointer object. For example:
>>> import array
>>> a = array.array('d',[1,2,3])
>>> a
array('d', [1.0, 2.0, 3.0])
>>> ptr_ = a.buffer_info()
>>> ptr
4298687200
>>> ctypes.cast(ptr, ctypes.POINTER(ctypes.c_double))
<__main__.LP_c_double object at 0x10069cd40>
>>>The from_ndarray() shows comparable conversion code for numpy arrays.By defining the DoubleArrayType class and using it in the type signature of avg(), asshown, the function can accept a variety of different array-like inputs:
>>> import sample
>>> sample.avg([1,2,3])
2.0
>>> sample.avg((1,2,3))
2.0
>>> import array
>>> sample.avg(array.array('d',[1,2,3]))
2.0
>>> import numpy
>>> sample.avg(numpy.array([1.0,2.0,3.0]))
2.0
>>>The last part of this recipe shows how to work with a simple C structure. For structures,you simply define a class that contains the appropriate fields and types like this:
class Point(ctypes.Structure):fields = [(‘x', ctypes.c_double),(‘y', ctypes.c_double)]
Once defined, you can use the class in type signatures as well as in code that needs toinstantiate and work with the structures. For example:
>>> p1 = sample.Point(1,2)
>>> p2 = sample.Point(4,5)
>>> p1.x
1.0
>>> p1.y
2.0
>>> sample.distance(p1,p2)
4.242640687119285
>>>A few final comments: ctypes is a useful library to know about if all you’re doing isaccessing a few C functions from Python. However, if you’re trying to access a largelibrary, you might want to look at alternative approaches, such as Swig (described inRecipe 15.9) or Cython (described in Recipe 15.10).
The main problem with a large library is that since ctypes isn’t entirely automatic, you’llhave to spend a fair bit of time writing out all of the type signatures, as shown in theexample. Depending on the complexity of the library, you might also have to write alarge number of small wrapper functions and supporting classes. Also, unless you fullyunderstand all of the low-level details of the C interface, including memory managementand error handling, it is often quite easy to make Python catastrophically crash with asegmentation fault, access violation, or some similar error.As an alternative to ctypes, you might also look at CFFI. CFFI provides much of thesame functionality, but uses C syntax and supports more advanced kinds of C code. Asof this writing, CFFI is still a relatively new project, but its use has been growing rapidly.There has even been some discussion of including it in the Python standard library insome future release. Thus, it’s definitely something to keep an eye on.