A weeks ago a was doing some performance test to [pyrex]. I made an application using genetic
algorithms with a very time consuming fitness function in python and I wanted to improve the
overall performance of the application. That's why I tried pyrex, a python-like programming
language that lets you mix python and C data types and the compiles it into a C extension for
python. The pyrex homepage says nothing about perfomance improvements, because the main purpose
of pyrex is developing python module wrapping C libraries in a easy way. It's more o less the
same thing that SWIG does. But besides this, I wanted to try it.
My Fitness function
The implemetation in pure python of my fitness function can be viewed below. The function is
simple, it receives two lists and try to count the times a given position of the list indiv
has exactly the same value in the traffic list. If this is true, a weight value is added to
count. The weight allow me to favour some position than others.
def compara(indiv,traffic):
wfeat=[0.1,0.1,0.1,0.1,0.5,0.1,0.4,0.4,0.2,0.3,0.7,0.7,0.7,0.7,0.1]
count=0
for instance in traffic:
for feature in xrange(0,len(indiv)):
if indiv[feature] >= 0:
if indiv[feature] == instance[feature]:
count+=(1 * wfeat[feature])
else:
count+=(0.25 * wfeat[feature])
else:
count+=(1 * wfeat[feature])
return count
This simple function is very time consuming because it's invoqued more than 30000 times. So
my idea was to implement this function y pyrex and see what happens.
The implementation in pyrex was very easy, is basically python with types. I have to put some array limits like in the
wfeat, and the function also has hardcode the bondary check to 14. Note the non-pythonic for implementation, this is the
way pyrex recomends to use for. Here is my compara.pyx
def compara (indiv,instance):
cdef int feature
cdef float wfeat[14]
cdef float count
wfeat[0]=0.1
wfeat[1]=0.1
wfeat[2]=0.1
wfeat[3]=0.1
wfeat[4]=0.5
wfeat[5]=0.1
wfeat[6]=0.4
wfeat[7]=0.4
wfeat[8]=0.2
wfeat[9]=0.3
wfeat[10]=0.7
wfeat[11]=0.7
wfeat[12]=0.7
wfeat[13]=0.7
count=0.0
for feature from 0 <= feature < 14:
if indiv[feature] >= 0:
if indiv[feature] == instance[feature]:
count=count+(1 * wfeat[feature])
else:
count=count+(0.25 * wfeat[feature])
else:
count=count+(1 * wfeat[feature])
return count
The next thing I do is to use the pyrex tool to build a C file ready to be built as a python module. I simply do:
pyrex compara.pyx
gcc -c -fPIC -I/usr/include/python2.4/ compara.c
After that I got a compara.so dynamic python module ready to be imported into my application. Yupppie! After that I tested
the new pyrex module and for my surprise the result was not the one I expected, the overall time of the application had
barelly changed :(. I was very dissapointed. Googling a bit looking for an answer for this bad result I found this [IBM
article] that explain my problem. Basically pyrex speedups python programs with a significant amount of numeric calculation.
But if a program spend more of its time doing libraries call is just not going to be benefited by pyrex. In order to speed up
the code you need to avoid the use of the most native python types as you can. In my case indiv and instance are
still python natives types and changing that is not a trivial task.
I have been using the [psyco] JIT compiler with excelente results and with a minimun source code modification needed, but psyco
faces the problem of being only supported for IA32 systems and I have a few AMD64 machines that I would like to use... I was
hoping pyrex saved me of this problem but this was not the case.
. That's not the case for me, but my girlfriend
Lili is a visual artist, so she painted the cabinets with some fancy geek
motives, including Ubuntu and Firefox logos among others.
