Sunday, April 13, 2014

Solved: cx_Freeze and OpenCV, app is portable!

Just managed to port my python OpenCv face detection app, Behave, to another mac and run ok using cx_freeze.

The trick is to copy the /lib directory from the opencv installation, into the application directory, then tell cx_freeze to use it.

In the base directory of your app:
mkdir lib
cp /usr/opencv-64/lib .
Change /usr/opencv-64 for the path to where opencv has been installed.

And then modify your setup.py:
from cx_Freeze import setup, Executable

setup(  name = "behave",
        version = "0.1",
        description = "Behave, face detection to help you",
        options = {'build_exe':
                    {'includes': ['numpy'],
                        'include_files': ['cascades/', 'lib/']}},
        executables = [Executable("capturebasic.py")])
The las entry in 'include_files' is the bit that tells OpenCv to bundle the whole folder with the app.
I'm sure there must be a more elegant way of doing this, but this one works: happy days!


Saturday, April 12, 2014

cx_Freeze and OpenCV, fixing numpy imports and cascades as data files.


After all the problems found trying cx_Freeze in Mac to create the app in 32-bits (ref) I decided to bend like a bamboo in the wind, and move to OpenCV-64 bit.

Now, a couple of problems found:

When packaging the app with:
python2.7 setup.py bdist_dmg
An error would arise trying to execute it:
open build/exe.macosx-10.6-intel-2.7/capturebasic
"cx-freeze numpy.core.multiarray failed to import"
This error is solved by adding numpy in your setup.py, the reason behind is that even though OpenCv's Python wrapper uses numpy, cx_freeze fails to detect it in its dependency crawler.

setup.py then looks like:
from cx_Freeze import setup, Executable
setup(  name = "behave",
        version = "0.1",
        description = "Behave, face detection to help you",
        options = {'build_exe':
                    {'includes': ['numpy']}},
        executables = [Executable("capturebasic.py", )])
The second problem I encounter after that is that the cascades directory, where the xml files that the face detector algorithm uses, were not included in the cx_freeze app's directory.
The error you can find is similar to:
cv2.error: ../cascadedetect.cpp:1578: error: (-215) !empty() in function detectMultiScale

That is, instantiating "cv2.CascadeClassifier(path_to_xml_file)" will give no errors if you pass a wrong path, but when it is used later in the code to actually detect something, it will complain.

How to solve this? you need to add the cascade files in the project:
from cx_Freeze import setup, Executable
setup(  name = "behave",
        version = "0.1",
        description = "Behave, face detection to help you",
        options = {'build_exe':
                    {'includes': ['numpy'],
                    'include_files': ['cascades/haarcascade_frontalface_alt.xml']}},
        executables = [Executable("capturebasic.py", )])
One more step is required, since we are using those haarcascade files as data files inside cx_freeze, we need to tell the app where to find them, adding a function to our code:
def find_data_file(filename):
    if getattr(sys, 'frozen', False):
        # The application is frozen
        datadir = os.path.dirname(sys.executable)
    else:
        # The application is not frozen
        # Change this bit to match where you store your data files:
        datadir = os.path.dirname(__file__)

    return os.path.join(datadir, filename)
And using it any time we try to access the cascades we are using.
Inside the code it looks like:
self.face_classifier = classifiers.CascadeClassifier(
           find_data_file('cascades/haarcascade_frontalface_alt.xml'))
More information here:
http://cx-freeze.readthedocs.org/en/latest/faq.html#using-data-files


Thursday, April 10, 2014

How to install OpenCv in Mac Os 10.8.5

Easy way that simply works, this will install 64bit version of opencv, using python2.7.

First, install git and get opencv from github:
git clone https://github.com/Itseez/opencv.git
Then:
cd opencv
mkdir 64-bit
cd 64-bit
make -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/opencv-64 -D PYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python2.7 -D INSTALL_PYTHON_EXAMPLES:BOOL=ON ..
make -j 8  #j 8 speeds up things by doing parallel jobs
sudo make install
CMAKE_INSTALL_PREFIX is the path where OpenCv libs will be installed.
FILEPATH is the path to the python you want OpenCv to use.

We need to set some env variables for when opencv library is imported inside our python code, in bold:
vim ~/.bash_profile
###-------
export PYTHONPATH=/usr/opencv-64/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages/:/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages:
export DYLD_LIBRARY_PATH=/usr/opencv-64/lib/
Last part (DYLD...) tells python where to find opencv libs when importing.

And that's it, you should be able to python; import cv2.

Tuesday, April 8, 2014

mach-o, but wrong architecture Part II

I've tried to use cx-freeze with the setup.py instead of the script, same same.
Ref:
http://cx-freeze.readthedocs.org/en/latest/distutils.html#distutils

How to install cx-freeze from source to use a 32 bit version of python, in this case python2.7-32.
1. Download source from:
http://cx-freeze.sourceforge.net/
2. Untar it. tar xfv or jus doubleclick
3. build/install:
python2.7-32 setup.py build
python2.7-32 setup.py install
Then you can run cx-freeze in your 32 bit version:
python2.7-32 setup.py bdist_dmg
 No matter what, the app generated is still a 64bit version:
$ python2.7-32 setup.py bdist_dmg
$ file capturebasic
capturebasic: Mach-O 64-bit executable x86_64
$ arch -i386 ./capturebasic
arch: posix_spawnp: ./capturebasic: Bad CPU type in executable
I think the problem is that I'm installing the 64bit version of cx-freeze, even if I am running python2.7-32 setup.py build.

So, what next?
- keep banging my head against this?
- move to 64bit opencv and forget?

Thursday, April 3, 2014

mach-o, but wrong architecture

Using cx-freeze to create a package to distribute behave, some problems with it.
>> python2.7-32 /Library/Frameworks/Python.framework/Versions/2.7/bin/cxfreeze capturebasic.py
[..]
>> ./capturebasic
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 27, in <module>
    exec code in m.__dict__
  File "capturebasic.py", line 1, in <module>
ImportError: dlopen(/Users/Luis/Dropbox/PyProjects/behave/dist/cv2.so, 2): no suitable image found.  Did find:
/Users/Luis/Dropbox/PyProjects/behave/dist/cv2.so: mach-o, but wrong architecture
This is the same error that I got when opencv has been built and installed in its 32-version and then you try to use from within a 64bits Python version.

I even tried to build/install cx-freeze in 32 bit:
  python2.7-32 setup.py build
  python2.7-32 setup.py install 
But same errors.

My best guess at the moment is that cxfreeze puts the 64-bit version in the app by default, and cannot import cv2 from opencv.

Options:
- try to force cxfreeze into python2.7-32
- re-install opencv, the reason why I installed the 32bit version was because it was the one compatible with pygame, since I'm not using pygame, maybe is a good chance to move to 64.

Sunday, March 30, 2014

OpenCv, calculate average fps in python

After having left the app to run for a couple of hours, I noticed that the memory had more than doubled.

From 70MB to 150MB of "Real Memory" as Mac's Activity Monitor puts it.

What was happening is that I had created a list that was growing with each frame in order to calculate the average of the frames.

I decided to get rid of this, first creating a circular counter in Python using a generator that resets to 0 when reaches a max value. Then initiating a fixed-size list that the generator will fill the fps in a circular manner.

It looks like this:
def circular_counter(max):
    """helper function that creates an eternal counter till a max value"""
    x = 0
    while True:
        if x == max:
            x = 0
        x += 1
        yield x

class CvTimer(object):
    def __init__(self):
        self.tick_frequency = cv2.getTickFrequency()
        self.tick_at_init = cv2.getTickCount()
        self.last_tick = self.tick_at_init
        self.fps_len = 100
        self.l_fps_history = [ 10 for x in range(self.fps_len)]
        self.fps_counter = circular_counter(self.fps_len)
    def reset(self):
        self.last_tick = cv2.getTickCount()
    def get_tick_now(self):
        return cv2.getTickCount()
    @property
    def fps(self):
        fps = self.tick_frequency / (self.get_tick_now() - self.last_tick)
        self.l_fps_history[self.fps_counter.next() - 1] = fps
        return fps
    @property
    def avg_fps(self):
        return sum(self.l_fps_history) / float(self.fps_len)
In your frame-by-frame while loop:
#Timecv:
cv2.putText(self.a_frame, "fps=%s avg=%s" % (timer.fps, timer.avg_fps),
            (10, 75), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255,255,255))
And finally in the app window:

Behave, declaration of principles

The idea of controlling yourself in front of computer helped by an app was born around three years ago while cycling to work. It was a sunny morning, precious situation in Ireland, riding on Strawberry Beds road direction Leixlip.

It was clear that had to use machine learning techniques, computer vision and Python. 

Three years have passed, today there is a running app that does the basics of what I had in mind, and it works.

It works so well that I use it as I code, the more I use it, the more need to code it to improve it, the more I code it, the more I need it... an unexpected vicious circle.

Behave is open source and free, its aim is to help people, serve as support to maintain a healthier back and habits in front of a computer. 

Because I firmly believe that health should be a right and not a benefit nor a business, this software is and it will be free.

The dream is to reach schools where youngsters are starting to spend time in front of the screen and to prevent them from back problems, and other habits, as they grow up :)

That's it, it is written.