Thursday, May 8, 2014

Webcam capture with OpenCV and Tkinter: widget.after(...)

So, how to show the webcam using OpenCV and Tkinter?

There seem to be two alternatives, one is using the widget.after(delay, function) method, the other one is multithreading/multiprocessing.

For widget.after option, this is the basic code:
import Tkinter as tk
import cv2
from PIL import Image, ImageTk

width, height = 800, 600
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)

root = tk.Tk()
root.bind('', lambda e: root.quit())
lmain = tk.Label(root)
lmain.pack()

def show_frame():
    _, frame = cap.read()
    frame = cv2.flip(frame, 1)
    cv2image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGBA)
    img = Image.fromarray(cv2image)
    imgtk = ImageTk.PhotoImage(image=img)
    lmain.imgtk = imgtk
    lmain.configure(image=imgtk)
    lmain.after(10, show_frame)

show_frame()
root.mainloop()
The after methods are ways of telling Tkinter to call functions after it has processed the events in the 'mainloop'.
When show_frame (Line 25) is called, a new frame is processed and the final call lmain.after(10, show_frame) outside the mainloop, will register 'show_frame' as after method to be called in the loop with the delay in milliseconds passed (10).

Later on, when inside the mainloop (Line 26), there is a check for if there are any 'after' methods at the end of the event processing, in this case it will find 'show_frame' and call it, the method will 'hook' itself to the mainloop, so it will be called semi-recursively.

So, think like this: the 'show_frame' will tell Tkinter mainloop: 'hey! I'm here, call me back in 10 milliseconds', tkinter will do its normal life, processing events, etc, and then after 10 milliseconds will be obedient and will call 'show_frame'. Method runs its code and at the end does the sneaky thing of saying again 'hey!, I'm here, call me...'

So, in each of those calls to the method, we take a frame captured by OpenCV's camera capturer. And that is the way you are able to capture frames and display them in Tkinker.

There is a bonus track also in all this:
   lmain.imgtk = imgtk
In spite of appearances it is not a superfluous piece of code, it is much needed, since the pseudo-recursive nature of the after call, it would garbage-collect the variable imgtk, we have to anchor it somewhere so it does not get deleted. We could also set it as "global" :|

Last word on widget.after: watch out for performance, the trick is on the delays set to the 'after' method. In this simple code if the delay of 10 is changed to 1, you can see the video capture with constant jumps in frames. With the value of 10 or even 100, it works smoothly.

I've got best results with the delay parameter when is set to more or less the time that takes the VideoCapture to take a frame. That is, we are talking about 'FPS' :) .

Refs to understand the after method:
http://effbot.org/tkinterbook/widget.htm#Tkinter.Widget.after-method

A bit more advanced, good read:

Sunday, May 4, 2014

New Behave version, user input issues.

New version of Behave is available.

The main difference for the user is that now it allows to auto adjust the face lower limit.

When you press space in the app, it will tell you to sit with your back straight for 5 seconds, during that time it will save your face position, calculate average position and size, and set the a lower limit for you, no manual input needed. And enforce it nicely :)

The code has also evolved a good deal, what started as basic functionality with initialisation of needed bits with a while(true) loop, moved towards being more modular via creating functions, and then pursuing further modularity using classes, and as last stage: expanding the user input towards a gui-event based paradigm.

What I'm finding is that more and more the user interface expands, the more complex the code becomes, losing part of its original orthogonality, that is, when I need to add a new handle for specific event, I usually need to touch other parts of the code to accommodate the new functionality.

What is happening?
The application is especially vulnerable to events that require different handling depending on the context... The code is littered with a multitude of global variables and flags that serve only one purpose -- handling the context.... With this representation, the context of the computation is represented ambiguously, so it is difficult to tell precisely in which mode the application is at any given time. Actually, the application has no notion of any single mode of operation, but rather tightly coupled and overlapping conditions of operation determined by values of the global variables and flags.

Found in Who moved my state, that reveals the weaknesses of event-driven programming and advocates the use of finite state machines.

As I see it, Behave's code should move from a main loop reacting to user keystrokes and mouse clicks, towards two separate modules with their respective loops in different threads.  One would run the frame capturing and what to do with it, the other would create the gui and facilitate start/stop and interact/modify with the first.

I'll be digging into finite state machines in the next days and how to implement them with Behave.

Some reading done:
http://infohost.nmt.edu/tcc/help/pubs/tkinter/tkinter.pdf
http://www.astro.washington.edu/users/rowen/ROTKFolklore.html
http://www.astro.washington.edu/users/rowen/TkinterSummary.html#Resources
http://www.ferg.org/thinking_in_tkinter/all_programs.html
Tutorial on guis using Tkinter and Python:
http://www.alan-g.me.uk/l2p/tutevent.htm
http://www.alan-g.me.uk/l2p/tutgui.htm

Finite state machine implementation in Python:
http://www.python-course.eu/finite_state_machine.php
These guys have also a Tkinter tutorial:
http://www.python-course.eu/python_tkinter.php

Sunday, April 13, 2014

Solved: cx_Freeze and OpenCV, app is portable!

Just managed to port my python OpenCv face detection app, Behave, to another mac and run ok using cx_freeze.

The trick is to copy the /lib directory from the opencv installation, into the application directory, then tell cx_freeze to use it.

In the base directory of your app:
mkdir lib
cp /usr/opencv-64/lib .
Change /usr/opencv-64 for the path to where opencv has been installed.

And then modify your setup.py:
from cx_Freeze import setup, Executable

setup(  name = "behave",
        version = "0.1",
        description = "Behave, face detection to help you",
        options = {'build_exe':
                    {'includes': ['numpy'],
                        'include_files': ['cascades/', 'lib/']}},
        executables = [Executable("capturebasic.py")])
The las entry in 'include_files' is the bit that tells OpenCv to bundle the whole folder with the app.
I'm sure there must be a more elegant way of doing this, but this one works: happy days!


Saturday, April 12, 2014

cx_Freeze and OpenCV, fixing numpy imports and cascades as data files.


After all the problems found trying cx_Freeze in Mac to create the app in 32-bits (ref) I decided to bend like a bamboo in the wind, and move to OpenCV-64 bit.

Now, a couple of problems found:

When packaging the app with:
python2.7 setup.py bdist_dmg
An error would arise trying to execute it:
open build/exe.macosx-10.6-intel-2.7/capturebasic
"cx-freeze numpy.core.multiarray failed to import"
This error is solved by adding numpy in your setup.py, the reason behind is that even though OpenCv's Python wrapper uses numpy, cx_freeze fails to detect it in its dependency crawler.

setup.py then looks like:
from cx_Freeze import setup, Executable
setup(  name = "behave",
        version = "0.1",
        description = "Behave, face detection to help you",
        options = {'build_exe':
                    {'includes': ['numpy']}},
        executables = [Executable("capturebasic.py", )])
The second problem I encounter after that is that the cascades directory, where the xml files that the face detector algorithm uses, were not included in the cx_freeze app's directory.
The error you can find is similar to:
cv2.error: ../cascadedetect.cpp:1578: error: (-215) !empty() in function detectMultiScale

That is, instantiating "cv2.CascadeClassifier(path_to_xml_file)" will give no errors if you pass a wrong path, but when it is used later in the code to actually detect something, it will complain.

How to solve this? you need to add the cascade files in the project:
from cx_Freeze import setup, Executable
setup(  name = "behave",
        version = "0.1",
        description = "Behave, face detection to help you",
        options = {'build_exe':
                    {'includes': ['numpy'],
                    'include_files': ['cascades/haarcascade_frontalface_alt.xml']}},
        executables = [Executable("capturebasic.py", )])
One more step is required, since we are using those haarcascade files as data files inside cx_freeze, we need to tell the app where to find them, adding a function to our code:
def find_data_file(filename):
    if getattr(sys, 'frozen', False):
        # The application is frozen
        datadir = os.path.dirname(sys.executable)
    else:
        # The application is not frozen
        # Change this bit to match where you store your data files:
        datadir = os.path.dirname(__file__)

    return os.path.join(datadir, filename)
And using it any time we try to access the cascades we are using.
Inside the code it looks like:
self.face_classifier = classifiers.CascadeClassifier(
           find_data_file('cascades/haarcascade_frontalface_alt.xml'))
More information here:
http://cx-freeze.readthedocs.org/en/latest/faq.html#using-data-files


Thursday, April 10, 2014

How to install OpenCv in Mac Os 10.8.5

Easy way that simply works, this will install 64bit version of opencv, using python2.7.

First, install git and get opencv from github:
git clone https://github.com/Itseez/opencv.git
Then:
cd opencv
mkdir 64-bit
cd 64-bit
make -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/opencv-64 -D PYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python2.7 -D INSTALL_PYTHON_EXAMPLES:BOOL=ON ..
make -j 8  #j 8 speeds up things by doing parallel jobs
sudo make install
CMAKE_INSTALL_PREFIX is the path where OpenCv libs will be installed.
FILEPATH is the path to the python you want OpenCv to use.

We need to set some env variables for when opencv library is imported inside our python code, in bold:
vim ~/.bash_profile
###-------
export PYTHONPATH=/usr/opencv-64/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages/:/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages:
export DYLD_LIBRARY_PATH=/usr/opencv-64/lib/
Last part (DYLD...) tells python where to find opencv libs when importing.

And that's it, you should be able to python; import cv2.

Tuesday, April 8, 2014

mach-o, but wrong architecture Part II

I've tried to use cx-freeze with the setup.py instead of the script, same same.
Ref:
http://cx-freeze.readthedocs.org/en/latest/distutils.html#distutils

How to install cx-freeze from source to use a 32 bit version of python, in this case python2.7-32.
1. Download source from:
http://cx-freeze.sourceforge.net/
2. Untar it. tar xfv or jus doubleclick
3. build/install:
python2.7-32 setup.py build
python2.7-32 setup.py install
Then you can run cx-freeze in your 32 bit version:
python2.7-32 setup.py bdist_dmg
 No matter what, the app generated is still a 64bit version:
$ python2.7-32 setup.py bdist_dmg
$ file capturebasic
capturebasic: Mach-O 64-bit executable x86_64
$ arch -i386 ./capturebasic
arch: posix_spawnp: ./capturebasic: Bad CPU type in executable
I think the problem is that I'm installing the 64bit version of cx-freeze, even if I am running python2.7-32 setup.py build.

So, what next?
- keep banging my head against this?
- move to 64bit opencv and forget?

Thursday, April 3, 2014

mach-o, but wrong architecture

Using cx-freeze to create a package to distribute behave, some problems with it.
>> python2.7-32 /Library/Frameworks/Python.framework/Versions/2.7/bin/cxfreeze capturebasic.py
[..]
>> ./capturebasic
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 27, in <module>
    exec code in m.__dict__
  File "capturebasic.py", line 1, in <module>
ImportError: dlopen(/Users/Luis/Dropbox/PyProjects/behave/dist/cv2.so, 2): no suitable image found.  Did find:
/Users/Luis/Dropbox/PyProjects/behave/dist/cv2.so: mach-o, but wrong architecture
This is the same error that I got when opencv has been built and installed in its 32-version and then you try to use from within a 64bits Python version.

I even tried to build/install cx-freeze in 32 bit:
  python2.7-32 setup.py build
  python2.7-32 setup.py install 
But same errors.

My best guess at the moment is that cxfreeze puts the 64-bit version in the app by default, and cannot import cv2 from opencv.

Options:
- try to force cxfreeze into python2.7-32
- re-install opencv, the reason why I installed the 32bit version was because it was the one compatible with pygame, since I'm not using pygame, maybe is a good chance to move to 64.