- français
- English
Quadrokinect
This project aims to create a system that will give you the ability to control a swarm of Parrot AR.Drones using Microsoft Kinect.
Started in september 2012 by Nikita Grishin
Supervised by: Alex Barchiesi
Git repo: https://github.com/sinlab-semester-2012/Quadrokinect
Presentation: http://prezi.com/-5l-jnnzpha0/quadrokinect-11122012/
Presentation video: https://vimeo.com/57151069
Warning: This project was tested only on Mac OS 10.7.x and 10.8.x!
Setup
Hardware
-
Microsoft Kinect
-
Parrot AR.Drone
-
Router
Any model can be used. For this wiki, the D-Link DSL-G804V was used.
Software
-
OpenNI Driver
Installation tutorial for Mac OS X:
http://developkinect.com/resource/mac-os-x/install-openni-nite-and-sensorkinect-mac-os-x
This is the driver for Microsoft Kinect.
-
OSCeleton
GitHub repo:
https://github.com/Sensebloom/OSCeleton
Installation instructions can be found on the GitHub.
This is a program that detects body coordinates and sends them via OSC to the server.
-
Java Virtual Machine
-
XQuartz
OSCeleton might need the X11 server, which is not part of Mac OS X anymore. XQuartz can be installed instead.
Project structure
The project consists of three parts: analysis of OSC messages coming from OSCeleton (i.e. the body coordinates) and intuitive controller realisation, communication with A.R.Drones, and a stabilization system.
The server is written in Java and Processing. Java is used mainly for encapsulation and the logic description. Processing is used for the graphical interface realisation, for comunication with A.R.Drones and Microsoft Kinect and for analyzing the video stream from A.R.Drones.
- The analysis of OSC messages, coming from OSCeleton, uses the oscP5 library. There are many realisations of controllers in this project, but all of them should draw the controller object on screen and update command listeners.
- The communication with A.R.Drones uses a modified version of ARDroneForP5 library. The original ARDroneForP5 is made for the first generation A.R.Drones, but during this project the second generation has been used. The modified version of ARDroneForP5 can be found on the project's GitHub repo.
- The stabilization system uses the NyARToolkit library. It gets the video stream from the "slave" A.R.Drone and looks for the Hiro marker, sticked on the master A.R.Drone. If it finds the marker, it will try to make the "slave" A.R.Drone follow him.
How to use
All instructions concerning the installation and launching of the project can be found on GitHub repo. In case of problems, e-mail nikita.grishin@epfl.ch.
The code is not commented enough, but all names are more or less meaningfull.
The class with the main method is "Quadrokinect.java" (Quadrokinect/src/application). All logic is described in "QuadroKinectSketch.java" (Quadrokinect/src/processing/sketch), and all commented lines can be uncommented if needed.
The package "Quadrokinect.ardrone" contains all classes needed for communication with A.R.Drones.
The package "Quadrokinect.gui" contains all JFrame objects used for showing the Processing sketches.
The package "Quadrokinect.listeners" contains all listeners (for the moment - only OSCListener).
The package "Quadrokinect.processing.sketch" contains all Processing sketches.
The package "Quadrokinect.processing.classes" contains all Processing classes. These are classes that will take an object of PApplet as the constructor argument.
The package "Quadrokinect.processing.classes.controllers" contains all the realisation of controllers and the abstract class "Controller.java" which must be implemented by all controllers.
The package "Quadrokinect.utils" contains static classes with some mathematical methods and some usefull constants.
Future work
Firstly, the stabilization system must be improved. The idea of marker detection could be keeped, but is not mandatory. Probably, the physics engine should be used.
The modifications of ARDroneForP5 should be finished. The code must be completed and supports both versions of A.R.Drones.
Also, the controller could be improved. It should be more intuitive and make user experience better.