Tag Archives: Studies

Leap2Arduino2Tx – Controlling a Drone Using the Leap Motion

Arvid Bräne - June 1st, 2015 - #Code, #Multirotors, #Open Source, #Studies

During the first half of 2015 I spent a lot of time on a school project I called Leap2Arduino2Tx (easy name to remember, right?) in a course called Digital Vision at Umeå University. The goal of the project was to be able to control a multirotor (or a drone, whatever you want to call it) by only using gestures from one hand. It might sound like something out of Minority Report, but the solution is actually quite simple!

Another goal of the project was also to do this without the need for any hardware modifications to the existing transmitter (in my case a Turnigy 9X). The reason for this is mostly that I want people with very little prior skill with electronics to be able to recreate this project, but also because I’m student with a limited budget. In this article I will cover a basic overview of the general parts in my solution, if you want to check out the code yourself you are more than welcome! Below is a short video  demonstrating how the controls work:

Solution

My solution consists of 6 parts in order for it to work; a hand, a Leap Motion, a computer (preferably a Mac), an Arduino (preferably an Arduino Uno or better), a transmitter (preferably a Turnigy 9X) and a drone of your choice, see image below.

Parts

The Leap Motion analyses the hand, sends data to a computer via USB that crunches the numbers and sends them to an Arduino via USB that converts numbers to a PPM pulse that then is transfered via a cable plugged into the trainer port to the transmitter that then sends the commands via 2,4 GHz to the drone. Simple!

This solution might look complicated, but it’s really just a long chain of converting numbers to other numbers; let’s break it down:

  1. The user moves his/hers hand (like in the video above) above the Leap Motion.
  2. The Leap Motion tracks the users hand using a depth-sensor and quickly figures out what motion the user is doing with his/hers hand at more than 60 times per second.
  3. The Leap Motion sends this data over USB to the computer that is running a Python program that is constantly recording this data and translating it into the same type of numbers as the transmitter is transmitting.
  4. The computer then sends these numbers (using CSV) via USB to an Arduino that reads these numbers and converts them into a (digital) PPM signal.
  5. Using one of the digital output pins the Arduino then sends these numbers to transmitter via the 3,5 mm trainer-port on the back of the transmitter.
  6. The transmitter (which is put into buddy box mode) then reads this signal (around every 22 ms) and then sends them to the drone via the standard 2,4 GHz band.
  7. The receiver on the drone takes these numbers and converts them to an analog PWM (if you are using PWM of course) signal which the control chip on your drone then interprets.

Bellow is a short video showing the necessary setup steps to get Leap2Arduino2Tx to work, detailed instructions can be found on the GitHub page.

Drawbacks

Since this is more of an MVP than a finished product it has some limitations and drawbacks, here are some:

  • The drone I used to test this out is a small 250 mm racing drone designed to go fast. It has no GPS which means it can’t stay in one point by itself, making it hard to control. The size of the quadrocopter is also an issue since its very responsive in its movements and easily influenced by the wind. This could be solved by building this solution for a more competent drone, such as a DJI Phantom.
  • The Arduino is limited in its USB-bus; it’s very hard to send large amounts of data and do precise PPM-conversions in real-time. This could be solved by excluding the Arduino and doing the PPM-conversion on the computer itself (something I didn’t have time to look into during the extent of the course) and sending it via USB to the transmitter directly.
  • I chose Python as a language for three reasons; I had some prior basic experience with it, it would be easy to make a GUI to show the stick movements and because the Leap SDK was available in Python. Since Python is a high-level programming language it requires a lot of resources (resources my five year old MacBook Pro didn’t really have), especially when using the Leap SDK which slows everything down.

Conclusion

My solution is defiantly not something I would use in its current state (see drawbacks), with that being said I think it has a huge potential as a proof-of-concept for others to develop upon.

During the project I emailed some large drone companies (3D Robotics, DJI etc.) to see if they wanted to help me in my research by giving/lending me a drone (I also included a link to the GitHub page so they could see what I was up to). Unfortunately none of these companies replied so I had to continue the work with my small racing drone. About one month after I sent these emails DJI released this video of the company using a Leap Motion to control one of their drones. I’m not accusing them of stealing my idea (which by the way has been done before, kind of), I just find it interesting that they decided to do it just now. The Leap Motion has been out for more than 3 years (?) and DJI has been making drones even longer, why the sudden interest?

If you feel like this is something you would try to replicate or build upon, you have my permission and blessing to check out the project on GitHub. If you want to dive even deeper into this project you might want to check out the Keynote or the Project Report.

One Month in Hong Kong

Arvid Bräne - September 24th, 2014 - #Personal, #Studies, #Travel, #University

So I moved to Hong Kong. Yes, the “small-half-island-jungle-full-with-people-that-used-to-be-an-english-colony-that-china-now-controls”-country is now my home. I’m currently one month into my four-month-semester-stay and I’ve just gotten used to the climate, food and culture. This is a short update on how things are going.

Living

sam the wiseI’ve been one of the fortunate to get a room at the student residence at CityU. I live in a 12 m2 room that I share with my roommate Sam, my bed is as hard as a rock, I pay for AC by the minute, I share my toilet with 4 people, I have no place to cook my own food, the WiFi feels like dial-up from the ’90’s and I wait for the elevator about 30-40 minutes per day; but I still consider myself very lucky. Living in Hong Kong is usually very expensive (in the top 10 most expensive cities in the world to live in) but as I live in a dorm I only pay about $4500 HKD (around $640 USD), and that’s for a whole semester! I can’t really complain about the distance to school either; for most of my courses it takes around 30 seconds to get to class (not counting the unbelievable amount of time I spend waiting for elevators).

School

One of the biggest differences so far is definitely school; even though City University of Hong Kong is a great school, it’s no where near Umeå University when it comes to the teaching quality and class participation. It might be the courses I’m taking (Media & Communication mostly); it might be that the serious stuff is still to come, but so far I feel way too relaxed about the workload.
cityu cmc City University of Hong Kong consists of four buildings: Academic Building 1, 2, 3 (on the left) and Run Run Shaw Creative Media Center (on the right).

Food & Culture

taxi-drivers do love there phones...The food is undoubtedly different and it’s not too uncommon for restaurants to have menus in only Cantonese, which makes ordering food that much harder. It takes some getting used to but most of the food is actually good, even though, in most cases, its looks beg to differ. The food (and most of the other stuff you buy for that matter) is fortunately quite cheap, compare to Sweden, so trying out new things doesn’t have to set you back a fortune. Even the public transportation is cheap and don’t even get me started on taxis.

Travel & Nature

One misconception of Hong Kong is that the country solely consists of the main city; this is obviously a load of bollocks since only 10% of the country is actual “city ground”. The jungle makes for great hiking and exploring. Catching a boat to any of the small islands (which feels like traveling to another country) outside the city is often cheaper than your average meal.

A panorama shot just above where I live, Kowloon Tong

A panorama shot just above where I live, Kowloon Tong

I’m planing to travel a lot during my time here; so far I’ve been to Macao and I have tickets to Taiwan for next week. Other places I’m planing to visit are South Korea, Cambodia, Singapore and hopefully Malaysia and/or The Philippines. This is going to be a hectic autumn, that’s for sure. See you in a couple of months Sweden and as my mother always says:

The best thing about living abroad is that you learn to appreciate home