Making LeapMotion and Arduino talk using Bluetooth

After having received the Dev Kit from LeapMotion team a while back, we enjoyed playing with the many demos, examples and libraries posted by fellow Leap Developers. Many asked for linking with H/W including Android, Raspberry Pi and Arduino. A few mustered the courage to get Hardware rolling using the Leap. Some examples included the control of a Labyrinth maze using the Leap (http://www.youtube.com/watch?v=eHpD3Wuj2Co).

In another recent development, Xavier (http://xseignard.github.io/2013/06/25/interfacing-leap-motion-with-arduino-thanks-to-nodejs/) successfully Interfaced the Leap and Arduino using the Node.js library. This was pretty cool, except for the fact that the Arduino had to be connected to the USB of the PC  from where it was receiving data using the Firmata protocol. A rather unfortunate(or fortunate) thing is that I burned up 2 of my USB ports leaving only one port working on my Laptop to which I had a choice of connecting either the Leap or the Arduino. So, in essence, we couldn’t try the above demo.

That set me thinking wouldn’t it be a lot cooler if we could use our Laptops’s inbuilt Bluetooth module to send Leap data wirelessly to a ‘Bluetooth enabled Arduino’ and have some fun? That idea, coupled with some intense coding, a boring Saturday, some electric shocks resulted in this:

For those already familiar with Java, Eclipse, LeapMotion, Bluetooth RXTX using Java, here’s the Source code with the references:

1. http://rxtx.qbang.org/wiki/index.php/Two_way_communcation_with_the_serial_port

2. http://mfizz.com/oss/rxtx-for-java (DLLs for 64 Bit Machine)

3. http://playground.arduino.cc//Interfacing/Java

4. Leap Java Example from SDK

Java Code:


package com.leaparduino.hello;

/******************************************************************************\
* Copyright (C) 2012-2013 Leap Motion, Inc. All rights reserved. *
* Leap Motion proprietary and confidential. Not for distribution. *
* Use subject to the terms of the Leap Motion SDK Agreement available at *
* https://developer.leapmotion.com/sdk_agreement, or another agreement *
* between Leap Motion and you, your company or other organization. *
\******************************************************************************/

import gnu.io.CommPort;
import gnu.io.CommPortIdentifier;
import gnu.io.SerialPort;

import java.io.IOException;
import java.io.OutputStream;

import com.leapmotion.leap.Controller;
import com.leapmotion.leap.Frame;
import com.leapmotion.leap.Listener;
//import java.lang.Math;
class SampleListenerMain extends Listener {

public void onInit(Controller controller) {
System.out.println("Initialized");
}

public void onConnect(Controller controller) {
System.out.println("Connected");
}

public void onDisconnect(Controller controller) {
System.out.println("Disconnected");
}

public void onExit(Controller controller) {
System.out.println("Exited");
}
public void onFrame(Controller controller) {
Frame frame = controller.frame();
System.out.println(frame.fingers().count());
LeapIntoArduino.writeToArduino(frame.fingers().count());
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

class LeapIntoArduino {
static OutputStream out = null;

public static void main(String[] args) {
//Connect to Arduino BT
try
{
(new LeapIntoArduino()).connect("COM12");
}
catch ( Exception e )
{
e.printStackTrace();
System.exit(0);
}

// Create a sample listener and controller
SampleListenerMain listener = new SampleListenerMain();
Controller controller = new Controller();

// Have the sample listener receive events from the controller
controller.addListener(listener);
// Keep this process running until Enter is pressed
System.out.println("Press Enter to quit...");
try {
System.in.read();
} catch (IOException e) {
e.printStackTrace();
}

// Remove the sample listener when done
controller.removeListener(listener);
}

void connect ( String portName ) throws Exception {

CommPortIdentifier portIdentifier = CommPortIdentifier.getPortIdentifier(portName);
if ( portIdentifier.isCurrentlyOwned() )
{
System.out.println("Error: Port is currently in use");
}
else
{
CommPort commPort = portIdentifier.open(this.getClass().getName(),2000);

if ( commPort instanceof SerialPort )
{
SerialPort serialPort = (SerialPort) commPort;
serialPort.setSerialPortParams(9600,SerialPort.DATABITS_8,SerialPort.STOPBITS_1,SerialPort.PARITY_NONE);
out = serialPort.getOutputStream();
}
else
{
System.out.println("Error: Only serial ports are handled by this example.");
}
}
}

public static void writeToArduino(int noOfFingers)
{
String tmpStr = Integer.toString(noOfFingers);;
byte bytes[] = tmpStr.getBytes();
try {
out.write(bytes);
} catch (IOException e) { }
}
}

Arduino Code:


#include <LiquidCrystal.h>
LiquidCrystal lcd(8,9,10,11,12,13);
char ch;
void setup()
{
Serial.begin(9600);
lcd.begin(16,2);
pinMode(2,OUTPUT);
pinMode(3,OUTPUT);
pinMode(4,OUTPUT);
digitalWrite(2,LOW);
digitalWrite(3,LOW);
digitalWrite(4,LOW);
}
void loop()
{
lcd.setCursor(0,0);
lcd.print("Remote Value:");

if (Serial.available()) {
lcd.setCursor(0,1);
ch = Serial.read();
lcd.print(ch);
if(ch == '1')
digitalWrite(2,HIGH);
if(ch == '2')
digitalWrite(2,LOW);
if(ch == '3')
digitalWrite(3,HIGH);
if(ch == '4')
digitalWrite(3,LOW);
if(ch == '5')
digitalWrite(4,HIGH);
if(ch == '6')
digitalWrite(4,LOW);
delay(50);
}
delay(10);
}

Quick Notes:

  1. The Bluetooth serial port in my Laptop was COM13, hence used that in the code. Please replace this with your COM PORT NO.
  2. The Arduino sketch includes code for LCD also. Since I didn’t have an extra serial port on Laptop to debug, I used this. This can be removed.
  3. The if conditions in Arduino can be replaced with well mannered if-else checks
  4. Please use the right version of rxtxSerial.dll (32 / 64 bit) in your Java JRE’s bin.
  5. Make sure you have added the LeapJava.jar (and also the DLLs) and RXTXcomm.jar to the Eclipse project

I’ll update a detailed step-by-step version soon. Meanwhile, if you’ve any questions, please ask.

Thanks!

18 thoughts on “Making LeapMotion and Arduino talk using Bluetooth

  1. Super ! It took a while for me to understand whats this when I first read the name of the Topic, but the video made it clear. Great job, keep going.

  2. HI! I’m Korean student and doing robot project by Processing and Arduino. So I use bluetooth module to send Leap Motion data to a bluetooth arduino shield. Then, I found some source about bluetooth, so found your source. However in my source, there are many error… if you have some time, I want to ask your help. Thank you!!

    1. Hi Kim, Yes, I’d be able to help you out. Please email me on syed [at] anwaarullah.com with further details and we can take it forward from there.

    1. Once you pair your PC’s Bluetooth with the remote Bluetooth Module that’s connected to the Arduino, you can find the COM port by opening up the Device Manager on Windows.

      Bluetooth COM Port

      Alternatively, you can open up the Bluetooth Adapter settings on PC and will be presented with a a list of COM ports of paired devices.
      COM Ports - Bluetooth Settings

      Hope that helps.

  3. Hey Anwaarullah this has been super helpful because I’m trying something very similar with Arduino and Leap Motion. The error I continue to run into in is a PortInUseException for the Bluetooth connection. I am confident I am using the right port and have the correct RXTX plugins for my Windows 7 64-bit computer. Any advice or help? I’ve been able to connect to my HC-06 Bluetooth adapter using Tera Term but nothing else. Thanks!

    1. Dave, can you please provide a screenshot of the COM ports that show up the Bluetooth Adapter of the PC (and also indicate which COM port you’re using from the list). Are you sure that the PC’s Bluetooth Adapter is not being used by any other process/service?

      1. Here are some screenshots of what I’m working with (http://imgur.com/a/MKb4a). I’ve been trying to connect to COM5 because when I used Tera Term, I was able to successfully connect via COM5 (the Bluetooth module LED went from blinking to steady). As far as I know the Bluetooth adapter on the Arduino has been properly connected and is not being used by any other process. The adapter in question can be found here (http://www.amazon.com/KEDSUM-Arduino-Wireless-Bluetooth-Transceiver/dp/B0093XAV4U?ie=UTF8&psc=1&redirect=true&ref_=oh_aui_detailpage_o01_s01). Is there a way to confirm the adapter is NOT being used by some other process? Thanks for the help and quick response!

  4. Assuming that you’ve already paired the JY-MCU Bluetooth Module with the PC, can you please try sending sample test data from TeraTerm (using COM5) and checking if data is being received on the Arduino Serial Port through the JY-MCU Module. Next step would be to have the Java program send over sample data and finally sending over Leap data.

  5. hi anwaarullah i have a question when you point 1 finger as you show in the video it will turn on the first light and when you point 2 finger it will turn off the light so my question was if you turn on the light and remove the finger in a leap motion will the light continue be on until you point again 2 fingers? because i did one of leap motion light control with 4 light by pointing fingers 1-4 but if i point 1 finger it will turn on light number 1 but if i remove that finger from the leap motion the light goes off

    1. Hi Isaac,

      Yes Isaac, the light will continue to be ON if I point 1 finger and remove it from above the LeapMotion.

      The code that plays with maintaining the state of the Light with respect to the number of fingers shown is there in the Arduino, so you’ll have to mod it accordingly to fix your case of it turning OFF after removing the single finger.

      If that doesn’t work, please share across the Arduino Code and ‘ll be happy to debug.

      Thanks.

Leave a Reply