Tuio Flash Blog

Creating custom Gestures

This article will focus on how you can make your own gestures in order to get most out of the library for your project.
The GestureManager detects gestures based on the TouchEvents and TuioEvents dispatched by the TuioManager. Every gesture has a kind of checklist that has to be completed from top to bottom in order dispatch the gesture's event to the stage. This process is called "saturation" and if the GestureManager reaches the bottom of the list it is fully "saturated". From now on I'll call every single item on the checklist "gesture step" and I'll call the checklist "gesture step sequence".
So in order to create your own gesture you simply have to define this gesture step sequence and have to implement a function that will be called if the sequence is saturated.
Before you can define your gesture step sequence you have to create a class for your gesture that extends the Gesture class. This could look somehow like this:

  1. package org.tuio.gestures {
  2.        
  3.    import flash.display.DisplayObject;
  4.    import org.tuio.*;
  5.        
  6.    public class OneDownOneMoveGesture extends Gesture {
  7.        
  8.       public function OneDownOneMoveGesture() {
  9.          // Here we'll add our steps.
  10.       }
  11.                
  12.       public override function dispatchGestureEvent(target:DisplayObject, gsg:GestureStepSequence):void {
  13.          // this function will be called if the gesture step sequence is saturated.
  14.       }
  15.    }
  16. }

This gesture won't do much now because we have no gesture step sequence or callback behaviour defined yet. So let's change that and add some steps.

  1. public function OneDownOneMoveGesture() {
  2.    this.addStep(new GestureStep(TouchEvent.TOUCH_DOWN, { tuioContainerAlias:"A", targetAlias:"A" } ));
  3.    this.addStep(new GestureStep(TouchEvent.TOUCH_MOVE, { die:true, tuioContainerAlias:"!B", targetAlias:"A" } ));
  4.    this.addStep(new GestureStep(TouchEvent.TOUCH_UP, { die:true, tuioContainerAlias:"A" } ));
  5.    this.addStep(new GestureStep(TouchEvent.TOUCH_MOVE, { tuioContainerAlias:"A", targetAlias:"A", goto:2 } ));
  6. }

As you can see steps are added via the function addStep(...). This function expects an instance of the GestureStep class which has two parameters. The first one is the name of the event and should be quite familiar if you ever happended to use events in as3. You can put all types of TuioEvents and TouchEvents in there. A step will be saturated as soon as the desired event is received. The second argument is an object which defines additional behaviours for the step. Here is a list of all the possible properties:

  • targetAlias: This is a String that is used as an alias name for the event's target. By using the same targetAlias for different GestureSteps you can make sure that the event was dispatched on the same target. If you use a "!" as the first character the value behind the name will be overwritten if it already has been set. This might be useful within loops.
  • tuioContainerAlias: This is again a String which if used for different GestureSteps makes sure that the tracked object that generated the event is the same. If you use a "!" as the first character the value behind the name will be overwritten if it already has been set.
  • frameIDAlias: This is also a String which can be used to make sure that the events were all generated by tracked objects from the same tuio frame. If you use a "!" as the first character the value behind the name will be overwritten if it already has been set. Also a failed match will cause the gesture to fail unlike the other alias types.
  • minDelay: Sets the minimum delay in ms for the specified event. After this the event is accepted.
  • maxDelay: Sets the maximum allowed delay in ms for the specified event. After this the gesture fails.
  • die: If set true the gesture fails if the GestureStep is saturated. GestureSteps that have die set true are optional and if the next GestureStep with die set false is saturated will be skipped.
  • goto: If set to a value between [1-n° of steps] the next check will occur on the specified GestureStep. This is basically for creating loops e.g. jump to an earlier step after reaching the final step.
  • optional: If set true the GestureStep will be skipped if it doesn't saturate or dies and the next GestureStep is saturated. If an optional GestureStep dies this won't cause the whole gesture to die.

Now that we have our gesture step sequence set up we have to take care of the actual event dispatching which is pretty straight forward.

  1. public override function dispatchGestureEvent(target:DisplayObject, gsg:GestureStepSequence):void {
  2.    gsg.getTarget("A").dispatchEvent(new Event("oneDownOneMove"));
  3. }

The dispatchGestureEvent function will receive two arguments when called. The first one is the target DisplayObject of the event that saturated the last step. The second one is the GestureStepSequence object that was saturated which is very handy because it gives you access to all the targets, tuioContainers and frameIDs saved under the aliases you used when specifying your gesture's steps. I used the getTarget(...) function to retrieve the target of the very first touch so i can dispatch the event on it. You can also use the GestureStepSequence to store and retrieve custom values with storeValue(...) and getValue(...). THis is very useful in looping GestureStepSequences.

Sometimes you might want to handle each step of the sequence or do some additional calculations after certain steps. This can be achieved very easily because after each step a GestureStepEvent is dispatched on you gesture. This could look like this:

  1. package org.tuio.gestures {
  2.        
  3.    import flash.display.DisplayObject;
  4.    import org.tuio.*;
  5.        
  6.    public class OneDownOneMoveGesture extends Gesture {
  7.        
  8.       public function OneDownOneMoveGesture() {
  9.          this.addEventListener(GestureStepEvent.SATURATED, handleSaturated);
  10.          //some steps maybe
  11.       }
  12.                
  13.       public override function dispatchGestureEvent(target:DisplayObject, gsg:GestureStepSequence):void {
  14.          // this function will be called if the gesture step sequence is saturated.
  15.       }
  16.                
  17.       private function handleSaturated(e:GestureStepEvent):void {
  18.          //handle the saturation
  19.          trace("step "+e.step+" saturated");
  20.       }        
  21.    }   
  22. }

Now you should be able to write your own gestures but if you happen to run into any problems or parts of the article aren't comprehensible write a comment or send me an email.

Comments (8) Trackbacks (1)
  1. Hi,

    at first I want to say, that this is a really great project!
    And now to my problem. I could not find a forum or something like that to post my question, so I will post it here.

    In my application I use the “DragGesture” to move Objects. This works perfectly for slow moves. But if I move my draggable objects fast they will lose focus.
    I think the reason is, that the first DragGesture Event ist dispatched when the finger is on the draggable object and with the next Event the finger is outside the draggable object.

    I think I have to make little changes in the OneFingerMoveGesture, but I did not know how/what to change.

    It would be great to get some help!
    Thank you!

    Patrick.

  2. Glad you like the library!
    I think your question has mostly been answered in this post: http://bubblebird.at/tuioflash/guides/using-the-gesturemanager/#comment-4132

    greets

  3. Thanks a lot!
    That worked for me!

    Greets.
    Patrick.

  4. Hi guys.

    Super good job on the library. It really helped in a latest project.

    TUIOClient is easy to configure and GestureManager is a must, albeit its documentation lays behind the latest releases.

    As for the gestures themselves, I wonder if it would be possible to unprotect the function addStep(s:GestureStep):void, making possible to extend the Gestures in a different package.

    Best,

    Thomas

  5. Hi thanks for the positive feedback. You are right, writing custom Gestures in another package would be usefull and a lot more intuitive I guess x) I’ll keep that in mind for the next release.

  6. Hi guys, i’d have question… I’ve created my own gestures to recognize 1 to 4 fingers move, they work fine, but i ran into a problem when I tried to find which direction they have moved.
    I have no loop in my gesture – i simply check if needed count of fingers touched down and if they all moved. Once the all get up, the gesture is complete. So im not tracking the move all the time the fingers are moving.
    So what would be the way to get the starting and ending coodrinates of the move to calculate the direction ?
    I tried to get the container i’ve been tracking gsg.getTuioContainer(“C”) and get the coordinates from there (container.x or container.X), but i guess the X is not total start of the move – its just last remembered point, and from the start there can be many of them.

  7. I’d suggest to listen for GestureStepEvents like explained above and use GestureStepSequence.storeValue() to store the position of the touch. When the Gesture is fully saturated you can retrieve those values with GestureStepSequence.getValue()
    container.X btw. is the velocity of the touch on the x-axis.

  8. Thx Gimmix – works just as needed :)
    If someone ever needed this, here is the function:

    private function handleSaturated(e:GestureStepEvent):void {
    if (e.step == 1) {
    var gsg:GestureStepSequence = e.group;
    var container:TuioContainer = gsg.getTuioContainer(“C”);

    gsg.storeValue(“touch_down_x”, container.x);
    gsg.storeValue(“touch_down_y”, container.y);
    }
    }

    and in dispatchGestureEvent function you call:

    var container:TuioContainer = gsg.getTuioContainer(“C”);
    trace(“total move in x axis: ” + (container.x – Number(gsg.getValue(“touch_down_x”))));


Leave a comment