Tuio Flash Blog
3May/108

How gestures will (probably) work

In this post I'll try to give you an overview on how the gesture stuff in the tuio-as3 library will work so we can get a discussion started on what things to improve or change.

Basically all you have to do is:

  1. GestureManager.init(stage, new TuioClient(new UDPConnector()));

That will start dispatching all the standard Gesture included in the library. Though there is no event for dispatching yet ...

That is cool and all but the key feature of the gesture package is that you can easily write your own Gestures based on all types of TuioEvents and TouchEvents that are dispatched by the TuioManager.
To give you a quick example here is the source code of the PressTabGesture:

  1. package org.tuio.gestures {
  2.    [...]
  3.    public class PressTapGesture extends Gesture {
  4.       public function PressTapGesture() {
  5.          this.addStep(new GestureStep(TouchEvent.TOUCH_DOWN, { tuioContainerAlias:"A" } ));
  6.          this.addStep(new GestureStep(TouchEvent.TOUCH_MOVE, { tuioContainerAlias:"A", die:true } ));
  7.          this.addStep(new GestureStep(TouchEvent.TOUCH_UP, { tuioContainerAlias:"A", die:true } ));
  8.          this.addStep(new GestureStep(TouchEvent.TAP, {goto:2} ));
  9.       }
  10.                
  11.       public override function dispatchGestureEvent(target:DisplayObject, gsg:GestureStepGroup):void {
  12.          trace("press tap " + getTimer());
  13.       }
  14.    }
  15. }

As you can see all you have to do is specify GestureSteps based on TouchEvents and TuioEvents and override the function dispatchGestureEvent which is called if the last step is reached. In this function you can simply dispatch the gesture's event to the stage or do some additional checks that can't be described with GestureSteps.

To create a GestureStep you have to specify an event type much like when adding an eventlistener. If the GestureStep receives the wanted event it is "saturated" and the next GestureStep is listening for its event. You can also specify an Object to narrow down the accepted events. This Object can currently have the following options:

  • targetAlias This is a String that is used as an alias name for the event's target. By using the same targetAlias for different GestureSteps you can make sure that the event was dispatched on the same target.
  • tuioContainerAlias This is again a String which if used for different GestureSteps makes sure that the tracked object that generated the event is the same.
  • frameIDAlias This is also a String which can be used to make sure that the events were all generated by tracked objects from the same tuio frame. If you use a "!" as the first character the value behind the name will be overwritten if it has already been set. This might be useful within loops. Also a failed match will cause the Gesture to fail unlike the other alias types.(This isn't fully implemented yet.)
  • minDelay Sets the minimum delay in ms for the specified event. After this the event is accepted.
  • maxDelay Sets the maximum allowed delay in ms for the specified event. After this the gesture fails.
  • die If set true the gesture fails if the GestureStep is saturated. GestureSteps that have die set true are optional and if the next GestureStep with die set false is saturated will be skipped.
  • goto If set to a value between [1-n° of steps] the next check will occur on the specified GestureStep. This is basically for creating loops e.g. jump to an earlier step after reaching the final step.

One thing I'll probably add is a set of callback functions to the option Object.

After you have written your own Gesture all you have to do is add it to the GestureManager like this:

  1. GestureManager.addGesture(new MyGesture());

I hope that I could give a you a nice little overview on how Gestures are going to work and that you like what has been done so far ;)

c & c are welcome and wanted

Update: frameIDAlias should work now ;)

Filed under: Development Leave a comment
Comments (8) Trackbacks (1)
  1. paul
    May 17th, 2010 - 08:33

    disculpa que escruba en español pero aun no aprendo ingles

    te escribo ya que estoy interesado en hacer un proyecto con flash y mensajes TUIO, y quisiera saber si me podrias orientar te dejo mi coreo
    por favor ayudame. solo quierohacer que se maneje con mensajes como raton con click´s y movimiento de scroler pero no lo se trabajr.
    ok que estes bien
    y espero tu respuesta

  2. Alex
    June 3rd, 2010 - 21:32

    Hi! I tried to get it work, but no results :(
    I set up TransformGestureEvent.GESTURE_ROTATE listener and tested it with TUIOSimulator. Everything was connected fine, Flash could fire other TUIO events, but not rotate gesture. All I could receive was two or three events with “rotation=0″. TransformGestureEvent.GESTURE_ZOOM didn’t work for me at all.
    May be I understand something wrong?

  3. gimmix
    June 3rd, 2010 - 22:28

    Which svn revision are you using? There were some bugs until the most recent one.
    In order to trigger an event you have to move both fingers at the same time afaik that isn’t really possible in the TUIOSimulator. As a quick fix you could change the extend statement of the ZoomGesture and RotateGesture classes from TwoFingerMoveGesture to OneDownOneMoveGesture.

    Then you have to change the if statement in the ZoomGesture from

    1. if(diffX < 0 || diffY < 0)

    to

    1. if(diffX <= 0 || diffY <= 0)

    And you have to change the first step of the OneDownOneMoveGesture to

    1. this.addStep(new GestureStep(TouchEvent.TOUCH_DOWN, { tuioContainerAlias:"A", targetAlias:"A" } ));

    then it should work ^^

  4. George
    June 25th, 2010 - 17:23

    Hi,
    great framework! Especially the UDP Connection in AIR 2.0 is unique up to today.

    I want to create a flick gesture, but after spending hours I decided to ask my question here.

    Below you can see a very generic version, without any die mechanism- I only want to know, how to get the distance of tuio container A between frame n and frame n+x. Is this realizable in version 0.7 ? I can calculate the frame distance between “down” and “up” via a frameAlias for the first step, but this data is not useful for my calculation.

    1. Is there a way to access/save the x,y position of a tuio container A when entering the first step ?
    I searched for a method like:GestureStepSequence.getTuioContainerAtFrame(“A”,12345)
    or a Event like GestureStepEvent.START to save my start values.

    2. Look at the code lines where I commented out my “die” mechanism. What`s wrong with that ? The gestures is never dying.

    Thanks for your information.

    Regards,
    George

    public class FlickGesture extends Gesture {

    public function FlickGesture() {
    this.addStep(new GestureStep(TouchEvent.TOUCH_DOWN, { tuioContainerAlias:”A”, targetAlias:”A” } ));
    //this.addStep(new GestureStep(TouchEvent.TOUCH_DOWN, { tuioContainerAlias:”B”,die:true} ));
    this.addStep(new GestureStep(TouchEvent.TOUCH_MOVE, {tuioContainerAlias:”A”}));
    //this.addStep(new GestureStep(TouchEvent.TOUCH_DOWN, { tuioContainerAlias:”B”,die:true} ));
    this.addStep(new GestureStep(TouchEvent.TOUCH_UP, {tuioContainerAlias:”A”,maxDelay:500}));
    }
    public override function dispatchGestureEvent(target:DisplayObject, gsg:GestureStepSequence):void {
    trace(“FLICK ” + getTimer());
    trace(gsg.getTuioContainer(“A”).x);

    }
    }

  5. gimmix
    June 25th, 2010 - 19:21

    Hi

    @1: You can add an eventlistener to your gesture listening for GestureStepEvent.SATURATED and checking the step property of the event to be “1″. -> Just like in the RotateGesture.

    @2: That’s probably because of the use of the tuioContainerAlias. Remove that and it should work.

    You can use the “X” and “Y” properties of the TuioContainer which contain the velocity information of the point’s move.

    I hope that helps a bit.

  6. George
    August 13th, 2010 - 15:35

    Hi, thanks for your help!
    More than a month later I started a next try. I think this should work- in my small test project it does.
    Anything curious to you? I placed some comments.

    1. package org.tuio.gestures {
    2.        
    3.         import flash.display.DisplayObject;
    4.         import flash.events.TransformGestureEvent;
    5.         import flash.utils.getTimer;
    6.        
    7.         import org.tuio.TouchEvent;
    8.         import org.tuio.TuioContainer;
    9.         import org.tuio.TuioEvent;
    10.        
    11.         public class OneFingerFlickGesture extends Gesture {
    12.                 private var _startTime:uint=0;
    13.                 public static const MIN_DISTANCE_HORIZONTAL:Number=0.05;
    14.                 public static const MAX_DISTANCE_VERTICAL:Number=0.1;
    15.                 public static const THRESHOLD_ACCELERATION:Number=0.2;
    16.                
    17.                 public function OneFingerFlickGesture(delay:uint=1000) {
    18.                         //register ANY target on touch_down, but never die if loosing target focus
    19.                         addStep(new GestureStep(TouchEvent.TOUCH_DOWN, {targetAlias:"!A",tuioContainerAlias:"A"} ));
    20.                         addStep(new GestureStep(TouchEvent.TOUCH_MOVE, { tuioContainerAlias:"A"} ));
    21.                        
    22.                         //die, other touch points are active
    23.                         addStep(new GestureStep(TouchEvent.TOUCH_DOWN, { tuioContainerAlias:"!B",die:true} ));
    24.                         addStep(new GestureStep(TouchEvent.TOUCH_UP, { tuioContainerAlias:"!B",die:true} ));
    25.                         addStep(new GestureStep(TouchEvent.TOUCH_MOVE, { tuioContainerAlias:"!B",die:true} ));
    26.                        
    27.                         addStep(new GestureStep(TouchEvent.TOUCH_UP, {tuioContainerAlias:"A", maxDelay:delay,goto:2} ));
    28.                        
    29.                         addEventListener(GestureStepEvent.SATURATED,onStepSaturated);
    30.                         addEventListener(GestureStepEvent.DEAD,onStepDead);
    31.                 }
    32.                 private function onStepDead(e:GestureStepEvent):void{
    33.                        
    34.                 }
    35.                 private function onStepSaturated(e:GestureStepEvent):void{
    36.                         //save initial position for threshold actions on satisfaction
    37.                         if(e.step==1){
    38.                                 var gsg:GestureStepSequence=e.group;
    39.                                 gsg.storeValue("aStartX",e.group.getTuioContainer("A").x);
    40.                                 gsg.storeValue("aStartY",e.group.getTuioContainer("A").y);
    41.                         }
    42.                 }
    43.                 public override function dispatchGestureEvent(target:DisplayObject, gsg:GestureStepSequence):void {
    44.                         var accX:Number=gsg.getTuioContainer("A").X
    45.                         var accY:Number=gsg.getTuioContainer("A").Y
    46.                         var deltaY:Number=Math.abs(Number(gsg.getValue("aStartY"))-gsg.getTuioContainer("A").y);
    47.                         var deltaX:Number=Math.abs(Number(gsg.getValue("aStartX"))-gsg.getTuioContainer("A").x);
    48.                        
    49.                         //simple threshold
    50.                         if(Math.abs(accX)MAX_DISTANCE_VERTICAL || deltaX&lt;MIN_DISTANCE_HORIZONTAL) return;
    51.                        
    52.                         trace(&quot;statisfied on target: &quot;+gsg.getTarget(&quot;A&quot;));
    53.                         gsg.getTarget(&quot;A&quot;).dispatchEvent(new TransformGestureEvent(TransformGestureEvent.GESTURE_SWIPE, true, false, null, 0, 0, 0, 0, 0,accX,accY));
    54.                 }
    55.                
    56.         }
    57.        
    58. }

    Regards
    George

  7. George
    August 13th, 2010 - 15:37

    I tried to use the tag..didn`t work. How can I use syntax highlighting ? Can you highlight my code ?

  8. admin
    August 14th, 2010 - 22:48

    yeah the code tag is a bit quirky :/ you also have to specify the lang=”actionscript3″ manually.
    I’ll have a look at you gesture tomorrow but it looks alright on a quick glance. Maybe specifing a targetAlias is unnecessary.


Leave a comment


Release 0.7 « Tuio Flash Blog
« Release 0.7 Some news from the development front »
 

Pages

Links

Categories