Tutorials‎ > ‎

Kinect Face Expression Tutorial (Part 1) - XAML & Kinect Technology

posted Oct 28, 2014, 2:38 AM by Sujono _   [ updated Aug 15, 2016, 11:32 PM by Surya Wang ]

        Hello, welcome to my Kinect tutorial. In this tutorial, we will learn about skeleton tracking, face tracking, and expression detection. I will divide this tutorial into three sections so you can understand more clearly. Also, I assumed you already know about basic C#, OOP and XML language. First, we will learn about Kinect technology and WPF.

1.     Introduction to Kinect Technology


        Kinect is an movement and sound detection device that produced by microsoft for video games. Kinect first introduced in 2010 as a free-controller game device. Microsoft also released Kinect SDK, so the game developers can use it with any programming language like C++, C#, or VB.NET. As a movement detection tool, kinect will yield 3 different output picture namely Infra-Red (IR), RGB, and depth image.
Here are various picture that describe about Kinect technology:



Kinect X-Box 360 Preview



RGB and Depth Image using Kinect Studio




Skeleton Tracking in Kinect

2.     Windows Presentation Foundation

        Windows Presentation Foundation (WPF) is a Windows client applications with resolution-independent and vector-based display. WPF use Extensible Application Markup Language (XAML) as its application appearance and it is included in the Microsoft .NET Framework, so you can build applications that incorporate other elements of the .NET Framework class library.
        XAML is an XML-based markup language that is used to create windows, dialog boxes, pages, and user controls, and to fill them with controls, shapes, and graphics. The following example uses XAML to implement the appearance of a window that contains a single button:

       MainWindow.xaml

<Window x:Class="WpfApplication1.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525">
    <Grid>
        <Button Content="Button" Height="70" HorizontalAlignment="Left" Margin="179,104,0,0" Name="button1" VerticalAlignment="Top" Width="151" Click="button1_Click" />
    </Grid>
    </Window>

        MainWindow.xaml.cs

namespace WpfApplication1
{
    public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();
        }

        private void button1_Click(object sender, RoutedEventArgs e)
        {
            MessageBox.Show("Hello WPF!");
        }
    }
}


Here is the result:


MainWindow Preview


Message Box Preview when user click the button

3.     Create Your WPF Design!

        In this step, we will create a WPF design with XAML. THe WPF design will consists of MainWindow and FaceTrackingViewer.

        a. MainWindow will be used as the user interface of program and consists of variety of button (Start Detect, Option, Back, and Exit), checkbox (Switch Hand Cursor and Full screen), label and Expression Progressbar. In this chapter, i'm also give these various images that are needed in the program. Here's the XAML code for MainWindow:

        MainWindow.xaml

  <Window
        x:Class="FaceTrackingBasics.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:local="clr-namespace:FaceTrackingBasics"
        xmlns:my="clr-namespace:Microsoft.Samples.Kinect.WpfViewers;assembly=Microsoft.Samples.Kinect.WpfViewers"
        Title="Face Expression Detect Program"
        Closed="WindowClosed"
        Height="735" Width="770" WindowState="Normal" WindowStartupLocation="CenterScreen">

    <Window.Resources>
        <SolidColorBrush x:Key="MediumGreyBrush" Color="#ff6e6e6e"/>
        <SolidColorBrush x:Key="KinectPurpleBrush" Color="#ff52318f"/>
        <SolidColorBrush x:Key="KinectBlueBrush" Color="#ff00BCF2"/>
        <Style TargetType="{x:Type Image}">
            <Setter Property="SnapsToDevicePixels" Value="True"/>
        </Style>
    </Window.Resources>

    <Grid Name="layoutGrid" Margin="10 0 10 10">
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto"/>
            <RowDefinition Height="*"/>
        </Grid.RowDefinitions>
        <DockPanel Grid.Row="0" Margin="0 0 0 20">
            <TextBlock DockPanel.Dock="Right" Margin="0 0 -1 0" VerticalAlignment="Bottom" Foreground="{StaticResource MediumGreyBrush}" FontFamily="Segoe UI" FontSize="30" TextAlignment="Center" Text="Face Expression Detect Program" Background="White"></TextBlock>
        </DockPanel>
        <Viewbox Grid.Row="1" Stretch="Uniform" HorizontalAlignment="Center">
            <Grid Name="MainGrid" Width="640" Height="480">
                <Image Name="ColorImage"/>
                <local:FaceTrackingViewer x:Name="faceTrackingViewer" Loaded="faceTrackingViewer_Loaded" />
                <Canvas Background="CadetBlue" Name="canvasDetect">
                    <my:KinectSensorChooser Name="kinectSensorChooser1" Height="480" Width="640" Loaded="kinectSensorChooser1_Loaded" FontSize="20" Canvas.Left="0" Canvas.Top="-2" />
                    <Image Canvas.Left="183" Canvas.Top="158" Height="55" Name="imageCheck" Stretch="Fill" Width="60" Source="/FaceTrackingBasics-WPF;component/Images/uncheckscreen.png" />
                    <Label Canvas.Left="246" Canvas.Top="158" Content="Full Screen" Height="53" FontSize="36" Name="lblFull" Width="247" />
                    <Image Canvas.Left="180" Canvas.Top="253" Height="55" Name="imageCheck2" Stretch="Fill" Width="60" Source="/FaceTrackingBasics-WPF;component/Images/uncheckscreen.png" />
                    <Label Canvas.Left="242" Canvas.Top="254" Content="Switch Hand Cursor" Height="53" FontSize="36" Name="lblBone" Width="329" />
                    <Label Canvas.Left="23" Canvas.Top="359" Content="Loading :" Height="110" Name="lblProcess" FontSize="36" Width="604" Foreground="Black"/>
                    <Label Content="Sujono © 2014 All Right Reserved" Height="39" HorizontalAlignment="Left" Margin="160,491,0,0" Name="label3" VerticalAlignment="Top" Canvas.Left="22" Canvas.Top="-15" FontSize="20" Width="309" />
                    <Image Canvas.Left="513" Canvas.Top="44" Height="84" Name="imageExit" Stretch="Fill" Width="84" Source="/FaceTrackingBasics-WPF;component/Images/exit.png" />
                    <Image Canvas.Left="19" Canvas.Top="169" Height="120" Name="imageBack" Stretch="Fill" Width="120" Source="/FaceTrackingBasics-WPF;component/Images/back.png" ImageFailed="image2_ImageFailed" />
                    <Image Canvas.Left="245" Canvas.Top="118" Height="101" Name="startDetect" Stretch="Fill" Width="160" Source="/FaceTrackingBasics-WPF;component/Images/startdetect2.png" />
                    <Image Canvas.Left="245" Canvas.Top="235" Height="101" Name="Option" Stretch="Fill" Width="160" Source="/FaceTrackingBasics-WPF;component/Images/option.png" />
                    <ProgressBar Canvas.Left="21" Canvas.Top="29" Height="45" Name="progressExpr" Width="130" Value="0" IsEnabled="False" />
                    <TextBlock Text="{Binding ElementName=progressExpr, Path=Value, StringFormat={}{0:0}%}" HorizontalAlignment="Center" VerticalAlignment="Center" Canvas.Left="67" Canvas.Top="34" Name="lblProgExpr" FontSize="25" Width="56" FontWeight="Bold" />
                    <Image Canvas.Left="529" Canvas.Top="29" Height="96" Name="imageExpr" Stretch="Fill" Width="101" />
                    <Label Canvas.Left="521" Canvas.Top="126" Height="56" Name="labelExpr" FontSize="30" Width="119" FontWeight="Bold" />
                    <Label Canvas.Left="144" Canvas.Top="140" Content="Are you sure want to exit?" Height="54" Name="lblConfirmExit" FontWeight="Bold" FontSize="30" Width="391" />
                    <Image Canvas.Left="141" Canvas.Top="235" Height="101" Name="imageYesExit" Stretch="Fill" Width="160" Source="/FaceTrackingBasics-WPF;component/Images/yesExit.png" />
                    <Image Canvas.Left="361" Canvas.Top="235" Height="101" Name="imageNoExit" Source="/FaceTrackingBasics-WPF;component/Images/noExit.png" Stretch="Fill" Width="160" />
                    <Image Canvas.Left="412" Canvas.Top="70" Height="69" Name="RightHand" Stretch="Fill" Width="84" Source="/FaceTrackingBasics-WPF;component/Images/jono.PNG"/>
                </Canvas>
              
            </Grid>
        </Viewbox>
   </Grid>
</Window>

Here are the explanation of MainWindow XAML code:
  • ViewBox : Defines a content decorator that can stretch and scale a single child to fill the available space.
  • Local : Defines an alias for other namespace (FaceTrackingViewer) that allow you to declare resources, controls, converters etc from the other namespace directly in your xaml.
  • Canvas : Defines an area within which you can explicitly position child objects, using coordinates that are relative to the Canvas area.
  • DockPanel : Defines layout panel that provides an easy docking of elements to the left, right, top, bottom or center of the panel. The dock side of an element is defined by the attached property DockPanel.Dock. To dock an element to the center of the panel, it must be the last child of the panel and the LastChildFill property must be set to true. Here's the preview:

DockPanel XAML

        b. FaceTrackingViewer will be used as the user control of face expression detection. Here's the XAML code for FaceTrackingViewer: 

        FaceTrackingViewer.xaml

<UserControl x:Class="FaceTrackingBasics.FaceTrackingViewer"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    mc:Ignorable="d" >
 </UserControl>

        UserControl is used to provides the base class for defining a new control that encapsulates related existing controls and provides its own logic. This is the result that we want after arrange elements in their cs file:

MainWindow Preview

        Okay, that's enough for this part. We will learn about Skeleton and Face Tracking in next part. Thank you :)

ċ
FaceTrackingViewer.xaml
(0k)
Sujono _,
Oct 28, 2014, 2:38 AM
ċ
Images.zip
(671k)
Sujono _,
Oct 28, 2014, 2:38 AM
ċ
MainWindow.xaml
(5k)
Sujono _,
Oct 28, 2014, 2:38 AM