MPML-FLASH Version 1.0

Last Update: 2004.2.20
Author: Zhenglu Yang

Download MPML-FLASH Demo(Bigger one 33M)

Download MPML-FLASH Demo(Zipped one 26M)

Attention: To view the demo, you need the Camtasia Studio software, which can be available from here.

Summary

This document is the reference of Flash verstion of MPML(Multimodal Presentation Markup Language), MPML-FLASH (Multimodal Presentation Markup Language for Flash). The figure of MPML-FLASH elements are shown as follows:

table of contents

root elements

mpml root element of MPML

mpml	root element of MPML

head elements

head	head information of MPML-FLASH document
meta	meta data description
title	the title name
spot	definition of a point on the screen
agent	the configuration of the agent to be used

body elements

body the definition of presentation flow

page the definition of one page for presentation

scene the definition of one scene for presentation
synchronous elements

seq synchronous script

par parallel script

pause pause script
agent script elements

emotion the configuration of agent emotion

move move the agent

play agent play actions

speak agent speaks

think agent think something

body	the definition of presentation flow
page	the definition of one page for presentation
scene	the definition of one scene for presentation

seq	synchronous script
par	parallel script
pause	pause script

emotion	the configuration of agent emotion
move	move the agent
play	agent play actions
speak	agent speaks
think	agent think something

background control elements

execute	execute a javascript command on the background page
consult	test a variable on the background page and choose what to do depending on its value.
test	test the variable of the parent <consult> and contain the script to execute in case the corresponding match is found.
wait	consult a javascript variable existing on the background page and wait until it is different from 0.
txt	used to allow a variable speech. It eads a javascript variable from the background and includes it in the speech

object elements

text text object in Flash

img image object in Flash

button button object in Flash

text	text object in Flash
img	image object in Flash
button	button object in Flash

root elements

mpml

MPML document root element

version: MPML document version

head: head information
body: presentation document body

explanation

Element <mpml> is MPML document root element. When define the MPML-FLASH document, set version to "Flash". In MPML-FLASH document contents, all elements are the children of element <mpml> .

<?xml version="1.0" encoding="shift_jis" ?>

Because in MPML-FLASH system, we employ client-server model, which use DOMXML at the server side as the parser of the MPML-FLASH, the XSLT file is not necessary.

sample

<?xml version="1.0" encoding="shift_jis" ?>

<mpml version="flash">
  <head>
    <!-- This is the head information. -->
  </head>
  <body>
    <!-- This is the presentation document body. -->
  </body>
</mpml>

head elements

head

information of head element

no attributes

meta: definition of meta
title: presentation title
spot: define a point on the screen. This point can be used later as spatial reference.
agent: configration of agent character

explanation

element<head> is the information all about head of flash presentation. In the element <head> there are all the informations that are supplement of the MPML-FLASH document.

meta

description of meta data.

name: name of the information
content: content of the information

empty element

explanation

In the element <meta> define the title, copyright and other properties of the presentation. In the element <meta> define the property name and correspond value respectively. name and content must be set as a group.

name properties are shown as follows:

attribute	explanation	example
titl	presentation Japanese title	MPML-FLASHについて
tite	presentation English title	about MPML-FLASH in English
aupk	presentation author name in Japanese	楊　征路
aupe	presentation author name in English	Zhenglu Yang
conf	name of Conference	MNSA 2004
cond	date of the presentation	2004
kywd	Japanese keyword	MPML-FLASH エージェント
kywe	English keyword	MPML-FLASH agent
writer	author of the script	Zhenglu Yang
wrid	date of written script	20030901
pred	date of the presenation demo	2004
vafd	valid deadline of the presentation(from)	2003
vatd	valid deadline of the presentation(to)	2004

sample

  <head>
    <meta name="titl" content="MPML-FLASH" />
    <meta name="aupe" content="Zhenglu Yang" />
    <meta name="wrid" content="20030901" />
  </head>

title

set tilte name

no attributes

explanation

element <title> is the definition of MPML-FLASH title.

sample

  <head>
    <title> about MPML-FLASH Ver1.0</title>
  </head>

spot

define a point on the screen. This point can be used later as spatial reference.

id: ID to use to refer to the spot.
name: name of the spot.
description: short explanation about the spot.
x: x coordinate of the spot.
y: y coordinate of the spot.
location: coordinate using "x,y" format.

empty element

explanation

Element <spot> defines a point on the screen. This point can be used later as spatial reference.

The default of this element is set to "0, 0".

sample

  <head>
    <spot id="spot1" name="middle" description="middle of the screen" x="400" y="300" />
  </head>

agent

configuration of the agent to be used in presentation

id: agent ID
profile-url: agent profile URL
location: first appear location of the agent

empty element

explanation

element <agent> is used to set the agent to be used in presentation

To control the agent in MPML-FLASH presentation, setting name of the agents can manage multiple agents.

In the element <agent> set id and then can use the profile of the agent by profile-url.

In the element <agent> the agent will appear at the place where location is set.

profile-url set the characteristic of every agent.

sample

  <head>
    <agent id="Agent" profile-url="andy.ap" location="StartPosition" />
  </head>

body elements

body

definition of presentation script body

no attribute

page: definition of present page

explanation

Element <body> defines the presentation flow. In <page> the script will be executed sequencely. <page> is appeared as the first child element in element <body>

page

definition of one presentation

id: page id
caption: page title
ref: page refence

scene: definition of presentation scene

explanation

Element <page> uses the reference which is defined by ref and is set id to become one page of the presentation. The page which is defined by ref can not change later.

caption page title is set.

The child elements of <page> <scene> can be defined multiple. In one page, the block which is defined by scene can be called more than once. MPML-FLASH presentation script must defined in <scene>.

sample

  <body>
    <!-- Page1 -->
    <page id="Page1" caption="ページ１" ref="test.jpg">
      <scene id="Scene1" caption="scene１">
        <!-- Scene1 content -->
      </scene>
      ...
    </page>

    <!-- Page2 -->
    <page id="Page2" caption="scene１" ref="test.jpg">
      <scene id="Scene1" caption="scene１">
        <!-- Scene1 content -->
      </scene>
    </page>
    ...
  </body>

scene

definition of one presentation scene

id: scene script ID
caption: scene title

seq: sequential presentation script

explanation

Element <scene> is referenced by defining id. In one page, the scene script is defined by seq.

<scene>  
  <seq>
    ...
  </seq>
</scene>

Scene script defines the presentation content. It must be written in <seq>

caption is used to define the scene title.

sample

  <body>
    <!-- Page1 (infinitely loop) -->
    <page id="Page1" ref="test.jpg">
      <scene id="Scene1">
        <!-- Scene1 content-->
        <seq>
          <jump to="Scene2" />
        </seq>
      </scene>

      <scene id="Scene2">
        <!-- Scene2 content -->
        <seq>
          <jump to="Scene1" />
        </seq>
      </scene>
    </page>
  </body>

synchronous elements

seq

synchronous script

begin: begin present time

par: parellel elements
scene script element: execute content

explanation

In element <seq> the script is executed sequentially.

The actions define in element <seq> are executed begin after the time[ms] begin assigns.

sample

  <!-- <seq> script is executed sequentially -->
  <seq>
    <play agent="Agent" act="greet" />
    <speak agent="Agent"> Please say 'Left' or 'Right'. </speak>
    <listen agent="Agent" />
  </seq>

par

parellel script

begin: parellel present start time

seq: sequentially execute element

explanation

The actions in element <par> are executed in parallel. If <seq> is written, the actions in <seq> is executed sequentially.

Element <par> can define multiple actions that are executed in parellel but this can not always right.

Element <par> can support actions that are executed after some time[ms] begin describes.

sample

  <seq>
    <!-- speak as while as greet -->
    <par>
      <seq>
        <move agent="Agent" how="walk" location="Center" />
        <play agent="Agent" act="greet" />
      </seq>
      <speak agent="Agent">Welcome to the MPML-FLASH. </speak>
    </par>
    <!-- after <seq>と<speak> are executed, it will continue do next actions from here -->
    ...
  </seq>

pause

pause script

id: ID to use to refer to the pause.
name: name of the pause.
description: short explanation about the pause.
duration: duration of the pause in milliseconds.

empty element

explanation

In element <pause> the presentation will pause for the indicated duration.

Make the indicated pause before going on.

sample

  <pause id="pause1" name="Pause before asking" description="Wait 1s before asking to the student what he thought of
   the presentation." duration="1000" />

agent script elements

emotion

configuration of agent emotion

agent: agent ID that need change emotion
type: emotion type
begin: execute start time

par: parelle execute element
agent script element: agent emotion content
scene script element: has no effect on controlling

explanation

Element <emotion> is the emotion of agent. The emotion is set by type.

Element <emotion> begins after the time[ms] which is set by begin. The result of combining element <emotion> and <par> can not be confirmed rightly.

sample

  <seq>
    <!-- set agent emotion to sad -->
    <emotion agent="Agent" type="sad"> 
      <speak agent="Agent"> I'm sorry. </speak>
    </emotion>
    <!-- after 0.5m the agent emotion will change to angry -->
    <emotion agent="Agent" type="angry" begin="500">
      <think agent="Agent"> How rude! </think>
    </emotion>
  </seq>

move

move agent

agent: agent ID that moves
begin: execute start time
how: actions executed while moving
location: place where agent moves to
speed: the speed rate of moving

explanation

Element <move> defines the move information of agent.

The move action of agent in MPML-FLASH are executed sequentially as follows:

agent turns direction to the destination
agent moves to the destination
agent turns direction at the destination

First, agent will turn direction to the location defines. This angle is computed by the location at present and the destination described by location.

Next, agent will move to the destination. While moving, it will act the action how defines. By adding the action like walking how defines, the natural actions will appear.

At last, agent will turn the direction to location defines.

These three actions are executed sequentially to achieve moving action.

The time need to turn and move is computed automatically by agent profile configuration, moving length, and so on. Element speed is used to scale the time needed.

Element <move> is executed after the time[ms] which begin defines.

sample

  <seq>
    <!-- Agent will move the center of the presentation window -->
    <move agent="Agent" how="walk" location="center" />
  </seq>

play

agent play some action

agent: agent ID that play some action
act: name of action
begin: execute start time

empty elements

explanation

Element <play> defines the actions that agent will play. The action name is described by act. The multiple actions in element <par> can not be always executed right in parellel.

The action that element <play> defines will execute after the time[ms] begin describes.

sample

  <seq>
    <!-- Agent plays greet action -->
    <play agent="Agent" act="greet" />
    <!-- Agent will play applaud after 1s -->
    <play agent="Agent" act="applaud" begin="1000" />
  </seq>

speak

agent speak

agent: agent ID that speaks
begin: execute start time

: speech content

emp: emphasize speech words

explanation

Element <speak> defines the contents that agent will speak. Element <speak> defines the action that will be executed after the time[ms] begin describes.

sample

  <seq>
    <!-- Speech content is described by text data -->
    <speak agent="Agent"> Hello, MPML-FLASH World. </speak>
    <!-- Agent will speak after 0.5s -->
    <speak agent="Agent" begin="500"> My name is Agent Richard. </speak>
  </seq>

think

agent thinks

agent: agent ID that thinks
begin: execute start time

: thought content

explanation

Element <think> defines the thought content of the agent. Only the thought content is appear and there is no output audio. Element <think> will be executed after the time[ms] begin defines.

sample

  <seq>
    <!-- Text data describes the thought content -->
    <think agent="Agent"> I'm OK? </think>
  </seq>

background control elements

execute

execute a javascript command on the background page

id: ID to use to refer to the execute.
name: name of the execute.
description: short explanation about the execute.
target: the address of the function to execute.

empty elements

explanation

Execute a javascript command on the background page. Can be used to synchronize the background events with the flow of the presentation.

sample

    <execute id="picture2" name="Picture change" description="Change 
    the current picture to the next one." target="changepicture()" />

consult

test a variable on the background page an dchoose what to do depending on its value.

id: ID to use to refer to the consult.
name: name of the consult.
description: short explanation about the consult.
target: the address of the variable to test.
mode: indicates what to do in case nothing is recognized. Can be "pass" to indicate to go to the next tag if nothing matches the value of the variable or "redo" to ask to check the variable until a match is found.

test

explanation

Test a variable on the background page and choose what to do depending of its value.

sample

    <consult id="consult1" name="Response to user" description="Check the anser of the student and give
     the appropriate answer to it." target="choice" mode="pass" >
          <test value="1" >
                ...
          </test>
          <test value="2" >
                ...
          </test>         
    </consult>

test

test the variable of the parent <consult> and contain the script to execute in case the corresponding match is found.

value: the value the variable should match to execute what is in this <test> tag.

empty elements

explanation

Test the variable of the parent <consult> and contain the script to execute in case the corresponding match is found.

sample

   <consult id="consult1" name="Response to user" description="Check the anser of the student and give
     the appropriate answer to it." target="choice" mode="pass" >
          <test value="1" >
                ...
          </test>
          <test value="2" >
                ...
          </test>         
   </consult>

txt

Used to allow a variable speech. It eads a javascript variable from the background and includes it in the speech.

target: the address of the variable to use in the background

empty elements

explanation

Used to allow a variable speech. It eads a javascript variable from the background and includes it in the speech.

sample

  <speak id="intro" name="introduction" description="introduction from robot." agent="agent4" >
       Welcome to my presentation <txt target="name" />!
       My name is Robby.
  </speak>

object elements

text

text object in Flash

id: object ID
x: x coordinate of the text to appear
y: y coordinate of the text to appear
height: height of the text to appear
src: content of the text to appear

empty elements

explanation

Element <text> defines the object text that display in Flash.

sample

    <!-- Show some text -->
    <text id="Title1" x="0" y="100" height="50" src="Multimodal Presentation Markup Language" />

img

image object in Flash

id: object ID
x: x coordinate of the image to appear
y: y coordinate of the image to appear
width: width of the image to appear
height: height of the image to appear
src: source file of the image to appear

empty elements

explanation

Element <img> defines the object image that display in Flash.

sample

    <!-- Show some image -->
    <img id="MPML_GUI" x="100" y="200" width="500" height="400" src="mpml3_gui.jpg" />

button

button object in Flash

id: object ID
x: x coordinate of the image to appear
y: y coordinate of the image to appear
value: button value that to appear
act: button click event

empty elements

explanation

Element <button> defines the object button that display in Flash.

sample

    <!-- Show some button -->
    <button id="Stop" x="780" y="680" value="Stop" act="Stop" />