Challenge: turn enabling technologies into working, usable systems.

Catch-2004 project

• Bi-lingual server-side implementation of speech recognition.
• Goals: interaction, task, user satisfaction enhancement
• Results: 30% gain in speed, slightly higher task accuracy
• Demo given of voice interaction component
  • "Do I have meetings today?"
  • "Yes, you have meetings with Paul and Patrik"
  • "What time does the first one start?"
  • response is "smart"
  • system detects language dynamically (Finnish, English)

Use case: Context rules on the client

• Work trip expense calculator
  • prompt user for expenses to add when a location change occurs **send aggregate of interaction back to expense handler on return

Example for reduced interactions

Spoken dialog:

• "How can I help you?"
• "I want to travel from Helsinki to Tampere tomorrow morning 1st class"

Based on this simple interaction, the system already knows a lot of things! For example:

• it can know whether to schedule a train, bus, or flight, based on the user history or the location of travel.
• departure time
• arrival time (user's calendar might have Tampere meeting info)
• seat (schedule nicest allowed by company policy)

Command/control

In an elevator - push the button, but why? Are we too scared to let it decide for us? Well, in the past we moved away from elevator operators...

Examples of things that can be done by a smart elevator that can sense user context:

• handheld calendar might have info about meeting on 7th floor in 10 mins... go there.
• it's 5pm, this user is going home now
• the user came in at basement, came by car
• only need explicit input for error correction
• essay: "Buttonless Elevators Creep People Out"

Another example: smart messaging:

• Having an engaging discussion in the cafeteria, meeting 5 minutes from now. Device can automatically prepare message to alert meeting attendees that you will be a few minutes late.

Simplified view

User intention/input + passive input (context, sensors, prefs, etc) = System behavior

Multimodal Integration

• Statistically merging user input (context and sensor data vs. data fusion)
• "semantic bridge"
• Example of multimodal interface: Speak: "From here" (tap location on screen) "To there" (tap location on screen)

SensorPlanet

• a global test platform for mobile-centric wireless sensor network research
• use cellphones+sensors to make large-scale testbed
• why cellphones - they are prevalent and powerful
• phone acts as sensor and/or sink node.
• green phones (environmental data collection)
• service manifolds (more users = much better)
• urban games, social networking

Nokia Remote Sensing Architecture (N-RSA)

Horizontal/Vertical Implementation

1. Passing codes, report spreading stats (Viral Research TK)
2. Building Urban stories (Manhattan Story Mash-up)
3. Virtual graffiti (Toijala students innovations)
4. Nokia RSA (Instant Meeting Room Access)
5. WLAN positioning (WLAN-based positiong, BT-based: Midwife Trial

Manhanttan Story Mashup

• collaborative and creative urban game
• send a text message with a short story
• respond with a photo
• 90 minutes
• 3142 images
• web players around the globe (4000 IPs)
• 115 mashup stories at Times Square

Key tech challaneges

• sensor data challenge
  • multidimensional
  • large amount
  • large object size (video, i.e.)
• mobile adaptation challenge
• integration challenge
• power challenge
• user interaction challenge

Discussion Point

• User wants control (or to feel in control)
• How do you convey the mode of interaction to the user?
• Comment: tradeoff with multi-modality, when you start to use multiple modes, because now user must "walk and chew gum".
  • How can we handle this tradeoff?
• What's not understood is cultural differences/cultural boundaries.
  • Can you make cultural analysis available based on speech, posture, faces, etc.