– –
- Digital wonderland at fingers' reach
- But cannot touch it
- Devices became portable
- But require our whole attention
- Either IN the screen our OUT in the world
A Humane Representation of Thought
- Technology constrain our bodies
- Only eye and fingers working
- Bodies have been neglected
- Tip of finger interaction is limiting. Not humane.
- Evolution made it so that we think with all our senses
Real World
We know how it works
Tangible User Interfaces
Augmented Reality
- Traditional way is to use video see-through
- Can also uses head mounted display
- However: requires hardware for user
Spatial Augmented Reality
Uses projectors or screens in the environments to display information spatially related to this environment- SAR instead uses projector or screens in the environment
- Link with Ubicomp
- Example mug from before
- Normal mug + projector...
- Creates augmented mug
Augmented Objects
- For example, we can augment a normal mug with different functions:
- Displaying remaining steeping time
- Displaying temperature of liquid inside
- Handle turns green when everything is OK
Augmented Objects
- It can also be used to create augmented objects
- In these two pictures, white physical objects
How does it work?
Augmented Objects
My objective is to move towards a real-world experience without having to remove technology. Twofold: - By including real world elements with our use of technology - Using technology as a way to reflect on ourselves
Interaction
How can we interact with digital content hosted on physical objects?
Direct Interaction
Direct Interaction
- Show pointing on real object using direct touch
- Tangible interaction is sometimes not enough
- So we want to use digital tools
- Such tools are often operated on screen
- What works great on screen is indirect interaction such as the mouse
Interaction
The evaluation of the use of 2D pointing devices – mouse and graphics tablet – in a pointing task in a SAR context compared to a screen condition.
The design, implementation and evaluation of a system enabling the interaction between a typical desktop computer environ- ment – traditional screens, mouse and keyboard – with tangible augmented objects, considering an object design scenario as a main thread.
Introspection
How can we use augmented objects to reveal hidden information about our own selves?
Introspection
Introspection
Introspection
Frey, J., Gervais, R., et al., 2014. Teegi: Tangible EEG interface, in: UIST’14.Gervais, R., Frey, J., et al., 2016. TOBE: Tangible Out-of-Body Experience, in: TEI’16.
Interaction
Standard way of pointing
Now what happens if...
Removing the screen
Does pointing still works without a screen?
Study
Questions
Differences between SCREEN and SAR conditions for pointing?
Does pointing in SAR follow Fitts' law?
Pointing technique
Pointing technique
Apparatus
- A: Circle-shaped cursor that follows the geometry of the real world
- B: Plane onto which cursor is mapped
- In SAR, plane is virtual
- In SCREEN condition, we use a wooden panel to create a screen there
- C: Guide displayed on the table to help know where the cursor is located
- D: Position of the user is known
- E: Projector
- Augment real cube in SAR condition
- Projects a virtual cube in SCREEN condition
SCREEN vs SAR
- On the left it is a simulated version of what is seen on the right
- Comparison of the view in both conditions
- The view of the cube is the same
- In SCREEN condition, note the virtual table is aligned with real table
Scene
- Scene was changing between trials
- Cube alone in different orientation
- Cube and a more complex shape
Procedures
Design
- Inefficiency: deviation from the most optimal path
Participants
- 16 participants
- Familiar with mice
- Little experience with graphics tablets
- No experience with SAR systems
Results
Time
Users were 11% faster using a screen vs SAR
- Screen faster than SAR by 11%
- Drop of performance not so important: still usable
- Screen probably provide context for interaction
- No dead spaces in midair for SCREEN condition
Inefficiency
- Input modality significant effect.
- Tablet is less efficient than mouse
- Explained by experience with mice vs graphics tablet
- See heatmap figure to show example of this
Fitts' law
- MT: Movement Time
- ID: Index of Difficulty
- D: Projected target distance in virtual screen
- W: Perceived target size
- We modeled the movement time with a linear regression.
Note: R2 = 0.8479
- We continued our investigation varying the different way to leverage the screen context
- Here, we keep the screen context for interaction and interact with a physical object when it is located in front of the screen
- Focus on hybrid environment
Metaphor
Working on a physical object becomes the same thing as working on a virtual version in a window
Pointing technique
- Same as with CurSAR
- Screen is tracked
- Behavior of cursor is coherent only for operator
- Cursor is visible by everyone
Applications
Painting / Progamming
Objects in Sync
Text Annotations
Complementary Representations
Towards the Desk as a Working Space
Direct-Indirect Combination
Improvised Working Surfaces
Introspection
Collaboration
EEG
- EEG: ElectroEncephaloGraphy
- Measures electrical activity at the surface of the scalp
- Can be used for
- Active control (BCIs): thinking about hand movements or imagining mental rotations
- Passive measurements
- Current tools are complex and difficult to approach
Objectives
- What can EEG measure?
- How does the brain work?
- let users explore their own brain
- in real-time
- Users were able to see different brain activity
- Filters were controlled using mini-teegis
Filters
Synchronized Blinks
Exposing the Signal Processing
Exploratory Study
Teegi Disco
Going Further
Inner Mirror
Building your own mirror
Metrics
Sensors & Signal Processing
- ECG
- Breathing
- EDA
- EEG
- EOG
3D Printing
Feedback
Customization
Use Case
In Summary
Contributions
I am very much interested continuing bringing technology closer to the real world
Because technology has had this tendency to focus on doing while part of the human experience is not only about achieving but also about being.
But I know that the times in my life I felt most alive is when I actually leave technology behind, like
when I go on a self-supported bike tour with Dominique for a few weeks, juste pedaling all day
and stoping at night to cook and sleep in a tent. When you are out there, you never miss technology.
But when you come back, even if you're still in holidays, technology grabs you back in such a way that you loose this sense of calm you had.
Designing for Calmness and Wellbeing
I think we should not have to choose between calm and technology. We should strive to have technology not only to create engagement and completing tasks, but also enabling us to feel calm at times.