My Blog

Zero-Input Interfaces: Using AI to Let You Control Devices Without Touch or Speech

Predictive AI Smell Blog Image

A User-Friendly Introductory Guide to 'Invisible Interaction' as an Emerging Technology

Touchscreen devices have changed how we interact with technology. The rise of voice recognition technology has allowed people to control their devices hands-free. New ways of interacting with technology through gesture, augmented reality (AR), and wearable technology have opened new forms of interaction between human beings and the digital world. However, advances in artificial intelligence (AI) are changing the direction of interaction even further.

Using 'Zero Input'

Zero input technology enables control of devices using only thoughts. There is no need for physical input (tapping or typing), vocal input (speaking), or gestures (waving). All you need is 'intent' and the device can use AI to infer what it is you want to accomplish.

The quick development of zero input interface (ZII) technologies will catch many people by surprise.

How a Zero Input Interface Works

A zero input interface is a device that:

  • does not require the user to give a conscious command
  • uses its sensors plus AI to predict someone's intention(s)
  • automatically interprets someone's predicted intention(s) and executes the appropriate action(s) as needed.

A good analogy would be an 'autopilot' for everyday technology interactions.

Examples of Zero Input Interfaces in Action

  • If someone looks away and loses concentration while listening to music, the music will stop playing automatically.
  • If a person's eyes get heavy (or they yawn), the screen on their phone dims automatically.
  • If a person receives a phone call and they wrap their wrist or arm around their body to answer it, the smartwatch will automatically connect.
  • If a person's laptop is open and turned on, and they get up to walk away from it, it will lock automatically when they leave.
  • If a person is feeling anxious or stressed and is in their smart home, the system will determine that their heart rate has increased and automatically cool down the room.

Technological Advancements Opening the Door for Zero-Input Interfaces Today

What Are the Required Technologies for Zero-Input Interfaces

To create a successful ZII, three technology areas need to have progressed sufficiently within the last three to five years:

Sensor Fusion

Your smartphone or watch utilises motion sensors, skin temperature, heart rate variability, directional gaze, breathing pattern, proximity, tension on muscles, and patterns of ambient noise as an integrated data source at all three steps in the ZII process.

Behavioural AI Models

The most advanced AI models can now interpret emotional states based on these factors. Using the aforementioned examples, AI determines an individual’s attention level, frustration level, interest level, cognitive load, engagement level, stress level, sleepiness level and habitual intention patterns based on context.

Predictive Contextual Engines

Using behavioural AI, predictive contextual engines today can make predictions, such as what a user is likely to do next, what a user may want to do, what interruptions will be helpful as opposed to annoying, and how an individual’s environment colours their decision-making process. This changes the way our devices will serve us, as instead of being passive tools, they will proactively support us.

How Do ZIIs Work (The Actual Process)

What Happens in the Background of a Zero-Input Interface?

Understanding the computing processes involved in the operation of ZIIs helps to understand how they are able to function within our lives. Here's how Zero-Input Interfaces work:

Step One: Sensing

Your ZII senses micro-signals, including:

  • slight fluctuations in wrist rotation
  • marked increases in eye blinking patterns
  • changes in an individual’s breath rate
  • activation of muscles
  • lateral movement(s)

Step Two: Interpretation

AI classifies the 'states' above into a contextual frame of reference, such as:

  • “The user appears confused”
  • “The user appears to be contemplative”
  • “The user is trying to look at their mobile device”
  • “The user is ready to take a break”
  • “The user is expecting to receive a notification”

Step Three: Prediction

The machine will make a prediction about your next intention:

  • "the user will be reaching for the next scene”
  • "the user has rejected this call"
  • "the user has reached the point where they wish to scroll”

Existing Prototypes of Real World Applications

The term “Zero-Input” may be new, but we have already developed the prototypes:

  • AirPods know when you are speaking and reduce the volume to zero (no taps and no buttons)
  • Apple Vision Pro will recognize fine eye movements with no hand gesture required (Just by looking)
  • Google Soli's radar will recognize all of your micro hand movements without having to touch the device (your hand movements can be very small but not be classified as gestures)
  • Smartwatches can now monitor your stress levels and provide reminders to breathe without you saying anything (the watch just does it)
  • Smartphones now recognize your presence and automatically unlock without you having to tap the screen or push any buttons (no touch)

These are the early versions of ZIIs (Zero-Input Interaction) and in the future, ZIIs will be enhanced 100X.

Phase Two: Intent Prediction

Predicting your intent will be the next step beyond sensing your intent.

What is to come:

  • The smartphone opens the app you were going to select
  • Lighting in the smart home will change as you focus on changing the light
  • Your AR/VR headset will automatically scroll to the next paragraph when it realizes you are about to read again
  • Your vehicle will automatically adjust your seat when you become uncomfortable
  • Your laptop will devise a layout of your applications that would most be familiar to your cognitive load

The device will become a mind-reader, although still not perfect, but extremely near to perfect.

How ZIIs Will Change How We Live Each Day

1. Zero-Input Smartphones

  • Unlocking devices based on your approach
  • Launching apps when you get close without needing to touch them
  • Automatically silencing notifications when you are in a focused state of mind
  • Launching quick actions based upon your emotional state

2. Zero-Input Smart Homes

  • Changing lighting according to your mood
  • Adjusting the air conditioning based on your degree of stress
  • Setting alarms based on when you are planning to leave
  • Playing music corresponding to your emotional rhythm

3. Zero-Input Wearables

  • Tracking your level of mental fatigue
  • Automatically prompting you to take breaks
  • Modifying the intensity of your workout based on your body's signals

4. Zero-Input Cars

  • Adjusting the temperature as soon as they detect discomfort from the person or driver in the vehicle
  • Stopping or pausing a song when the driver's stress level spikes
  • Indicating that a driver intends to change lanes before the turn indicator (blinker) is turned on
  • Using proximity plus biometric information to unlock and secure items

Zero-Input is Not Magic but Rather the Equation of Mathematics, Biology and Psychology Working Together

Zero-Input Uses:

1. Intent Modeling

Mapping the patterns of:

  • Posture
  • Gaze
  • Microexpressions
  • Movement Sequences

to generate predictive actions.

2. Cognitive State Estimation

Estimating:

  • Attention Level
  • Level of Motivation
  • Emotional Load
  • Decision Readiness

3. Environmental Mapping

Tracking Environmental Contexts:

  • Location
  • Noise Level
  • Lighting Conditions
  • Time of Day
  • Social Contexts

Artificial Intelligence (AI) merges all of the above signals into a "Probability of Intent".

Once the probability has crossed a specific threshold, the requested action will be executed.

This is the Scientific Background of the Concept of Zero-Input.

Things That People Don't Normally Talk About That Are at Risk

  • ⚠️ Automation to the point of nonuseability: Your product could execute tasks automating information about yourself without your explicit consent.
  • ⚠️ Unauthorized Access to Personal Privacy: Devices that use emotion recognition rely on sensitive, personal information.
  • ⚠️ Misunderstood Intentions: AI may be able to stop playing songs when you’re listening, ignore calls you wanted, confuse worry with nothing.
  • ⚠️ Confusion in Society: Devices that may automatically complete actions and the triggers of those actions cannot be seen by humans. Users will no longer understand when their devices take action.
  • ⚠️ Inability to Regain Control of Device Manually: Users probably won’t be as capable of learning how to manually operate their devices anymore.

These problems require a sound ethical framework.

Why Zero-Input Is the Future

The reason is that the next generation of technology will be less demanding of you.

And it's not simply how fast your devices respond; all you need to do is request a task to be performed without any effort on your part.

Touching was good.

Speaking was even better.

What’s left? No-input! The ideal combination of the two is:

  • Providing the ability to perform tasks with little or no physical interaction.
  • The minimal amount of cognitive energy needed to remain productive.
  • Reducing the number of decisions needed to effectively manage one’s time.
  • The best way to get devices to respond to you is by establishing common ground.

Final Words: Zero-Input Will be the Mark of the Future

Zero-Input Interfaces will never be presented as Fiction.

They are the logical evolution of the Human-AI Experience!

Devices of the Future will:

  • Know.
  • Learn.
  • Create Plans.
  • Make Decisions.
  • And the only way for you to do either will be to guide your devices with your mind!

Similar concept: Ethical AI Pets

Also check: AI Time Capsules