SIGNAL-AV

Accessible human-intent perception for autonomous vehicles.

SIGNAL-AV is an Unmute1AI project designed to translate visual human communication — beginning with American Sign Language (ASL) and safety-critical gestures — into structured, machine-readable intent for autonomous vehicle systems.

Mission

Autonomous vehicles can detect people.

SIGNAL-AV is designed to help them understand when people are intentionally communicating with them.

Making Every Signal Accessible to All.

Core Flow

Camera / Visual Input
        ↓
Hand + Pose + Facial Features
        ↓
Temporal Sign Analysis
        ↓
Intent Recognition
        ↓
Confidence + Spatial Context
        ↓
Safety Policy Gate
        ↓
Structured Vehicle Intent

Initial Intent Vocabulary

• STOP
• GO
• WAIT
• HELP
• DANGER
• COME
• LEFT
• RIGHT
• UNKNOWN

Example Output

{
  "intent": "STOP",
  "source": "pedestrian_sign",
  "language": "ASL",
  "confidence": 0.94,
  "spatial_relation": "front_right",
  "safety_class": "advisory"
}

Safety Architecture

SIGNAL-AV does not directly control vehicle motion.

It produces a structured human-intent observation that can be evaluated alongside vehicle perception, localization, trajectory planning, environmental context, and safety policies.

Ambiguous or low-confidence communication should resolve to UNKNOWN rather than forcing an interpretation.

Design Principles

Accessibility First

Human communication should not become invisible simply because it is visual instead of spoken.

Intent Before Language

SIGNAL-AV is designed to identify meaning and intent rather than perform simple word-for-word translation.

Multimodal Understanding

SIGNAL-AV is designed to reason across:

• hand movement
• hand shape
• body pose
• facial expression
• temporal motion
• spatial context

Safety-Constrained Output

Recognized communication becomes an advisory perception signal rather than an unrestricted vehicle command.

Local-First Architecture

The architecture is designed for edge processing where practical to reduce latency and unnecessary transmission of raw visual data.

Intended Use Cases

• Deaf and Hard-of-Hearing pedestrian communication
• ASL interaction with autonomous vehicles
• traffic-control gestures
• emergency communication
• accessibility-aware robotaxis
• autonomous shuttles
• smart transportation systems
• human-to-robot communication

Architecture Goal

HUMAN
  ↓
Visual Signal
  ↓
SIGNAL-AV
  ↓
Structured Intent
  ↓
Safety + Confidence Validation
  ↓
AV Reasoning System
  ↓
Vehicle Response

Status

Early development / research prototype.

Current priorities:

1. Define the intent schema
2. Build the temporal visual perception pipeline
3. Establish ASL and gesture evaluation datasets
4. Implement confidence and UNKNOWN handling
5. Create simulation adapters
6. Benchmark recognition latency and accuracy
7. Validate accessibility and safety behavior

Unmute1AI

SIGNAL-AV is part of the Unmute1AI accessibility ecosystem.

Accessibility First. Always.

Making Every Signal Accessible to All.