> For the complete documentation index, see [llms.txt](https://sws3009.gitbook.io/sws3009-tutorial/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sws3009.gitbook.io/sws3009-tutorial/10-july-consultation/group-7.md).

# Group 7

Smart Guide Dog Companion

**Project Objective:** The project aims to develop a Smart Guide Dog Companion to assist visually impaired individuals by providing navigation, emotional support, and daily living assistance through advanced technologies.

**Key Features:**

1. **Navigation Module:**
   * Utilizes sensors to detect the environment and plan safe routes.
   * Employs path planning and obstacle avoidance algorithms.
   * Integrates voice interaction for navigation commands.
2. **Emotional Support:**
   * Analyzes voice, behavior, and facial expressions for emotional recognition.
   * Provides predefined comfort phrases and connects to remote psychological consultations if needed.
3. **Daily Living Assistance:**
   * Manages medication reminders and confirmations.
   * Integrates with smart home controls for lighting, temperature, and security.

**Deep Learning Components:**

1. **Data Collection:**
   * Obstacle data from various road scenarios.
   * Body language and facial recognition data using datasets like FER-2013, AffectNet, and CK+.
2. **Model Selection:**
   * Uses Wav2Vec 2.0 or Whisper for audio analysis, OpenPose or MediaPipe for body language detection, and DeepFace for facial expression recognition.
   * Combines models using frameworks like Hugging Face’s Transformers or DeepMind's Perceiver.
3. **Input Preprocessing:**
   * Converts audio to spectrograms, normalizes signals, and segments audio.
   * Extracts and normalizes keypoints for body and facial features.
   * Tokenizes and encodes text, removing noise and handling stop words.
4. **Result Processing:**
   * Outputs probabilistic scores for detected emotions and assesses their severity.
   * Generates comforting words using TTS models like Tacotron or WaveNet.
   * Integrates with real-time communication tools for remote psychological support.

**Robotics Components:**

1. **System Implementation:**
   * The model is implemented on a smart car with a camera for computer vision, ultrasonic sensors for obstacle detection, and a microphone for voice commands.
2. **Autonomy:**
   * The car navigates autonomously using navigation algorithms and communicates via a web interface for real-time monitoring and decision-making.

**Teacher's Comments:**

1. **Deep Learning:**
   * Focus on obstacle avoidance using YOLO and consider video analysis or map design instead of SLAM for localization.
   * Implement a smaller version of the environment using Navilens.
2. **Robotics:**
   * Increase human-robot interaction to enhance user experience.

We would like to see the juction detection in next meeting. Please give a prototype of physical condition between the robot and the blind people.