2024

Designing AI Agent System in VR Software Testing

Supervisor: Shuqing Li     Sep. 2024 – Dec. 2024
Position: Research Assistant

  • Designing an AI Agent for automated VR game testing and assisting in developing the task execution framework.
  • Learning and implementing Retrieval-Augmented Generation (RAG) and Model-Based Testing frameworks for the Agent.

1Designing AI Agent System in VR Software Testing

Sep. 2024 – Dec. 2024

3

  • Designing an AI Agent for automated VR game testing and assisting in developing the task execution framework.
  • Learning and implementing Retrieval-Augmented Generation (RAG) and Model-Based Testing frameworks for the Agent.

ComboBench: Can LLMs Manipulate Physical Devices to Play VR Games?

Supervisor: Prof. Michael R. Lyu     Apr. 2024 – Aug. 2024
Position: Summer Research Internship

  • Benchmark Implementation: Curated and structured 270 VR game tasks from four top-rated VR games, establishing the first benchmark for evaluating LLMs performance in immersive VR environments.
  • Experiment: Benchmarked 7 cutting-edge LLMs on VR tasks, assessing their effectiveness in completing complex game objectives.
  • Data Analysis: Conducted in-depth data analysis, assisting in the development of 3 scoring systems for robust LLMs performance evaluation in VR settings.
  • Survey: Designed and distributed questionnaires using Qualtrics to gather human evaluations of VR tasks, comparing them to AI model results.

1ComboBench: Can LLMs Manipulate Physical Devices to Play VR Games?

Apr. 2024 – Aug. 2024

3

  • Benchmark Implementation: Curated and structured 270 VR game tasks from four top-rated VR games, establishing the first benchmark for evaluating LLMs performance in immersive VR environments.
  • Experiment: Benchmarked 7 cutting-edge LLMs on VR tasks, assessing their effectiveness in completing complex game objectives.
  • Data Analysis: Conducted in-depth data analysis, assisting in the development of 3 scoring systems for robust LLMs performance evaluation in VR settings.
  • Survey: Designed and distributed questionnaires using Qualtrics to gather human evaluations of VR tasks, comparing them to AI model results.

2023

An AI-enhanced Adaptive and Individualized eLearning System for Mathematics Foundation Courses in the Faculty of Engineering

Supervisor: Dr. Dongkun Han     Jun. 2023 – Sep. 2023
Position: student helper

  • Data Collection: Collected, organized, and managed comprehensive background data on Hong Kong secondary schools and students.
  • Model Implementation: Assisted in implementing different classification algorithms for predicting students’ learning levels.
  • Model Tuning: Assisted in tuning parameters and organizing training data to enhance model performance.

1An AI-enhanced Adaptive and Individualized eLearning System for Mathematics Foundation Courses in the Faculty of Engineering

Jun. 2023 – Sep. 2023

3

  • Data Collection: Collected, organized, and managed comprehensive background data on Hong Kong secondary schools and students.
  • Model Implementation: Assisted in implementing different classification algorithms for predicting students’ learning levels.
  • Model Tuning: Assisted in tuning parameters and organizing training data to enhance model performance.