Syllabus

Status of Syllabus

• This is the Fall 2026 draft. The syllabus is subject to change, provided the class is informed.

Updates to Syllabus

Professor

Francisco R. Ortega, Ph.D. (NUI Lab).

• Associate Professor (tenured)
• Office: CSB‑360. Available for Zoom/Teams.
• Office Hours: TBD (additional times by appointment)
• Phone: 1‑970‑491‑7445 (I don't use this number much – use Teams)
• Email: fortega@colostate.edu

Note: Office hours begin the second week of classes. My preferred method of communication for quick questions is Teams. Use the class email for course matters. Office hours are subject to change; if this happens I will announce it via Teams.

Graduate Teaching Assistant(s)

TBD — GTA name(s), office hours, and contact will be posted here and announced in the first week.

Overview

CS 567 is a graduate course on Human-Centered Computing for 3D spatial interfaces. It brings together 3D user interfaces (AR/VR), spatial AI, and human-centered AI, and teaches you to research and build in this space using agentic AI. A 3DUI is the medium of communication between a user performing three-dimensional interaction and a computer: one where the input, the output, or both are in 3D space.

The course centers on three things.

1. Using agentic AI for XR and HC-AI research and coding

A core skill of this course — and one that is useful well beyond it — is working with agentic AI. We use agentic coding and research agents (Claude Code, OpenAI Codex, opencode, and others, including agentic AI inside VS Code), together with a small multi-agent, multi-machine stack that you will learn to set up and run:

• amux — runs agent sessions in isolated git worktrees per machine, with a web dashboard, so several agents can work in parallel without colliding.
• hcom — a cross-machine, end-to-end-encrypted (XChaCha20) agent-to-agent bus, so agents on different machines can coordinate.
• mosquitto — a private MQTT broker the bus syncs through (network-restricted and password-authenticated).

You will learn to combine local AI models with frontier models and to use this setup for three purposes: coding, system setup, and research. This is Human-Centered AI in practice — you stay the decision-maker and remain fully accountable for what the agents produce, while learning where they help, where they fail, and how to keep a human in control.

A defining feature of CS 567. Hands-on use of agentic AI — the agents, the multi-machine stack above, and local + frontier models — for research and coding is both a learning objective and a core feature of this course.

2. 3D User Interfaces and Spatial AI

The theory and practice of input devices, interaction techniques, and multimodal interaction for 3D user interfaces, and their applications in augmented and virtual reality (AR/VR/XR). Most head-mounted displays will be Meta Quest 3 (with video-see-through AR); some optical-see-through AR (e.g., HoloLens 2) and additional input devices may be available.

Spatial AI is the perception-and-memory side: first-person (“egocentric”) AI on wearable glasses. These come in two forms: non-display glasses with a camera, microphones, and an on-device assistant but no screen (e.g., Ray-Ban or Oakley Meta), and display glasses that put information in the lens (e.g., Meta Ray-Ban Display, Orion). Display glasses also differ in whether the display is monoscopic (a single 2D overlay, like the Ray-Ban Display’s monocular right-lens screen) or stereoscopic with binocular depth (like Orion). We look at how these devices sense and remember the world, how on-glasses assistants answer questions and recall context (egocentric vision-language models with retrieval and memory), and how this connects to spatial intelligence and world models.

Some of these devices break from the classic definition of AR. Azuma (1997) requires three things: it combines real and virtual, it is interactive in real time, and it is registered in 3D. Non-display glasses meet none of these in the visual field, since they assist through audio and AI, and a monoscopic display shows 2D, head-locked content that is not registered in 3D.

Azuma, R. T. (1997). A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, 6(4), 355–385.

3. Human-Centered AI

Designing AI that keeps people in control: interpretable, mixed-initiative, human-in-the-loop, and aligned with human needs — including how smaller models can still be useful given compute constraints versus frontier models.

A human-centered question runs through the whole course: how do people make decisions as model capacity changes, from small on-device language models (SLMs) to large frontier LLMs? As capability grows, so do cost, latency, privacy exposure, and the chance of confident-but-wrong answers — so what a person should delegate, verify, or keep under direct control shifts along the spectrum. We treat that as a design question, not a fixed answer.

You can also do non VR/AR work. As a graduate course, the emphasis is on reading state-of-the-art and foundational research and on carrying out a substantial project from proposal through implementation and a research paper. Students may use any game engine, language (e.g., Rust), or toolkit they prefer; the class emphasizes interaction and intelligence in 3D space.

NIST Collaboration

This semester, the National Institute of Standards and Technology (NIST) — whose researchers have worked extensively with CAVE systems and AR/VR — will collaborate with our class to propose real research projects. NIST is also recruiting interns in this space.

Important: The NIST internship requires U.S. nationality. However, every student in the course — regardless of citizenship — may work on the NIST-proposed projects. The internship is a separate opportunity from the coursework. Project topics, scope, and NIST points of contact will be shared early in the semester and posted on the Project page.

Faculty Collaboration

You are also welcome to work on a problem with your own faculty advisor (or another faculty member), as long as the project is related to this course — that is, it advances 3D user interfaces, Human-Centered Spatial AI, or the use of agentic AI for that research. Talk to me early so we can scope it to the course requirements (proposal, progress, paper, and implementation). Your work and authorship remain your own, and you stay accountable for what you submit.

Course Objectives

By the end of CS 567, you should be able to:

1. Understand the theory and concepts behind input devices, interaction techniques, and multimodal interaction for 3D user interfaces.
2. Read, evaluate, and synthesize current research across 3D user interfaces and Human-Centered Spatial AI.
3. Design and reason about AI-enabled spatial systems that keep humans in control.
4. Use agentic AI tools (Claude Code, opencode, and others), a multi-agent / multi-machine stack (amux, hcom, mosquitto), and local + frontier AI models effectively and responsibly for research, coding, and system setup — keeping a human in control and staying accountable for the results.
5. Plan and execute a graduate-level research project, including an implementation and a written research paper.
6. Communicate research progress clearly through regular presentations and writing.

Use of AI in CS 567

AI is allowed in this course, but it is optional — you are never required to use it. If you do use it, you remain fully responsible and accountable for everything you submit. AI is a tool, not a substitute for understanding.

• Coding: Use AI to explain concepts, scaffold, and debug. You may not submit work you cannot explain. If I ask you about your code or results and you can't account for them, you may receive a zero. The work is yours; "the AI did it" is not a defense.
• Writing: You may use AI to help you write and edit (including tools like Grammarly). The ideas, claims, and citations remain your responsibility, including verifying that every reference exists and is relevant. Weak or unsound AI-generated text increases the risk of rejection and lowers your grade.

For the paper, we follow the current ACM Policy on Authorship. Two points matter:

• Writing assistance does not need to be disclosed. ACM no longer requires you to disclose AI used to assist with writing the submission.
• Research conduct must be disclosed. When AI is used in the research itself — for example data analysis, running statistics, selecting or generating data, implementing models, running simulations, or producing code, datasets, or figures that affect your conclusions — you must describe that use in detail in the methods section.

In short: AI help with writing and editing needs no disclosure; AI used to produce results does. Every named author is fully accountable for the integrity of the work, whatever its source. (If you target a non-ACM venue, check that venue's policy.) Critical thinking is the point: you can only judge AI output if you have the foundational knowledge to evaluate it.

Email, Canvas, and Teams

Email, Canvas, Teams, and this syllabus (plus the Project page) are critical. Check the syllabus, Teams, and Canvas regularly. Sessions may be recorded, but I can't guarantee the system's functionality.

How to communicate

Be respectful and concise, but provide enough information for me to understand your question. Please follow these guidelines:

• Use Teams for a faster response. Use email for private or essential matters where timing is less critical. When emailing, CC the GTA(s) and begin the subject line with "CS-567". If I don't reply within 48 hours, email again. Use "URGENT" only when truly urgent (not for a 911-type emergency).
• Do not use the Canvas messaging system to contact the TAs or the instructor.
• Unless it is a private matter, use the class email: TBD (the address will be posted in the first week).
• All official announcements are sent via Microsoft Teams. You will be added in the first week. Use the correct channel, and be respectful to your peers, TAs, and instructor. Hate speech or harassment will not be tolerated.
• Please address me as Professor, Professor Ortega, Dr. Ortega, or Francisco — your preference. My goal is for you to develop professional communication skills.
• My pronouns are He/His/Him. If I get your pronouns or the pronunciation of your name wrong, please correct me — in private or in class. I have an accent and some hearing/speech differences, so feel free to correct me at any time; it is totally fine.
• I don't know everything about HCI, VR, or AI — these are broad, fast-moving fields. I will always come back with an answer when I can find one.

If a student behaves in a manner considered prohibited conduct, I reserve the right to ask that student to leave immediately and to report the behavior to the appropriate department.

Format & Expectations

This is a graduate course built around reading, discussion, and a semester-long project. As a Human-Computer Interaction (HCI) / 3DUI course, it involves more reading and writing than a typical systems course, while still requiring you to build. Expect to read research papers every week and to write throughout the semester. Professional and scholarly maturity is expected at this level.

This year the evaluation is different from prior offerings. There are no midterm exams, and the only quiz is a low-stakes syllabus quiz. Instead, your grade is driven by continuous evidence of progress: project checkpoints and presentations, a proposal, and a final project (paper + implementation), plus a single cumulative final exam. See the Grading Breakdown for the full list.

Project Teams

You may complete the project solo or in a group of 2. Both the implementation and the research paper are required deliverables either way. Details, milestones, and the rubric will be posted on the Project page and on Canvas by the end of the second week.

Prerequisites

Graduate standing in Computer Science, or instructor consent. Programming maturity is assumed. See your advisor or the CS website for details.

Grading Scale

A+: Given only to top students who have gone above and beyond (and already have an A). For your GPA, an A+ is the same as an A.

I reserve the right to curve in your favor per deliverable or for the class. This is uncommon, but if it happens I will announce it.

Grading Breakdown

Your grade is built from the components below. Most of the weight is on continuous, demonstrated progress toward a project and paper.

There are no midterm exams; the only quiz is the low-stakes syllabus quiz. Rubrics and exact dates are released on Canvas and the Project page.

Final Exam

There is one cumulative final exam, worth 10%.

Final exam date/time are confirmed by the Registrar and cannot be changed. The exam concentrates on theory and concepts from lectures, readings, and discussion. If a proctoring tool is used, any suspicious activity will be reported.

The AI Dilemma

How computer scientists should use AI to aid coding and writing — and these are different — is an open question I take seriously, alongside the ethical considerations (energy, copyright, confidentiality). I will share my current thinking, as of Fall 2026, and we will discuss it in class. If you let AI do everything, you will not learn, and you will not be able to maintain what you "wrote." Use it to support your learning, not to replace it.

Late Work and Make-ups

Because the grade is built on continuous progress, deadlines for checkpoints and presentations matter. A 24-hour late policy with a 10% penalty applies to progress deliverables unless stated otherwise; presentations and the final exam cannot be made up except for valid, verifiable reasons. If I extend a deliverable, the penalty may be waived. I will tell you in advance when a deliverable has no late policy.

Respect for Others

Be respectful of everyone's right to be who they are, including members of the LGBTQ+ community, minorities (race, ethnicity, etc.), and people of differing gender, culture, personality, and political views. This class deals with humans, so opinions will sometimes differ; we will always use evidence and science to work through the issues. Order in class is required.

If a student behaves in a manner considered prohibited conduct, I reserve the right to ask that student to leave immediately and to report the behavior to the appropriate department.

Accommodations

1. Contact me as soon as you receive your memo from the SDC to discuss your needs. Please read the SDC section of the syllabus before our conversation.
2. Our conversation should address the specific requests in the SDC memo, their applicability, and reasonable arrangements for the course.
3. All students are expected to submit all deliverables as defined in the syllabus. Time extensions may be limited by the continuous, progress-based structure of the course; extension as a reasonable accommodation will be considered.
4. Accommodations cannot fundamentally alter the nature of a course.
5. Retroactive accommodations are generally not permitted.

Teaching Philosophy and Other Matters

While my classes don't share one syllabus, I keep some standard practices:

1. My objective is for you to learn the core principles of the course. For CS 567, those are the theory and practice of 3D interaction, and how Human-Centered Spatial AI reshapes it.
2. I aim for a science-of-learning approach (repetition, spacing, retrieval). I recommend A Guide to Effective Studying and Learning by Matt Rhodes et al.
3. I start from trust. I will not try to trick you on exams or deliverables. This does not mean cheating is acceptable — it is not — and you must know the rules for this class.
4. AI detectors are unreliable and can disproportionately harm historically marginalized groups; I do not use them as proof. You must still learn to work with AI and to learn with AI. Each class has distinct rules; understand and respect them.
5. There is, by design, a separation between students and me — I can affect your grade. We are not friends, even when I'm friendly. TAs do not date students, and professors don't date students or TAs. Please be respectful of the TAs.
6. When in doubt, ask me — preferably in writing.
7. I am the arbiter of this class, but you may appeal a grade or contact my superiors. As a professional lesson: try to resolve issues with me first.
8. Your grade evaluates where you were during this class in a given period. A B or a C does not mean you are a bad computer scientist; it says nothing about you as a person.
9. I will not tolerate sexual harassment, abuse, physical violence, or intimidation, and I will report it. A safe environment (safe from harm, not from debate) is critical to learning.
10. I want you to have as much fun as you can while there is grading!

About Cheating (Academic Integrity)

Don't cheat. You know what cheating looks like, and so do I. Make sure you understand the CSU Academic Honor Principle. On the final exam, all work must be your own. For the project, collaboration within your team is expected, but work shared across teams must be cited, and you must attribute any code or text you did not write yourself. Misrepresenting output, or submitting another team's work as your own, is a violation.

A good rule of thumb for collaboration across teams: if you are talking in plain English about ideas, you are probably fine; if you are dictating code line-by-line to another team, you have crossed the line. When in doubt, ask me, and err on the side of caution.

Strict Rules Against Copying

Cheating will be reported. You will only lose points for a deliverable you choose not to submit. However, submitting plagiarized work may result in an F for the class and additional consequences.

© 2026 Dr. Francisco R. Ortega. All rights reserved.