Through the Looking Glass: The Hidden Design Decisions of Adaptive AI

Colleges and universities are adopting adaptive AI platforms — recommendation engines, intelligent tutoring systems, early-alert tools — at a growing pace. These systems are personalization tools. They are also instructional design tools, making decisions that have traditionally belonged to faculty and designers.

AI systems are deciding what content shows, what gets bypassed, when alerts fire, and what counts as mastery — and these decisions are typically invisible to the institution that licensed the platform. Letting the algorithm make these decisions sounds convenient and effective, but new research shows four in ten AI-driven educational tools produce weaker results for students from underrepresented backgrounds.¹

Adaptive systems make instructional choices that accumulate across a learner's path through a program. By the time their effects appear in outcomes data, the opportunity for timely design intervention has passed. Ultimately, the institution is responsible for these outcomes — and for the decisions that produced them.

What Adaptive AI Decides

Adaptive AI in higher education covers several distinct tool types: recommendation engines that surface content based on prior behavior, dynamic sequencing systems that adjust difficulty in real time, intelligent tutoring systems that respond to learner inputs, and early-alert platforms that flag students predicted to be at risk. Institutions adopt these tools for real reasons — class sizes that make individual monitoring impossible, support staff stretched past capacity, and evidence that earlier intervention improves retention.

View Full Size Image

Image alt. text: Learning Environment Model showing where adaptive AI influences content, interaction, practice, feedback, and evidence.

What these tools share is that they make instructional decisions at the component level: what information surfaces, when practice is triggered and at what difficulty, what feedback to provide, what counts as evidence of mastery. In early-alert and advising systems, they also decide which learners get routed toward human support.

In a standard course design process, those decisions are documented — visible to the whole team, revisable at any stage, traceable back to the learning goals that motivated them. When an algorithm makes them, institutions receive outcomes data. The reasoning behind those outcomes — what was weighted, what was skipped, why one learner was flagged and another was not — typically remains inaccessible. The institution cannot inspect what the algorithm decided the way it can inspect any other instructional decision made on its behalf.

What the Research Shows About How These Systems Behave

Optimizing for success can suppress productive difficulty.

To adapt in real time, the AI needs a measurable target — engagement, completion, problem accuracy. It identifies content the learner can succeed at and eases difficulty when performance drops. For learners, durable understanding depends on productive struggle — working through material at the edge of current capability, where the effort required builds retention and transfer.⁵ AI systems optimize for success metrics which may reduce this type of struggle.

More prediction errors for underrepresented students.

Adaptive AI systems learn from whoever succeeded and failed before. When certain groups are underrepresented in that historical record, the model's predictions for those groups carry larger error. Models built on historical data were nearly twice as likely to falsely predict failure for underrepresented racial and socioeconomic groups.² This is a difficult problem that common correction techniques have not been able to solve, and the pattern holds across multiple educational AI contexts.^2,3

Institutions can't examine decisions they can't see.

When the logic behind a system's decisions isn't visible, there is nothing to examine when some students start getting different results. When the institution receives outcomes data, it has no way to trace a pattern back to a cause — which means it has no way to address it. Transparency and explainability are foundational principles for this reason: accountability requires decisions that can be inspected.⁴

These three findings sit at different layers of the same system: what it is optimizing for, what it produces, and what is visible inside it. Each has a different consequence. Together they describe a system that is difficult to govern at any layer without deliberate institutional design.

Where This Becomes a Design Problem

Adaptive AI deployments don't come with audit infrastructure built in. The system presents results — completion rates, performance patterns, who got flagged for intervention — without surfacing the logic that produced them. Learners using the system don't know why one recommendation appears and another does not. Faculty have limited visibility into how it routes individual students. The institution is the only party that sees outcomes across the student population and carries accountability for what those outcomes look like.

The decisions the system makes at the component level — what information surfaces, when to introduce practice exercises, what counts as evidence — accumulate across a learner's path. They shape the learning experience in ways that are invisible without a structured way to look. The patterns documented above are what that accumulation produces.

The Scaffold That Never Comes Down established that adaptive AI is in part a sequencing tool, and the sequence design has to remain visible to the human designer. Metacognition 2.0 makes the parallel point at the learner level: when a system is continuously interrogating the learner's performance, the learner needs the metacognitive skill to interrogate the system's recommendations in return.

The diagram below shows where the algorithm enters the LEM design process and which components it touches. The audit conversation belongs at the upstream stages — where human intent is set — before the algorithm begins making decisions on the institution's behalf.

The EDUCAUSE AI Ethics framework names eight principles for institutional AI deployment. Four of them — Transparency, Nondiscrimination, Respect for Autonomy, and Accountability — speak directly to the audit work an adaptive AI deployment requires.⁴

The Adaptive AI Audit

The four EDUCAUSE principles named in the previous section translate into four practical questions an institution can ask before adopting an adaptive platform — and return to on a regular basis while the platform is in use.

Four Audit Dimensions for Adaptive AI Adoption

These four questions are not a one-time exercise. They are the basis for a recurring institutional conversation — one that opens before adoption and returns annually, or whenever the system updates. Adaptive systems update their models over time. The institution's review process needs to update with them. A vendor's answer at procurement may not hold six months into deployment.

➡ Download the Adaptive AI Audit Guide — an institutional planning tool for the strategy phase of any adaptive AI adoption. It belongs at the strategy table, before procurement decisions begin, and on a return visit every time the system updates.

Working With Program Teams

In our work helping institutions evaluate adaptive learning platforms, the conversation typically opens with feature comparison — which system has the best dashboard, the most content integrations, the most flexible reporting. The harder questions don't appear on the vendor's feature sheet: what design decisions is the system making on behalf of the institution, whether those decisions are inspectable, and who is accountable for the outcomes they accumulate. These are strategy questions, and LX Studio can help you answer them at procurement, or through investigation afterward.

Stay Connected

Subscribe to the LX Studio newsletter for research-grounded insights.

This post is part of a series on designing learning environments for the age of AI:

• Metacognition 2.0: Teaching Learners to Think About How AI Thinks
• Brains on Borrowed Time: What Students Lose - and Gain - When AI Thinks for Them
• Beyond the Red Pen: Designing AI Feedback Loops That Actually Teach
• Rethinking Assessment in the Age of Generative AI
• Trust but Verify: Redefining Academic Integrity in an AI Enabled Classroom
• AI: The Scaffold That Never Comes Down

References

View our resource library that inspired The Human Side of AI in Learning Series.

1. Judijanto, L. (2025). Beyond access: Cultural, ethical, and infrastructural challenges of AI in marginalised education contexts. European Journal of Contemporary Education and E-Learning, 3(6), 83–98.

2. Gándara, D., Anahideh, H., Ison, M. P., & Picchiarini, L. (2024). Inside the black box: Detecting and mitigating algorithmic bias across racialized groups in college student-success prediction. AERA Open, 10.

3. Baker, R. S., & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32, 1052–1092.

4. Georgieva, I., & Stuart, K. (2025). Ethics is the edge: The future of AI in higher education. EDUCAUSE Review. https://er.educause.edu/articles/2025/6/ethics-is-the-edge-the-future-of-ai-in-higher-education

5. Bjork, R. A., & Bjork, E. L. (2020). Desirable difficulties in theory and practice. Journal of Applied Research in Memory and Cognition, 9(4), 475–479.

AI Use Disclosure: This article was developed with the assistance of generative AI for drafting, refinement, and editorial support. The author reviewed, revised, and vetted the content and takes responsibility for the accuracy, conclusions, and final published version.