Case Studies: Illustration of AI Tool Evaluation
Building upon the evaluation framework detailed in the previous section, we now present a small sample of case studies that demonstrate how faculty have implemented AI tools to address specific educational challenges. These narratives bridge the gap between theoretical assessment and practical application, showing how educators move from evaluating AI capabilities to purposeful implementation in real educational settings. Rather than treating evaluation as a pre-adoption phase, these narratives also emphasize the importance of continued critical engagement with AI tools after adoption—through iterative review, classroom observation, and policy revision.
The four case studies in this collection showcase diverse applications across computer science education and administrative contexts.
- Learning Technologies Specialist Abby Bechtel’s exploration of Microsoft Copilot for faculty reporting highlights AI’s potential for reducing administrative burden.
- Associate Professor Xin Ye’s implementation, using Notebook LM to support students in a programming course, illustrates how different AI tools can assist with notetaking, organization, synthesis, and scaffolded learning.
- Associate Professor Julie Cuccio Slichko’s pilot study of Brisk demonstrates a measured approach to integrating AI into teacher preparation.
- Dr. Cory Look made a reflective use of ChatGPT to scaffold student critique and creative ideation.
What distinguishes these case studies is their problem-first approach to technology integration. Each implementation begins with a clear instructional or administrative challenge—difficulty organizing course materials, struggles understanding explicit instruction components, or the time-intensive nature of required documentation. The selected AI tools were then evaluated specifically for their capacity to address these challenges, rather than being adopted for their novelty or technical capabilities alone.
We must note here that evaluating tools only before adoption is not sufficient. It is important to revisit and revise AI use policies, practices, and pedagogical framing after initial adoption or implementation. Faculty should consider their own and their students’ responsibilities in monitoring AI use, establishing boundaries, and learning from unexpected outcomes. The contributors’ reflective practice reveals these important insights about AI’s current educational capabilities and limitations. Across diverse implementations, we see common themes emerge: AI tools can effectively supplement but not replace traditional educational resources; they require thoughtful integration with clear guidelines for ethical use; and their greatest value often lies in reducing cognitive load for routine tasks while preserving human judgment for more complex aspects of teaching and learning.
In all cases, meaningful outcomes require not just evaluating the tool, but also teaching students how to use it responsibly, interpret its outputs critically, and understand its limits. These case studies offer modest but concrete examples of how educators can approach AI implementation thoughtfully. By documenting both successes and limitations, they provide colleagues with realistic models that acknowledge both the potential and constraints of current AI technologies in educational contexts. They also reinforce the need to center student rights, educational equity, and critical thinking in all phases of AI use. Faculty must not only evaluate tools for effectiveness but also consider how to teach students to use AI meaningfully, ethically, and with growing awareness of its limits. More importantly, these cases emphasize a cycle of use, evaluation, reflection, and adaptation—an approach that supports not only effective teaching but also student agency and data ethics.
