Concerns and Considerations

Over-Reliance and Passive Learning

AI tutors, while promising, carry a significant risk of fostering passive learning and over-reliance among students. This often happens when students are tempted to delegate all their work to the AI or outsource cognitive tasks rather than actively engaging with the material. They might use the AI as a “crutch,” copying and pasting answers or relying on it to summarize texts, outline papers, or analyze information without truly grappling with the underlying concepts. This passive approach means students are less likely to critically assess and interrogate AI outputs, instead accepting them without verifying facts, which can lead to neglecting knowledge reinforcement and a potential decline in learning performance (Mollick & Mollick, 2024; Morrone, 2024; Palmer, 2024).

The consequences of this over-reliance are substantial, primarily stifling the development of essential skills. When AI provides quick answers and detailed analyses, it can undermine critical thinking, creativity, and reflection, leading to a shallow understanding of complex concepts. Students may miss out on learning valuable skills like writing, summarizing, analyzing, and drawing conclusions because the AI performs these tasks for them. Studies even suggest that while AI can initially improve performance, it can substantially inhibit learning in the long run, with students performing significantly worse when AI access is removed. This dependency can create “volatile learning patterns that become erased without the presence of AI,” hindering genuine knowledge retention (Bastani et al., 2024; Memarian & Doleck, 2023; Mollick & Mollick, 2024; Wang & Fan, 2025).

To counteract these issues, educators must implement strategies that promote active learning and a “human-in-the-loop” approach. Students need to be taught to critically assess and interrogate AI outputs, actively verifying facts and challenging the AI’s suggestions. Prompt engineering is key here, with prompts designed to spark debate and encourage students to ask good questions rather than just receiving answers. It’s vital to clearly communicate that AI is a supplementary tool for exploration, not a replacement for critical thinking. Promoting critical AI literacy, which includes understanding AI’s limitations, biases, and tendency to “hallucinate,” is essential to student success. Furthermore, designing AI-resilient assignments that cannot be easily completed by AI, such as personal reflections, can encourage deeper engagement. Ultimately, emphasizing the irreplaceable value of human skills, context, and oversight ensures that AI tutors enhance, rather than hinder, genuine learning (Chauncey & McKenna, 2023; Slimi, 2023; Vee, 2025).

Limitations of AI Tutors

AI tutors are designed to mimic human tutors, offering personalized and adaptable learning by leveraging AI algorithms. They can provide direct instruction, offer immediate feedback, generate custom questions and examples, and monitor student progress to identify knowledge gaps. Unlike traditional resources, AI can simulate human-like intelligence to analyze, synthesize, and generate insights, providing on-demand expertise. However, both faculty and students must understand that AI’s “reasoning” is an illusion of automatic syntax generation, not genuine human-like intelligence. AI tutors are effective for quick information checks but often fall short when addressing profound, fundamental misunderstandings that require deeper human intervention (Bailey & Warner, 2024; Solomon, 2025).

Environmental Impact

Faculty who are considering whether to use an AI tutor with their students should make themselves aware of the environmental impact of AI use and weigh the benefits of AI tutors against those negative impacts. The first and second editions of the SUNY FACT2 Guide to Optimizing AI in Higher Education include additional information about environmental impact that can help instructors make an informed decision.

Hallucinations

AI hallucinations, also known as “confabulation,” refer to the tendency of Large Language Models (LLMs) to produce information that is incorrect yet sounds entirely plausible. These errors can be deeply embedded in the AI’s output, making them difficult to detect, and are particularly common when the AI is asked for detailed information, such as quotes, sources, or citations, often leading to the fabrication of non-existent references. This phenomenon is further complicated by “behavioral drift,” where an AI’s responses can vary over time due to updates or changes in prompting. Underlying this issue is the nature of self-supervised learning, where models like GPT-3 may generate synthetic information based on patterns learned from their training dataset, rather than strictly adhering to factual data, which can result in the application of “wrong knowledge” (Chauncey & McKenna, 2023; Mollick & Mollick, 2024).

In educational settings, the risk of AI hallucinations presents a significant challenge, potentially having detrimental effects on student learning if AI-generated hints or responses contain errors. When students rely on AI tutors without critical engagement, there’s a risk of “outsourcing thinking” rather than truly engaging with the material. AI tutors might convey only superficial knowledge or provide subtly incorrect answers that students, lacking the necessary expertise, may not be able to identify as inaccurate. This can hinder learning performance, perception, and higher-order thinking, and some studies even suggest a reduction in creative writing abilities with AI use. The risk of confabulation is particularly high when AI functions in a tutoring role, as incorrect guidance can actively derail the benefits of teaching (Kestin et al., 2024; Pardos & Bhandari, 2024; Wang & Fan, 2025).

Biases

Concerns about the ways in which AI outputs reflect biases have been well-documented and are described extensively in the first and second editions of the SUNY FACT2 Guide to Optimizing AI in Higher Education. It is important to be aware of the impact these biases can have for students’ use of AI tutors. If AI-generated hints or responses contain errors or reflect biases, they can hinder learning outcomes and negatively affect equitable teaching and learning processes. For instance, language models might generate examples or scenarios that reflect cultural or socioeconomic biases, potentially disadvantaging learners from diverse backgrounds. Students, often unaware of these inaccuracies or lacking the knowledge to spot biased content, might over-rely on the AI, stifling critical thinking. Moreover, AI-generated biased information often sounds authoritative and credible, making these subtle and malign errors difficult for users to detect. Instructors who choose to implement an AI tutor should inform themselves and their students of these potential biases and provide guidance to help students think critically about AI outputs.

Equity and Access

Highly effective LLMs, such as GPT-4, are proprietary, and their cost can raise inclusivity concerns for low-income students. The first and second editions of the SUNY FACT2 Guide to Optimizing AI in Higher Education provide a more in-depth consideration of equity and access concerns for AI in higher education. AI tutors present opportunities for bridging these disparities by offering centralized learning opportunities, especially in areas with teacher shortages, and instructors can fine-tune LLMs with diverse data to significantly narrow performance gaps across different languages and cultures, thereby enhancing inclusivity in educational tasks (Kwak & Pardos, 2024; Memarian & Doleck, 2023; Morrone, 2024).