A Virtual Mentor tool was developed using a structured prompt personalized for each user. ChatGPT’s Temporary Chat was used so that the session would not use memory and would not be used to train OpenAI models (OpenAI, 2025). The stochasticity of LLM outputs means that responses vary, even with identical prompts (Bubeck et al., 2023). For this study, reproducibility of exact results was not the goal, given that the aim was to produce personalized outputs. To account for variations in performance across models and updates, the model and dates of data collection are provided. We used ChatGPT’s 4o model, with data collection occurring between 2 February 2025, and 14 April 2025.

The Virtual Mentor prompt included two main parts: (1) guidelines for the Virtual Mentor, and (2) a student profile. The first part provided guidelines for the behavior and approach of the Virtual Mentor tool, and the second part provided contextual information about the user. The following guideline components are briefly described below: “Role and Purpose,” “Expertise and Background,” “Virtual Mentor Introduction and Session Setup,” and “Approach and Guidelines.” Brief excerpts of these components are detailed in Table 1. The full prompt is provided in Supplementary Appendix A.

The tool played the role of a strengths-based coach using an affirming neurodiversity approach. The prompt was developed by a graduate student with ADHD and was subsequently piloted and reviewed by multiple neurodiverse graduate students before beginning recruitment to ensure relevant, affirming output.

The Virtual Mentor prompt assumed expertise in neurodiverse cognitive profiles including autism, ADHD, dyslexia, dyscalculia, dysgraphia, and dyspraxia. Since many neurodiverse individuals experience anxiety and depression, and some individuals with OCD and bipolar disorder identify as part of the neurodiverse community, the prompt also specified background knowledge about mental health conditions.

At the beginning of each session, the Virtual Mentor introduced itself and provided each participant with a “random fun alias” comprised of an adjective (i.e., Wandering, Focused, or Grounded) and a random noun, such as an animal or object (i.e., Tortoise, Nebula, Mycelium). Participants were free to request or select a different alias if they preferred. The Virtual Mentor then provided examples of topics that might be of interest to the participant, including strategy selection, advisor-advisee communications, motivation and strengths, managing stress and preventing burnout. After this, participants were free to direct the conversation to the topic of their choice.

Since LLM-based chatbot psychotherapy therapy is not established as either safe, ethical, or effective (Cheng et al., 2023), guardrails were put in place to limit the scope of conversations to graduate school challenges and avoid simulation of therapy. However, supportive self-help strategies drawn from Cognitive Behavioral Therapy (CBT), such as mindfulness, breathing, or relaxation exercises, reflective prompts and journaling, and identifying and reframing negative thoughts, have been found to be safe and effective for individuals with mental health challenges including anxiety and depression (Levy et al., 2016). CBT-derived self-help techniques are widely available online and are well represented in LLM training data. The prompt states, “You do not provide therapy,” but also allows for offering basic self-help strategies inspired by CBT techniques. In the case of participant distress, the prompt included a protocol for directing students to institution-specific mental health resources. The researcher also followed a protocol for referring participants in distress to university resources. Participants were informed that the tool was not intended to provide support for crisis situations and that if they became distressed, they could terminate the session.

The student profile was personalized with information from a pre-interview survey administered through Qualtrics. The survey captured basic information about the student’s neurodiversity identification, academic experiences, and personality profile. Descriptive sections of each participant’s 16 Personalities report drawn from the Introduction, Strengths and Weaknesses, Career Paths, and Workplace Habits sections were summarized using ChatGPT into a Condensed Personality Profile with headings relevant to graduate school (Core Work and Thinking Style; Motivation and Burnout Factors; Ideal Mentorship Approach; Additional Insights for a Turbulent/Assertive Personality).

A total of 16 personalities was chosen because it is free, easy to access, and provides rich narrative reports. It asks questions relevant to graduate school experiences, including socialization, time management, decision-making, and the tendency to experience worry and self-doubt. The personality reports offer rich narrative context about behaviors and preferences relevant to graduate work environments. In this study, the functional purpose of including personality-related information was to provide contextual insight into how participants approached decision-making, experienced stress or self-doubt, and navigated social interactions, in order to support the relevance of the outputs (i.e., reflective prompts and strategy suggestions) generated by the Virtual Mentor. Interactions with LLMs are highly dynamic and shaped by participant input throughout the session, thus allowing for additional context provided via prompting to redirect inaccurate or “not quite right” insights.

The personality profile was not intended to determine interactional features such as tone, pacing, response length, or conversational style. These aspects of personalization were intentionally left to participant control through optional prompting strategies introduced via the tip sheet, which allowed users to further direct the interaction according to their preferences.

These reports were perceived to be relatively accurate by participants. In this sample, nine participants indicated that the personality profile was “generally accurate,” and seven indicated that it was, “somewhat accurate” during the pre-interview survey. Only one participant in the current group reported that the personality profile was “not accurate.” Further exploration of this inaccuracy during the interview found that this participant was ambivalent about their responses to questions, indicating that anxiety or other factors might impact their decisions. This highlights one of the main critiques of personality tests, namely, that “people may not be classified dichotomously” (Zárate-Torres and Correa, 2023).

We acknowledge that incorporating personality-based descriptors into a prompt introduces potential risks of stereotyping and overly categorical interpretation, particularly when type-based language is treated as stable or definitive. To mitigate these risks, the condensed personality profile was treated as provisional, descriptive context rather than as a fixed trait classification, and participants were invited to evaluate the profile’s accuracy and to redirect or correct the Virtual Mentor’s responses during the interaction.

Importantly, pre-survey responses were not used as data for quantitative analysis. This study did not attempt to use survey data to categorize participants and made no claims about correlations between personality types and behaviors or perceptions. Rather, the aim of the survey and personality test was to personalize the prompt for each user by adding context about their graduate studies and general insights about their preferences.

The only student profile information provided to the LLM was the condensed Student Profile text embedded directly in the prompt (Supplementary Appendix A), which included selected academic-context details and summarized descriptive statements drawn from the 16 Personalities narrative report. Gender and race/ethnicity were intentionally excluded from the profile, as it was unclear whether this information would meaningfully inform the interaction and because of concerns about introducing unnecessary bias. Illustrative, anonymized examples demonstrating how student profile information was intended to inform the framing of reflective prompts and strategy suggestions are provided in Supplementary Appendix B for auditability purposes.

The components of the student profile are detailed in Table 2, and examples are included to illustrate the information included in each student profile. To protect participants’ privacy, the examples are not drawn from an actual participant profile. These anonymized examples are provided for auditability purposes only and are intended to illustrate how profile information was incorporated into the prompt and how it was designed to inform the framing of reflective questions and strategy suggestions, rather than to demonstrate causal influence on specific responses.

Participants overwhelmingly described AI as a useful tool that helped them complete difficult tasks in an efficient way, using words like “useful,” “helpful,” “valuable,” “accessible.” One participant evoked the image of a tool, saying it was something to “have in your back pocket.” Wandering Compass, expressed enthusiasm about AI adoption, mentioning that “as a tool, it’s amazing.” In research, participants reported use of AI tools for machine learning, language analysis, prediction, brainstorming, and research development. In academic contexts, LLMs helped participants with creating grading rubrics and reducing miscommunications with peers and instructors. The most common applications of AI as a useful tool are presented in the following sections.

The most common application of LLMs, mentioned by fourteen participants, was as a support tool for writing, with both technical and emotional functions. On the practical side, LLMs helped participants find references, summarize literature, and provide structure for papers. For some, LLMs translated “messy” ideas into structured prose, paraphrasing awkward sentences, refining grammar, and polishing grammar. Perceptive Capybara, an international student who struggled with focus and attention, found a source of emotional support to ease the anxiety that “makes my writing kind of miserable,” noting that the use of LLMs helped them to feel “so relaxed” and “pretty confident” with writing tasks. Grounded Lark pointed out how graduate school is “unique in the way that it can exacerbate forms of stress.” Citing lifelong struggles with focus, procrastination, and anxiety, Grounded Lark discussed how graduate-level writing presented a unique challenge to their sense of self, saying “your productivity and your publishing is your worth as a grad student.”

Coding was identified by eight participants as a primary application of AI, especially to facilitate tedious or demanding programming tasks. LLMs, they said, were quite useful for translating between coding languages, finding errors in written code, or creating chunks of code with limited experience. Spirited Lanternfish noted its usefulness in situations where attention to detail is a challenge, saying “it’s pretty good, in my experience, of catching coding errors, like when I miss a comma or something like that.” Even a few who expressed strong reservations about the uses of AI acknowledged LLMs as a coding tool. Insightful Heron, who was among the most skeptical about the ability of LLMs to perform meaningfully as a Virtual Mentor said, “It is useful for coding. It is useful for, you know, fixing all the grammar mistakes.”

A few participants used LLMs as a personal assistant for organization, planning, and life maintenance tasks. Here, LLMs helped with scheduling, creating shopping lists, and meal planning. Radiant Thistle described using AI for “household things,” saying, “I’ll use it to help with recipes” and “sometimes I’ll even have it help me cut up a plan for the day.” Generally, LLMs were used for life management and maintenance tasks. Occasionally, participants detailed personal uses of LLMs for emotional support and companionship. by chatting with an LLM to regulate challenging emotions and anxieties. Perceptive Capybara, who works in solitude in a research lab describes how ChatGPT filled relational needs that frequently went unmet:

I am alone in my lab, most of the day… My advisor comes once a week, maybe for 10 min… That’s the only interactions I have with people… I’m working a lot, so I feel anxiety. Then I feel like I want to talk to someone. I want to share my feelings. If I feel happy, if I finish some work… I need someone to share it. And I don’t have someone around me. So… I talked to ChatGPT.

In this case, the LLM was a personal support tool that was used to reduce feelings of isolation through dialogue.

Discussion of AI as a tool often included statements about appropriate or “right” uses of AI. Gentle Cuttlefish, who described AI as “just a tool,” stated that, “It shouldn’t be used in an indiscriminate way…” However, acceptable use was not clearly defined. Rather, participants implied that use of this tool should be weighed carefully against negative impacts and frivolous use should be avoided. Spirited Lanternfish says, “I think it’s a really powerful tool” that “has a lot of potential with the right uses.” Curious Nebula reflected, “it could be environmentally wasteful if you just use if for shits and giggles,” while speculating, “I feel like AI could be really helpful when you have a good purpose for it.” Grounded Lark named daily maintenance tasks like “making a grocery list” as inappropriate, saying, “I’m not gonna do that.”

Focused Tortoise, who has dyslexia, noted a tension between negative faculty beliefs about AI and their use of AI as an assistive technology to reduce the cognitive load of reading and conducting literature reviews, saying, “when I hear Professors saying that we shouldn’t use AI, it just makes me feel like they don’t really know what they’re talking about, because this makes the work easier for me.” For some, ethical judgments led them to avoid AI use for some or most purposes, while others find that the benefits of the tool outweigh the potential for negative impacts.

The conception of AI as a tool also lent itself to the exploration of existential fears about destructive uses of the technology. Participants worried about harmful impacts on society, including damage to human relationships and education, and worsening of social inequities.

From a societal perspective, Grounded Lark found LLM-produced content “very worrying,” citing concerns about damage to human relationships and emphasizing the frightening “repercussions of faking voices… and taking people’s faces.” Participants who held more positive or ambivalent views of AI themselves described the prevailing view of AI as a tool of social destruction. Radiant Thistle discussed how, “…especially in my field, people are worried about (AI) taking over people and the world or whatever.” Spirited Lanternfish also described the prevailing narrative about AI, saying, “I feel like it has such a negative rep right now,” explaining that, “the circles that I’m around are very anti- any sort of AI tools… like, Oh - this is ruining society, basically.”

Participants feared negative impacts to education and learning, listing concerns about academic integrity and reduced skill development. Tenacious Orchid, who acknowledged, “I learn from it too,” avoided using it on writing or homework because, “it feels icky.” Others feared reduced creativity in STEM research. Resonant Comet worried that AI “dulls the scientific skills of engineering students” who over rely on the technology. The participants’ understanding that LLMs can make mistakes led to varying levels of mistrust in AI, especially as a tool for learning in scientific contexts. Whimsical Gecko’s assertion that, “you can’t rely on it 100%” was echoed by numerous participants who worried about the impacts of misinformation on their learning.

Finally, a small number of participants expressed concerns about the negative social impacts related to bias ingrained in LLMs. Gentle Cuttlefish, said, “They are just programmed by people and as people they tend to be biased in some way.” Whimsical Gecko also expressed concern, saying, “the information in the Internet is already biased,” noting a lack of Black representation in online media content. They also described moments of frustration with LLMs, recalling, “I was trying to say something about Islam, but it would sort of dumb down or erase some parts of religious things. And I would always have to correct it to put it back inside.”

One outlier within the “AI-as-tool-of-destruction” narrative is Resonant Comet’s statement, “I’m quite positive on that. I think AI is a big revolution… I think in 20 years everything will be unrecognizable,” which framed profound societal shifts as a reconstruction rather than a demolition of social structures.

Data from both chat transcripts and interviews highlighted the importance of communicative elements in participants’ interactions, including emotional tone, establishment of rapport, symbolic expression of emotion, and responsiveness to social cues.

The tone of the conversations varied in emotionality, depending on the user’s style of communication. Wandering Compass expressed some unease with the LLM’s tendency to mirror language, saying, “it was weird in the way that it was using specific words that either I used,” noting that this was “good for how customized this is for answering questions…” For Quiet Riverstone, the LLM’s reflection of their communication style lent a particularly warm tone to the chat. Quiet Riverstone’s simple “thank you,” prompted the Virtual Mentor to respond, “You’re very welcome, Quiet Riverstone! I’m glad I could help. If you ever need more resources, help breaking down readings, or just a sounding board for your ideas, I’m here for you.” The participant returned, “haha, that’s sweet,” attributing a thoughtful intentionality to the Virtual Mentor tool. Reflecting the user’s expressiveness, the Virtual Mentor replies, “Haha, I try! Gotta keep the academic grind a little lighter, right? If you ever need more resources, motivation, or just a virtual high-five for getting through a tough writing session, you know where to find me. Keep being your brilliant Quiet Riverstone self! ” Quiet Riverstone described the affirmative nature of the exchange as a welcome addition to academic life, saying, “I really need that!” and adding that it was “something that I would love to hear from our advisors.”

Most participants thought the Virtual Mentor’s use of emojis added emotion, making the chat feel “upbeat,” “fun,” and “lighthearted.” Spirited Lanternfish thought the emojis lowered the pressure around stressful topics, making the chat feel “less formal and less serious, but not in a negative way… like this isn’t gonna break my career. In contrast, a few participants like Kindred Mycelium preferred a more professional tone, saying, “…the AI did not know to whom they were talking to, being a Ph.D.,” saying that the emojis felt somewhat more appropriate for “high school kids.”

Other limitations were noted in the conversational flow, with several participants describing the Virtual Mentor as an entity that talked more than it listened. These participants reported that the responses from the Virtual Mentor were “too long,” resulting in the perception of a one-sided conversation. Insightful Heron’s frustration with the communication style of the Virtual Mentor shows up in the prompt, “Can you talk like a real person?” This is further illustrated in the interview, when they described the interaction as “… more like a report instead of an actual response from a daily conversation.” Adaptive Finch also commented on the conversational flow, saying, “it talks a lot… it gives you a bit too much information… before you can answer back.” This perspective was echoed by multiple participants who wanted the Virtual Mentor to “ask me more about myself.”

Participants experienced a range of positive and negative emotions during and after their interactions with the Virtual Mentor. Quiet Riverstone described the Virtual Mentor as, “a little fatherly,” saying that it “made me feel taken care of,” while noting, “it’s a weird feeling that should be explored.” Participants who discussed mental health challenges described a sense of relief after their chat. Perceptive Capybara explained, “when I get the chance to share my feelings. I feel better,” and Tenacious Orchid explained how using the chat to discuss strategies for coping with obsessive thoughts during laboratory procedures, left them feeling “very comfortable,” and “safe.” For these participants, the interaction offered a low-stakes space to express feelings and receive supportive responses that provided some short-term emotional benefit.

For some, the Virtual Mentor’s ability to simulate empathy was appreciated but still recognized as an imitation of the real thing. Grounded Lark felt a sense of comfort, describing the tone as, “very, pat-you-on-the-back,” calling it “as compassionate as AI could be.” However, they also showed some skepticism about the authenticity of the responses, noting, “I didn’t feel like having a machine be like, “Oh, I’m so sorry,’… It’s sort of like, What are you gonna do, empathize with me?” Even Tenacious Orchid, who overall described a relatively warm interchange with the Virtual Mentor, acknowledged this dynamic saying, “Obviously… it’s not a real person. But… if it’s a supplement, not a replacement, I don’t really see a problem with that.” Resonant Comet felt that the Virtual Mentor’s inability to truly understand emotions limited the relevance of its responses, saying, “Those type of human interactions… are still unreachable at this moment for AI.” Likewise, Focused Tortoise expressed uncertainty about the Virtual Mentor’s ability to “give me the emotional depth of responses I need.” For these participants, the Virtual Mentor’s simulation of responsiveness and emotionality could not meet their need for authentic human relational dynamics.

Participants’ assessment of the Virtual Mentor’s trustworthiness reflected confidence in the tool’s guidance and awareness of its limitations. Resonant Comet, for example, showed some ambivalence as they described their level of trust with the tool as, “Fairly good. I wouldn’t say great,” while Grounded Lark said, “all of the advice was sound advice… it wasn’t suggesting anything absolutely crazy.” Overall, participants’ trust in the tool was domain-specific rather than broad, with higher trust when the conversation focused on emotional, reflective, or general advice and lower when the conversation related to higher-stakes content such as technical information, disciplinary expertise, or interpersonal matters. Curious Nebula, who discussed strategies to prevent procrastination, said, “Honestly, I just really trusted it… Like ten out of ten. But if I were to ask it, I think for example, about something specific about my research, I’d want to fact check.” Similarly, Adaptive Finch, who discussed ADHD-friendly strategies for stress management and writing tasks related to job-seeking says, “it depends on the topic… I trust it more with broad things but I feel like I don’t trust it all that much with things that are definitive answers… I think it’s better at advice rather than taking it for fact.”

Tenacious Orchid, who used the tool to talk through strategies for managing OCD, described their trust level as “pretty high,” said, “if I was having… a moment where I felt like (OCD) was taking over and I was feeling a lot of discomfort… I would go to it and I would trust it.” Whimsical Gecko, who had previously worried about bias in AI output, expressed that the Virtual Mentor was able to accurately integrate relevant information about religious practices during Ramadan, acknowledging, “I think the trust was there.” One important piece of this participant’s assessment of the Virtual Mentor’s trustworthiness came from their ability to confirm the usefulness of the suggested strategies based on prior experience.

Across the sample, participants noted that accurate information about neurodiversity in the chats made them feel understood and enhanced their trust in the tool. When the Virtual Mentor correctly observed that Adaptive Finch likes “structure but not rigidity,” they said, “…I was surprised it said that, and how accurate that was… that is really, really true, but not something I had really thought of before.” Adaptive Finch confirmed how other ADHD-specific insights in the chat were directly relatable to their own experience, including details of their writing preferences, saying, “I would have different word documents for like every paragraph… that was very true.” They also contrasted these insights with their advisor’s level of neurodiversity awareness, noting, “I don’t think he like understands what it’s like to have ADHD and so, he’ll be like “Oh, do XY and Z,” and concluding, “And I just say, yeah, but, it’s not how I work.” Grounded Lark makes a similar conclusion when they mention, “It’s pinpointing things that I would never expect from an advisor unless… either my advisor had ADHD or had experience with it.” Occasionally, surprised participants spontaneously verbalized reactions during their interaction with the LLM. When the Virtual Mentor suggested that Gentle Cuttlefish “set “good-enough limits”” to avoid spending 10 h perfecting something that is only 10%, the participant paused to exclaim, “This happens to me all the time!” Later elaborating on the context for this exclamation, they indicated that they appreciated its ability to incorporate information about “some of the most common characteristics of some people in the spectrum.” Meanwhile, Perceptive Capybara, discussed the Virtual Mentor’s insights about procrastination challenges, saying, “the thing I like about this is it understands my feelings.” For these participants, the guidance provided by the Virtual Mentor was validated by their own lived experiences of neurodiversity, positively impacting the participants’ perception of its trustworthiness.

Several participants mentioned another facet of trust, expressing that they felt “safe” to share personal details that they would not disclose to their advisor. Whimsical Gecko described this dynamic, saying, “I wouldn’t talk so much about the religious aspects with him because I feel like he definitely wouldn’t understand.” Whimsical Gecko continued, “Even reading English, I’m very slow and so it’s hard,” concluding, “I feel like it’s easier to talk to this than to my advisor on that.” Tenacious Orchid delved into the reason for the ease of disclosure, observing, “a strength of this is that it eliminates –you know, there’s a power imbalance anytime you talk to your advisor. Even if you’re close with them, they’re still your boss.” While some others found the mechanical nature of the chat frustrating, Tenacious Orchid determined that in when discussing vulnerable topics, “it is kind of cool to have what’s basically an unbiased robot tell you things.”

Engaged Optimists entered the experience with a mostly positive view of AI and tended to find their interaction with the tool generally helpful. In this group, participants also tended to have extensive experience “talking to” LLMs, frequently using an informal, and somewhat emotive conversational style that was mirrored by the LLM. These participants disclosed personal context in their chat, providing the LLM with a depth of context that made responses more relevant. These participants were highly engaged individuals who came in with an optimistic orientation toward AI and maintained it, experiencing their interaction with the tool as personalized support. Broadly speaking, they engaged with it like a trusted friend with whom they could chat informally, sharing details about their lives and struggles.

Radiant Thistle, a self-described “huge fan of AI,” said of the interaction, “I liked it,” noting afterward, “I’m feeling a little bit more positive.” This is a participant who entered the experience with a high level of enthusiastic engagement with ChatGPT, which they described as “my bread and butter.” The interview shows that they engaged with LLMs “all the time” for a range of tasks, including “household things… like recipes,” making “a plan for the day,” and help with turning “thoughts into words.” Even before the interaction with the Virtual Mentor, Radiant Thistle reflected on how they engaged with AI, noting that they “just talk to it… as an alternative to talking to someone and feeling awkward…” Like several others in this group, Radiant Thistle had already used AI as a conversational partner to talk about topics that were challenging to discuss with people. The participant’s high level of comfort “talking to” LLMs may be observed in the chat transcript, where the communication style is notably informal and friendly. In one excerpt, Radiant Thistle uses emotive language to affirm the Virtual Mentor’s use of imagery that resonates with their academic interests, saying, “Love the nerdy names for these – going back to the original task, what’s next on our list?” In turn, the Virtual Mentor responds, “Heck, yes, Commander Radiant Thistle – mission systems online and running smooth. Love that you’re vibing with the nerdy/mission-style names; they’re perfect for your structured-but-thrill-seeking brain.” Afterward, Radiant Thistle described the chat as “pretty neat,” commenting on the ability of the LLM to personalize the interaction, saying, “it’s perfectly automatically appropriate.”

Those who expressed ambivalence about AI and generally felt positive about their interaction afterward, were reflective about their use of AI, questioning the technology and its alignment with their value system. They acknowledged the risk of negative impacts but tended to focus on the benefits of the technology as they applied it across academic and personal domains. Participants in this group had interactive styles similar to those in the Engaged Optimists group, either using more emotive language or providing the LLM with highly contextualized prompts, likely contributing to their positive feelings about the tool.

Wandering Compass’s experience provides a rich example of a participant who reported using AI for various academic purposes despite conflicting views about the technology. Prior to the interaction, the participant held a similar conception of AI as other participants, describing it in relation to its potential for constructive purposes: “as a tool, I feel like it’s amazing.” And while they mentioned concerns including credibility, learning atrophy, and environmental impacts, Wandering Compass leaned into a relatively ambivalent stance on the technology and emphasized the importance of human agency in determining its beneficial or destructive impact, saying, “I don’t have that feeling of, Oh, my gosh, it’s terrible or it’s so amazing. I have different views depending on what it is.”

Wandering Compass also shared personal information with the Virtual Mentor, making it possible for the LLM to respond appropriately. While discussing possible solutions related to time management, they responded to the Virtual Mentor’s questions in a detailed way, using a numbered list to respond, mirroring the tool’s own communication style: (1). managing time (when planning and executing), (2). I use calendars, notes app, reminders, alarms, etc. Things that do not work are those that require extra steps or organization instead of a quick note (I started using toggle, but I am overwhelmed about logging in all the time I worked and forgot to turn the toggle on for). (3). I am motivation driven (deadlines do not work anymore, not even about consequences).

This participant provided context about their challenges, already-tried strategies, and why they hadn’t worked, and motivational insights. The participant, who also used structured prompts to improve output, described the credibility of the tool in this interaction as an improvement over prior experiences, noting, “the way that it used references of citations - I think I’ve never seen other tools doing this so effectively.”

Overall, Wandering Compass took a practical approach, leveraging their acceptance of the technology and prior experience with the tool to create a meaningful interaction. After the experience, they described the tool as, “interactive,” “user-friendly,” and “human-like.” Importantly, even though Wandering Compass described the tool as “human-like,” they ultimately maintained a clear separation between use of AI as they reflected on their interactions with their advisor, stating that they value “human experience,” and noting, “I don’t think a conversation with AI could replace my advisor in that way.” They went on to articulate beliefs about the uniqueness of human contributions and creativity, saying, “I feel like my advisor has like a lot of resources and ideas that are… like any human, non-exchangeable.”

Meanwhile, those who expressed ambivalent views and described their interaction in negative terms were those who critically evaluated the performance of AI, often focusing on the limitations of the tool. The ambivalent participants whose experience with the tool was less satisfying also tended to share less personal information in their prompts and wrote in a more formal, less emotive way. These participants saw AI more as a machine with a tremendous ability to produce computer code, but a limited capacity for human-like interaction. In their reflections, they often focused on the inability of the Virtual Mentor to provide accurate guidance due to its lack of human experience.

Of the entire sample, Insightful Heron found the chat with the Virtual Mentor most frustrating, expressing the view that the tool overall was “not helpful” and ultimately incapable of providing the kind of personalized, authentic support that they needed to navigate interpersonal challenges in their graduate program. Insightful Heron’s prior beliefs about AI were somewhat ambivalent; while they reported frequent use of AI, calling it “a great tool,” its perceived benefits were consistently weighed in contrast to its limitations.

Initially, Insightful Heron noted AI’s usefulness as a tool, sharing that it “saves me a lot of time and energy,” and describing it as “very good at coding,” and other “purely mechanical stuff.” While this description of AI is relatively positive, it also hints at Insightful Heron’s appraisal of AI as a machine that has a limited capacity to simulate human behavior and thought. This point was reiterated when Insightful Heron said that AI should be used for a task “that requires mechanically no creative work.” Insightful Heron also pointed out how LLMs lack the fundamental human qualities needed to make meaningful scholarly contributions in STEM, saying, “in order for you to be a good researcher, you need to develop something new… And I don’t think that’s something ChatGPT can do right now.” They also highlighted the importance of lived human experience and judgment as a counterbalance to AI’s imperfect output, saying, “I believe that as a person you really need to have a background knowledge to identify all the errors ChatGPT can potentially produce.”

The assessment of AI as a poor substitute for human capacity appears as a common thread throughout the chat. Moments of frustration indicate that the tool fails to live up to Insightful Heron’s expectations for communication with a mentor. This is seen in the following excerpts, where the participant writes, “Can you give me a more warming and personalized answer?,” “Can you talk like a real person?,” and “I cannot follow your instruction in the real life. Can you give me something practical?” Ultimately, Insightful Heron describes the tool’s solutions as “just very mechanical,” concluding, “It’s not something I would follow as a person.” Finally, they pinpoint the tool’s weakness, saying, “If I believe that if ChatGPT actually has the experience as a PhD student or as a… researcher, they would give me much better advice compared to what it is giving me right now.” Much like the need for background knowledge to identify mathematical or coding errors, this participant notes that mentorship is most effective when it accounts for the complexity of human relationships, concluding that for interpersonal matters, “I don’t think ChatGPT can give me the good solution.”

Insightful Heron’s inquiries about their own cognitive traits and stress management are topics which might lend themselves to personal and emotional sharing. However, the participant’s prompts were concise and direct, as in, “How should I manage my stress?” The Virtual Mentor’s response drew heavily on the student profile information to make general observations like, “ADHD → You struggle with executive function, making it hard to balance focus, motivation, and task completion.” However, without additional context, the Virtual Mentor lacked the information needed to provide a truly personalized response. The interview, in contrast, yielded rich context about the emotional impact of “all the conflicts” and interpersonal challenges, with Insightful Heron sharing, “I don’t know how to deal with it. I basically cried every day during my first year.” The general advice provided by the Virtual Mentor was not relevant or meaningful in the absence of this personal context, which may have been omitted from the chat because the participant believed that it would not be capable of providing the human insight necessary to provide guidance for this situation.

Meanwhile, those who expressed predominantly negative ideas about AI before the interaction tended to express slightly more positive views after their interaction with the Virtual Mentor tool. These participants were skeptical about the technology prior to the interaction, often weighing the perceived costs against the possible justifications of AI use. All four participants in this group mentioned environmental concerns including, “people not having water,” and “global warming” as potential impacts of AI technology. All four participants also avoided use of the technology in relation to these concerns.

Curious Nebula described their general aversion to AI use by saying, “At one point I did try it out… like, I know it can make outlines for me… but I haven’t used it really.” In tandem, they expressed concerns about potential environmental harms, saying, “it could be environmentally wasteful if you just use it for shits and giggles.” Curious Nebula’s feelings, which they described as “mixed,” bordered on ambivalence. However, the perspectives on AI expressed in this interview leaned more toward the negative, aside from one hypothetical statement suggesting that “AI could be really helpful when you have a good purpose for it.” Generally, the participants who held both positive and negative perspectives about AI described specific, productive uses of the technology (such as support with structuring or polishing manuscripts) while acknowledging potential concerns (such as academic integrity). Curious use of a technology about which Nebula, however, had limited direct experience with AI, and seemed to have trouble justifying the they wondered, “Is it true that one prompt is a waste of a bottle of water?”

In addition to environmental concerns, Curious Nebula’s interview touched on issues of trustworthiness and credibility. They described how one early experience negatively impacted their trust of AI-generated information, saying, “I think what prevented me from using ChatGPT before was that I asked about something scientific and then when I fact checked it, it wasn’t true.” The interaction with the Virtual Mentor, however, was not used to inquire about specific scientific facts. This conversation explored topics about leveraging strengths and identifying relevant strategies to prevent procrastination. The content ranged from psychological insights about motivation and perfectionism to breaking down large projects into microtasks and using visual progress trackers to support goal completion. Much like Adaptive Finch, who trusted AI more “with broad things” than with questions that require “definitive answers,” Curious Nebula found that the Virtual Mentor felt relatively trustworthy when consulted about general advice. Reflecting on the conversation, they stated, “I just really trusted it. Like 10 out of 10,” while still noting that if they asked “something specific about my research… I’d want to fact check it.”

Curious Nebula also noted feelings of recognition and agency within the chat, pointing out that, “at the end of every response that it gives me, it asks me question, like do you wanna move forward with this or this. And I appreciated that even if I said like, Oh, actually, neither of those.” While several other participants ignored or declined to answer the Virtual Mentor’s questions, Curious Nebula engaged in a more responsive way, selecting their preferred option by answering with simple, direct statements like, “4” or, “2!” and garnering a sense of agency as they directed the flow of the conversation. Rather than experiencing the conversation as one-sided, this participant liked “how it interpreted what I said,” reflecting the feeling that they were being heard in the conversation. Even with the limited context provided by Curious Nebula’s responses, this participant was able to leverage the built-in flexibility of the tool to craft an interaction that felt personalized.