How Do Students Use ChatGPT?
Role: UX Researcher
Project Sponsored by ITHAKA via UC Berkeley
Context & Background
Ithaka, parent company of JSTOR, builds tools to support higher education and the people involved in it, such as students, faculty, researchers, and librarians. JSTOR hosts academic journals, books, and primary sources, largely focused in the humanities fields.
Ithaka would like to analyze developing technologies and their potential uses in higher education: this includes OpenAI’s ChatGPT, the newly released AI-powered language model designed to provide personalized responses to various types of inputs.
Although many academic staff have weighed in publicly on ChatGPT’s potential effects on higher education, little is known about how students actually use this tool. On behalf of Ithaka, this research conducted is an exploratory and generative study to learn more about student use of ChatGPT and AI tools in the context of their studies.
Although many academic staff have weighed in publicly on ChatGPT’s potential effects on higher education, little is known about how students actually use this tool. On behalf of Ithaka, this research conducted is an exploratory and generative study to learn more about student use of ChatGPT and AI tools in the context of their studies.
Objective
Our goal is to understand how and why students use ChatGPT for academic purposes. We also seek to learn the strengths and weaknesses of ChatGPT, and future implications of ChatGPT on learning.
To gain insight into our goals, we asked the following questions:
To gain insight into our goals, we asked the following questions:
1. How do students use ChatGPT?
2. What motivates students to use ChatGPT?
3. What concerns students about ChatGPT?
4. What are the strengths/challenges of using ChatGPT?
5. How does ChatGPT change student learning?
Recruitment & Sampling
We identified two core population segments that we believed we were able to capture.
Primary segment
Humanities and Social Studies undergraduate, graduate and PhD students already using ChatGPT as a learning tool
Secondary Segment
The broader population(not limited to Humanities and social studies) of undergraduate, graduate and PhD students who are already using ChatGPT as a learning tool.
Primary Recruitment strategies
1. Connecting with the department admins
2. Spreading fliers throughout campus
3. email blasts
4. social media outreach (Slack, Reddit, Discord).
2. Spreading fliers throughout campus
3. email blasts
4. social media outreach (Slack, Reddit, Discord).
Through our recruitment screener we selected 15 participants for our interviews and contextual inquiries and surveyed 60 students. These participants came from a breadth of disciplines including STEM, business and pre-health.
Students were incentivized to participate in the screener by being notified that they would receive a $25.00 gift for participating in a 60 minute interview or contextual inquiry if selected. Students were not compensated for completing the anonymous quantitative survey or screener.
Methodology
Semi-Stuctured Interview
We held 9 one-hour semi-structured interview, conducted remotely on Zoom with one interviewer and one notetaker from the research team interfacing with one participant.
Contextual Inquiry
We recruited 5 students to participate in a one-hour exercise where we observed them using ChatGPT in the context of their assignments.
Quantitative Survey
To understand the prevalence of chatGPT among students, We successfully recruited 60 students to complete the survey.
Sorting Through The Data
After an extensive interviewing process, we began processing each of our interviews into categories noting arising themes, commonalities, and items that seemed to "stick out." As we began to chart each interviewee, we began to notice that certain themes spanned across multiple interviews.
We began to combine the insights of our individual interviews into overarching themes, and began to extract and synthesize our key findings.
Our Findings
The greatest motivation of students’ usage of ChatGPT is that it saves time.
This was both evident in our survey data, and our qualitative interviews. Of our survey respondents, all students (100%) who had used ChatGPT reported that they use ChatGPT in order to save time.
ChatGPT seems to have greater usability than Search Engines and University Resources.
“Google cannot understand my problem like ChatGPT.”
Students use ChatGPT to help them with them with writing, understanding concepts, computer programming, and finding sources.
students' trust in chatGPT's quality drops for long form prompts and opinion generation.
Students are finding it difficult to distinguish between when it is appropriate to use chatGPT and when it is not.
Students wanted to strike the balance between “using it versus abusing it” - using the tool to accomplish their goals without compromising on other priorities, both external - academic integrity, their reputation among peers - and internal - feelings of ownership over their work, absorption of material, and skill development.
Moving Forward
New Plagiarism Guidelines
At present, students are actively mediating their own usage of ChatGPT to avoid plagiarism, however with the use of ChatGPT, the line between plagiarism and original work begins to blur and vary from student to student.
Informed Educators
Educational staff, professors and administrators should educate themselves on new technologies like ChatGPT before formulating assumptions.
The more informed our educators are, the better they may be able to help and understand their students.
Contributors:
Lucy Chen
Nereida Navalesca
Mia Schneider-Martin
Dean Rene
Nereida Navalesca
Mia Schneider-Martin
Dean Rene
For more in-depth data and information, please feel free to click through our presentation or read our paper by clicking the buttons below!