Artificial Intelligence

Access the presentation slides

Purpose

"Paper Stacks & AI Hacks: Using Artificial Intelligence to Lighten the Load" presentation features a multi-semester model to show how AI can be used to analyze student progression in a class, make changes to assignments or structure as needed, and then compare student progress over time to make further improvements. Ginny Chaisson and Jessica Lee have partnered together to show how the process of reassessing class content with CLO alignment is an ongoing process towards, which can be made easier with the help of AI.

Jessica is a Technical Librarian who uses AI to further the research process and analyze trends in library usage to evaluate services and library collections. Ginny is an English Instructor who uses AI to help create course content and integrates AI use within assignments. Their mutual usage of AI and focus on data driven improvements led to their collaboration on this presentation.

First Analysis:

Background

Ginny had been co-teaching for a few semester before teaching co-requisite classes on her own. When she had to create her first course on her own, she turned to AI. She enlisted the help of ChatGPT to create course content that aligned with her CLOs, but she found the content to be very plain with large paragraphs of text and bullet points. ChatGPT also made simple charts with terms, relationships, and punctuation, but it was still visually plain. This content was reviewed for accuracy, adjusted to include OER materials, and uploaded into the course modules.

Target Goal

Ginny found that over time in her English I classes, students seemed to have the lowest scores across the conjunction and transition modules leading up to the first draft of their final paper. The conjunction module has 3 assessments, and the transitions module has 1 assessment. Ginny and Jessica worked together to look over the class data and see what changes could be made using the help of AI for analysis.

Class Environment

6 students were randomly selected from each of 4 classes taught by Ginny in one semester. (24 students total for the first analysis.) The 4 classes represented 8 week and 16 week course load. 16 week classes are in-person and 8 week classes are online and in-person. The online and in-person course structure is the same, including content and materials. Online classes are asynchronous where students have deadlines but no set class meeting time.

Target Areas

Ginny provides scores for the randomly selected students who completed the chosen conjunction and transition assessment and the first draft of the final paper. The table also included modality of in-person or online and if the class was 16 weeks or 8 weeks. So a sample student record from an in-person, 8 week class would have scores of 35 out of 60 on the conjunctions assignment, 38 out of 90 on the transition assignment, and 75 out of 100 on the first draft. This sample is also listed in the table below:

Because the students have different modalities and class lengths, Jessica wanted to see what the differences were in students who are:

• In-person versus online for 8 weeks
• 8 week versus 16 week classes for in-person students
• Assignment correlation (success of student over time as one grade feeds into the next)

ChatGPT First Analysis

After plugging the data into ChatGPT, Jessica got the following analysis back:

• 8 week student overall performed best on Draft 1, both in-person and online
• 16 week students scores lowest overall, especially on Draft 1
• Online students have weaker conjunctions scores than in-person students but perform better on Draft 1
• For 16-week students, the skills assignments aren’t translating cleanly into essay quality yet.
• For 8-week students, the accelerated format may push them to connect skills directly into essays, especially conjunctions (in-person) and transitions (online).
• Online students benefit strongly from transition practice, while in-person students benefit strongly from conjunction practice.

Review the entire chatlog with ChatGPT for the first and second analysis.

From Feedback to Context to Prompt

Ginny took this feedback and returned to ChatGPT to get its help with creating a prompt for ChatGPT. Ginny supplied that students represent a variety of the following: 

• class readiness
• confidence levels
• learning needs and styles
• native languages

Ginny then asked ChatGPT to include in the prompt that feedback should reflect changes to clarity, accessibility, and engagement. ChaptGPT returned with the following prompt for itself:

"You are an instructional coach. Critique the following lesson for clarity, accessibility, and engagement. Suggest 3 improvements so it can reach a variety of learners, including ESL students, visual learners, and first-generation college students. Provide specific, practical changes, not general advice.”

ChatGPT Practical Improvements

Ginny uploaded this prompt, the conjunctions and transitions assignments and the coursework initially created by ChatGPT. The 3 practical improvements recommended by ChatGPT were:

• update the reference charts
• simplify terminology
• increase interactive, low-stakes practice

With each suggestion, ChatGPT provided benefits to the students:

• updating the chart to add color coding and include symbols that help convey meaning for terms plus provide examples. ESL and visual learners will see patterns more clearly, and all student have a quick reference guide.
• simplifying terms by including plain language definitions and including a quick tip area for final review. This makes abstract grammar terms more concrete for first-generation and ESL students.
• implementing interactive practice. Builds confidence, lowers anxiety for first-gen students, and adds peer interaction to keep engagement high. (For this recommendation, Ginny added a short practice on transitions using the chatbot feature in SchoolAI.)

Review the entire chatlog with ChatGPT for more details on improvements recommended and suggestions.

Second Analysis:

Between the first and second semester, Ginny applied the changed recommended by ChatGPT to her assignments and content. After the students completed the conjunctions and transitions assignments and first draft, Ginny compiled the scores from the 16 week class to do a second analysis. Jessica took the data and continued the chatlog with ChatGPT from the first analysis, including context that the changed recommended were implemented. The 6 students from the first semester were compared to 6 randomly selected students from the second semester. The conjunctions assignment saw an average increase of 3.84 points from 44.83 to 48.67. The conjunctions assignment saw an average slight decrease of 0.5 points from 71.33 to 70.83. The first draft saw an average increase of 14.67 points from 71.5 to 86.17. This comparison is also listed in the table below:

**We would like to point out that the results here are from a small sample size. We will run the analysis again in the future based on pulling more students from 16 week, in-person classes.

Using the same format as in the first analysis, Jessica asked ChatGPT to make further recommendations based on the reported scores from the second semester. ChatGPT made the following recommendations:

• focus on transitions for improvements (as this area had the lowest student score increase)
• strengthen paragraph-level practice (as transitions are most beneficial connecting sentences within paragraphs and conjunctions are most beneficial connecting clauses within sentences)
• expand interactive, low-stakes practice (the transitions module can benefit from the increase in interactive practice that was included in the conjunctions module)

The Cyclical Process of Assessment

This process is a multi-semester model that shows the cyclical proccess of assessment. While this project was conducted at the semester-to-semester level, it must be noted that reassessment and focus shifting is an on-going everyday process. Intervening within the semester or module or assignment will uphold the most desired results as you are able to make change in real-time. Assessment does not have a start and end place, but Ginny and Jessica recommend jumping in the process when you can. By continuing the journey of realigning your class to your learning objectives, you can ultimately improve the experience for you and your students in your class and lead everyone to success.

Check out the following list of AI resources designed with educators and professionals in mind:

School AI

Schoolai.com offers instructors custom tools for classroom engagement, student content assessment, and content creation based on your curriculum.

AI for Education Prompt Library

Explore AI for Education's prompt library for help figuring out how to ask GenAI bots for help with lesson plans, administrative tasks, rubrics, discussion questions, and more for education.

AI Mood Meter Analyzer - Cause Writer AI

This text analyzer can break down messages into primary and secondary emotions, explain them, then offer areas for improvement based on the desired intent of the message. This resource is great for making sure your message comes across how you intend it to. 

Microsoft Copilot Tutorial

All Fletcher faculty email accounts are subscribed to Microsoft Copilot, an AI imbedded into the Microsoft Apps. Use the one-hour self-paced tutorial above to learn how to use Microsoft Copilot to boost productivity and creativity.