The educational landscape is undergoing a seismic shift with rapid advancements in artificial intelligence (AI) and machine learning technologies. Throughout history, some educators have met technological innovations with skepticism and resistance before eventually embracing their benefits, potentially stunting their students’ growth. For instance, calculators in the 1960s and 1970s initially sparked concerns about the decline of students’ arithmetic skills. As late as the early 2000s the expression “You won’t be walking around with a calculator in your pocket!” was (oh so wrongly) uttered by many an intransigent math teacher. Similarly, the rise of the internet in the 1990s led to apprehensions about superficial learning, the credibility of online sources, and increased plagiarism. However, as these technologies evolved and proved their value, they have become integral to enhancing education.
That utility hasn’t come about haphazardly. It’s the result of educators seeing the value (and drawbacks!) of these tools and figuring out the best ways to implement them in the educational landscape. Calculators haven’t replaced math teachers and the internet hasn’t replaced the need for learning information literacy. By understanding and embracing the potential of AI tools, educators can adapt their teaching methods to capitalize on the benefits of these tools, rather than futilely resisting change. Likewise, understanding the blindspots and failings of AI tools is key to using them effectively.
AI tools will change how students learn. If ChatGPT can earn a B- (or higher!) on written work assigned to students, perhaps that assignment isn’t worthy of anyone’s time, student or teacher. In this blog post, we want to share the results of our experiments with AI. We’ll show you where it can shine, where it struggles, and share some best practice tips to make you more effective.
Note on Methodology and Technology: AI tools are changing and improving day by day. The results of our tests and examples are from May of 2023 using Open AI’s ChatGPT.
Takeaway 1: Length Counts
When asking ChatGPT to help provide feedback, it is important to provide a length count or specify the level of detail required. Without clear instructions, ChatGPT tends to generate excessively detailed responses. For example, when asking for feedback on the introductory paragraph of an essay, it would respond with five paragraphs worth of feedback. We’ve found giving it a suggested maximum word count to be helpful. It does much better with hard numbers (100 words) vs. concepts (a paragraph) as asking for “a paragraph” was often ignored by the AI.
Takeaway 2: Tell the AI Who It Is
To elicit more relevant and context-specific feedback, it is beneficial to give ChatGPT a frame of mind to reference. By setting a specific scenario or perspective, such as “imagine you are a high school English teacher,” you enable ChatGPT to provide feedback through the lens of that particular role. This helps align the generated responses with your expectations and educational context. Our tests found that ChatGPT tends to default to a “newspaper critic” voice when asked to provide feedback or constructive criticism. Rather than engaging with the student, it communicates like an impersonal observer.
Notice the different results from these two prompts.
Prompt: Analyze the following introduction to a paper and give constructive criticism and praise in a single paragraph:
Result: The introduction presents a clear narrative about the importance of advertising and market segmentation in successfully launching a product in the British market, which is commendable. However, improvements could be made in several areas. The writing could be more concise and avoid redundancy; phrases like “we must advertise” and “we must concentrate on a good ad campaign” essentially communicate the same idea…
Prompt: Imagine you are a high school English teacher. A student named Joseph is writing an essay about advertising. Give him personalized feedback in 100 words or less about the introduction to his essay. It should be a mix of positive and constructive criticism. The tone should be professional but personable.
Result: Dear Joseph, Great start on your essay discussing various aspects of advertising. Focus on a clear thesis statement, and consider breaking your introduction into several paragraphs for clarity. Watch for grammar errors and work on transitions between sentences for smooth flow. Remember to include the “approximate prices” if it’s pertinent to your argument. Keep refining; you’re on the right track.
Takeaway 3: Tell it Who Is Receiving its Feedback
In addition to giving it a reference of the voice it should be writing from, it’s helpful to tell the AI who it is writing to. While ChatGPT is capable of analyzing and providing analysis by default, explicitly instructing it to offer personalized feedback enhances its effectiveness.
Example prompt: “In less than 100 words, craft a response as if you were this student’s teacher addressing him directly. It should contain constructive criticism, with praise before and after the criticism.”
Result: I want to start by applauding your in-depth analysis of advertising strategies for Pakanawa in the British market. It’s evident that you’ve put considerable thought into the importance of advertising to raise product awareness and capture the attention of specific consumer segments. I do have a suggestion to enhance your work: consider organizing your ideas into separate paragraphs to improve readability. Additionally, backing up your arguments with concrete evidence and examples would strengthen your points. Keep up the great work, and don’t forget to review and polish your writing for clarity.”
Takeaway 4: GPT-4 Is Better At Feedback
As of writing, GPT-4 is the latest model available in public beta from Open AI. There are pros and cons to the two models currently available. For example, GPT-3.5 operates much faster and doesn’t have a message limit (25 messages per 3 hours for GPT-4). However, our testing found that GPT-4 did better at crafting feedback that seemed genuine and personalized.
As a case in point, we asked both models to provide feedback to a student about a book review essay. This was the prompt: “Imagine you are a high school English teacher providing feedback on an essay. In 200 words or less, provide constructive praise and suggestions for improvement. Here is the text of the essay:”
GPT-3.5
Praise:
The essay provides a thorough analysis of the theme of hierarchy versus friendship in “The Kite Runner.” The use of relevant quotes from the novel demonstrates a good understanding of the text. The connection between the social sense of hierarchy and racial discrimination is well-explained and adds depth to the discussion. The essay effectively highlights the struggles and conflicts faced by the characters, particularly Amir, in navigating their relationships within the hierarchical society. The comparison between Amir’s relationship with Hassan and Baba’s relationship with Ali is insightful and supports the argument.
Suggestions for improvement:
To enhance the essay, consider organizing the ideas into paragraphs for better readability and clarity. Each paragraph could focus on a specific aspect of the topic and provide supporting evidence and analysis. Additionally, it would be helpful to provide more context and background information about the novel, including the author’s name and the publication year, in the introductory paragraph. This would give readers a better understanding of the text being analyzed. Lastly, consider expanding on the analysis of specific scenes or events to provide more depth and engage the reader further.
GPT-4
Dear John,
Your essay provides a thorough analysis of the complex relationship between Amir and Hassan in “The Kite Runner” and how it reflects the socio-cultural dynamics in Afghanistan. You’ve done an excellent job of using direct quotations from the text to support your arguments.
For future work, consider refining your thesis statement for clarity. It seems like you are suggesting that Amir’s actions should not be considered wrong given the societal context, but this argument isn’t fully developed. You might elaborate on why society’s norms might excuse his behavior, while also considering opposing views.
The structure of the essay could also benefit from a more focused approach. Instead of attempting to cover all instances of hierarchy and discrimination, perhaps pick a few key examples and delve into them more deeply, analyzing the implications for the characters and their society.
Lastly, remember to proofread carefully. While your essay is generally well-written, there are minor grammatical errors and awkward sentence structures that detract from the overall flow of your analysis.
Keep up the good work!
Takeaway 5: Plugins and Web-browsing – Limited Uses But Not There Yet
By now, Open AI has rolled out access to beta features to all paid users of ChatGPT. There are two optional modes that can be enabled with the GPT-4 model to take advantage of this. One allows ChatGPT to browse the web to find references, read content, etc. The other enabled ChatGPT to interact with services that extend its feature set.
On the whole, these features still have a ways to go to be useful and reliable. For example, the AskYourPDF plugin allows ChatGPT to work with data right from a PDF document. (It also supports other formats like Word). Ideally, this would save teachers time checking for routine issues like compliance with formatting guidelines. However, the results when asking it to do that were mixed. In one test it provided seven observations, of which only four were usable. And on another, it struggled to get anything correct and seemed to have issues reading the PDF (even though it was the same PDF it had previously done an ok job on.) While AI can’t handle this task automatically, Annotate PRO’s College Edition Library has pre-written comments with references to help streamline giving feedback on MLA, APA, and CMS style guidelines.
Likewise, web browsing still seems to have some major flaws. When asked to browse specific websites it responds with an error more times than not. At times the results were downright comical. We asked it to provide some helpful references from our free and ad-free 11guide as feedback on a student essay. The essay was about the movie The Last King of Scotland. The prompt: Using information from 11trees.com/guide provide 3 suggestions for articles to help the author of this paper:
We got this comical result (it was significantly longer but you get the idea). No mention of trees or lemons are found anywhere in the essay.
Result: Here are some tips for growing a lemon tree in your climate:
- Select the right variety: In USDA zone 11, many citrus trees can grow, including lemon trees. The Improved Meyer Lemon is a popular variety for your zone due to its ability to withstand the heat and its prolific fruit production.
A Use for Web-Browsing and AP
We’re sure these tools will improve over time. They’ve just recently been released to the wider public for testing. However, as of this writing, we’ve not found many effective uses for them. One use we did find is for digesting current events and turning them into a discussion prompt. We pointed it to NPR’s headline article of the day, which was discussing the subject of banning books in school libraries. We fed ChatGPT the following prompt and a link to the article:
Prompt: Imagine you are a high school history teacher who wants students to discuss current events in a historical context. Create three thought-provoking discussion prompt questions that students can choose from. The questions should center on personal experiences and viewpoints. It should be based on ideas from this article: https://www.npr.org/2023/05/23/1177636289/georgia-school-book-bans-may-hostile-environment
Results:
- Censorship and Discrimination: Reflecting on the Department of Education’s findings that the Georgia school district’s book screening process may have created a hostile environment for some students, do you believe that the act of banning books could be viewed as a form of discrimination? Compare this situation with historical instances of censorship and discrimination. How do your personal beliefs align or conflict with these historical examples?
- Role of School Administrators in Content Monitoring: The school district’s media committee rejected a proposal to allow parents to grant or deny permission for their child to read books with sexual or LGBTQ+ content, arguing that students would find ways around the system and librarians would be forced into the role of gatekeepers. What is your personal stance on this issue? Should parents have the final say on what their children can read in school, or should this responsibility fall to educators? Can you draw any parallels between your views and historical debates about the role of school administrators in monitoring content?
- Public Opinion and Educational Policy: Given the diversity of public opinions on the topic of book bans in schools as observed in the public comments at the Forsyth board meeting, where do you stand on this issue? Do you agree or disagree with the viewpoints expressed at the meeting? Reflect on how public opinions have shaped educational policies in the past and compare this with your own opinion on the matter. Are there any historical events or movements that have influenced your viewpoint?
We think this is a great way to use AI tools to more efficiently allocate a teacher’s limited time. The time that would’ve been spent on crafting these questions can now be spent on guiding the discussion with students, helping them learn to think critically and clearly express themselves. Personalizing the discussion questions to relate them to their own experiences and opinions rather than purely factual analysis makes it harder for students to have ChatGPT craft responses for them. Rather than trying to ban or catch students using new tools (which will never work) find ways to create assignments that require them to express themselves rather than rehashing information. 11trees also offers a free tool to help with this, our Discussion Moderator Edition Library which is designed to help teachers foster higher-quality discussions in their classes.
Conclusion
AI is not going to replace the role of the teacher. But teachers will need to adapt to changing technologies or risk hindering their students’ progress. Using tools like ChatGPT is easy and effective when done right. And doing so frees up teachers from monotonous, routine tasks to spend more time fostering an environment that will allow students to flourish. Our motto at 11trees is “Feedback is a teacher’s superpower.” Much like using Annotate PRO helps teachers be more efficient and effective at providing feedback, AI tools are another arrow in the quiver to make the feedback process more potent. In a future article, we will cover ways to harness the power of AP directly with tools like ChatGPT.