Can computers really grade your IELTS Writing and Speaking?
Knowledge Bank is available in My course
Edit data

E-mail

Phone

Ukraine

Here are suggested study plans for you
to make the most of the course

Self-study plan

PROGRESS

0%

Log out

Can computers really grade your IELTS Writing and Speaking?

Artificial intelligence in IELTS preparation

Artificial Intelligence (AI) touches many parts of our daily life. Chatbots, voice assistants, robots give us extra support when we need it. They are no more a part of sci-fi. Information and communications technology has also changed the way we teach and learn languages. Students can attend classes and access learning materials remotely. This has made education more affordable and flexible. More engaging digital textbooks that contain hyperlinks, interactive presentations, and videos are replacing paper-based textbooks. Digital learning environment also allows students to set their own pace of study and teachers to track students’ progress more efficiently.

In fact, artificial intelligence algorithms make it possible to advance eLearning in every field. How does AI work in English language assessment? Can it really improve and test productive skills such as speaking and writing that we need for IELTS exam and that have always been assessed by humans?

Evaluating writing

Automated scoring technology has already gained global acceptance since Ellis Batten Page argued for the possibility of scoring essays by computer, and in 1968 published his successful work with a program called Project Essay Grade (PEG). Optical mark recognition and bubble-card reader popularized in the early 1970s caused a dramatic shift in testing reading and listening skills through multiple-choice item type. The terms automated essay grading (AEG) and automated essay scoring (AES) were introduced in the 1990s and typically refer to computer scoring of writing in high-stakes tests.

In order to have the writing scored, the test-taker needs to type at a computer. Depending on the purpose of the test, an AES system might need to process the essay before evaluating it. For example, if the candidate has written an essay in capital letters, they will be turned into small letters in order not be perceived as acronyms. In addition, spelling dictionaries will ensure that “organise” and “organize”, for example, are both scored as correct in international tests of English such as IELTS.

There are two typical AES models:

Prompt-specific AES

 

Generic AES
Each prompt or essay title must be administered to test-takers from the target population in order to collect a sample of responses representing the entire range of ability/performance. These responses will be used to develop the scoring model. It make take 100 – 1,000 responses depending on the complexity of the writing task, the rating scales, and the type of modeling techniques. Expert judges assign rubric-based scores to each response in the training set, and the model is optimized to predict these scores.

A unique combination of features is developed to optimally predict human scores for each prompt.

Prompt-specific models can more accurately evaluate such concepts as content of the essay, completion of the task, organization and coherence, appropriacy of register and authorial voice.

These models are trained only once and apply to all essay prompts; they apply the same set of features and feature-weighting to score every prompt on some pre-determined scale. Generic models tend to rely more on surface features such as frequency of grammar errors, punctuation errors, number and location of discourse markers, metrics of sentence complexity, word frequency etc.

In order to build the scoring model, it is necessary to take the three steps. First, it is needed to identify a set of variables relevant to the construct that is going to be assessed. For, example, we can use such variable as lexical density to score a feature called “Vocabulary range and accuracy”. Next, it is necessary to analyse and decompose a large number of training essays into a set of statistics. What does this mean? The number of words in the top 1,000 frequency level, the 2,000 level and so on. Third, use the statistics in models in order to find out how well they predict the expert human scores. After that, the best combination of variables and their weightings are identified.

Take IELTS with confidence

Start your IELTS Grade online course

GET STARTED FOR FREE

According to a recent survey, modern AES systems try to score different dimensions of an essay’s quality in order to provide feedback to users. These dimensions include the following items:

  • Grammaticality: if the writer follows grammar rules
  • Usage: using of prepositions, word usage
  • Mechanics: if the writer follows rules for spelling, punctuation, capitalization
  • Style: how the writer chooses words and varies sentence structures
  • Relevance: how relevant of the content to the prompt
  • Organization: how well the writer structures the essay
  • Development: development of ideas with examples
  • Cohesion: appropriate use of transition phrases
  • Coherence: appropriate transitions between ideas
  • Thesis Clarity: clarity of the thesis
  • Persuasiveness: if the major argument is convincing

Such kind of feedback intends to help the writer identify which aspects of the essay need improvement.

How come it comes under Artificial Intelligence? The time, machine can grade human written essays, which requires some expertise, we can tell that this is Artificial Intelligence. Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it learn for themselves.

The underlying idea of this approach relies on training of binary classifiers to distinguish “good” from “bad” essays and on using the scores to rank essays and assign grades to them.   The process consists of two steps: a training phase, in which the grading rules are acquired using various algorithms, and a testing phase, in which the rules gathered in the first step are used to determine the most probable grade for a particular essay. A larger set of documents enables the acquisition of a larger set of rules during the training phase, thus a higher accuracy in grading. A major part of these techniques, giving training to the systems and later stage, making the systems to learn from new essays or experience is nothing but machine learning.

A growing number of statistical learning methods have been applied to solve the problem of automated text categorization in the last few years, including regression models, nearest neighbor classifiers, Bayes belief networks, decision trees, rule learning algorithms, neural networks and inductive learning systems.

In our IELTS Academic self-study course , we use AI to check and evaluate your reports and essays. You also receive immediate feedback on your writing as you submit it to us. Our automated checking system checks all of your practice tasks as you proceed and gives you instant feedback on the issues you have to improve.

READ ABOUT: Tips for note completion questions in IELTS Listening

Scoring of speaking

While automated systems for evaluation of writing, reading and listening skills have been available for some time, systems that evaluate speaking are only now becoming more common. How do they work? Can they really evaluate competencies we need for spoken communication?

In such test as IELTS, the Speaking section is delivered through face-to-face communication of the examiner and the candidate. Most of Cambridge exams still need two examiners who will test two candidates at the same time. Cambridge English is also exploring the option to deliver direct oral proficiency interviews remotely using video-conferencing technology. However, how can one prepare for such exams and get instant feedback on speaking if the speaking partner is not available? Automated scoring of speaking will help. Such systems can evaluate what was said and how it was said unlike the systems for evaluation and remediation of pronunciation.

There are three models that can make automated scoring possible and accurate together with test and task design.

  • Acoustic model can estimate the likeliest phonemes or words among the number of possibilities. It is the main component of speech recognizer. Such model has to be trained or optimized on a set of speech data including radio podcasts and audio from Youtube. It involves pairing the audio speech with its transcriptions, i.e. sounds are associated with their orthographic representations. Speaker’s accent is also important. That is why it’s better if the speech data matches the target population of the test.
  • Language model comprises the vocabulary that is likely to be used in the answer. Such words are called n-grams in the context of automated writing or speaking scoring. They include the set unigram, e.g. “apple”, bigram “an apple” and trigram “eating an apple” etc. The language model is made of frequencies of such n-grams. For example, if the task is to describe a picture of a girl eating an apple, the unigrams, bigrams and trigrams are likely to appear in candidate’s response quite often. Based on AI logic, a candidate is more likely to belong to a certain proficiency level if they use similar vocabulary to learners at that level.
  • Scoring model refers to the method for selecting features from the speech recognition process and applying them to predict human ratings, i.e. how these features are combined and formulated to predict human ratings.

READ ABOUT: The most important part of IELTS Speaking

Is AI better than a teacher?

It is important to remember that automated scoring technology does not make computers behave like humans. Rather, we can program the machines to identify and quantify some features in speaking and writing, combine them and weight them according to multiple dimensions, and identify which specific features and their weightings best predict the score a human teacher or examiner would provide. Thus, using AI has some benefits.

Considering your needs. Thanks to using an AI for learning languages and exam preparation, we can take into consideration the needs of a bigger number of IELTS candidates. Our AI-powered language learning platform allows you to work at your own pace, repeating topics and emphasizing things you need to improve, engaging you with the tasks you’re best at. Data also allows our IELTS teachers to understand what is going on in the minds of our students and predict their future performance.

Providing instant feedback. With AI exam preparation, feedback comes immediately. This is quite beneficial because you do not need to wait for a few days for the results. It allows you to see the mistakes you made, and you will remember how and realize why you made them. Our platform evaluates essays and reports automatically right after you have turned them in, pointing out errors and suggesting ways to avoid them in the future. This allows you to correct your mistakes, improve your writing shortly and do better in future tests. This also allows our IELTS teachers to see if you need additional guidance.

No fears. When preparing for IELTS or any other exam, most students are afraid of failing it. When failing, we often feel ashamed and demotivated. AI doesn’t reprimand or criticize learners, it evaluates you without judging. When you get your feedback, you know what and how to work on. Additionally, knowing your drawbacks, you can have a Skype session with the IELTS tutor. However, most of the routines are done by the platform.

Teachers become guides and advisers. Our IELTS teachers have more time for analyzing and using the data gained from your learning process to improve the course and the platform, coordinate your learning process and give you more support as all of the grading and the paperwork are done by the AI.

Flexibility. Thanks to using AI on our platform, you are able to study and prepare for your IELTS from any place in the world at your own pace, set your own goals, and follow a study-plan. We offer a study plan to organise and guide your IELTS Academic preparation process.

We believe that the fast and powerful tool such as AI combined with the human judgment of examiners, annotators, assessment experts, data analysts and teachers will reduce costs, save your time, improve efficiency and give you a faster turnaround for your desired IELTS results.

Subscribe to our IELTS digest

READ ABOUT: How to check your IELTS writing

Download your Academic Writing Guide on how to use linking words

Your writing guide is in your inbox!
Get studying now :)

By using the site, I confirm that I have read and agree to the terms of the user license agreement Privacy Policy, Public Offer, Terms of Use.

Agree

×