Computational Linguists Help Africa Try to Close the AI Language Gap

Introduction

The fact that African languages are underrepresented in the digital AI ecosystem has gained international attention. On July 29, 2025, Nature published a news article stating that

“More than 2,000 languages spoken in Africa are being neglected in the artificial intelligence (AI) era. For example, ChatGPT recognizes only 10–20% of sentences written in Hausa, a language spoken by 94 million people in Nigeria. These languages are under-represented in large language models (LLMs) because of a lack of training data.” (source: AI models are neglecting African languages — scientists want to change that)

Another example is BBC News, released on September 4, 2025, stating that

“Although Africa is home to a huge proportion of the world’s languages – well over a quarter according to some estimates – many are missing when it comes to the development of artificial intelligence (AI). This is both an issue of a lack of investment and readily available data. Most AI tools, such as ChatGPT, used today are trained on English as well as other European and Chinese languages. These have vast quantities of online text to draw from. But as many African languages are mostly spoken rather than written down, there is a lack of text to train AI on to make it useful for speakers of those languages. For millions across the continent this means being left out.” (source: Lost in translation – How Africa is trying to close the AI language gap)

To address this problem, linguists and computer scientists are collaborating to create AI-ready datasets in 18 African languages via The African Next Voices project. Funded by the Bill and Melinda Gates Foundation ($2.2-million grant), the project involves recording 9,000 hours of speech across 18 African languages in Kenya, Nigeria, and South Africa. The goal is to create a comprehensive dataset that can be utilized for developing AI tools, such as translation and transcription services, which are particularly beneficial for local communities and their specific needs. The project emphasizes the importance of capturing everyday language use to ensure that AI technologies reflect the realities of African societies. The 18 African languages selected represent only a fraction of the over 2,000 languages spoken across the continent, but project contributors aim to include more languages in the future.

Role of Computational Linguists in the Project

Computational linguists play a critical role in the African Next Voices project. Their key contributions include:

Data Curation and Annotation: They guide the transcription and translation of over 9,000 hours of recorded speech in languages like Kikuyu, Dholuo, Hausa, Yoruba, and isiZulu, ensuring linguistic accuracy and cultural relevance. This involves working with native speakers to capture authentic, everyday language use in contexts like farming, healthcare, and education.
Dataset Design: They help design structured datasets that are AI-ready, aligning the collected speech data with formats suitable for training large language models (LLMs) for tasks like speech recognition and translation. This includes ensuring data quality through review and validation processes.
Bias Mitigation: By leveraging their expertise in linguistic diversity, computational linguists work to prevent biases in AI models by curating datasets that reflect the true linguistic and cultural nuances of African languages, which are often oral and underrepresented in digital text.
Collaboration with Technical Teams: They work alongside computer scientists and AI experts to integrate linguistic knowledge into model training and evaluation, ensuring the datasets support accurate translation, transcription, and conversational AI applications.

Their involvement is essential to making African languages accessible in AI technologies, fostering digital inclusion, and preserving cultural heritage.

Final Thoughts

From the perspective of a U.S. high school student interested in pursuing computational linguistics in college, inspired by African Next Voices, here are some final thoughts and conclusions:

Impactful Career Path: Computational linguistics offers a unique opportunity to blend language, culture, and technology. For a student like me, the African Next Voices project highlights how this field can drive social good by preserving underrepresented languages and enabling AI to serve diverse communities, which could be deeply motivating.
Global Relevance: The project underscores the global demand for linguistic diversity in AI. As a future computational linguist, I can contribute to bridging digital divides, making technology accessible to millions in Africa and beyond, which is both a technical and humanitarian pursuit.
Skill Development: The work involves collaboration with native speakers, data annotation, and AI model training/evaluation, suggesting I’ll need strong skills in linguistics, programming (e.g., Python), and cross-cultural communication. Strengthening linguistics knowledge and enhancing coding skills could give me a head start.
Challenges and Opportunities: The vast linguistic diversity (over 2,000 African languages) presents challenges like handling oral traditions or limited digital resources. This complexity is exciting, as it offers a chance to innovate in dataset creation and bias mitigation, areas where I could contribute and grow.
Inspiration for Study: The focus on real-world applications (such as healthcare, education, and farming) aligns with my interest in studying computational linguistics in college and working on inclusive AI that serves people.

In short, as a high school student, I can see computational linguistics as a field where I can build tools that help people communicate and learn. I hope this post encourages you to look into the project and consider how you might contribute to similar initiatives in the future!

— Andrew

4,811 hits

September 16, 2025 0

How NLP Helps Robots Handle Interruptions: A Summary of JHU Research

I recently came across an awesome study from Johns Hopkins University describing how computational linguistics and NLP can make robots better conversational partners by teaching them how to handle interruptions, a feature that feels basic for humans but is surprisingly hard for machines.

What the Study Found

Researchers trained a social robot powered by a large language model (LLM) to manage real-time interruptions based on speaker intent. They categorized interruptions into four types: Agreement, Assistance, Clarification, and Disruption.

By analyzing human conversations from interviews to informal discussions, they designed strategies tailored to each interruption type. For example:

If someone agrees or helps, the robot pauses, nods, and resumes speaking.
When someone asks for clarification, the robot explains and continues.
For disruptive interruptions, the robot can either hold the floor to summarize its remaining points before yielding to the human user, or it can stop talking immediately.

How NLP Powers This System

The robot uses an LLM to:

Detect overlapping speech
Classify the interrupter’s intent
Select the appropriate response strategy

In tests involving tasks and conversations, the system correctly interpreted interruptions about 89% of the time and responded appropriately 93.7% of the time.

Why This Matters in NLP and Computational Linguistics

This work highlights how computational linguistics and NLP are essential to human-robot interaction.

NLP does more than generate responses; it helps robots understand nuance, context, and intent.
Developing systems like this requires understanding pause cues, intonation, and conversational flow, all core to computational linguistics.
It shows how multimodal AI, combining language with behavior, can enable more natural and effective interactions.

What I Found Most Interesting

The researchers noted that users didn’t like when the robot “held the floor” too long during disruptive interruptions. It reminded me how pragmatic context matters. Just like people expect some rules in human conversations, robots need these conversational skills too.

Looking Ahead

This research expands what NLP can do in real-world settings like healthcare, education, and social assistants. For someone like me who loves robots and language, it shows how computational linguistics helps build smarter, more human-friendly AI systems.

If you want to dive deeper, check out the full report from Johns Hopkins:
Talking robots learn to manage human interruptions

— Andrew

August 7, 2025 0

Shared Task at DravidianLangTech 2025

In 2025, I had the privilege of participating in the shared task on Sentiment Analysis in Tamil and Tulu as part of the DravidianLangTech@NAACL 2025 conference. The task was both challenging and enlightening, as it required applying machine learning techniques to multilingual data with varying sentiment nuances. This post highlights the work I did, the methodology I followed, and the results I achieved.

The Task at Hand

The goal of the task was to classify text into one of four sentiment categories: Positive, Negative, Mixed Feelings, and Unknown State. The datasets provided were in Tamil and Tulu, which made it a fascinating opportunity to work with underrepresented languages.

Methodology

I implemented a pipeline to preprocess the data, tokenize it, train a transformer-based model, and evaluate its performance. My choice of model was XLM-RoBERTa, a multilingual transformer capable of handling text from various languages effectively. Below is a concise breakdown of my approach:

Data Loading and Inspection:
- Used training, validation, and test datasets in .xlsx format.
- Inspected the data for missing values and label distributions.
Text Cleaning:
- Created a custom function to clean text by removing unwanted characters, punctuation, and emojis.
- Removed common stopwords to focus on meaningful content.
Tokenization:
- Tokenized the cleaned text using the pre-trained XLM-RoBERTa tokenizer with a maximum sequence length of 128.
Model Setup:
- Leveraged XLM-RoBERTaForSequenceClassification with 4 output labels.
- Configured TrainingArguments to train for 3 epochs with evaluation at the end of each epoch.
Evaluation:
- Evaluated the model on the validation set, achieving a Validation Accuracy of 59.12%.
Saved Model:
- Saved the trained model and tokenizer for reuse.

Results

After training the model for three epochs, the validation accuracy was 59.12%. While there is room for improvement, this score demonstrates the model’s capability to handle complex sentiment nuances in low-resource languages like Tamil.

The Code

Below is an overview of the steps in the code:

Preprocessing: Cleaned and tokenized the text to prepare it for model input.
Model Training: Used Hugging Face’s Trainer API to simplify the training process.
Evaluation: Compared predictions against ground truth to compute accuracy.

To make this process more accessible, I’ve attached the complete code as a downloadable file. However, for a quick overview, here’s a snippet from the code that demonstrates how the text was tokenized:

# Tokenize text data using the XLM-RoBERTa tokenizer
def tokenize_text(data, tokenizer, max_length=128):
    return tokenizer(
        data,
        truncation=True,
        padding='max_length',
        max_length=max_length,
        return_tensors="pt"
    )

train_tokenized = tokenize_text(train['cleaned'].tolist(), tokenizer)
val_tokenized = tokenize_text(val['cleaned'].tolist(), tokenizer)

This function ensures the input text is prepared correctly for the transformer model.

Reflections

Participating in this shared task was a rewarding experience. It highlighted the complexities of working with low-resource languages and the potential of transformers in tackling these challenges. Although the accuracy could be improved with hyperparameter tuning and advanced preprocessing, the results are a promising step forward.

Download the Code

I’ve attached the full code used for this shared task. Feel free to download it and explore the implementation in detail.

XLM-RoBERTa Model And Eval Download

If you’re interested in multilingual NLP or sentiment analysis, I’d love to hear your thoughts or suggestions on improving this approach! Leave a comment below or connect with me via the blog.

January 26, 2025 0

Exploring the Intersection of AI and Human Creativity: A Review of Deep Thinking by Garry Kasparov

Recently, I had the opportunity to read Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins by Garry Kasparov. While this book doesn’t directly tie into my work in computational linguistics, it still resonated with me due to its exploration of artificial intelligence (AI), a field closely related to many of my interests. The book combines my passions for chess and technology, and while its primary focus is on AI in the realm of chess, it touches on broader themes that align with my curiosity about how AI and human creativity intersect.

In Deep Thinking, the legendary chess grandmaster Garry Kasparov delves into his personal journey with artificial intelligence, particularly focusing on his famous matches against the machine Deep Blue. This book is not just a chronicle of those historic encounters; it’s an exploration of how AI impacts human creativity, decision-making, and the psychological experience of competition.

Kasparov’s narrative offers more than just an inside look at high-level chess; it provides an insightful commentary on the evolving relationship between humans and technology. Deep Thinking is a must-read for those interested in the intersection of AI and human ingenuity, especially for chess enthusiasts who want to understand the psychological and emotional impacts of playing against a machine.

Kasparov’s main argument is clear: While AI has transformed chess, it still cannot replicate the creativity, reasoning, and emotional depth that humans bring to the game. AI can calculate moves and offer solutions, but it lacks the underlying rationale and context that makes human play unique. As Kasparov reflects, even the most advanced chess programs can’t explain why a move is brilliant—they just make it. This inability to reason and articulate is a crucial distinction he highlights throughout the book, particularly in Chapter 4, where he emphasizes that AI lacks the emotional engagement that a human player experiences.

For Kasparov, the real challenge comes not just from the machine’s power but from its lack of emotional depth. In Chapter 5, he shares how the experience of being crushed by an AI, which feels no satisfaction or fear, is difficult to process emotionally. It’s this emotional disconnect that underscores the difference between the human and machine experience, not only in chess but in any form of creative endeavor. The machine may be able to play at the highest level, but it doesn’t feel the game the way humans do.

Kasparov’s exploration of AI in chess is enriched by his experiences with earlier machines like Deep Thought, where he learns that “a machine learning system is only as good as its data.” This idea touches on a broader theme in the book: the idea that AI is limited by the input it receives. The system is as powerful as the information it processes, but it can never go beyond that data to create something entirely new or outside the parameters defined for it.

By the book’s conclusion, Kasparov pivots to a broader, more philosophical discussion: Can AI make us more human? He argues that technology, when used properly, has the potential to free us from mundane tasks, allowing us to be more creative. It is a hopeful perspective, envisioning a future where humans and machines collaborate rather than compete.

However, Deep Thinking does have its weaknesses. The book’s technical nature and reliance on chess-specific terminology may alienate readers unfamiliar with the game or the intricacies of AI. Kasparov makes an effort to explain these concepts, but his heavy use of jargon can make it difficult for casual readers to fully engage with the material. Additionally, while his critique of AI is compelling, it sometimes feels one-sided, focusing mainly on AI’s limitations without fully exploring how it can complement human creativity.

Despite these drawbacks, Deep Thinking remains a fascinating and thought-provoking read for those passionate about chess, AI, and the future of human creativity. Kasparov’s firsthand insights into the psychological toll of competing against a machine and his reflections on the evolving role of AI in both chess and society make this book a significant contribution to the ongoing conversation about technology and humanity.

In conclusion, Deep Thinking is a compelling exploration of AI’s role in chess and human creativity. While it may be a challenging read for those new to the fields of chess or AI, it offers invaluable insights for those looking to explore the intersection of technology and human potential. If you’re a chess enthusiast, an AI aficionado, or simply curious about how machines and humans can co-evolve creatively, Kasparov’s book is a must-read.

December 14, 2024 0

I am back!

This will be a short post since I’m planning to post a more in-depth discussion on one thing that I’ve been up to over the summer. Between writing a research paper (currently under review by the Journal of High School Science) and founding a nonprofit called Student Echo, I’ve been keeping myself busy. Despite all this, I plan to post shorter updates more frequently here. Sorry for the wait—assuming anyone was actually waiting—but hey, here you go.

Here’s a bit more about what’s been keeping me occupied:
My Research Paper
Title: Comparing Performance of LLMs vs. Dedicated Neural Networks in Analyzing the Sentiment of Survey Responses
Abstract: Interpreting sentiment in open-ended survey data is a challenging but crucial task in the age of digital information. This paper studies the capabilities of three LLMs, Gemini-1.5-Flash, Llama-3-70B, and GPT-4o, comparing them to dedicated, sentiment analysis neural networks, namely RoBERTa-base-sentiment and DeBERTa-v3-base-absa. These models were evaluated on their accuracy along with other metrics (precision, recall, and F1-score) in determining the underlying sentiment of responses from two COVID-19 surveys. The results revealed that despite being designed for broader applications, all three LLMs generally outperformed specialized neural networks, with the caveat that RoBERTa was the most precise at detecting negative sentiment. While LLMs are more resource-intensive than dedicated neural networks, their enhanced accuracy demonstrates their evolving potential and justifies the increased resource costs in sentiment analysis.

My Nonprofit: Student Echo
Website: https://www.student-echo.org/
Student-Echo.org is a student-led non-profit organization with the mission of amplifying students’ voices through student-designed questionnaires, AI-based technology, and close collaboration among students, teachers, and school district educators.

November 28, 2024 0