Tag: llm

The AI Gap: How Socioeconomic Status Shapes Language Technology Use — A Perspective from Best Social Impact Paper at ACL 2025

The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025) recently finished in Vienna, Austria from July 27 to August 1. The conference announced a few awards, one of which is Best Social Impact Paper. This award was given to two papers:

  1. AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset (by Charles Nimo et al.)
  2. The AI Gap: How Socioeconomic Status Affects Language Technology Interactions (by Elisa Bassignana, Amanda Cercas Curry, and Dirk Hovy).

In this blog post, I’ll talk about the second paper and share the findings from the paper and my thoughts on the topic. You can read the full paper here: https://aclanthology.org/2025.acl-long.914.pdf

What the Paper is About

This paper investigates how socioeconomic status (SES) influences interactions with language technologies, particularly large language models (LLMs) like ChatGPT, highlighting an emerging “AI Gap” that could exacerbate social inequalities. Drawing from the Technology Acceptance Model and prior work on digital divides, the authors argue that SES shapes technology adoption through factors like access, digital literacy, and linguistic habits, potentially biasing LLMs toward higher-SES patterns and underrepresenting lower-SES users.

Methods

The study surveys 1,000 English-speaking participants from the UK and US via Prolific, stratified by self-reported SES using the MacArthur scale (binned as low: 1-3, middle: 4-7, upper: 8-10). It collects sociodemographic data, usage patterns of language technologies (e.g., spell checkers, AI chatbots), and 6,482 real prompts from prior LLM interactions. Analysis includes statistical tests (e.g., chi-square for usage differences), linguistic metrics (e.g., prompt length, concreteness via Brysbaert et al.’s word ratings), topic modeling (using embeddings, UMAP, HDBSCAN, and GPT-4 for cluster descriptions), and markers of anthropomorphism (e.g., phatic expressions like “hi” and politeness markers like “thank you”).

Key Findings

  • Usage Patterns: Higher-SES individuals access more devices daily (e.g., laptops, smartwatches) and use LLMs more frequently (e.g., daily vs. rarely for lower SES). They employ LLMs for work/education (e.g., coding, data analysis, writing) and technical contexts, while lower-SES users favor entertainment, brainstorming, and general knowledge queries. Statistically significant differences exist in frequency (p < 0.001), contexts (p < 0.001), and tasks (p < 0.001).
  • Linguistic Differences in Prompts: Higher-SES prompts are shorter (avg. 18.4 words vs. 27.0 for low SES; p < 0.05) and more abstract (concreteness score: 2.57 vs. 2.66; p < 0.05). Lower-SES prompts show higher anthropomorphism (e.g., more phatic expressions) and concrete language. A bag-of-words classifier distinguishes SES groups (Macro-F1 39.25 vs. baseline 25.02).
  • Topics and Framing: Common topics (e.g., translation, mental health, medical advice, writing, text editing, finance, job, food) appear across groups, but framing varies—e.g., lower SES seeks debt reduction or low-skill jobs; higher SES focuses on investments, travel itineraries, or inclusivity. About 45% of prompts resemble search-engine queries, suggesting LLMs are replacing traditional searches.
  • User Perceptions: Trends indicate lower-SES users anthropomorphize more (e.g., metaphorical verbs like “ask”), while higher-SES use jargon (e.g., “generate”), though not statistically significant.

Discussion and Implications

The findings underscore how SES stratifies LLM use, with higher-SES benefiting more in professional/educational contexts, potentially widening inequalities as LLMs optimize for their patterns. Benchmarks may overlook lower-SES styles, leading to biases. The authors advocate the development of inclusive NLP technologies to accommodate different SES needs and habitus and mitigate the existing AI Gap.

Limitations and Ethics

Limited to Prolific crowdworkers (skewed middle/low SES, tech-savvy), subjective SES measures, and potential LLM-generated responses. Ethical compliance includes GDPR anonymity, opt-outs, and fair compensation (£9/hour).

Overall, the paper reveals SES-driven disparities in technology interactions, urging NLP development to address linguistic and habitual differences for equitable access and reduced digital divides.

My Takeaway

As a high school student who spends a lot of time thinking about fairness in AI, I find this paper important because it reminds us that bias is not just about language or culture, it can also be tied to socioeconomic status. This is something I had not thought much about before. If AI systems are trained mostly on data from higher SES groups, they might misunderstand or underperform for people from lower SES backgrounds. That could affect how well people can use AI for education, job searching, or even just getting accurate information online.

For me, the takeaway is that AI researchers need to test their models with SES diversity in mind, just like they do with gender or language diversity. And as someone interested in computational linguistics, it is inspiring to see that work like this is getting recognized with awards at ACL.

— Andrew

1

Speeding Up AI for Everyone: The PaPaformer Model Making Language Tech Work on Phones and Low-Power Devices

AI has become more capable than ever, but many of the most advanced tools still require massive cloud servers to run. That means if you want ChatGPT-level performance, you usually need a reliable internet connection and a lot of computing power behind the scenes. But what if you could have that kind of AI right on your phone, even without Wi‑Fi?

That’s where the PaPaformer model comes in.

What is the PaPaformer Model?
PaPaformer is a new AI architecture developed to train large language models more efficiently and make them small enough to run smoothly on low-power devices like smartphones, tablets, or even embedded systems. You can read more about it in the original paper here: PaPaformer: Language Model from Pre-trained Parallel Paths.

Unlike most large models today that require powerful cloud servers to process requests, PaPaformer is designed so the model can be stored and run directly on your device. This means you can use advanced language technology without a constant internet connection. It also helps protect privacy, since your data stays local instead of being sent to the cloud for processing.

Why It Matters
By making AI lighter and more portable, PaPaformer could bring powerful language tools to more people around the world, including those with limited internet access or older devices. It could also make AI faster to respond, since it does not have to constantly send data back and forth to the cloud.

Examples in Action
Imagine using ChatGPT-style features on a budget smartphone in a remote area. With most current apps, like the regular ChatGPT app, you still need a strong internet connection because the AI runs on servers, not your device. But with a PaPaformer-powered tool, the AI would actually run locally, meaning you could:

  • Translate between languages instantly, even without Wi‑Fi
  • Use a speech-to-text tool for endangered languages that works entirely on your device
  • Let teachers translate lessons in real time for students in rural schools without relying on an internet connection
  • Help students write essays in multiple languages privately, without sending drafts to a remote server

This offline capability is the big difference. It is not just accessing AI through the cloud, it is carrying the AI with you wherever you go.

Looking Ahead
If PaPaformer and similar approaches keep improving, we could see a future where advanced AI is available to anyone, anywhere, without needing expensive devices or constant internet access. For someone like me, interested in computational linguistics, this could also open up new possibilities for preserving languages, creating translation tools, and making language technology more inclusive worldwide.

— Andrew

0

Is the Increasing Trend of Leveraging LLMs like ChatGPT in Writing Research Papers Concerning?

On August 4, 2025, Science published a tech news piece titled “One-fifth of computer science papers may include AI content,” written by Phie Jacobs, a general assignment reporter at Science. The article reports on a large-scale analysis conducted by researchers at Stanford University and the University of California, Santa Barbara. They examined over 1 million abstracts and introductions and found that by September 2024, 22.5% of computer science papers showed signs of input from large language models such as ChatGPT. The researchers used statistical modeling to detect common word patterns linked to AI-generated writing.

This caught my attention because I was surprised at how common AI-generated content has already become in academic research. I agree with the concern raised in the article, particularly this point:

Although the new study primarily looked at abstracts and introductions, Dmitry Kobak (University of Tübingen data scientist) worries authors will increasingly rely on AI to write sections of scientific papers that reference related works. That could eventually cause these sections to become more similar to one another and create a “vicious cycle” in the future, in which new LLMs are trained on content generated by other LLMs.

From my own experience writing research papers over the past few years, I can see why this concern is valid. If you have followed my blog, you know I have published two research papers and am currently working on a third. While working on my papers, I occasionally used ChatGPT (including its Deep Research) to help find peer-reviewed sources for citations instead of relying solely on search engines like Google Scholar. However, I quickly realized that depending on ChatGPT for this task can be risky. In my case, about 30% of the citations it provided were inaccurate, which meant I had to verify each one manually. For reliable academic sourcing, I found Google Scholar much more trustworthy because current LLMs are still prone to “hallucinations.” You may have encountered other AI tools like Consensus AI, a search engine tailored for scientific research and limited to peer-reviewed academic papers only. Compared to ChatGPT Deep Research, it’s faster and more reliable for academic queries, but I strongly recommend always verifying AI outputs, as both tools can occasionally produce inaccuracies.

The Science article also highlights that AI usage varies significantly across disciplines. “The amount of artificial intelligence (AI)-modified sentences in scientific papers had surged by September 2024, almost two years after the release of ChatGPT, according to an analysis.” The table below shows estimates of AI usage by field, with certain disciplines adopting AI much faster than others. James Zou, a computational biologist at Stanford University, suggests these differences may reflect varying levels of familiarity with AI technology.

While the study from Stanford and UCSB is quite solid, Data Scientist Kobak pointed out that the estimates above could be underreported. One reason for this is that some authors may have started removing “red flag” words from manuscripts to avoid detection. For example, the word “delve” became more common right after ChatGPT launched, but its usage dropped sharply once it became widely recognized as a hallmark of AI-generated text.

If you want to read the full article, you can find it here: Science – One-fifth of computer science papers may include AI content.

— Andrew

Update: Here is another more recent report from Nature.

0

How NLP Helps Robots Handle Interruptions: A Summary of JHU Research

I recently came across an awesome study from Johns Hopkins University describing how computational linguistics and NLP can make robots better conversational partners by teaching them how to handle interruptions, a feature that feels basic for humans but is surprisingly hard for machines.

What the Study Found

Researchers trained a social robot powered by a large language model (LLM) to manage real-time interruptions based on speaker intent. They categorized interruptions into four types: Agreement, Assistance, Clarification, and Disruption.

By analyzing human conversations from interviews to informal discussions, they designed strategies tailored to each interruption type. For example:

  • If someone agrees or helps, the robot pauses, nods, and resumes speaking.
  • When someone asks for clarification, the robot explains and continues.
  • For disruptive interruptions, the robot can either hold the floor to summarize its remaining points before yielding to the human user, or it can stop talking immediately.

How NLP Powers This System

The robot uses an LLM to:

  1. Detect overlapping speech
  2. Classify the interrupter’s intent
  3. Select the appropriate response strategy

In tests involving tasks and conversations, the system correctly interpreted interruptions about 89% of the time and responded appropriately 93.7% of the time.

Why This Matters in NLP and Computational Linguistics

This work highlights how computational linguistics and NLP are essential to human-robot interaction.

  • NLP does more than generate responses; it helps robots understand nuance, context, and intent.
  • Developing systems like this requires understanding pause cues, intonation, and conversational flow, all core to computational linguistics.
  • It shows how multimodal AI, combining language with behavior, can enable more natural and effective interactions.

What I Found Most Interesting

The researchers noted that users didn’t like when the robot “held the floor” too long during disruptive interruptions. It reminded me how pragmatic context matters. Just like people expect some rules in human conversations, robots need these conversational skills too.

Looking Ahead

This research expands what NLP can do in real-world settings like healthcare, education, and social assistants. For someone like me who loves robots and language, it shows how computational linguistics helps build smarter, more human-friendly AI systems.

If you want to dive deeper, check out the full report from Johns Hopkins:
Talking robots learn to manage human interruptions

— Andrew

0

WAIC 2025: What Geoffrey Hinton’s “Tiger” Warning Taught Me About AI’s Future

At the end of July (7/26 – 7/28), Shanghai hosted the 2025 World Artificial Intelligence Conference (WAIC), drawing over 1,200 participants from more than 40 countries. Even though I wasn’t there, I followed the conference closely, especially the keynote from Geoffrey Hinton, the so-called “Godfather of AI.” His message? AI is advancing faster than we expect, and we need global cooperation to make sure it stays aligned with human values.

Hinton’s talk was historic. It was his first public appearance in China, and he even stood throughout his address despite back pain, which was noted by local media. One quote really stuck with me: “Humans have grown accustomed to being the most intelligent species in the world – what if that’s no longer the case?” That’s a big question, and as someone who’s diving deeper into computational linguistics and large language models, I felt both amazed and a little uneasy.

His warning compared superintelligent AI to a tiger we’re raising as a pet. If we’re not careful, he said, “the tiger” might one day turn on us. The point wasn’t to scare everyone, but to highlight why we can’t rely on simply pulling the plug if AI systems surpass human intelligence. Hinton believes we need to train AI to be good from the beginning because shutting it down later might not be an option.

WAIC 2025 wasn’t all doom and gloom though. Hinton also talked about the huge potential of AI to accelerate science. For example, he highlighted DeepMind’s AlphaFold as a breakthrough that solved a major biology challenge, predicting protein structures. That shows how powerful AI can be when guided properly.

What stood out the most was the recurring theme of cooperation. Hinton and others, like former Google CEO Eric Schmidt, emphasized the need for global partnerships on AI safety and ethics. Hinton even signed the “Shanghai AI Safety Consensus” with other experts to support international collaboration. The message was clear: no single country can or should handle AI’s future alone.

As a high school student passionate about AI and language, I’m still learning how these pieces fit together. But events like WAIC remind me that the future of AI isn’t just about building smarter systems, it’s also about making sure they work for everyone.

For those interested, here’s a more detailed summary of Hinton’s latest speech: Pandaily Report on WAIC 2025

You can also explore the official WAIC website here: https://www.worldaic.com.cn/

— Andrew

0

ACL 2025 New Theme Track: Generalization in NLP Models

The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025) will be happening in Vienna, Austria from July 27 to August 1. I won’t be attending in person, but as someone planning to study and do research in computational linguistics and NLP in college, I’ve been following the conference closely to keep up with the latest trends.

One exciting thing about this year’s ACL is its new theme track: Generalization of NLP Models. According to the official announcement:

“Following the success of the ACL 2020–2024 Theme tracks, we are happy to announce that ACL 2025 will have a new theme with the goal of reflecting and stimulating discussion about the current state of development of the field of NLP.

Generalization is crucial for ensuring that models behave robustly, reliably, and fairly when making predictions on data different from their training data. Achieving good generalization is critically important for models used in real-world applications, as they should emulate human-like behavior. Humans are known for their ability to generalize well, and models should aspire to this standard.

The theme track invites empirical and theoretical research and position and survey papers reflecting on the Generalization of NLP Models. The possible topics of discussion include (but are not limited to) the following:

  • How can we enhance the generalization of NLP models across various dimensions—compositional, structural, cross-task, cross-lingual, cross-domain, and robustness?
  • What factors affect the generalization of NLP models?
  • What are the most effective methods for evaluating the generalization capabilities of NLP models?
  • While Large Language Models (LLMs) significantly enhance the generalization of NLP models, what are the key limitations of LLMs in this regard?

The theme track submissions can be either long or short. We anticipate having a special session for this theme at the conference and a Thematic Paper Award in addition to other categories of awards.”

This year’s focus on generalization really highlights where the field is going—toward more robust, ethical, and real-world-ready NLP systems. It’s not just about making cool models anymore, but about making sure they work well across different languages, cultures, and use cases.

If you’re into reading papers like I am, especially ones that dig into how NLP systems can perform reliably on new or unexpected inputs, this theme track will be full of insights. I’m looking forward to checking out the accepted papers when they’re released.

You can read more at the official conference page: ACL 2025 Theme Track Announcement

— Andrew

0

Is It Legal to Train AI on Books? A High School Researcher’s Take on the Anthropic Ruling

As someone who’s been exploring computational linguistics and large language models (LLMs), I’ve always wondered: How legal is it, really, to train AI on books or copyrighted material? This question came up while I was learning about how LLMs are trained using massive datasets, including books, articles, and other written works. It turns out the legal side is just as complex as the technical side.

A major U.S. court case in June 2025 helped answer this question, at least for now. In this post, I’ll break down what happened and what it means for researchers, developers, and creators.

The Big Picture: Copyright, Fair Use, and AI

In the U.S., books and intellectual property (IP) are protected under copyright law. That means you can’t just use someone’s novel or article however you want, especially if it’s for a commercial product.

However, there’s something called fair use, which allows limited use of copyrighted material without permission. Whether something qualifies as fair use depends on four factors:

  1. The purpose of the use (such as commercial vs. educational)
  2. The nature of the original work
  3. The amount used
  4. The effect on the market value of the original

LLM developers often argue that training models is “transformative.” In other words, the model doesn’t copy the books word for word. Instead, it learns patterns from large collections of text and generates new responses based on those patterns.

Until recently, this argument hadn’t been fully tested in court.

What Just Happened: The Anthropic Case (June 24, 2025)

In a landmark decision, U.S. District Judge William Alsup ruled that AI company Anthropic did not violate copyright law when it trained its Claude language model on books. The case was brought by authors Andrea Bartz, Charles Graeber, and Kirk Wallace Johnson, who argued that Anthropic had used their work without permission.

  • Andrea Bartz: The Lost Night: A Novel
  • Charles Graeber: The Good Nurse: A True Story of Medicine, Madness, and Murder
  • Kirk Wallace Johnson: The Fisherman and the Dragon: Fear, Greed, and a Fight for Justice on the Gulf Coast

Judge Alsup ruled that Anthropic’s use of the books qualified as fair use. He called the training process “exceedingly transformative” and explained that the model did not attempt to reproduce the authors’ styles or specific wording. Instead, the model learned patterns and structures in order to generate new language, similar to how a human might read and learn from books before writing something original.

However, the court also found that Anthropic made a serious mistake. The company had copied and stored more than 7 million pirated books in a central data library. Judge Alsup ruled that this was not fair use and was a clear violation of copyright law. A trial is scheduled for December 2025 to determine possible penalties, which could be up to $150,000 per work.

Why This Case Matters

This is the first major U.S. court ruling on whether training generative AI on copyrighted works can qualify as fair use. The result was mixed. On one hand, the training process itself was ruled legal. On the other hand, obtaining the data illegally was not.

This means AI companies can argue that their training methods are transformative, but they still need to be careful about where their data comes from. Using pirated books, even if the outcome is transformative, still violates copyright law.

Other lawsuits are still ongoing. Companies like OpenAI, Meta, and Microsoft are also facing legal challenges from authors and publishers. These cases may be decided differently, depending on how courts interpret fair use.

My Thoughts as a Student Researcher

To be honest, I understand both sides. As someone who is really excited about the possibilities of LLMs and has worked on research projects involving language models, I think it’s important to be able to learn from large and diverse datasets.

At the same time, I respect the work of authors and creators. Writing a book takes a lot of effort, and it’s only fair that their rights are protected. If AI systems are going to benefit from their work, then maybe there should be a system that gives proper credit or compensation.

For student researchers like me, this case is a reminder to be careful and thoughtful about where our data comes from. It also raises big questions about what responsible AI development looks like, not just in terms of what is allowed by law, but also what is fair and ethical.

Wrapping It Up

The Anthropic ruling is a big step toward defining the legal boundaries for training AI on copyrighted material. It confirmed that training can be legal under fair use if it is transformative, but it also made clear that sourcing content from pirated platforms is still a violation of copyright law.

This case does not settle the global debate, but it does provide some clarity for researchers and developers in the U.S. Going forward, the challenge will be finding a balance between supporting innovation and respecting the rights of creators.

— Andrew

Update (September 5, 2025):

AI startup Anthropic will pay at least $1.5 billion to settle a copyright infringement lawsuit over its use of books downloaded from the Internet to train its Claude AI models. The federal case, filed last year in California by several authors, accused Anthropic of illegally scraping millions of works from ebook piracy sites. As part of the settlement, Anthropic has agreed to destroy datasets containing illegally accessed works. (Read the full report)

0

My First Solo Publication: A Case Study on Sentiment Analysis in Survey Data

I’m excited to share that my first solo-authored research paper has just been published in the National High School Journal of Science! 🎉

The paper is titled “A Case Study of Sentiment Analysis on Survey Data Using LLMs versus Dedicated Neural Networks”, and it explores a question I’ve been curious about for a while: how do large language models (like GPT-4o or LLaMA-3) compare to task-specific neural networks when it comes to analyzing open-ended survey responses?

If you’ve read some of my earlier posts—like my reflection on the DravidianLangTech shared task or my thoughts on Jonathan Dunn’s NLP book—you’ll know that sentiment analysis has become a recurring theme in my work. From experimenting with XLM-RoBERTa on Tamil and Tulu to digging into how NLP can support corpus linguistics, this paper feels like the natural next step in that exploration.

Why This Matters to Me

Survey responses are messy. They’re full of nuance, ambiguity, and context—and yet they’re also where we hear people’s honest voices. I’ve always thought it would be powerful if AI could help us make sense of that kind of data, especially in educational or public health settings where understanding sentiment could lead to real change.

In this paper, I compare how LLMs and dedicated models handle that challenge. I won’t go into the technical details here (the paper does that!), but one thing that stood out to me was how surprisingly effective LLMs are—even without task-specific fine-tuning.

That said, they come with trade-offs: higher computational cost, more complexity, and the constant need to assess bias and interpretability. There’s still a lot to unpack in this space.

Looking Ahead

This paper marks a milestone for me, not just academically but personally. It brings together things I’ve been learning in courses, competitions, side projects, and books—and puts them into conversation with each other. I’m incredibly grateful to the mentors and collaborators who supported me along the way.

If you’re interested in sentiment analysis, NLP for survey data, or just want to see what a high school research paper can look like in this space, I’d love for you to take a look:
🔗 Read the full paper here

Thanks again for following along this journey. Stay tuned!

0

Shared Task at DravidianLangTech 2025

In 2025, I had the privilege of participating in the shared task on Sentiment Analysis in Tamil and Tulu as part of the DravidianLangTech@NAACL 2025 conference. The task was both challenging and enlightening, as it required applying machine learning techniques to multilingual data with varying sentiment nuances. This post highlights the work I did, the methodology I followed, and the results I achieved.

The Task at Hand

The goal of the task was to classify text into one of four sentiment categories: Positive, Negative, Mixed Feelings, and Unknown State. The datasets provided were in Tamil and Tulu, which made it a fascinating opportunity to work with underrepresented languages.

Methodology

I implemented a pipeline to preprocess the data, tokenize it, train a transformer-based model, and evaluate its performance. My choice of model was XLM-RoBERTa, a multilingual transformer capable of handling text from various languages effectively. Below is a concise breakdown of my approach:

  1. Data Loading and Inspection:
    • Used training, validation, and test datasets in .xlsx format.
    • Inspected the data for missing values and label distributions.
  2. Text Cleaning:
    • Created a custom function to clean text by removing unwanted characters, punctuation, and emojis.
    • Removed common stopwords to focus on meaningful content.
  3. Tokenization:
    • Tokenized the cleaned text using the pre-trained XLM-RoBERTa tokenizer with a maximum sequence length of 128.
  4. Model Setup:
    • Leveraged XLM-RoBERTaForSequenceClassification with 4 output labels.
    • Configured TrainingArguments to train for 3 epochs with evaluation at the end of each epoch.
  5. Evaluation:
    • Evaluated the model on the validation set, achieving a Validation Accuracy of 59.12%.
  6. Saved Model:
    • Saved the trained model and tokenizer for reuse.

Results

After training the model for three epochs, the validation accuracy was 59.12%. While there is room for improvement, this score demonstrates the model’s capability to handle complex sentiment nuances in low-resource languages like Tamil.

The Code

Below is an overview of the steps in the code:

  • Preprocessing: Cleaned and tokenized the text to prepare it for model input.
  • Model Training: Used Hugging Face’s Trainer API to simplify the training process.
  • Evaluation: Compared predictions against ground truth to compute accuracy.

To make this process more accessible, I’ve attached the complete code as a downloadable file. However, for a quick overview, here’s a snippet from the code that demonstrates how the text was tokenized:

# Tokenize text data using the XLM-RoBERTa tokenizer
def tokenize_text(data, tokenizer, max_length=128):
    return tokenizer(
        data,
        truncation=True,
        padding='max_length',
        max_length=max_length,
        return_tensors="pt"
    )

train_tokenized = tokenize_text(train['cleaned'].tolist(), tokenizer)
val_tokenized = tokenize_text(val['cleaned'].tolist(), tokenizer)

This function ensures the input text is prepared correctly for the transformer model.

Reflections

Participating in this shared task was a rewarding experience. It highlighted the complexities of working with low-resource languages and the potential of transformers in tackling these challenges. Although the accuracy could be improved with hyperparameter tuning and advanced preprocessing, the results are a promising step forward.

Download the Code

I’ve attached the full code used for this shared task. Feel free to download it and explore the implementation in detail.

XLM-RoBERTa Model And EvalDownload

If you’re interested in multilingual NLP or sentiment analysis, I’d love to hear your thoughts or suggestions on improving this approach! Leave a comment below or connect with me via the blog.

0

I am back!

This will be a short post since I’m planning to post a more in-depth discussion on one thing that I’ve been up to over the summer. Between writing a research paper (currently under review by the Journal of High School Science) and founding a nonprofit called Student Echo, I’ve been keeping myself busy. Despite all this, I plan to post shorter updates more frequently here. Sorry for the wait—assuming anyone was actually waiting—but hey, here you go.

Here’s a bit more about what’s been keeping me occupied:
My Research Paper
Title: Comparing Performance of LLMs vs. Dedicated Neural Networks in Analyzing the Sentiment of Survey Responses
Abstract: Interpreting sentiment in open-ended survey data is a challenging but crucial task in the age of digital information. This paper studies the capabilities of three LLMs, Gemini-1.5-Flash, Llama-3-70B, and GPT-4o, comparing them to dedicated, sentiment analysis neural networks, namely RoBERTa-base-sentiment and DeBERTa-v3-base-absa. These models were evaluated on their accuracy along with other metrics (precision, recall, and F1-score) in determining the underlying sentiment of responses from two COVID-19 surveys. The results revealed that despite being designed for broader applications, all three LLMs generally outperformed specialized neural networks, with the caveat that RoBERTa was the most precise at detecting negative sentiment. While LLMs are more resource-intensive than dedicated neural networks, their enhanced accuracy demonstrates their evolving potential and justifies the increased resource costs in sentiment analysis.

My Nonprofit: Student Echo
Website: https://www.student-echo.org/
Student-Echo.org is a student-led non-profit organization with the mission of amplifying students’ voices through student-designed questionnaires, AI-based technology, and close collaboration among students, teachers, and school district educators.

0

Blog at WordPress.com.

Up ↑