A Book That Expanded My Perspective on NLP: Natural Language Processing for Corpus Linguistics by Jonathan Dunn

Book Link: https://doi.org/10.1017/9781009070447

As I dive deeper into the fascinating world of Natural Language Processing (NLP), I often come across resources that reshape my understanding of the field. One such recent discovery is Jonathan Dunn’s Natural Language Processing for Corpus Linguistics. This book, a part of the Elements in Corpus Linguistics series by Cambridge University Press, stands out for its seamless integration of computational methods with traditional linguistic analysis.

A Quick Overview

The book serves as a guide to applying NLP techniques to corpus linguistics, especially in dealing with large-scale corpora that are beyond the scope of traditional manual analysis. It discusses how models like text classification and text similarity can help address linguistic problems such as categorization (e.g., identifying part-of-speech tags) and comparison (e.g., measuring stylistic similarities between authors).

What I found particularly intriguing is its structure, which is built around five compelling case studies:

Corpus-Based Sociolinguistics: Exploring geographic and social variations in language use.
Corpus Stylistics: Understanding authorship through stylistic differences in texts.
Usage-Based Grammar: Analyzing syntax and semantics via computational models.
Multilingualism Online: Investigating underrepresented languages in digital spaces.
Socioeconomic Indicators: Applying corpus analysis to non-linguistic fields like politics and sentiment in customer reviews.

The book is as much a practical resource as it is theoretical. Accompanied by Python notebooks and a stand-alone Python package, it provides hands-on tools to implement the discussed methods—a feature that makes it especially appealing to readers with a technical bent.

A Personal Connection

My journey with this book is a bit more personal. While exploring NLP, I had the chance to meet Jonathan Dunn, who shared invaluable insights about this field. One of his students, Sidney Wong, recommended this book to me as a starting point for understanding how computational methods can expand corpus linguistics. It has since become a cornerstone of my learning in this area.

What Makes It Unique

Two aspects of Dunn’s book particularly resonated with me:

Ethical Considerations: As corpus sizes grow, so do the ethical dilemmas associated with their use. From privacy issues to biases in computational models, the book doesn’t shy away from discussing the darker side of large-scale text analysis. This balance between innovation and responsibility is a critical takeaway for anyone venturing into NLP.
Interdisciplinary Approach: Whether you’re a linguist looking to incorporate computational methods or a computer scientist aiming to understand linguistic principles, this book bridges the gap between the two disciplines beautifully. It encourages a collaborative perspective, which is essential in fields as expansive as NLP and corpus linguistics.

Who Should Read It?

If you’re a student, researcher, or practitioner with an interest in exploring how NLP can scale linguistic analysis, this book is for you. Its accessibility makes it suitable for beginners, while the advanced discussions and hands-on code offer plenty for seasoned professionals to learn from.

For me, Natural Language Processing for Corpus Linguistics isn’t just a book—it’s a toolkit, a mentor, and an inspiration rolled into one. As I continue my journey in NLP, I find myself revisiting its chapters for insights and ideas.

December 26, 2024 0

Exploring the Intersection of AI and Human Creativity: A Review of Deep Thinking by Garry Kasparov

Recently, I had the opportunity to read Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins by Garry Kasparov. While this book doesn’t directly tie into my work in computational linguistics, it still resonated with me due to its exploration of artificial intelligence (AI), a field closely related to many of my interests. The book combines my passions for chess and technology, and while its primary focus is on AI in the realm of chess, it touches on broader themes that align with my curiosity about how AI and human creativity intersect.

In Deep Thinking, the legendary chess grandmaster Garry Kasparov delves into his personal journey with artificial intelligence, particularly focusing on his famous matches against the machine Deep Blue. This book is not just a chronicle of those historic encounters; it’s an exploration of how AI impacts human creativity, decision-making, and the psychological experience of competition.

Kasparov’s narrative offers more than just an inside look at high-level chess; it provides an insightful commentary on the evolving relationship between humans and technology. Deep Thinking is a must-read for those interested in the intersection of AI and human ingenuity, especially for chess enthusiasts who want to understand the psychological and emotional impacts of playing against a machine.

Kasparov’s main argument is clear: While AI has transformed chess, it still cannot replicate the creativity, reasoning, and emotional depth that humans bring to the game. AI can calculate moves and offer solutions, but it lacks the underlying rationale and context that makes human play unique. As Kasparov reflects, even the most advanced chess programs can’t explain why a move is brilliant—they just make it. This inability to reason and articulate is a crucial distinction he highlights throughout the book, particularly in Chapter 4, where he emphasizes that AI lacks the emotional engagement that a human player experiences.

For Kasparov, the real challenge comes not just from the machine’s power but from its lack of emotional depth. In Chapter 5, he shares how the experience of being crushed by an AI, which feels no satisfaction or fear, is difficult to process emotionally. It’s this emotional disconnect that underscores the difference between the human and machine experience, not only in chess but in any form of creative endeavor. The machine may be able to play at the highest level, but it doesn’t feel the game the way humans do.

Kasparov’s exploration of AI in chess is enriched by his experiences with earlier machines like Deep Thought, where he learns that “a machine learning system is only as good as its data.” This idea touches on a broader theme in the book: the idea that AI is limited by the input it receives. The system is as powerful as the information it processes, but it can never go beyond that data to create something entirely new or outside the parameters defined for it.

By the book’s conclusion, Kasparov pivots to a broader, more philosophical discussion: Can AI make us more human? He argues that technology, when used properly, has the potential to free us from mundane tasks, allowing us to be more creative. It is a hopeful perspective, envisioning a future where humans and machines collaborate rather than compete.

However, Deep Thinking does have its weaknesses. The book’s technical nature and reliance on chess-specific terminology may alienate readers unfamiliar with the game or the intricacies of AI. Kasparov makes an effort to explain these concepts, but his heavy use of jargon can make it difficult for casual readers to fully engage with the material. Additionally, while his critique of AI is compelling, it sometimes feels one-sided, focusing mainly on AI’s limitations without fully exploring how it can complement human creativity.

Despite these drawbacks, Deep Thinking remains a fascinating and thought-provoking read for those passionate about chess, AI, and the future of human creativity. Kasparov’s firsthand insights into the psychological toll of competing against a machine and his reflections on the evolving role of AI in both chess and society make this book a significant contribution to the ongoing conversation about technology and humanity.

In conclusion, Deep Thinking is a compelling exploration of AI’s role in chess and human creativity. While it may be a challenging read for those new to the fields of chess or AI, it offers invaluable insights for those looking to explore the intersection of technology and human potential. If you’re a chess enthusiast, an AI aficionado, or simply curious about how machines and humans can co-evolve creatively, Kasparov’s book is a must-read.

December 14, 2024 0

Insights from My Ling 234 Summer Class at UW

This summer, I got my first taste of college life—or at least, the online version—through Ling 234 at UW. If you’re imagining grand lecture halls and bustling campus energy, this was not that. Instead, it was me, my laptop, and a series of online modules. But don’t let the format fool you—this class packed a lot of depth.

I took Ling 234 to get a deeper understanding of the linguistic concepts that underpin computational linguistics. As someone interested in the intersection of language and technology, I wanted to explore the “deep end” of linguistics: how societies perceive language, how languages vary and change, and how they influence identity. Understanding topics like language ideologies, multilingualism, and even the sociolinguistics of dialects helps ground the technical aspects of computational linguistics in real-world human language complexities.

One of the most valuable connections I found was how language variation and sociolinguistic factors affect language processing. For example, concepts like dialect variation, multilingualism, and even gendered language use are critical when developing systems that work across diverse language contexts. Computational linguistics relies on handling these nuances, whether in sentiment analysis, machine translation, or speech recognition. The insights I gained from this course are stepping stones to building more inclusive and accurate models in AI.

If you’re curious, I’ve attached my notes for the class. They’re comprehensive and detail everything from the mechanics of language contact to the challenges of language revitalization. While these notes may not be everyone’s cup of tea, they represent a foundational step in my journey toward understanding language through both linguistic and computational lenses.

Ling 234 may not have had the traditional “college experience” feel, but it exceeded my expectations in laying the groundwork for integrating linguistics into computational approaches. It wasn’t just a class—it was a valuable perspective shift.

LING 234 A Summer Course Notes Download

November 29, 2024 1

I am back!

This will be a short post since I’m planning to post a more in-depth discussion on one thing that I’ve been up to over the summer. Between writing a research paper (currently under review by the Journal of High School Science) and founding a nonprofit called Student Echo, I’ve been keeping myself busy. Despite all this, I plan to post shorter updates more frequently here. Sorry for the wait—assuming anyone was actually waiting—but hey, here you go.

Here’s a bit more about what’s been keeping me occupied:
My Research Paper
Title: Comparing Performance of LLMs vs. Dedicated Neural Networks in Analyzing the Sentiment of Survey Responses
Abstract: Interpreting sentiment in open-ended survey data is a challenging but crucial task in the age of digital information. This paper studies the capabilities of three LLMs, Gemini-1.5-Flash, Llama-3-70B, and GPT-4o, comparing them to dedicated, sentiment analysis neural networks, namely RoBERTa-base-sentiment and DeBERTa-v3-base-absa. These models were evaluated on their accuracy along with other metrics (precision, recall, and F1-score) in determining the underlying sentiment of responses from two COVID-19 surveys. The results revealed that despite being designed for broader applications, all three LLMs generally outperformed specialized neural networks, with the caveat that RoBERTa was the most precise at detecting negative sentiment. While LLMs are more resource-intensive than dedicated neural networks, their enhanced accuracy demonstrates their evolving potential and justifies the increased resource costs in sentiment analysis.

My Nonprofit: Student Echo
Website: https://www.student-echo.org/
Student-Echo.org is a student-led non-profit organization with the mission of amplifying students’ voices through student-designed questionnaires, AI-based technology, and close collaboration among students, teachers, and school district educators.

November 28, 2024 0

The World of Kaggle

Sorry for taking so long to write another blog post! I have been busy with finishing up and starting the new season of Vex Robotics, High Stakes. For more information, you can watch the game reveal video on YouTube. Link: https://www.youtube.com/watch?v=Sx6HJSpopeQ

So, what is Kaggle? Kaggle is a popular online platform for data science and machine learning enthusiasts to practice and enhance their skills. It offers a variety of resources, including datasets, notebooks, and learning tutorials. One of the key features of Kaggle is its competitions, which are hosted by companies and research institutions. These competitions challenge participants to solve real-world problems using data science and machine learning techniques. Participants can work individually or in teams to create models and submit their solutions. Competitions often come with monetary prizes and recognition, providing valuable opportunities for learning, networking, and career advancement in the data science community.
Over the past few months, I have been competing in various competitions and, as you might find in my Linguistics courses listed in this blog, I have taken some of Kaggle’s courses. If you would like to see some of my work, you can visit me at my Kaggle profile.

What Makes Kaggle So Interesting?

To me, Kaggle serves as a way to test my knowledge and skills in machine learning and data processing. These topics are key in the neural network and artificial intelligence side of Computational Linguistics. Though I am not in it for the prizes, I still find joy in seeing myself progress on the public leaderboard and observing how my shared notebooks can foster a community where like-minded people can share questions or advice. This is what I enjoy about Kaggle—the community and its competitive yet supportive atmosphere that allows people to learn.
Kaggle competitions are particularly valuable because they provide real-world datasets that are often messy and complex, mimicking the challenges faced in professional data science roles. The platform encourages experimentation and innovation, pushing participants to think creatively and critically. Additionally, Kaggle’s discussion forums are a treasure trove of insights, where experts and beginners alike can exchange ideas, troubleshoot problems, and discuss strategies.

Where Should You Start?

Kaggle is friendly to both beginners and experts, offering different levels of competitions and a wide range of courses. Here are some competitions and courses that I would recommend:
Competitions: These competitions are ongoing, so you can take as much time as you need.
– House Prices – Advanced Regression Techniques: It is your job to predict the sales price for each house. For each Id in the test set, you must predict the value of the Sale Price variable.
– Natural Language Processing with Disaster Tweets: Predict which Tweets are about real disasters and which ones are not
– Titanic – Machine Learning from Disaster: Predict survival on the Titanic and get familiar with ML basics
Courses:
– Intro to Machine Learning: Learn the core ideas in machine learning and build your first models.
– Intro to Deep Learning: Use TensorFlow and Keras to build and train neural networks for structured data.
– Intro to AI Ethics: Explore practical tools to guide the moral design of AI systems.

Conclusion

Diving into Kaggle has been an enriching experience, both for honing my technical skills and for being part of a vibrant community. Whether you’re just starting out or looking to advance your expertise, Kaggle offers a wealth of resources and challenges to help you grow. The blend of real-world problems, collaborative learning, and the thrill of competition makes it an ideal platform for anyone passionate about data science and machine learning.

As I continue to explore the realms of Computational Linguistics and AI, I find that the lessons learned and the connections made on Kaggle are invaluable. I encourage you to take the plunge, participate in competitions, and engage with the community. Who knows? You might discover a new passion or take your career to new heights.

Feel free to check out my work on my Kaggle profile and join me on this exciting journey of learning and discovery. Happy Kaggling!

July 5, 2024 0

Devin: the world’s first AI software engineer by Cognition

Cognition recently unveiled Devin AI, the world’s inaugural AI software engineer, marking a significant breakthrough in the field. Devin AI stands out for its unique ability to convert commands into fully operational websites. While it might appear that this innovation could negatively impact the demand for human software engineers, Devin AI is designed to act primarily as an advanced virtual assistant for software development.

Understanding Devin’s Functionality

Devin AI utilizes a novel, forward-thinking method to tackle complex software development tasks. It employs a combination of a personalized browser, a code editor, and a command line interface, enabling it to process tasks in a manner similar to a human software engineer. Devin starts by accessing its extensive knowledge base. If necessary, it uses its personalized browser to search for additional information needed to complete the task. Finally, it writes the code and applies common debugging techniques to fix any issues that arise, effectively mimicking the human approach to software development.

The Advantages of using Devin

The primary reason for using Devin lies in its proficiency in addressing bugs, enhancing applications, and performing other straightforward engineering duties. Beyond these basic tasks, Devin is capable of handling more complex challenges. These include mastering new technologies and optimizing extensive language models. Cognition’s demonstrations highlight Devin’s remarkable ability to evaluate various API providers’ capabilities. Moreover, Devin can visually present the results on a website it creates, showcasing its versatile skill set. This versatility makes Devin an invaluable tool for both routine and complex software engineering projects.

In a demonstration, Cognition showcased Devin’s impressive capability to assess and test the functionalities of multiple API providers, adeptly presenting the outcomes through a visually engaging website it constructed.

Today we're excited to introduce Devin, the first AI software engineer.

Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork.

Devin is… pic.twitter.com/ladBicxEat
— Cognition (@cognition_labs) March 12, 2024

Conclusion

Despite the initial buzz around Devin being positioned as the AI destined to supplant human roles in the software engineering realm, a deeper exploration into its functionalities and design philosophy suggests a different narrative. Devin is engineered to act as a powerful augmentative tool for software engineers rather than a replacement. Its intrinsic value lies in its compatibility with the human elements of software development, evidenced by its capability to automate mundane tasks, streamline complex workflows, and enhance the overall efficiency of the development process. Through its innovative use of AI to tackle both routine and complex engineering challenges, Devin stands out as a testament to the potential of human-AI collaboration, aiming not to eclipse human ingenuity but to expand its reach and impact within the technological ecosystem.

April 10, 2024 0

Nvidia: Understanding Its Rise to Popularity

In the fast-paced world of technology, Nvidia has emerged as a powerhouse, captivating the attention of tech enthusiasts and investors alike. Known primarily for its cutting-edge graphics processing units (GPUs), Nvidia has become synonymous with high-performance computing, gaming, and artificial intelligence (AI). But what is Nvidia, and why has it become such a focal point in the tech industry today? In this blog post I seek to answer questions such as this and expand your knowledge about the rise of this company.

Nvidia’s Origins and Offerings

Nvidia’s ascent from a hopeful startup to a powerhouse in the technology sector is a story of strategic foresight, impeccable timing, and innovative breakthroughs. In 1993, Nvidia was brought to life by three forward-thinking individuals. They identified a niche in enhancing computing through superior graphic processing, a domain they believed was pivotal for the future of computing. Their anticipation of market trends and the emerging video game sector’s demand for advanced graphic solutions laid the groundwork for Nvidia’s remarkable growth trajectory.

An interesting tidbit about the company’s name: Originally, Nvidia was simply known as “NV”, an abbreviation for “next version”, reflecting the founders’ ambition for continual innovation. The final name, Nvidia, was derived by merging “NV” with “Invidia”, the Latin word for envy, crafting a unique and memorable brand identity.

Nvidia’s first major breakthrough came with the introduction of its inaugural graphics accelerator. Unlike its competitors, including giants like Microsoft, Nvidia opted for a more efficient approach by focusing on quadratic primitives processing. This strategic decision, though initially financially burdensome, proved to be a game-changer. Within four months of its launch, the product’s sales skyrocketed to a million units, firmly establishing Nvidia’s reputation in the tech industry and setting the stage for its future successes.

Nvidia in the World of AI and Beyond

As the realms of business and artificial intelligence increasingly intertwine, companies are betting big on AI as the next frontier of innovation. This trend has been significantly accelerated by the global pandemic, which not only transformed operational paradigms across industries but also catalyzed advancements in Nvidia’s GPU technology. In the wake of these developments, it became evident that Nvidia’s GPUs were not just exceptional for rendering graphics but also exceptionally suited for the demands of AI software development, particularly in training sophisticated AI models.

This realization marked a pivotal shift, positioning Nvidia’s GPUs as the processors of choice over traditional CPUs, like those offered by Intel, especially within the burgeoning AI sector. The unique capabilities of Nvidia’s technology made it indispensable for a range of tech companies venturing into AI, from startups to tech giants. These companies rely on the immense processing power of GPUs to handle the complex computations required for AI applications, from natural language processing to autonomous vehicle navigation.

Furthermore, the demand for Nvidia’s GPUs has skyrocketed as AI-focused companies require an ever-increasing number of chips to power their innovative solutions. This surge in demand underscores Nvidia’s influential role in the stock market and its contributions to shaping the future of technology. Nvidia’s growth trajectory and its central position in the supply chain for AI technologies highlight the critical role AI is poised to play in determining the success and strategic direction of companies across the globe. As Nvidia continues to lead in GPU development, its impact extends beyond the tech industry, signaling a broader shift towards an AI-driven future where Nvidia’s innovations are at the heart of transformative change.

Conclusion

Nvidia’s journey from a small startup to a titan in the tech world is a testament to innovation and vision. As we’ve seen, their evolution from pioneering graphics processing to dominating the AI landscape showcases their ability to not just keep pace with technological advancements but to anticipate and shape them. Nvidia’s GPUs, now at the heart of AI development, highlight the company’s significant impact on technology today and its potential to drive future innovations. As we look to the future, Nvidia’s story reminds us of the power of foresight and adaptability in the ever-evolving tech industry, signaling exciting possibilities for what’s next.

March 27, 2024 0

Sora: New OpenAI Text to Video Technology

Introduction to Sora: A Leap in AI-Driven Visual Arts

OpenAI has recently unveiled its pioneering text-to-video technology, Sora, marking a significant milestone in the fusion of artificial intelligence with the realm of visual creativity. This innovative technology is celebrated for its ability to generate characters and scenes that are rich in emotion, seamlessly narrating stories across successive frames. Despite its prowess in creating complex visual narratives, Sora faces challenges in accurately simulating physics and maintaining spatial consistency over time.

The Synergy of ChatGPT and DALL·E 3

At the heart of Sora lies the integration of OpenAI’s advanced language model, ChatGPT, and the imaginative power of DALL·E 3. This combination not only adheres to OpenAI’s rigorous guidelines but also ensures the technology’s application remains ethical and constructive. The collaboration between these two technologies exemplifies OpenAI’s commitment to leveraging artificial intelligence to foster creativity and prevent misuse.

Showcasing Sora’s Capabilities

OpenAI’s official website features a diverse array of examples demonstrating Sora’s capabilities. Among these, a standout video portrays the imagined daily life in Lagos, Nigeria, in 2056, captured as though through a mobile phone camera. This example not only illustrates Sora’s creative potential but also its capacity to engage and inspire with futuristic visions. For those interested in experiencing this remarkable depiction firsthand, further details and the video can be found directly on OpenAI’s platform or through the following link. https://cdn.openai.com/sora/videos/lagos.mp4

Beyond Video Generation: Image to Video Transformation

Expanding Creative Horizons

Sora’s ability to transform a single image into a detailed video sequence represents a significant expansion of its creative capabilities. This function allows for the enhancement of a static image into a dynamic narrative, showcasing an impressive level of accuracy and attention to detail.

NLP’s Role in Advancing AGI

While Sora’s image-to-video feature may not directly advance computational linguistics or NLP, it highlights the role of NLP in pushing the boundaries of artificial general intelligence (AGI). By building upon the existing training of ChatGPT, Sora exemplifies how NLP can be applied to achieve remarkable milestones in AGI, illustrating the potential for language understanding to merge with visual creativity.

Conclusion: Envisioning the Future of AI Integration

Sora’s development reflects the growing synergy between linguistic intelligence and visual artistry in the field of artificial intelligence. By enabling the generation of videos from text instructions and the transformation of images into videos, Sora represents a significant stride towards the realization of integrated AI systems. As we look towards the future, the continued fusion of these technologies promises to unlock new realms of creativity and innovation in AI applications.

February 19, 2024 0

NLP Python Tutorial (Happy Late New Year!)

Happy late New Year, fellow computational linguistics enthusiasts! I trust that your 2023 was a year of delightful experiences and learning. As we step into 2024, may it be a year where all your dreams and goals come to fruition!

In the spirit of continuous learning and exploration, I’m excited to share my latest endeavor with you: an engaging NLP Python Course on YouTube crafted by codebasics. I’ve been steadily navigating through the wealth of knowledge it offers and felt compelled to bring this incredible learning resource to your attention.

This course isn’t just another tutorial; it’s a comprehensive journey into the world of Natural Language Processing, tailored for those starting or looking to solidify their understanding in the field. So, let’s dive into what makes this course a standout and how it can be the catalyst for your NLP mastery in 2024!

Why this Course?

As I navigated through various computational linguistics topics via Kaggle and Coursera, I hadn’t deeply explored a specific domain until my intrigue in Natural Language Processing (NLP) led me to the beginner-level course by codebasics. So, why this course? It elegantly lays down the complexities of NLP in digestible segments while also pointing learners towards additional resources to bolster their understanding. This approach is particularly effective, serving a broad spectrum of learners, from those taking their first step in NLP to those looking to refresh their foundational knowledge. It’s a well-crafted guide that not only initiates beginners into the world of NLP but also enriches the learning curve for the more experienced, making it a perfect fit for my journey into NLP.

What is in this Course?

It’s a complete package that takes you through all the essential NLP concepts, coupled with neat coding sessions and hands-on exercises. You’ll be playing around with Python and some of the coolest libraries out there like Spacy, NLTK, Hugging Face, Dialogflow, and TensorFlow.

This playlist isn’t just about watching; it’s about doing. You’ll get your hands dirty with real coding and solve practical problems as you go along. From getting the gist of text analysis to building your very own chatbots, the course has a bit of everything for anyone curious about how machines understand human language.

It’s structured to be super learner-friendly, so don’t worry if you’re just starting out. By the end of it, you’ll be more confident in tackling NLP projects and ready to dive deeper into this fascinating field. Get ready to kickstart your NLP adventure with lots of code, fun, and learning!

Continuing the Learning Journey

Finished the NLP course? Here’s how to keep advancing:

Deepen Your NLP Skills: Now that you’ve got the basics, try tackling more advanced topics. Look for resources on sentiment analysis or dive into creating more complex language models.
Learn Machine Learning: It’s a big deal in tech and super useful in NLP. Start with an introductory machine learning course to understand the algorithms that power advanced NLP.
Explore Translation: If languages intrigue you, explore how machines handle translation and multilingual communication. It’s a challenging but rewarding area.
Broaden Your Horizons: Platforms like Coursera, edX, or Udemy offer a plethora of courses in tech, including advanced NLP and related fields.
Hands-On Practice: Apply your skills in real-world projects or join online data science competitions. Websites like Kaggle can provide practical experience and community engagement.
Stay Connected: Join forums, follow tech blogs, or attend online workshops to keep up with the latest in tech and NLP.

Keep exploring and building your skills. The journey in computational linguistics is as exciting as it is endless!

January 6, 2024 0