Vasu Chetty

If you write computer code daily in your job or research, especially as a graduate student, the attached book, *"The C Programming Language"* by Brian W. Kernighan and Dennis M. Ritchie, is an absolute must-read. Even if you never write a single line of C code, the lessons you’ll learn in programming patterns and minimizing verbose coding are invaluable. I strongly believe that adopting pure functions in a C style, even in languages like Python, is one of the best approaches to coding. Coupled with well-structured data types and cohesive composition, it leads to clear and efficient code. Enjoy the read! #AI #Agriculture

64

9 Comments

I'm happy to finally share my latest article on Towards Data Science as part of my Machine Learning Basics Series. 😌 Writing about the K Nearest Neighbors Regressor turned out to be more involved than I initially thought. While the regressor itself is quite similar to the classifier, getting deep into nearest neighbor search algorithms like KD Trees and Ball Trees was a new challenge. I’ve spent a fair amount of time creating various visuals to illustrate these concepts clearly—hopefully helping you distinguish between the different types of trees without confusion. In this article, I aim to: - Introduce the K Nearest Neighbors Regressor and how it makes prediction - Explore the intricacies of KD Trees and Ball Trees for efficient nearest neighbor searches - Break down the concepts in a way that’s approachable, whether you're new to machine learning or looking to solidify your understanding It was a lot of tree-making, but I hope it pays off in making these ideas clearer. #MachineLearning #KNNRegressor #DataScience #KDTrees #BallTrees

21

3 Comments

Here is an example of a hand-drawn ERD I created. While it's true that SQLite doesn't offer a built-in option for generating ERDs, unless you use third-party tools like DB Browser or DBeaver, I believe it is essential to understand the relationships between your tables using SQL tools. An ERD is an valuable resource in this regard. In my view, the best approach—when applicable—is to manually create an ERD using a pen and paper, as I have done in the example. This practice significantly enhances your productivity as a data analyst or data scientist. Often, gathering insights about table relationships through a hands-on process, like writing, proves far more effective than relying on automated methods. Transitional techniques, such as manual ERD creation, are not necessarily a waste of time. In fact, and this is rarely discussed, they are often more efficient and provide deeper insights. #DataScientist #DataAnalyst #SQLite #ERD

1

Unbelievable - Coding questions in an assessment for 10th class AI asking students to write Simple Programs for (1) Sentiment Analysis (2) Spam Detection (3) Chatbot and (4) Handwritten Digit Recognition. This is all in one exam. Looking at the fill-in-the-blank questions above that carry 1 point, it is crazy that these programs carry 4 points each. Assessment Design remains a big challenge in the field! Source in the comment. #ai #coding #assessment #exam #test

23

12 Comments

Just read about Contextual Retrieval on Anthropic's blog—it's fascinating how much precision it adds to chunked document retrieval! The idea is simple yet powerful: adding context to individual document chunks before embedding them or creating a BM25 index. For example, instead of a standalone chunk like "The company's revenue grew by 3% over the previous quarter," the contextualized version might be: "This chunk is from an SEC filing on ACME Corp’s Q2 2023 performance; the previous quarter's revenue was $314 million. The company's revenue grew by 3% over the previous quarter." The added context frames the information, making it more relevant and easier for the retrieval model to understand the chunk’s significance within the larger document. 🔥 Key takeaways for implementation: - Automatic Context Generation: Anthropic uses Claude with prompts tailored to condense document-wide insights into chunk-specific context, ensuring high-quality retrieval without manual annotation. - Flexible Tuning: Chunk size, embedding models, and custom prompts tailored to the domain all impact performance—so testing different setups is crucial. - Reranking for Precision: Retrieving the top-150 chunks, then reranking and narrowing to the top-20, helps reduce processing load and enhances relevance, combining accuracy with efficiency. Ultimately, this approach integrates methods like contextual embeddings, BM25, and reranking for stacked performance gains, transforming standard document retrieval into a highly precise tool. Highly recommend checking out Anthropic's post if you're interested in advanced retrieval techniques! #ContextualRetrieval #AI #MachineLearning #InformationRetrieval #NaturalLanguageProcessing #NLP #Embeddings #RAG #BM25 #Claude #DataScience #DocumentSearch

24

A simple, but powerful tree for GPU poor folks like us to decide what quant to use. From Sam McLeod - https://lnkd.in/gURB5mgp

8

1 Comment

Hot Take: we use notebooks in Data Science because Python became the lingua franca and Python has significant whitespace. Saw a critique of using notebooks, so I thought I'd give my reason why they are so prominent in DS. A REPL-based development workflow works great for DS because you need to quickly look at the data, etc, to make sure that whatever you're writing is feasible. You need to check means, etc, to make sure the data makes sense. (Why am I getting the control mean is NaN?? Oh wait, I subset the data to treatment only up there... etc) Python makes the normal REPL border-line impossible to use like a REPL. You can't copy-paste most functions into it as you'd naturally write them because if there is a line break anywhere in the function, it'll fail. So, notebooks are useful because they fix Python's REPL. They let you try snippets out, re-evaluate the fixed function, and try again, etc. They inherit the initial tab position, so if you copy code in and it's all shifted over a few tabs, it's fine. The Python REPL will be like "UNEXPECTED INDENT..." So notebooks are just a REPL-driven method of development a la Lisp, but fixing the original sin of Python. What if Mr. van Rossum had accepted the word "end" into his heart? Jupiter would be our solar system's largest planet and nothing more.

15

8 Comments

For news recommendation, it has different concerns compared to other application scenarios of personalized recommender systems. First of all, the news being recommended needs to be “new” or fresh enough to remain relevant. It must have timeliness and should not quickly become outdated (e.g., recommending yesterday’s weather). Because of this, the item pool is comparatively smaller than in other application scenarios. As a result, you can put more focus to the ranking stage rather than the matching stage. Moreover, in many cases, recommending a fresher item can be more valuable than recommending one that simply aligns with the user’s preferences. This is why implementing a time-decay mechanism in the re-ranking process is worth considering. However, beyond the freshness of items, there are also challenges related to users. One of the biggest concerns is tracking user behavior. News platforms typically have a large proportion of incognito users, which makes it harder to track their behavior. Consequently, models often struggle to accurately capture their interests. That said, techniques like “FingerprintJS” can help by identifying users more reliably and durably than cookie-based methods. These techniques can provide better data to the models, yet it’s important to remember that a recommender system is more than just the model itself. There are also systemic improvements that can significantly enhance its performance. For example, encouraging users to register and log in can be a practical strategy. Once users register or log in, you gain access to their background information, which allows for better profiling. This additional side information can help the recommender system learn user preferences more effectively and provide more personalized recommendations. In summary, different application scenarios of recommender systems come with different challenges. And it’s essential to adopt a systematic approach that considers not just the models being used but also strategies to improve user engagement.

6

I had the joy of reading a recent preprint, "Questionable practices in machine learning" by Leech et al. It echoes many of the sentiments I've had for the past couple of years. I wrote some thoughts about it here: https://lnkd.in/e_HkFY9n

15

3 Comments

This seems like a neat way to train & learn.

10

Was messing around with Julia today. It's improved so much in terms of quality of life type things since I last used it for anything involved ~5 years ago. Dig the emphasis on multiple dispatch, especially in a language targeting numerical applications because standard math notation uses multiple dispatch (a lot), e.g. the "+" operator: a+b (scalars), A+B (matrices), a (scalar) + B (matrix); or multiplication: ab (scalar product), AB (matrix product), fg (function composition). So it makes translating math to code natural. (Okay, it occasionally takes the math thing a little too far, e.g. to concatenate strings you use "*" instead of "+" like most other languages in this group. Why? Well, because concatenation isn't commutative and multiplication symbols are usually used for generic noncommuntative operators. I mean... sure lol.) https://julialang.org/

8

3 Comments

Ever found yourself torn between Langchain and LlamaIndex, unsure which GenAI framework to use? What if you didn't have to choose? 🤔 I just wrote a blog post where I show how to combine the best of both frameworks to build an Agentic RAG System. This can be a great starting point for someone wanting to implement Agents for their GenAI use cases. 🤖 You can check it out here: https://lnkd.in/g8fqq5GG

15

1 Comment

Reflections on Sparse Representations in Language Models Reading about the use of sparse autoencoders in language models revealed an intriguing balance between interpretability and efficiency. Sparse representations enhance feature disentanglement, aiding in understanding model behaviors, especially in AI safety and bias detection. However, for practical applications where interpretability is secondary, sparse representations might introduce computational redundancy and inefficiency. Thus, while sparse autoencoders offer valuable insights for research and safety, more compact representations could be preferable for deployment. Balancing these aspects is crucial, potentially through adaptive approaches that optimize for both interpretability during research and efficiency in real-world applications. https://lnkd.in/egPP7dEg

1

A new (to me!) way to measure relationships between variables - I have previously worked with Pearson and Kendall but this one is new. This article explores its applications, and I can already think of past projects where it would have been valuable. Intriguing stuff! #ksaai #datanalysis #correlation

58

Self Alignment using Instruction Backtranslation Recently, it's become common practice to fine-tune LLMs on proprietary datasets or domain-specific data. However, this process requires labeled data – often referred to as ground truth – and getting humans to label this data is both time-consuming and expensive. A recent paper, 'Self Alignment by Instruction Backtranslation',  where researchers have come up with an innovative approach to generate high-quality training data. Check it out here at: https://lnkd.in/dfx5yyha In simple terms, the process starts with a model fine-tuned on a small amount of seed data. The seed model constructs training examples by generating instruction prompts for web corpus. High-quality examples from among these candidates are selected and used to fine-tune a more aligned model. The cool part? When they tried this method with LLaMA, the resulting model outperformed all other LLaMA-based models on the Alpaca leaderboard not relying on data distillation. I've put together a code implementation in my GitHub repo. Feel free to check it out and let me know what you think! #GenAI #LLMs #Llama #Fine-tuning #MachineLearning #NLP #DataScience

47

Haven't posted much recently - but wanted to share about three papers I've been deep into studying lately: auto-encoders for time series clustering, utility and enjoyment of video games over time, and matrix factorization for gene selection. I love how these seemingly unrelated concepts can come together. While all three papers have unique problem statements and methods, I've found its really helped to consider such different ideas in tandem. Since studying I've had a few succesful ideas combining learnings from each paper, coupled with even more confusion and ideas that do not work (like, at all). But, I find that has been my journey of learning mathematics and I'm trying not to be so hard on myself when I don't understanding things on the 1st, 2nd, or even N'th try. That being said, being able to combine methods used for gene selection with that of video game utility however has been a pretty great payoff! Links to the papers below.

28

9 Comments

🏠 How Zillow lost $500 million because of machine learning ↓ 🤖 Machine learning isn't a magic bullet. Zillow's iBuying blunder is a classic case study. 📅 2018-2021: Zillow used ML to predict home prices, buy properties, and sell for profit. 📊 Peak inventory: 9,680 homes 💸 The result? A $500 million loss and program closure. 🧮 What went wrong? • ML models were too optimistic, leading to property overpayment • Acquisition and renovation costs exceeded predictions • Market volatility was underestimated 📚 Key lessons: ⚠️ If best and worst cases both show "gains," it's a red flag. 🔍 ML can't fix a flawed underlying business model. 💡 AI hype doesn't guarantee success. (AI-powered toilet, anyone? 🚽) 🤔 What's your takeaway from Zillow's ML misadventure? Share below! ↓ #MachineLearning #BusinessLessons #AIHype #RealEstate #datascience

5

1 Comment

🔮RAG + KG: Powerful Combination for Translation Multi-AI Agents Large language models (LLMs) have shown great promise in machine translation, but they still struggle with contextually dependent terms, such as new or domain-specific words. This leads to inconsistencies and errors that are difficult to address. Existing solutions often depend on manual identification of such terms, which is impractical given the complexity and evolving nature of language. While RetrievalAugmented Generation (RAG) could provide some assistance, its application to translation is limited by issues such as hallucinations from information overload. =================== In this paper, researchers propose CRAT, a novel multi-agent translation framework that: 》 leverages RAG and causality enhanced self-reflection to address these challenges. This framework consists of several specialized agents: ✱ the Unknown Terms Identification agent detects unknown terms within the context, ✱ the Knowledge Graph (KG) Constructor agent extracts relevant internal knowledge about these terms and retrieves bilingual information from external sources, ✱ the Causality enhanced Judge agent validates the accuracy of the information, ✱ Translator agent incorporates the refined information into the final output. =================== This automated process allows for more precise and consistent handling of key terms during translation. Their results show that CRAT significantly improves translation accuracy, particularly in handling context-sensitive terms and emerging vocabulary. --- If your company is looking for an AI consultant to accelerate business growth, or if you are looking to build your career in AI, send me a message and I can help

4

Sometimes model training is expensive. How do you balance between reprodcibility and efficiency? 1. When you are training traditional ML models that don't take long enough to train (few mins to few hours). ✅ Retrain your model using the training pipeline rather than re-using the model trained by notebooks. Even if it gives the same results. 2. When your model takes several hours or days to train ✅ Do experimentation on smaller dataset and do full run only using pipelines. This doesn't have to be the case for deployments in lower environment. It is fine to deploy notebook generated models during development but make sure you are able to reproduce the same results on the same data using the training pipeline before you move to production.

5