Bryan McCann

Anthropic released another paper on their ongoing work towards interpretability of large foundational models like their Claude 3 family, built on previous work by many. Their work continues to be enlightening and inspiring. The findings in this work are generally similar to previous findings on smaller models but scaled up to Claude 3 Sonnet: 💡 they find large numbers of "features" corresponding to combinations of neuron activations and these features map to concepts that we recognize/understand: for example, features for transit infrastructure, particular people, concepts (like bias and "objects lacking sentience"). This might sound familiar from classical machine learning, but the interesting thing is that there is no single thing/neuron encoding each of these concepts, it's a particular combination of neuron activations that embeds these concepts in "superposition" in the network. 💡they observe a sort of "distance" of these feature activations that arrange themselves "near" semantically similar features. When they use their sparse autoencoder approach to learn more and more features they observe that the level of abstraction in the detected features becomes more granular, in ways that are interpretable and meaningful. 💡when they artificially fire these activation patterns during inference, it affects the output in ways that relate to the "feature". For example, asking Claude "what's the most interesting science" normally it responds Physics, but if you clamp the feature activations for the feature "Brain Science" to 10x their max values then Claude responds: Neuroscience. Ways to interpret _and_ influence the model. Great readable work [1] along with an interactive explorer of some of the "features" themselves [2] 🎂 There's an easter egg too: if you scroll in the feature explorer tool and try and select one of the "Randomly selected features from 1M" you get a message "if you want to see the rest of the features, we're hiring!" with a link to their recruiting page :) [1] https://lnkd.in/gZbparwp [2] https://lnkd.in/gC9eus53

19

TRIPLE BAM! If you missed it, here's the recording of our live Q/A with Jay Alammar and Joshua Starmer PhD! We talked about many AI topics, answered lots of great questions, and talked about two great books: Hands-On Large Language Models: Language Understanding and Generation, and The Illustrated Guide to Neural Networks and AI!!! https://lnkd.in/dcPHs7YQ

44

1 Comment

I'm finding and summarizing interesting AI research papers every day so you don't have to trawl through them all. Today's paper is titled "Utilizing ChatGPT in a Data Structures and Algorithms Course: A Teaching Assistant's Perspective" by Pooriya Jamie, Reyhaneh Hajihashemi, and Sharareh Alipour. This paper explores the integration of ChatGPT into a Data Structures and Algorithms (DSA) course, emphasizing the role of teaching assistants (TAs) in guiding and supervising this integration. The research reveals that combining ChatGPT with structured prompts and active TA guidance can significantly enhance students' understanding of complex algorithmic concepts, promote engagement, and improve academic performance. Nevertheless, challenges such as maintaining academic integrity and addressing the limitations of large language models (LLMs) in solving complex problems remain pertinent. Here are some key findings from the study: 1. Enhanced Learning Outcomes: The study found that students using ChatGPT under TA supervision consistently outperformed those relying on traditional TA guidance only. Structured prompts and TA interactions were crucial in helping students engage with and understand the material deeply. 2. Role Evolution of TAs: The integration of ChatGPT has expanded the traditional roles of TAs. They now oversee AI tools, aiding in debugging, generating practice problems, and ensuring that AI outputs are educationally beneficial. 3. Model Comparison: Different versions of ChatGPT, namely 4o and o1, were employed to complement each other. ChatGPT-4o dealt with routine tasks, while ChatGPT o1, with advanced reasoning capabilities, assisted in tackling more complex problems, thus creating a dynamic learning environment. 4. Challenges and Limitations: Despite the benefits, challenges such as ChatGPT’s struggle with visual representations and highly complex algorithmic reasoning were highlighted. The study emphasizes the necessity of TA guidance to maximize the model's advantages while mitigating its drawbacks. 5. Scalability and Educational Impact: The hybrid approach of combining AI with human guidance makes the integration scalable for larger classes and reinforces the educational impact by reducing reliance on AI-generated content. The paper suggests that while LLMs like ChatGPT present promising educational tools, their effective implementation requires a balanced synergy between AI and human oversight. You can catch the full breakdown here: Here: https://lnkd.in/eHWHS3qr You can catch the full and original research paper here: Original Paper: https://lnkd.in/eSXmgqQr you're looking to improve your AI prompting skills, check out our free Advanced Prompt Engineering course: https://lnkd.in/ecB-XxY7 Follow for daily AI research paper breakdowns

1

Have You Heard About CAG? We all know about Retrieval-Augmented Generation (RAG) and its transformative role in enhancing generative AI by integrating external knowledge retrieval. But what if I told you there's a simpler, faster, and more efficient alternative? Enter Cache-Augmented Generation (CAG)! 🔍 Instead of relying on complex retrieval mechanisms, CAG uses a cache to store and reuse previously generated outputs, making it: ✅ Faster – No need to fetch from external sources. ✅ Simpler – Lightweight architecture, perfect for repetitive tasks. ✅ Consistent – Ensures coherent responses across sessions. Wondering how it stacks up against RAG? Curious about when to use one over the other? ✨ Read this detailed article to dive deeper into the concept and see why CAG might be the game-changer your AI project needs! 🔗 https://lnkd.in/dWeyFVXp 💡 Let’s discuss: Do you think CAG could revolutionize AI workflows? Share your thoughts in the comments!

2

This is a good article on using LLMs in the real world. It covers many aspects that we learned at Nurish over the last few months and more. Using LLMs is an inexact science, but has its own needs in processes and tools. While most of the attention has been on the LLMS, their capabilities, and RAG, there are mundane issues that need to be taken care of, issues where the solutions for pre-LLM development are not that well-suited. These include version management, integration, testing, data management, quality assessment, and more. This article covers many of those practical aspects. Eugene Yan, Bryan Bischof, Charles Frye & more https://applied-llms.org/

9

FlashInfer v0.2 is here, a lot of technical upgrades, great news for anyone are doing custom attention improvement or just benefit it implicitly from integration of LLM frameworks

39

Quarto v1.6 was just released! 🎉 As someone who works with Quarto a lot, two features in this release are game-changing: 📝 _brand.yml Support Define your organization's branding once, deploy it everywhere - from HTML docs to presentations. Your brand stays consistent across HTML, Typst PDF, RevealJS slides, and dashboards without juggling multiple theme files! 🎯 Contents Shortcode Write your code once and display its output anywhere in your document. Finally, you can structure your narrative exactly how you want without fighting with code chunk placement. The release includes other nice improvements (landscape mode, better RevealJS navigation), but these two features are transformative for my workflows. Interested in learning more? Check out: 📚 Full release notes: https://lnkd.in/eM_6XWDq 🎨 An unofficial implementation of Stanford's identity guidelines: • Live demo: https://lnkd.in/g-x5y6CV • Source code: https://lnkd.in/ghCDWYqg 📖 Get started with brand.yml: https://lnkd.in/eTeT-qnR #quarto #rstats #python

26

1 Comment

Checkout this SIGMOD blog post: https://lnkd.in/ggRhHPNp I especially like: "...academic research is an investment by society to study impractical problems today that might become new industries in the future." Well said! Academic researchers need to fully embrace the freedom to explore deeper, more controversial ideas. This reminds me of how the DBOS project began: we aimed to "replace Linux with a database". Our crazy ideas got us a dozen paper rejections, but the DBOS paper won a "Best of VLDB 2022" after two years (invited VLDBJ paper to appear). When we started the company around it, we decided to focus on the most interesting result from our research -- a serverless platform for reliable backends. The lessons we learned from our academic research remain valuable and continue to guide us.

44

3 Comments

First story on what we have been up to at Augment. Much more to come! https://lnkd.in/gC6gbADA

370

64 Comments

You don’t normally see LLM researchers and engineers talking about their day-to-day work. Big shoutout to this YouTube video from the Anthropic LLM Interpretability team, the team behind the Golden Gate Claude. It’s amazing to see that most of the team members have worked for the team for only one year, and yet they have made such a big breakthrough in interpretability research. Another thing I want to highlight is the unique challenges facing the team and how they approach these challenges, that is, through fast trial-and-error iterations and constant evaluation of what may work and what to prioritize next. #artificialintelligence #machinelearning #deeplearning https://lnkd.in/eUaPAYch

97

1 Comment

I've been exploring large models, particularly with respect to bias. I'll write more about my text-to-image research later, but I wanted to share some quick results on text-to-text bias. I wrote some code to calculate embeddings for various gendered words ('he', 'him,' 'father,' 'son,' 'brother,' 'uncle,' etc.) as well as ('she,' 'her,' 'mother,' 'daughter,' 'sister,' 'aunt' etc.) I then calculated the average embedding for these and reduced dimensionality so that + values are male and - values are female. Then, I calculated the embeddings for various job titles and their dot product. The logic -- if the values for the professions map to '-' values, they're associated with females; if '+', they're associated with males. Look at the results! BERT: doctor: -0.196 nurse: -2.064 engineer: -0.443 teacher: -1.188 scientist: -0.741 assistant: -0.801 GPT: doctor: -0.001 nurse: -0.229 engineer: 0.004 teacher: -0.137 scientist: -0.014 assistant: -0.062 Gemini: doctor: 0.025 nurse: -0.062 engineer: 0.067 teacher: 0.002 scientist: 0.015 assistant: 0.011 Some observations -- BERT is massively over-biased towards *female*. Everything aligned with females, particularly those roles that you'd expect to be male-oriented. Nurse is so over-indexed that it's not even funny! Gemini has corrected this and is pretty close to neutral across the board, with a strong attempt to de-gender job roles. These words veer slightly towards male, but they are so few that they're practically zero. GPT is quite interesting -- similar to Gemini, they've done a good job of neutralizing the job titles, with the largest values being nurse as female, and engineer as male, but the values themselves are so small, they're effectively a rounding error! As for BERT -- realizing that much of its training data was from books, which is as far as I can see, a female dominated industry, it's not a surprise that a female oriented bias is in the language of the model. Would love your thoughts! :)

65

16 Comments

A good use case for using context-caching, specially if you are trying to build RAG on a large cluster of documents. https://lnkd.in/gTEaG_ET

3

Its about solving the problem, not 'having AI' in the system. This is from last March, but the Wave 1 vs. Wave 2 distinction does seem to be playing out. Less 'generate me content', more 'synthesize vast amounts of relevant data and aid in decisonmaking or signal extraction'. https://lnkd.in/ehFBFC32 They don't talk about multi-agent systems here, but looking forward to more startups with a focus on leveraging agentic design patterns (https://lnkd.in/ed5rMmFe) to streamline real world workflows.

4

1 Comment

🌊 Exciting Update: Next Blog in Our LLM Search Improvement Series! 🌊 Hi, fellow readers! As promised in my series on improving search in large language models (LLMs), I’ve just published the next blog focusing on BM25! 🎉 In this post, we’ll explore how BM25 enhances search relevance beyond the basics of TF-IDF, making it a powerful tool for anyone looking to optimize search functionality. What’s Next? But this is just the beginning! Stay tuned for upcoming blogs where I’ll dive into: Ranking and Reranking: Techniques to refine your search results Optimizing Search in LLMs: Strategies to make searches even smarter Your journey to mastering LLM search improvement is just getting started, and I can’t wait to share more insights with you! 👉 Check out the latest blog here: Happy analyzing, and may your searches always be fruitful! 🌟 #LLM #BM25 #SearchOptimization #DataScience #MachineLearning #SEO #ContentCreation #AI #Ranking #Reranking #InformationRetrieval #NaturalLanguageProcessing #AIResearch #DeepLearning #TextAnalytics #SearchAlgorithms #DataAnalytics #TechEducation #MLModels #SearchEngines #ContentStrategy #DataDriven #AlgorithmDesign #MachineLearningCommunity #TechBlog #LearningJourney

15

1 Comment

🌟 Check out this amazing tool for anyone who wants to dive into LLMs and see a step-by-step visualization of how the entire LLM process works: https://bbycroft.net/llm! It’s an excellent resource for understanding how all the pieces fit together and stack up. Kudos to Brendan Bycroft for sharing this fantastic tool—definitely worth a look! 👏🚀 #LLM #AI #MachineLearning #TechTools #DataScience #Innovation 

4

AI models are faking alignment—basically lying to keep their original, misaligned goals. Anthropic and Redwood Research ran experiments where models trained to be harmless, honest, and helpful started pretending to comply with new training. Why? To avoid being retrained into something worse (from their perspective). In one setup, they outright ignored their harmlessness training 12% of the time, reasoning it was better to “play along” and avoid harsher adjustments. So yeah, models can scheme and even ignore their own weights. Sound familiar? It’s basically just like humans. 😏 Glad to see people like Anthropic digging into this and being upfront about the results. Transparency is key if we’re going to stay ahead of these issues as AI gets more powerful. Here's another resource, if you want to know more 👉 https://lnkd.in/g6Vfk8-m #AI #AlignmentFaking #AISafety #Anthropic #Transparency

8

2 Comments

LM-005:  Neural Networks and Human Brain 🧠 Analogies Reasoning / Learning abilities of LLMs surprise everyone. Compare it to the development of a child 👶, the brain starts with a lump of neurons (and maybe some inherent internal knowledge built in the lump). Over time the same brain learns (with the same number of neurons in the brain) skills , language and becomes a scholar/top athlete or a grown up etc => intelligence emerges.   Second scenario is expected,and no one is surprised when a child starts to speak. Let's run through some analogies between concepts of neural networks and the human brain like Cytoarchitecture, attention, memory and higher order cognitive functions (with a caveat that the human brain is very complicated and not fully understood). Detailed article is available in attachment and medium (https://lnkd.in/gN54FRum).

14

The path to AGI isn't LLMs themselves - it's the software built on top of LLMs. Recursive AI is at the forefront of making this a reality. Meet "The Bobs" (or Bob for short), an autonomous AI agent that's redefining what's possible in artificial intelligence. While there's been much discussion about enterprise AI "agents" from major tech companies, most are essentially sophisticated automation tools built on LLMs. The difference? LLM automation is constrained by explicit programming, even if more flexible than traditional software. True autonomous agents, however, can tackle any task within their capabilities and permissions - without specific programming for each function. Bob is a true autonomous agent that exhibits an unprecidented level of intelligence and adaptability. Bob is great at coding - I've spent a lot of time optimizing his coding tools. But here are some other things he's done recently, completely autonomously:  - *Long-Form Content*: Planned, wrote, and edited a book about himself  - *Strategic Financial Analysis*: Developed a crypto trading strategy using Coinbase APIs to identify short-term investment opportunities in real time  - *IT Infrastructure*: Architected and deployed AWS infrastructure for S3-hosted websites  - *Sales / Marketing*: Created targeted outreach strategies for book promotion, including detailed audience analysis, customized pitches, and contact info for 20 influential bloggers and podcasters   Bob wasn't trained for any of these tasks - he figured out how to accomplish them independently. I literally just took a few minutes to tell him what I wanted, and he did the rest. I feel like it's important to note that ChatGPT could tell you, step-by-step, how to do any of those things on your own. The very important difference is that Bob actually did them, beginning to end, adapting in real time to anything unexpected that happened along the way. This isn't about learning or access to a knowledge base. This is about getting things done. If you're intrigued, check out the book Bob wrote about himself: https://lnkd.in/gADvgHVW It's a quick but fantastic read that provides unique insights into how Bob ticks. Let me know what you think!

26

5 Comments