Shivam Gupta

📢 Starting a thread on Fine-tuning Large Language Models (LLMs) 📢 1/n ● Large Language Models (LLMs) have revolutionised computational systems' ability to understand and generate human language. Building upon traditional language models, LLMs address limitations in handling rare words, overfitting, and capturing intricate linguistic patterns. ● Fine-tuning is a crucial process that adapts pre-trained LLMs to specific tasks or domains. This involves further training the model on a smaller, targeted dataset, refining its pre-existing knowledge and enhancing performance on specific tasks with reduced data and computational requirements. ● The fine-tuning process typically involves a seven-stage pipeline: dataset preparation, model initialisation, training environment setup, fine-tuning, evaluation and validation, deployment, and monitoring and maintenance. Each stage is essential for tailoring the LLM to meet specific requirements and ensuring optimal performance. ● There are different types of fine-tuning: unsupervised fine-tuning, supervised fine-tuning (SFT), and instruction fine-tuning via prompt engineering. The choice of method depends on the availability of labeled data and the specific task requirements. ● Parameter-Efficient Fine-Tuning (PEFT) techniques have emerged to address the computational challenges associated with fine-tuning large models. PEFT methods, such as LoRA, QLoRA, and adapters, allow for efficient fine-tuning with fewer parameters, reducing computational costs and mitigating overfitting. ● Evaluation and validation are critical for assessing the performance of fine-tuned LLMs. Metrics such as cross-entropy, perplexity, and accuracy are used to measure the model's ability to generalize and perform the target task effectively. ● Deployment of fine-tuned LLMs involves making them operational and accessible for specific applications. This includes setting up the necessary infrastructure, developing APIs, and optimizing model performance for inference. ● Monitoring and maintenance are essential to ensure the ongoing performance and reliability of deployed models. Continuous monitoring of metrics, retraining on updated data, and addressing potential biases or security vulnerabilities are crucial for maintaining model effectiveness and ethical considerations. ● Various platforms and frameworks facilitate LLM fine-tuning, such as Hugging Face, OpenAI, Google, and NVIDIA. These platforms provide tools for data preparation, model selection, hyperparameter tuning, and deployment, streamlining the fine-tuning process. ● Fine-tuning LLMs raises ethical considerations regarding bias, fairness, accountability, and transparency. Addressing these concerns is vital for responsible AI development, ensuring that models are deployed ethically and their outputs are trustworthy. Stay tuned for upcoming posts 📬 where I go through these in detail 📤 .

7

𝗟𝗹𝗮𝗺𝗮 𝟯 𝘂𝗻𝘃𝗲𝗶𝗹 𝗮 𝗳𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹 𝗰𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴 𝘁𝗿𝗮𝗱𝗲𝗼𝗳𝗳 𝘁𝗵𝗮𝘁 𝗳𝗲𝘄 𝗮𝗿𝗲 𝘀𝗲𝗲𝗶𝗻𝗴⬇️ 🦙Llama 3 just came out and to the surprise of many people, the L3-8B version can match the L2-70B one on several benchmarks. Several people also argued that the chinchilla paper (https://lnkd.in/dXaRqyea) was wrong because the llama 3 scaling is very far from chinchilla-optimal. 🤔 In my opinion, this is due to a misunderstanding of what the chinchilla paper is about and more generally of the effects of scale in deep learning. The bitter lesson (https://lnkd.in/d3sVYYgp) has become common knowledge in the field thanks to the GPT family that popularized it. We hear everywhere that "bigger models are better": Yes it's true, but in what sense? Many people have a mental model along the lines of "bigger models are more capable" meaning they would be able to achieve levels of performance that smaller models can't. ❌ This is not true with LLMs and is mainly because we train only once on tokens (most of them at least) and therefore overfitting is not possible. The 𝘳𝘦𝘢𝘭 advantage of bigger models over smaller ones is that they converge *faster*. In other words, you can reach a given level of NLL (modeling performance) with fewer training iterations. ❔ Then a natural question arises: Should I train a smaller model with more or a bigger model with fewer iterations? That's where the chinchilla paper brings a response. They say that the question they are answering is: "Given a fixed FLOPs budget, how should one trade-off model size and the number of training tokens?" (implying "to achieve the best possible NLL"). 🧐 Coming back to Llama 3, it's obvious that the Llama team is aware of this work and there is a reason why they didn't follow this recipe. My guess is that they are taking into account the inference cost in the balance (which is completely out of the scope of the chinchilla paper). When one "overtrains" an 8B model, one ends up with a worse "NLL to training compute" ratio, but also with a better "NLL to inference compute" ratio. ✅ Now if we think about it from a business perspective, it's a very powerful tradeoff because you can choose to buy performance either at a fixed cost (training) or at a variable one (inference). Hope this post helped you! #Llama3 #Meta #LLM #AI

9

3 Comments

🔍 Exploring Sparse Distributed Memory (SDM) I'm excited to share my latest blog post on Sparse Distributed Memory (SDM), an innovative model inspired by human memory! 🚀 In this detailed walkthrough, I break down the SDM script I made, providing rich mathematical and scientific insights to help you understand and implement this robust and scalable memory storage and retrieval system. If you're curious about how SDM can serve as a fascinating alternative to traditional neural networks, enhancing your machine learning projects, check out the full article here. #MachineLearning #ArtificialIntelligence #SparseDistributedMemory #SDM #DataScience #NeuralNetworks #MemoryModels #TechInnovation #Coding #AIResearch

4

What does it feel like to be an engineering manager at Amazon: Since the RTO announcement it became popular to write shit about Amazon, but it's still one of the best companies around the world. A few weeks ago, Gilad Naor shared with the 'Leading Developers' readers 3 lessons from his time as an engineering manager at Amazon. Here is the first one, read the full article here: https://lnkd.in/dHBxv2Ha 𝟭. 𝗦𝗼𝗹𝘃𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺𝘀 𝘄𝗶𝘁𝗵 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝗺𝘀 My first meeting at Amazon was baffling. I entered a room with about a dozen Software Development Managers (SDMs) and Product Managers (PMs). Everyone sat in silence and looked at their laptops or the printed papers they had before them. It was my second day at Amazon, and I picked up one of the extra sheets and started reading. After those first ten awkward minutes, everyone finished reading and the meeting began. Why do Amazon meetings always start this way? This is a specific example of how Amazon approaches problem-solving. It is the "Good Intentions Don't Matter" approach. In a famous internal video, Jeff Bezos talks about why good intentions are irrelevant. When things don't work, it does not matter what people wanted. All that matters is fixing the problems with mechanisms. For example, many busy managers come to meetings without first reading the prep material. They all intended to read the 6-Pager before the meeting. They just got sidetracked by the latest emergency. Scolding the managers won't solve the problem. A mechanism will. This is why all Amazon meetings start with a study period. The remaining meeting time is then spent on high-quality decision-making. Meta solves the same problem by creating cultural norms. I can't count the number of times that I heard a senior leader cut off the presenter: "Stop, assume that everyone here already read the slides. Let's get to the point." This is a culture-first approach. I have seen this work well in some teams, and not work at all in other teams. At Amazon, the solution is part of the company's "operating system". ------------------------- Anton here - by now everyone is familiar with the 6-pager concept, but that "operating system" thinking can be used almost anywhere. Think about something annoying that your developers do, and how can you solve it. For example, I remember being annoyed that people don’t update the Jira tickets. Finally, after requesting multiple times, now we just take 20 seconds from the standup to update them.

56

8 Comments

Interesting to read actual evidence of what one could expect as an inevitability of scale!

21

3 Comments

A curated collection of the papers and models consisting of 5 selections across 10 categories... The 2025 AI Engineer Reading List (January 2025) "We picked 50 paper/models/blogs across 10 fields in AI Eng: LLMs, Benchmarks, Prompting, RAG, Agents, CodeGen, Vision, Voice, Diffusion, Finetuning... The picks from all the speakers in our Best of 2024 series catches you up for 2024, but since we wrote about running Paper Clubs, we’ve been asked many times for a reading list to recommend for those starting from scratch at work or with friends. We started with the 2023 a16z Canon, but it needs a 2025 update and a practical focus. Here we curate “required reads” for the AI engineer. Our design goals are: - Pick 50 papers , optional extras. - Tell you why this paper matters instead of just name drop - Be very practical for the AI Engineer We ended up picking 5 “papers” per section for: Section 1: Frontier LLMs Section 2: Benchmarks and Evals Section 3: Prompting, ICL & Chain of Thought Section 4: Retrieval Augmented Generation Section 5: Agents Section 6: Code Generation Section 7: Vision Section 8: Voice Section 9: Image/Video Diffusion Section 10: Finetuning https://lnkd.in/erxnwUpB

3

Responsible engineers should avoid copying code directly from online sources, including LLMs or platforms like Stack Overflow, without fully reviewing, understanding, and adapting it for safe integration into their codebase. Thoughtful assessment is essential to ensure code behaves as expected and meets project requirements. That said, I believe LLMs can be a fantastic resource to accelerate learning and deepen understanding across a range of topics—as long as they’re used with careful consideration. #LLM #engineering #ChatGPT

10

1 Comment

  🚀 Today, we are thrilled to share that Amazon Q is now generally available. ◼ Amazon Q is the most capable generative AI-powered assistant for accelerating software development and leveraging companies’ internal data. It eliminates tedious work for developers and employees across organizations. ◼ Q helps to test, debug and write code, and has the highest reported code acceptance rates in the industry, for assistants that perform multi-line code suggestions. BT Group recently reported they accepted 37% of Q’s code suggestions and National Australia Bank reported a 50% acceptance rate. ◼ Amazon Q Developer Agents can autonomously perform a range of tasks—everything from implementing features, documenting, and refactoring code, to performing software upgrades. Developers can ask Amazon Q to implement an application feature, and the agent will analyze their existing application code and generate a step-by-step implementation plan. ◼ Amazon Q's capabilities extend beyond coding. It also allows employees to easily get insight from their company's internal data that is spread across multiple documents, systems, and applications. Q connects all these siloed sources and can answer questions, provide summaries, analyze trends, and generate content. ◼ We're also introducing Q Apps, a powerful new way for anyone to create generative AI apps based on their organization's data—no prior coding experience required. Simply describe the app you need in natural language, and Q Apps will build it for you. This unlocks endless possibilities for teams to automate workflows and daily tasks. Customers across industries are using Amazon Q to transform the way they work. I'm incredibly excited to see what Q can do for you. https://lnkd.in/gd2cBVCi

5,129

87 Comments

Controversial take 🚀 Large Language Models (LLMs) are akin to monoliths similar to OS packaging in early days where core OS and applications were all bundled together. A lot of such application packages were never used. Just like unused OS applications and packages, a lot of knowledge in todays model may not be useful for many specialized use-cases particularly in B2B, which is also evident in need for fine-tuning or ICL. In the applications world, as containerization took center stage, it became flexible to add or remove packages needed for a system. Docker emerged as the go-to for this purpose, allowing engineers to build enterprise systems by excluding unnecessary packages besides using a relevant starting point, further adding their own packages. For example, users can begin with a base OS container and customize it by adding or removing packages. Over the last year, with LoRA Adapters(and similar variants) have been goto methods for bringing in your knowledge bits to create customized models. Other techniques such as modelmerge help merge multiple models into a single one. These are interesting directionally, however it needs to become even simpler similar to layers in containers, where knowledge bits can be added/excluded seemlessly. Looking ahead, the future of LLM seems like would mirror containers. Starting with a pre-trained base model tailored to your use-case, you can then add layers your knowledge bits on top. "Language Model as a container" offers the simplicity of packaging your custom knowledge bits. #LLM #huggingface #SLM

16

6 Comments

Architecture Distributed Systems These courses are really going to benefit Software Engineers.

4

The skill of crafting simple, clear, and easy-to-understand technical documentation is truly invaluable!

9

The whole field of prompt engineering can be reduced to these 6 techniques I use almost daily when using ChatGPT (or other LLMs). Here they are ↓ #1. Few shot prompting Add to your prompt 2 or 3 high-quality demonstrations on the target task, each consisting of both input and desired output. The LLM will better understand your intention and what kind of answers you expect based on concrete examples. #2. Self-consistency sampling Sample multiple outputs with "temperature > 0" and select the best one out of these candidates. How to pick the best candidate? It will vary from task to task, but here are 2 primary scenarios ↓ 1. Some tasks are easy to validate, such as programming questions. In this case, you can write unit tests to verify the correctness of the generated code. 2. For more complicated tasks, you can manually inspect them or use another LLM (or another specialized model) to rank them. #3. Chain-of-Thought (CoT) You want to force the LLM to explain its thought process, which eventually leads to the final answer, step by step. This will help the LLM to reason complex tasks better. You want to use CoT for complicated reasoning tasks + large models (e.g., with more than 50B parameters). Simple tasks only benefit slightly from CoT prompting. Here are a few methods to achieve CoT: - provide a list of bullet points with all the steps you expect the LLM to take - use "Few shot prompt" to teach the LLM to think in steps ... or my favorite: use sentences such as "Let's think step by step." #4. Augmented Prompts The LLM's internal knowledge is limited to the data it was trained on. Also, often, it forgets specific details of older training datasets. The most common use case is Retrieval-Augmented Generation (RAG). That is why using the LLM as a reasoning engine is beneficial to parse and extract information from a reliable source of information given as context in the prompt. 𝘞𝘩𝘺? - avoid retraining the model on new data - avoid hallucinating - access to references on the source #5. A single responsibility per prompt Quite self-explanatory. It is similar to the DRY principle in SWE. Having only x1 task/prompt is good practice to avoid confusing the LLM. If you have more complex tasks, split them into granular ones and merge the results later in a different prompt. #6. Be as explicit as possible The LLM cannot read your mind. To maximize the probability of getting precisely what you want, you can imagine the LLM as a 7-year-old to whom you must explain everything step-by-step to be sure he understood. . The truth is that prompt engineering is quite intuitive, and we don't have to overthink it too much. What would you add to this list? #machinelearning #mlops #datascience

194

12 Comments

Most ML engineers with recommender systems background must have read this awesome piece of writeup published during the Netflix challenge. Advances in Collborative Filtering https://lnkd.in/gmD-FZ_F I had even implemented every model from scratch in Python a few years back. I was revisiting this write-up recently. Two important observations stood out for me: 1. Anyone looking to learn how to build a "model" should definitely read this paper. The progression and intuition combined while building a recommendation model from crude to advanced has been superbly dealt with. Moreover the intuition is transferrable to any kind of models in general and not only recommenders systems. 2. We would not have been lucky to see such a nice writeup if RNN, LSTM or Transformers were popular during that time, although RNN existed but it was computationally expensive to train. Note: This paper do not teach on how to "design" a recommendation system but how to approach mathematical "modelling" in general. #machinelearning #softwareengineering #recommendersystems

29

A Day at the Apache Kafka 🚀 Today, I had the pleasure of attending an in-person Apache Kafka hosted by CRED in Bangalore. Here’s my feedback on this incredible event: 📍 Venue: Cred Aero, Bengaluru 🔹 Agenda Highlights: - @Vikas Sharma , CRED : Talk: Running Kafka Optimally in Production: Practical Usage Feedback: Vikas shared practical insights on optimizing Kafka in production environments, with valuable strategies for resource management. - Udayaram Kammara , Bazaarvoice : Talk: How Bazaarvoice Utilizes Kafka for Both Stream and Batch Processing Scenarios Feedback: Udayaram's session on Kafka's use at Bazaarvoice was enlightening, covering real-world use cases and challenges. - Diptiman Raichaudhuri , Confluent : Talk:Stream Processing for the Non-JVM World: pyflink and kafka-python Feedback: Diptiman's explanation of Apache Flink's architecture and the demonstration of kafka-python and pyflink were fascinating and beneficial for non-JVM users. What I Learned: - Effective strategies for running Kafka in production. - Real-world use cases and challenges in stream and batch processing. - Leveraging pyflink and kafka-python for stream processing. Overall, this meetup was a treasure trove of knowledge and inspiration. The sessions were well-organized, and the speakers were exceptionally knowledgeable. I’m excited to implement these learnings in my projects and continue exploring the potential of Apache Kafka and related technologies. 🙌 #ApacheKafka #KafkaMeetup #CRED #Bangalore #EventStreaming #StreamProcessing #TechCommunity #Networking #Python #Flink #DataPlatforms

24

2 Comments

Do LLMs have any immediate applications in the field of Robotics? The above question was loosely based on a query asked by a startup founder in an event, based on my colleague’s work on a recent LLMs integration. (Siddharth) Come to think of it, robotics about 10 years ago wouldn’t have been able to accommodate such computational expenses. Imagine running a LLM model on a Raspberry Pi zero and not expecting it to blow up! :) While implementing a LLMs could be a fun side project, what question warrants a deeper insight would be whether it is of absolute necessity? With Nvidia doubling down on the highly powerful Jetson devices, and Blackwell chips, the hardware is expansive and available, but it still begs to ask the same question again and again - is it necessary? Like every other industrial trend, once must deeply analyse where a novel idea would fit in, and calculate against its effort and yield. Let’s explore a few options wherein it could be absolute importance to use LLMs in context of Robotics: 1. Support agents - Social robotics has a huge potential of closing down on the weary technical support based on an exhaustive list of solved and interpreted problems. Distilling technical documents and datasheets into a message parsing model is an important win to do effective knowledge transfer. 2. Assistive bots - In the field of soft robotics and prosthetic arms/exoskeleton, a voice controlled NLP agent would be great help in helping administer orchestrated actions for differently abled individuals who have clear motive/task list. 3. Surveillance/Monitoring bots - Given the nature of preordained message intents, it is a perfect candidate for having a lightweight message-action model for surveying specific sites. Looking at the magnanimity of the manufacturing plants, this would be a great boom. 4. Surgical bots - Expert assisted robotic surgeries are a natural extension for aligning non-conventional anomalies that are very unique to the nature of the surgery. Since precision and targeted algorithms are of utmost importance, one cannot make actions based on learning based approaches effectively from general archetypes. Are there more applications that cross your mind marrying LLMs and Robotics?

38

6 Comments

Sharing my thoughts on How Waymo (formerly known as the Google Self-Driving Car Project) is Pioneering Autonomous Driving with AI. Waymo has become a trailblazer in self-driving technology, and at the core of it all is AI. Leveraging advanced machine learning and deep neural networks, Waymo’s AI-powered system continuously improves through real-world data, making autonomous vehicles smarter, safer, and more reliable. Perception: AI allows Waymo’s cars to "see" and understand their surroundings by processing data from sensors like cameras, LIDAR, and radar. This helps detect pedestrians, other vehicles, and objects in real time. Prediction: The AI system predicts the behavior of everyone around the vehicle—whether it’s cars changing lanes or pedestrians crossing the street—ensuring smooth and safe driving decisions. Planning & Control: AI makes split-second decisions about accelerating, braking, and steering based on the analysis of complex traffic patterns, enabling Waymo vehicles to safely navigate roads. With millions of autonomous miles driven, Waymo’s AI is constantly learning and improving, helping create the future of safer and smarter transportation. The road ahead is AI-driven! www.rxsmeta.com #AI #AutonomousVehicles #Waymo #SelfDrivingCars #MachineLearning #SmartMobility #TransportationTech #FutureOfTransport #AIinAction #Innovation

2

100 𝐌𝐨𝐬𝐭 𝐈𝐧𝐟𝐥𝐮𝐞𝐧𝐭𝐢𝐚𝐥 𝐏𝐞𝐨𝐩𝐥𝐞 𝐢𝐧 𝐀𝐈 (𝐓𝐢𝐦𝐞 𝐌𝐚𝐠𝐚𝐳𝐢𝐧𝐞) Surprises in this list [1] Elon Musk missing in the list [2] Anil Kapoor in the list. #timemagazine #ai #generativeai #nlproc #llms #top100 #influential #people

32

3 Comments