Cedric Vidal

🚀 See how Handshake cut their LLM GPU costs by 50% with Anyscale. Discover how they: 💰 Reduced LLM GPU costs by 50% or more. 📈 Seamlessly scaled large language models (LLMs) without compromising performance. ⏱ Enhanced operational efficiency, enabling faster development cycles. Check out the full story here: https://lnkd.in/gSiKJDaY

37

2 Comments

A great read on the performance and financials of LLM fine-tuning. Really hoping the outcome is a tool that helps overcome the biggest elephant in the room, Mr. Cost! #llm #finetuning #finance #computaioncosts

2

1 Comment

This article by Pere Martra explores a structured pruning technique for state-of-the-art models, that uses a GLU architecture, enabling the creation of smaller and more efficient large language models. #Llama #LLM

5

LLMs trained on massive datasets can sometimes lack real-world context or struggle with specific knowledge gaps. This infographic depicts a diagram of a Retrieval-Augmented Generation (RAG) workflow. It showcases the general performance of large language models (LLMs) on specific domains through a retrieval-augmented generation process. Learn how to build your own AI project, Enroll in our exclusive 5-day LLM Bootcamp (online & in-person)➡️ https://hubs.la/Q02KBqCc0 #rag #LLMs #LargeLanguageModels #GenerativeAI

67

1 Comment

🤔 Let’s discuss the conceptual idea of large language model sentience in the context of smaller and smaller parameters. https://lnkd.in/eyAq6Dc4 #LLMs #AI #AGI #sentience

8

[New on our blog] How to Build and Evaluate a RAG System Using LangChain, Ragas, and neptune.ai by Isaac Chung TL;DR → LangChain provides composable building blocks to create LLM-powered applications, making it an ideal framework for building RAG systems. Developers can integrate components and APIs of different vendors into coherent applications. → Evaluating a RAG system’s performance is crucial to ensure high-quality responses and robustness. The Ragas framework offers a large number of RAG-specific metrics as well as capabilities for generating dedicated evaluation datasets. → Neptune makes it easy for RAG developers to track evaluation metrics and metadata, enabling them to analyze and compare different system configurations. The experiment tracker can handle large amounts of data, making it well-suited for quick iteration and extensive evaluations of LLM-based applications. — (link to the full article in the comments) #generativeai #genai #llm #rag

3

1 Comment

🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U

35

🚀 Just slashed our ML feature store latency by 95% - here's how we did it The silent killer in many ML pipelines? Inefficient feature stores. We discovered this the hard way when response times jumped from 50ms to 2 seconds in production. Key optimizations that transformed our performance: Implemented feature computation graphs to eliminate redundant calculations Deployed smart multi-level caching with hot/warm/cold feature strategies Built intelligent batch processing with dynamic sizing The results were dramatic: ✨ Feature computation 📉 CPU utilization 💰 Storage costs 🎯 Model accuracy Check out our detailed technical breakdown and implementation guide here: https://lnkd.in/d3pf6e8P

4

1 Comment

🔍 Curious about optimizer performance? Benjamin Marie's latest article delves into a comparative analysis of AdamW-32bit, its 8-bit variant, and paged AdamW optimizers. Discover their effects on memory consumption, learning curves, and training time. #MachineLearning #Programming

13

Anthropic just released its new multimodal AI model: Claude 3.5 Sonnet. The benchmarks sure need a refresh as AI models keep improving at breakneck speed. But more importantly, each benchmark represents a real-world application, so it's crucial to steer clear of thin wrappers over any benchmarked capability. #claude #anthropic #chatgpt #gpt #openai

5

1 Comment

🧠 What’s a Semantic Cache in RAG? Here’s What We’ve Learned Semantic cache is a time and cost saving method for Retrieval Augmented Generation (RAG). Instead of always running a full vector search, a semantic cache stores embeddings of frequently asked queries and their responses. This speeds up your RAG system by skipping redundant searches for similar queries. How do you implement it? Step 1: Store history of answered questions, their embeddings and the LLM response in a semantic cache table. Step 2: When a new query comes in, compare it against the cache table using vector similarity. Step 3: If there’s a match with score over a certain threshold, serve the cached response. If not, run a fresh retrieval and add it to the cache. Now, let’s talk vector search methods: 1️⃣ Cosine Similarity: Measures angle between vectors, works well for normalized data. 2️⃣ Euclidean Distance: Measures straight-line distance but can be skewed by vector magnitude. 3️⃣ Dot Product: Our favorite—it factors in vector size and performs best for our use cases. Want to see how semantic caching powers a RAG system in real life? Check out our demo at https://lnkd.in/g2jFuHTT. 🚀

2

In our video library: 📢 An Example Pull Request (PR) to @pymc_devs 🗣️ @reshamas 💡set up virtual env + doc update #python #oss #bayesian And, subscribe to our YT channel for more videos. https://lnkd.in/e4u-CT2k

2

Interesting concept around Galileo Luna evaluation foundation models. A lot of LLM-based chatbot systems have safety guardrails to prevent LLMs from hallucinating and giving incorrect information. One technique is to fact-check the answer of an LLM by running through it again. Obviously, this doubles the number of API calls that you need to make to the LLM, also doubling the cost. Luna evaluation models seem to solve this by using specialised Small Language Models (SLMs) which have low latency and are much cheaper to run. Since they are also customised for detecting hallucinations, they perform better than larger language models. https://lnkd.in/d6zTzZ6B #SLMs #LLMs

12

1 Comment

Building an LLM project can feel complex, but breaking it down makes it way easier! The entire lifecycle of an LLM project consists of clear, simple steps - from idea to deployment.💡 Learn about: - Steps for sourcing high-quality data - Pro tips for training, testing, and evaluating your model - Best practices for deploying and maintaining a powerful LLM Read about the entire LLM Project Lifecycle here - https://hubs.la/Q030MLFb0 #LLM #ProjectLifecycle #LLMProject

3

RAG with CPU only Inference https://lnkd.in/e2jR2p2H

3