Jin Huang

TensorFlow vs. PyTorch: Finding the best Deep Learning framework for your AI journey 🚀 Which framework will boost your career goals? Whether you’re excited about diving into cutting-edge research with PyTorch or eager to create scalable AI models with TensorFlow, each option has its own perks! From how easy they are to use and the support you can get from the community to their flexibility and scalability, understanding these differences can really set you up for success in the fast-paced world of AI. I think Juliette Carreiro's article for @ironhack is a fantastic resource for anyone starting their journey in Data Science and Machine Learning! On another note, I’m still looking for a German-speaking Data Manager to join a leading company on an exciting automotive project in Frankfurt. If this sounds like a good fit for you or someone you know, please reach out here or at jelgueta@redsofa.com. Happy to share more details! #DeepLearning #AI #DataScience #MachineLearning #TensorFlow #PyTorch #CareerInTech #Hiring #ironhack #redsofa #datamanager

12

1 Comment

📢 New Breakthrough in Time Series Forecasting: TSMamba Model 📢 Researchers from Tsinghua University and several esteemed institutions have introduced a novel time series foundation model named TSMamba. This model leverages the Mamba architecture to tackle the complexity of time series forecasting across diverse domains. Unlike traditional models reliant on quadratic complexity architectures like Transformers, TSMamba offers a linear complexity alternative, enhancing both efficiency and scalability. Key Highlights: - Bidirectional Encoding: TSMamba incorporates both forward and backward Mamba encoders to capture time dependencies effectively. - Two-Stage Transfer Learning: To reduce the need for extensive data and computational resources, TSMamba utilizes pretrained Mamba models, allowing for effective adaptation to specific time series tasks. - Cross-Channel Attention: A compressed attention module captures dependencies across channels, enhancing predictive accuracy on multivariate datasets. Experiments show that TSMamba performs on par or even better than current state-of-the-art models, especially in scenarios with limited training data, such as zero-shot and full-shot forecasting. This approach could revolutionize forecasting across industries, from finance to healthcare, by offering more data-efficient and adaptable solutions. 📚 Read more about the research here: https://lnkd.in/d_k3sfgv #TimeSeries #Forecasting #MachineLearning

52

The BayLearn 2024 abstract submission deadline has been extended to Aug 5, 2024. If you are working on cutting-edge ML research, please consider submitting an abstract. BayLearn 2024 will be an in-person event, held on Oct 10, 2024: https://baylearn.org This is a great opportunity for student researchers to showcase their latest research work and get good feedback. Submissions are non-archival and the review process is quite informal. #machinelearning #research

69

An amazing article on using  Low Rank Adaptation (LoRA) and QLoRA (Quantized Low-Rank Adaptation)  for fine-tuning LLMs. https://lnkd.in/gHu7hfAE #lora #qlora #machinelearning #llms

16

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

180

2 Comments

I remember when this all started. Faris - invited me to catch up at our favorite spot around Blok M. I had no idea he’d just wrapped up a procedure at Pondok Indah Hospital! (He really knows how to add drama to our catch-ups.) We’d been on separate paths and projects since "the winding down project", but there he was, reminding me of our dream to ride this AI wave together again. It was a plan we’d hatched with Sahal Zain & Gurnoor Dhillon when the world of AI was still mostly theoretical and jargon-heavy. Somehow, the conversation continued with Gurnoor Dhillon while he was waiting for his meeting with someone who would eventually become one of CalvinBall’s key early adopters. It felt like everything was coming together, like a series of seemingly random dots connecting to form a bigger picture. Seeing how far we’ve come as a team at CalvinBall Technologies is truly exciting. What started as theoretical discussions has now evolved into practical Gen AI solutions that are solving everyday problems and hold endless possibilities to shape the industry. 👨‍🚀 👩‍🚀 Here’s to pushing boundaries and achieving new milestones together! Gurnoor Dhillon, Sandeep Ramesh, Faris -, Sahal Zain, Eva Zuliya, Ratu Nadiah Khairunnisa 🚀 #GenAI #AIInnovation #TechJourney #ArtificialIntelligence #TechTransformation #CalvinballTech #Calvinballers

47

2 Comments

Both AI research and engineering use Pytorch, but there is not a single shared library between the RAG and agent research and engineering communities. I have read hundreds of papers on RAG and agents, almost none of which use frameworks like Langchain and LlamaIndex. I have talked to many ML engineers building RAG and agent products; most people only use the utility functions of these tools like text splitters or output parsers. The majority of people who use these two popular frameworks are software engineers who don't have much background in AI. Building RAG and agent applications is no different from training a model; you just currently manually tune your prompts. You need evaluations, you need iterations, you need to get the in-context learning working well and gather more data to automate your prompting process and to move into model finetuning. The problem with researchers’ code is that it has no abstraction at all. It only works on the benchmarking dataset. They lack powerful string processing that can generalize the tool calls. Their code is hard to read as it is not organized. Thus, it is really hard to adapt to production. The problems with the frameworks are that they have abstracted so many layers but are not supported with really strong engineering to provide robust adaptation to different use cases. They are more like no-code solutions, which are perfect for people to use out-of-the-box, but terrible for products and research. There can be a solution great for both the ML research and the product community: a library that provides strong string processing, an easy-to-understand tool interface to support tool calls, various output formats, and various access to different models with great monitoring capabilities like TensorBoard for Pytorch. That is the mission and purpose of LightRAG. Let researchers and engineers work on what matters: prompts, datasets, evaluations, and model finetuning without building everything from scratch alone with subpar solutions or using an overly engineered framework which requires them to spend more time figuring out how to customize instead of on what matters. I know I'm not a popular voice, but this is super important, and I think the majority of the community is moving in the wrong direction, even some of those people I looked up to and considered the best in the field. If you are passionate about a library that really unites RAG and Agent research and engineering, please work with me. I know I'm not able to do this alone.

1,375

127 Comments

🚨 New paper w/ interesting insights for fine-tuning and GPU poor folks 🚨 🚀 Exploring Fine-Tuning for LLMs: Full Fine-Tuning vs. #LoRA - Are They Truly Equivalent? 🤔 As LLMs grow, fine-tuning them for specific tasks becomes challenging due to high computational costs. Recently, Low-Rank Adaptation (LoRA) has become a popular alternative, promising similar performance with fewer parameters. But are LoRA-tuned models the same as fully fine-tuned ones? MIT’s latest study dives deep into this question, revealing some surprising differences! 🔍 📝Key Takeaways: 1. Spectral Differences 🧬: LoRA introduces “intruder dimensions” not found in fully fine-tuned models. These alter the model’s weight structure, suggesting that LoRA and full fine-tuning leverage different parts of the parameter space, even if they hit similar accuracy on specific tasks. 2. Generalization & Robustness 🌐: Models fine-tuned with LoRA may struggle with new or sequential tasks, forgetting previously learned knowledge more easily. Full fine-tuning, however, shows better retention and robustness in shifting environments. 3. Effective Rank & Optimal Performance ⚙️: LoRA’s rank matters! Mid-range ranks (like r=64) perform well on tasks and retain pre-trained knowledge, while lower or higher ranks can lead to greater forgetting. Smart parameter tuning (like setting α=2r) can help LoRA resemble full fine-tuning more closely. 4. Why It Matters 💡: LoRA offers incredible parameter efficiency, but its unique trade-offs might affect its use in scenarios needing high adaptability and long-term memory. With adjustments, LoRA can bridge some of these gaps, but full fine-tuning remains the go-to for complex or multi-tasking demands.

25

1 Comment

More opportunities

12

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

15

Can LLM preference tuning degrade Multi-Agent system performance? In a recent tweet, Andrej Karpathy questioned why all LLMs sound similar, prompting Sebastian Rascha to speculate about the uniformity of alignment datasets. Aligning user preferences is a last-mile step in training large language models. Regardless of the pre-training datasets used, could the preference datasets used during alignment compromise the effectiveness of LLMs? This issue could adversely affect multi-agent systems incorporating multiple LLMs in their design.

7

1 Comment

Here's the technical breakdown of the AI Penalty Challenge authored by Alok Pattani (check out this video if you haven't seen the demo yet: https://lnkd.in/gsD3xjJ2) Key Takeaways: ☁️ Firestore for triggering slow-mo video capture on Android phones & uploading to Google Cloud via Firebase Storage. 🧠 Cloud AutoML for training custom object detectors (ball & goal tracking) to calculate speed & accuracy. 🗣️ Gemini Pro 1.5 on Vertex AI for personalized coaching feedback based on multiple data points. 🎨 Imagen on Vertex AI for instant, prompt-driven image editing for cool player pics. More technical insights from the project to come... #AIChallenge #MachineLearning #SportsTech #GoogleCloud #footballNOTsoccer

89

5 Comments

April was an exciting month for AI at Meta! We launched MTIA v2 https://lnkd.in/gvC-UgVN, Llama3 https://lnkd.in/gdbfXa-6, presented a tutorial and paper on the PyTorch2 compiler at ASPLOS https://lnkd.in/gsT9DUr5, released PyTorch 2.3 https://lnkd.in/gYn3Vnzm and, to top it off, we launched the PyTorch ecosystem solution for mobile and edge deployments, ExecuTorch Alpha https://lnkd.in/gWv96jpm optimized for Large Language Models. What better than to combine all of these... running Llama3 on an a mobile phone exported with the PT2 Compiler's torch.export, and optimized for mobile deployment. And you can do all of this in an easy-to-use self-service format starting today, for both iPhone and Android as well as many other mobile/edge devices. The video below shows Llama3 running on an iPhone. (Makers will love how well models run on Raspberry Pi 5!) PyTorch is being developed by a multi-disciplinary team comprising ML engineers, accelerator experts, compiler developers, hardware architects, chip designers, HPC developers, mobile developers, and specialists and generalists that are comfortable across many of the layers involved in building end-to-end solutions. Even better -- if you're excited by the possibilities of AI, and solving the system design challenges of making AI run well across all hardware types, we are looking for YOU! The Pytorch team has openings across PyTorch core, compilers, accelerators and HW/SW co-design https://lnkd.in/gu2Y_KpY https://lnkd.in/gQVXBP6P and a broad range of positions that involve PyTorch from model development all the way to hardware deployments https://lnkd.in/gjDdUh5w #PyTorch #ExecuTorch #Llama3 #AICompilers #MTIA #AcceleratedAI #MetaAI #Meta

64

Resharing it: This is a great summary shared by the LinkedIn engineering team about their #llmops and LLM deployment work and their key learnings. #llms #chatgpt4 #gemini #aiml #mlops #llmops https://lnkd.in/d2bH5GRR

2

🚀 **Just Published: Unlocking the Power of Large Language Models with Chunking Techniques!** 🚀 🔍 Ever wondered how large language models (LLMs) handle long documents while keeping context intact? The answer lies in **chunking**! In my latest blog, I explore the key chunking methods that boost LLM performance, covering everything from **simple sentence splits** to **adaptive AI-based chunking**. With each technique, I’ve included **code samples** to help you put these methods into action! 💡 Whether you're working in **Generative AI**, **NLP**, or **text summarization**, this guide will help you unlock new potential with LLMs. 📈 **Curious to know how it works? Check out the full blog!** https://lnkd.in/gniDQYwW --- #GenerativeAI #AI #MachineLearning #LLM #NLP #AIResearch #DataScience #Chunking #TextProcessing #AIEngineering --- For maximum engagement, tag a few companies or groups if possible, like: - **@OpenAI** - **@Google AI** - **@Hugging Face** - **@DeepMind**    Excited to share these insights and can’t wait to hear your thoughts! #llm #openai #genai #chunking #langchain

12

2 Comments

In his latest post from #TheBatch, Andrew Ng offers his argument on why #California's proposed law SB-1047 is likely to stifle #AI builders and impose unnecessary guardrails on #AI innovation. The law established a “hazardous capability” designation that makes *builders* of AI models liable if someone uses their models to do harm. Andrew argues that laws should not regulate the technology (the tool itself) but its application (how it is implemented).   It's a thoughtful read --> DeepLearning.AI #AIgovernance #AIregulation https://lnkd.in/eBD-9rja

3

A new week brings a fresh opportunity to learn something new! If you have worked with more complex PyTorch models, especially when implementing LLMs, you may have encountered PyTorch buffers. To help you understand them better, I've recorded a short 13-minute hands-on coding video this weekend. In this video, I explain what PyTorch buffers are, when we need them, and how to use them effectively: https://lnkd.in/giAVBGAy

711

21 Comments