Zico Kolter

✨ Exciting insights from Yann Dubois's recent lecture at Stanford University on building Large Language Models (#LLMs) 🟢 Components of LLMs: Focus on architecture, training algorithms, data collection, evaluation methods, and system optimization for efficient model deployment. 🟢 Pre-training vs. Post-training: Understand the classical language modeling paradigm and the emerging trend of fine-tuning models like ChatGPT to function as AI assistants. 🟢 Importance of Tokenization: Effective tokenization strategies are crucial for handling diverse languages and ensuring model performance. 💎 This lecture is a must-watch for anyone interested in #AI advancements! 👉 https://lnkd.in/dcevp3cJ

44

1 Comment

A New Theory for Managing Chaotic Systems: The Proposal of Weighted Stochastic Riccati Equations Many systems around us involve inherent "chaos." For instance, in remote robotic control systems that experience random communication delays, designing a control law to achieve a goal, such as moving a robot to a target location, requires considering the chaotic nature of these delays. However, chaotic systems are mathematically challenging to handle, making traditional methods limited in scope, computationally intensive, and lacking generality. To address this issue, Y. Ito from TCRDL, in collaboration with Kyoto University, has developed the Weighted Stochastic Riccati Equation (WSR), a theoretical framework for designing control laws that accommodate diverse goals while accounting for stochastic characteristics. By incorporating "weights" into stochastic elements, the WSR equations enable the design of various control strategies. These strategies range from simple ones that average out the chaos to more complex ones, such as "moving slowly to avoid falling" or "prioritizing speed even at the risk of instability." This groundbreaking research is expected to have applications in fields involving chaotic systems, such as robotics and energy systems, contributing to the future development of safe and efficient system designs. https://lnkd.in/gmbYEzn9 https://lnkd.in/gK8GSJEH https://lnkd.in/g93HRugw

9

✨ Exciting developments at the intersection of Large Language Models (#LLMs) and Reinforcement Learning (#RL)! 🚀 Let's see how pre-trained LLMs can be leveraged to enhance RL in several key ways: 🔹 LLMs for Pre-processing Input: If the input observations sent to the agent are in natural language, it is natural to use an LLM to process them, in order to compute a more compact representation. 🔹 LLMs for Reward Functions: Designing reward functions can be challenging, but LLMs can help. For example, LLMs can provide preference judgements (like a human would) for RLHF (Reinforcement Learning from Human Feedback) systems, thereby generating training data for a reward model. 🔹 LLMs for World Models: Instead of learning world models from scratch, pre-trained LLMs can serve as world models. For example, code as a reward. Where prompt a VLM with an initial and goal image, and ask it to describe the corresponding sequence of tasks needed to reach the goal. Then ask the LLM to synthesize code that checks for completion of each subtask. These reward functions are then “verified” by applying them to an offline set of expert and random trajectories; a good reward function should allocate a high reward to the expert trajectories and a low reward to the random ones. Finally, the reward functions are used as auxiliary rewards inside an RL agent. 🔹 LLMs for world models: There are many papers that use transformers or diffusion models to represent the world model, and learn them from data collected by the RL agent. 🔹 LLMs for Policies: LLMs can be used as policies, either by pre-training on state-action sequences or by using in-context learning with prompts. In-context learning allows LLMs to "reason" step-by-step before predicting an action, and this can be enhanced by retrieving relevant past examples. ⚡ In summary, LLMs are proving to be versatile tools in RL, offering benefits from input processing to policy generation. This is a rapidly evolving field with much potential, stay tuned for more! 👉 https://lnkd.in/dRKRem9P #ReinforcementLearning #LargeLanguageModels #AI #MachineLearning

24

🤔 Openness and success: correlation? causation? …vis-a-vis Waymo! 🤔 Here's an innocent question… Does a culture of openness align with a culture of success? Back in 2019, a 10-year-old Waymo published its first round of automated driving datasets. Cut to today, and the company not only creates and openly shares some of the largest and most diverse data sets, but it also calls on the hive mind to spur innovation with its annual Open Dataset Challenges. And they've gone gang busters in launching robotaxis... is there something there? Okay, my questions aren't so innocent. Though I have no affiliation with Waymo, anyone who follows me already knows I believe a culture of openness does not just align with success, but is essential.

41

1 Comment

check the new paper which explores Expectation Value Samplers (EVSs) - quantum-based generative models - and proposes a novel approach that enhances their #expressivity while reducing resource demands. the potential for quantum computing to power #genAI is still unfolding, and this work sheds light on a concrete area where quantum might offer advantage in the future. kudos Kevin Shen, Andrii Kurkin and all the co-authors from Universiteit Leiden and BMW Group!

76

I recently tested NotebookLM with a collection of papers on fast formation, battery data science, and cost modeling for battery manufacturing that I have been collecting. While it sounded "human", it reinforced a recurring challenge in the industry—the hype around data science “revolutionizing” battery manufacturing often overshadows the practical hurdles. THE glaring issue is data scarcity. Most studies analyze datasets with dozens to a few hundred cells—hardly enough to build robust, scalable machine learning models. These datasets are often lab-scale, lacking real-world aging effects and variability seen in factory production. On the flip side, relying solely on factory-produced cells introduces bias, limiting insights to what’s easiest to measure rather than what’s most impactful. It’s like trying to build an autonomous vehicle by only training on empty freeways at 2 AM on Christmas—you might get great results in isolation, but it won’t prepare you for real-world challenges. The battery community needs to shift back from training models and focus on scalable data collection strategies with the volume, velocity, and variability required for meaningful predictions. Until we have useful datasets, the gap between industry optimism and factory-scale reality will persist

42

4 Comments

Pretrained large networks, sometimes called foundational models, are becoming increasingly useful in our research these days. Many works are pointing out how these models are advanced and can be generalized to many different tasks. I generally take the more humble view and accept that these methods fall somewhere between far better than starting from a randomly initialized model to can be helpful for planning. I gave a lecture in my class to piece together the very recent progress on using large or pretrained models for robot control. There are likely more works to include, but I suspect this outline will help others catch up to the progress made in using large models for robotics. More source information on the lecture is available here. https://lnkd.in/gcX8vgr7 I also look forward to presenting this recent research in a more condensed format at #WorldAiSummit this week, where I will talk about how we can use generalist robots to learn the tech tree of the universe. https://lnkd.in/gEEB_5kZ

65

1 Comment

Prediction of Redox Potentials Using Machine Learning https://lnkd.in/gPCtEDtU A study conducted by Ryotaro Jinnouchi from TCRDL, in collaboration with the University of Vienna and VASP Software, has been published in npj Computational Materials. Redox potential is a crucial property for understanding electrochemical phenomena in secondary batteries, fuel cells, electrolysis, and artificial photosynthesis. However, predicting this property using first-principles calculations, which rely on wave equations, is extremely challenging and requires substantial computational resources. Traditional methods for calculating redox potential demand 1 million core hours (equivalent to 114 years of computation on a single core). In this study, the researchers developed a method using machine learning to incrementally improve the accuracy from coarse models to highly precise ones. This approach achieved high-accuracy results in 1/50th of the time required by conventional methods. The method was validated with three types of metal ions, showing a high degree of agreement with experimental values. This technique is expected to form the foundation for future first-principles calculations in electrochemistry. 4o

14

Amazing conversation with Dr. Serrano in today's episode. I'd like to learn more from his book!

1

Falkonry (Suhas Mehta and Ashis Khan) and iba AG (Eric Snyder) will present how steel makers are pivoting to AI & Data driven decision support system at AISTech 2024 today (May 6, 2024) - please join us if you are interested. I co-authored this paper with Eric Snyder of iba AG.

64

5 Comments

😖 Identifying the leading LLM using performance on academic benchmarks is flawed. Why? 👉 No guarantee the benchmark data wasn't used to train the LLM, directly or indirectly 👉 Performance on academic benchmark does not equate with utility in the real world. The data is completely different. 👉 Academic benchmarks are small by industry standards 👉 "When a metric becomes a target, it ceases to be a good metric" So, which rankings do I trust? 🤩 Blind human ratings on the LMSys Chatbot Arena! On the Chatbot arena, a user can chat about any topic with two anonymous chatbots. Then, the user rates the response that is better. Whichever chatbot is rated as "better" gains some points, the other loses points. This is used to create a ranking based on ELO ratings. Meta's Llama-3 is already climbing the leaderboard rankings! How exciting! Still, there are issues with deciding which LLM to use for your enterprise use case based on Chatbot Arena rankings, but that a discussion is for another post 😄

65

6 Comments

Amazing findings by Hugh Zhang, Michele L., Summer Yue, and others at Scale AI, addressing a critical concern regarding Large Language Models (LLMs) evaluation and the challenge of data contamination! With LLMs achieving remarkable success on various benchmarks, concerns have arisen about the extent to which this performance reflects true reasoning ability versus exposure to similar benchmark questions. In response, the paper https://lnkd.in/gttAW-g2 undertook an investigation, introducing Grade School Math 1000 (GSM1k) to closely mirror established benchmarks like GSM8k. Their findings shed light on the prevalence of overfitting, particularly among models like Mistral and Phi, which exhibit accuracy drops of up to 13% on GSM1k compared to GSM8k. However, models at the forefront, such as Gemini, Claude, and GPT, demonstrate minimal signs of overfitting. This research not only underscores the importance of robust benchmarking but also prompts critical reflections on the evaluation of LLM performance. Kudos to the team for their work! #llm #research #genai #machinelearning #ai

15

5 Comments

⚛️ It’s #Physics Time: The Cooper Problem – The Birth of Electron Pairing in Superconductivity ⚛️ 🔍 Historical Context In 1956, physicist Leon Cooper made a groundbreaking observation that would lay the foundation for the BCS theory of superconductivity. He discovered that even a weak attractive interaction between electrons in a lattice could lead to something extraordinary: an instability in the conventional Fermi sea of electrons. This phenomenon became known as the Cooper problem, and it set the stage for explaining superconductivity at a microscopic level. ✨ What’s the Cooper Problem? Imagine adding two electrons to a sea of free electrons (the Fermi sea). Normally, electrons repel each other due to their negative charges, but in a lattice, an attractive interaction can emerge, mediated by lattice vibrations called phonons. Cooper realized that this tiny attraction changes everything. These electrons can pair up to form a Cooper pair, reducing the energy of the system. This pairing happens because the electrons form a quantum state with integer spin, turning them into bosons, which can condense into the same quantum state without scattering off lattice atoms. The result? Superconductivity – the ability of a material to conduct electricity without resistance. 📸 The Mathematical Derivation I’ve attached a mathematical derivation of the Cooper problem, where the Schrödinger equation for two electrons in the Fermi sea reveals how they reduce the system's energy by pairing. Check it out! 💡 Real-World Applications The concept of Cooper pairs underpins the entire field of superconductivity, which has transformed science and technology: 1️⃣ MRI Machines: Superconducting magnets provide high-resolution imaging in healthcare. 2️⃣ Particle Accelerators: Superconducting materials create the powerful magnets that guide particle beams. 3️⃣ Quantum Computing: Superconducting qubits are at the heart of cutting-edge quantum processors. 4️⃣ Lossless Power Transmission: Superconductors enable efficient energy transport with zero resistance. 🔭 Outlook From Theory to Revolution The Cooper problem was just the beginning. Building on this insight, Bardeen, Cooper, and Schrieffer formulated the BCS theory, explaining superconductivity in full detail and earning a Nobel Prize. Today, researchers are exploring high-temperature superconductors and their mysterious mechanisms, with the goal of bringing superconductivity to room temperature – a discovery that could revolutionize energy and computing. 🔗 The Cooper problem demonstrates how a small insight can transform our understanding of the universe. It’s a reminder of the beauty and power of theoretical physics! #Physics #Superconductivity #CooperProblem #QuantumPhysics #BCSTheory #Innovation

33

Happy about our latest paper titled "Deduplicator: When Computation Reuse Meets Load Balancing at the Network Edge", which was recently accepted for publication by IFIP Networking 2024. All the props go to my senior Ph.D. student, Md. Washik Al Azad. This is the 9th (!) publication during his Ph.D., which is a substantial accomplishment for a systems student. The rest of his papers have been published at IEEE PerCom (best paper runner up award), the IEEE Internet of Things Journal (impact factor > 10), IEEE CLOUD, and the IEEE Wireless Communications Magazine (impact factor > 10), among other venues. In this paper, we explore whether we can achieve the contradictory in nature functions of load balancing and computation reuse at the network edge. To this end, we propose an edge middlebox, called the Deduplicator. Our experimental results demonstrate that we can facilitate computation reuse and distribute the load among edge servers at the same time. You can find a preprint of our paper on arXiv: https://lnkd.in/dkQqq9XM Our code is also available on GitHub: https://lnkd.in/dq3HXJSZ #research #cloudcomputing #edgecomputing #loadbalancing #distributedsystems #systems #computernetworks #middleboxes

45

6 Comments

Very interesting edge-orientation GNN framework from a recent arXiv paper that improves performance over HodgeNets. It seems fairly easy to extend this new model to TDL frameworks with Hodge Laplacians as boundary operators for invariance, equivariance, and fusion pooling: https://lnkd.in/e-vjbtmj

19

2 Comments

“Nougat can easily parse PDF-format academic documents, extract mathematical formulas and tables, and convert them into structured data for subsequent processing and analysis.“ Read “Advanced RAG Retrieval Strategy: Embedded Tables“ by zhaozhiming on Medium: https://lnkd.in/dFBUMW3n

4

 A pre-print of my summer intern's recent paper, entitled “Domain decomposition-based coupling of operator inference reduced order models via the Schwarz alternating method”, is available on ArXiv here: https://lnkd.in/g8qZ94CA. The paper presents and evaluates an alternating Schwarz-based approach for coupling together subdomain-local reduced order models constructed via non-intrusive Operator Inference (OpInf) with each other and with subdomain-local full order models (FOMs).

53