Cyril Gorlla

Interesting paper from ICML 24 for scalable RL. Stop Regressing: Training Value Functions via Classification for Scalable Deep RL https://lnkd.in/ghukW_sU

8

A substantial piece of work from Apple led by Iman Mirzadeh, keivan alizadeh vahid, Oncel Tuzel, Samy Bengio, Mehrdad Farajtabar and Hooman Shahrokhi supports the thesis that transformer-based LLMs do not reason--they perform pattern-matching. The team from  also provides a great benchmarking tool--GSM-Symbolic for assessing mathematical and formal reasoning. This supports my anecdotal findings when I evaluated o1 https://lnkd.in/ec72fecb as well as my thoughts on the incompleteness of our assessments of AGI https://lnkd.in/ehBT3HB3

16

2 Comments

🚀 Advancing Information Retrieval with APEER: Automating Prompt Engineering for LLMs! 🚀 I recently delved into a fascinating paper on APEER (Automatic Prompt Engineering Enhances LLM Reranking), a method transforming the way we approach relevance ranking in Information Retrieval (IR) using Large Language Models (LLMs). 🌟 🔍 Challenges in IR with LLMs: Current IR methods with LLMs often rely heavily on human-crafted prompts for zero-shot relevance ranking. This process is time-consuming, subjective, and lacks scalability. The complexities of integrating query and passage pairs and conducting comprehensive relevance assessments further complicate the effectiveness of existing methods. 💡 APEER: A Game-Changer APEER addresses these challenges by automating prompt engineering through iterative feedback and preference optimization. Here's how it stands out: Feedback Optimization: Continuously refine prompts based on performance metrics. Preference Optimization: Improves prompts by learning from positive and negative examples. 🌟 Key Highlights: Reduced Human Effort: APEER minimizes the need for manual prompt crafting, making the process more efficient and less reliant on human expertise. Enhanced Performance: Demonstrates significant improvements in LLM performance on IR tasks, with notable gains in metrics like nDCG@10. For example, APEER achieved an average improvement of 5.29 nDCG@10 on eight BEIR datasets over manual prompts on the LLaMA3 model. Better Transferability: Shows consistent outperformance across diverse datasets and LLM architectures, including GPT-4, LLaMA3, and Qwen2. 📊 Robust Validation: APEER has been rigorously tested on multiple datasets such as MS MARCO, TREC-DL, and BEIR, ensuring its robustness and effectiveness in various IR scenarios. 🌐 Implications: This advancement represents a significant step forward in optimizing LLM prompts for complex relevance ranking tasks. By reducing manual intervention and enhancing the efficiency of LLMs, APEER paves the way for more scalable and accurate applications in real-world IR tasks. 🛠️ Conclusion: APEER's automated approach to prompt engineering is a major leap towards more effective and scalable IR systems. It highlights the potential of integrating iterative feedback and preference optimization to refine LLM performance, offering promising avenues for future research and practical applications. Looking forward to seeing how APEER influences the landscape of Information Retrieval and beyond! #InformationRetrieval #LLM #AI #MachineLearning #NLP #DataScience #Research

1

In my previous blog post (link in the comments), I tackled the issue of dead neurons and vanishing gradients during neural network training. By analyzing the math behind the tanh activation function, we discussed an informal solution: multiplying hidden layer weights by certain "magic numbers." Surprisingly, it worked! But this raised a bigger question: What happens in more complex networks or with different activation functions? Unfortunately, these ad hoc techniques fall short in such scenarios. Enter the formal solutions: Kaiming Initialization and Batch Normalization. Kaiming Initialization addresses vanishing gradients by setting weights to maintain proper signal flow, while Batch Normalization stabilizes training by normalizing activations layer by layer. These techniques solve the problem of dead neurons, enabling networks to learn and optimize efficiently across any setting. In my latest blog post, I apply these methods to train a trigram language model and demonstrate through code how they eliminate dead neurons and enhance learning. Curious to dive deeper? Check out the full blog post—complete with code in the GitHub repo. I'd love to hear your thoughts and insights!

12

3 Comments

I thought it was very interesting to hear Andrej Karpathy talk at GPU MODE IRL 2024 about llm.c and the conversion of PyTorch code into C/CUDA, as well as the potential implications of this work for future LLM progress. The idea of converting high-level deep learning frameworks like PyTorch into low-level, highly efficient C/CUDA code could drastically improve model performance and execution speed, especially in specialized environments. It will be exciting to see how software engineering (SWE) practices evolve over the coming years as this becomes more widely adopted. One of the most intriguing aspects is the potential for a model to generate a specific kernel for a use case at someone's company in code that has no dependencies and could be fully optimized for that specific task. This capability could not only make models more efficient by reducing overhead but also streamline the workflow for developers, allowing them to focus on higher-level design and problem-solving. Essentially, it introduces a whole new paradigm for how we think about integrating AI into SWE roles. Instead of relying on pre-built libraries or packages, engineers might one day leverage AI models to dynamically create specialized, hyper-optimized code that meets their unique needs, reducing both the time and complexity of development. This shift could also lead to a greater level of automation, where AI models help developers by taking over tasks like performance optimization or low-level code generation, potentially reshaping the way we approach software development as a whole. You can listen to the keynote here: https://lnkd.in/eqJYS8Gb #AI #MachineLearning #DeepLearning #PyTorch #CUDA #SoftwareEngineering #C #LLM

2

𝐍𝐞𝐰 𝐁𝐥𝐨𝐠 𝐏𝐨𝐬𝐭 𝐀𝐥𝐞𝐫𝐭: 𝐂𝐚𝐥𝐢𝐛𝐫𝐚𝐭𝐢𝐨𝐧 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬 𝐟𝐨𝐫 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬🚀 I'm excited to share my latest blog post where we delve into the critical aspect of AI - Calibration Techniques for Language Models. This piece highlights why meticulous calibration is essential and various ways to get calibrated response from LLM models 𝐊𝐞𝐲 𝐦𝐞𝐭𝐡𝐨𝐝𝐬: 𝐕𝐞𝐫𝐛𝐚𝐥𝐢𝐳𝐞𝐝 𝐂𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞  - How asking models to rate their confidence can lead to more trustworthy AI. 𝐒𝐞𝐥𝐟-𝐂𝐨𝐧𝐬𝐢𝐬𝐭𝐞𝐧𝐜𝐲-𝐁𝐚𝐬𝐞𝐝 𝐂𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞 - Using consistency in model responses to gauge confidence. 𝐋𝐨𝐠𝐢𝐭-𝐁𝐚𝐬𝐞𝐝 𝐀𝐩𝐩𝐫𝐨𝐚𝐜𝐡𝐞𝐬 - Techniques such as Platt Scaling and Temperature Scaling to refine probabilistic outputs. 𝐒𝐮𝐫𝐫𝐨𝐠𝐚𝐭𝐞 𝐌𝐨𝐝𝐞𝐥 𝐚𝐧𝐝 𝐅𝐢𝐧𝐞-𝐓𝐮𝐧𝐢𝐧𝐠 𝐌𝐞𝐭𝐡𝐨𝐝𝐬 - Advanced strategies to enhance prediction reliability and pinpoint uncertainty. Accuracy and confidence in AI responses are crucial, especially in critical applications. Each technique offers a unique approach to better align model outputs with real-world probabilities. For a deeper insight into these techniques and detailed examples, check out the full blog post! 👀 https://lnkd.in/d9emAMEb Do share your experiences and feedback! 💡 #AI #MachineLearning #LanguageModels #ArtificialIntelligence

15

Beautiful visual guide to quantization, which is becoming a super important technique for compressing LLMs. This is such a fun guide with lots of visuals to build intuition about quantization. Highly recommended! https://lnkd.in/eb-eg3Ca ↓ Follow my weekly summary of the top AI and LLM papers. Read by 70K+ AI researchers and developers: https://lnkd.in/e6ajg945

177

This paper examines the economics and equity challenges of implementing LLMs in healthcare, focusing on three pathways: training from scratch, fine-tuning pretrained models, and using out-of-the-box models, and evaluates their trade-offs in cost, customization, and accessibility. 1️⃣ Training from scratch offers maximum customization and transparency by building LLMs on specific healthcare data but requires significant resources in infrastructure, workforce, and upfront costs. 2️⃣ Fine-tuning pretrained models balances cost and customization by adapting existing LLMs with healthcare-specific data but risks biases from the original training data and has limited transparency. 3️⃣ Using out-of-the-box models enables rapid deployment with minimal infrastructure and workforce needs but offers limited customization and transparency, often relying on general-purpose LLMs. 4️⃣ Cloud service providers play a critical role in scaling LLMs through managed services, federated learning, and hardware accelerators, but affordability and equitable access remain challenges. 5️⃣ Privacy, ethics, and regulatory considerations, including data deidentification and compliance with standards such as HIPAA and GDPR, are essential for LLM adoption in healthcare. 6️⃣ Potential applications of LLMs include chatbots, clinical decision support, and data summarization, but ongoing efforts are needed to mitigate biases, toxicity, and ensure model transparency. ✍🏻 Radha Nagarajan, FHIMSS, FAMIA, Midori Kondo, Franz Salas, Emre Sezgin, Yuan Yao, Vanessa K., Sandip Godambe, Naqi Khan, MD, MS, Alfonso Limon, Ph.D., Graham Stephenson, Sharief Taraman, MD, DABPN, DABPM, FAAP, Nephi Walton, Louis Ehwerhemuepha, Jay Pandit, Deepti Pandita, Michael Weiss, Charles Golden, Adam Gold, John Henderson, Angela Shippy MD, Leo Anthony Celi, Bill Hogan, Eric Oermann, Terence Sanger, Steven Martel, MD. Economics and Equity of Large Language Models: Health Care Perspective. Journal of Medical Internet Research. 2024. DOI: 10.2196/64226 📄 Planning to submit your research to a peer-reviewed, open-access journal? Use the discount code AMJB200 at checkout to save $200 USD on the article processing charge at any JMIR Publications journal. Start your submission here: https://lnkd.in/deuAEN2r. It's important to me that you know this isn't a sponsored post. I'm simply committed to keeping you informed about opportunities to disseminate your research effectively.

111

16 Comments

𝐌𝐮𝐬𝐭-𝐫𝐞𝐚𝐝, 𝐟𝐫𝐞𝐞 𝐛𝐨𝐨𝐤 𝐭𝐡𝐚𝐭 𝐩𝐫𝐨𝐯𝐢𝐝𝐞𝐬 𝐚 𝐩𝐚𝐭𝐡𝐰𝐚𝐲 𝐟𝐫𝐨𝐦 𝐆𝐀𝐈 𝐭𝐨 𝐀𝐆𝐈! ​ Yann LeCun has expressed skepticism about the potential of LLMs to achieve Artificial General Intelligence (AGI), citing limitations in memory, planning, and grounding in real-world understanding. However, we argue that 𝗟𝗟𝗠 𝗖𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝘃𝗲 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 (𝗟𝗖𝗜)—particularly with multiple multimodal LLMs working together—offers a promising architecture to address these challenges and progress toward AGI. ​ Two key architectural innovations underlie LCI: ​ 𝐒𝐞𝐩𝐚𝐫𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐔𝐧𝐜𝐨𝐧𝐬𝐜𝐢𝐨𝐮𝐬 𝐏𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠 𝐚𝐧𝐝 𝐂𝐨𝐧𝐬𝐜𝐢𝐨𝐮𝐬 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 𝐚𝐧𝐝 𝐏𝐥𝐚𝐧𝐧𝐢𝐧𝐠: Inspired by the human mind’s dual-layer structure, LCI divides its architecture into foundational and adaptive layers. The foundation layer, akin to unconscious processing, leverages extensive datasets to build robust pattern recognition, encoding essential responses much like human instincts. The adaptive layer, mirroring conscious thought, enables rapid adaptation and contextual reasoning. This dual-layer approach allows LLMs to perform complex tasks with foundational knowledge while adjusting to new information on the fly, much as a child learns new concepts within a developed cognitive framework. ​ 𝐃𝐢𝐬𝐭𝐢𝐧𝐜𝐭 𝐑𝐨𝐥𝐞𝐬 𝐟𝐨𝐫 𝐄𝐚𝐜𝐡 𝐋𝐋𝐌-𝐀𝐠𝐞𝐧𝐭: Each LLM-agent is assigned a specific role, such as executive (knowledge processing), legislative (behavior guardrails), and judicial (adapting to cultural norms). ​ Additionally, our validation of the LCI framework has been thoroughly tested through real-world deployment in applications across various domains, including healthcare, sales planning, investment, and debiasing news. These empirical deployments highlight the practical viability and adaptability of the LCI framework, showcasing its ability to enhance reasoning and decision-making through collaborative intelligence. ​ The eleven aphorisms presented in this free book illustrate the philosophical foundation of the LCI framework, backed by empirical studies. Through LCI, we envision a clear pathway to AGI, rooted in collaborative intelligence and flexible, context-aware adaptation. https://lnkd.in/gn74DkNj (free) https://lnkd.in/gszSvfqP ($6.98 printing fee)

9

A Machine Learning algorithm near to my heart: Principal Component Analysis! This dimension reduction method allows analysts identify the most important patterns in a dataset based on relative covariance. Many a nights spent on labeling faces and figuring out PCA... Special thanks to Dr. Karl Ricanek Jr. and Dr. Eric Patterson for introducing me to the method. At the end of the lecture, I discuss how PCA was used in combination to Dr. Tim Cootes' work on Active Appearance Models to generate artificial faces based on PCAs of different age demographics. https://lnkd.in/e3xf4QNS #ai #MachineLearning

1

This ArXiv pre-print posted in August 2024 (by Sun et al. from Sakana and Emergence AI) tells a very similar story "Transformers Layers as Painters" where they show strong redundancy of inner layers in the transformer, but input and output layers are extremely important https://lnkd.in/ePxK5grg They address these questions: 1. Do layers use the same representation space? (§3.1) 2. Are all the layers necessary? (§3.2) 3. Are middle layers all doing the same function? (§3.3) 4. Does the layer order matter? (§3.4) 5. Can we run the layers in parallel? (§3.5) 6. Does order matter for some tasks more than others? (§3.6) 7. Does looping help parallelized layers? (§3.7) 8. Which variants harm performance the least? (§3.8)

10

A fascinating exploration of the interplay between Computational Argumentation and Machine Learning is presented in a recent arXiv paper by Rago, \v{C}yras, Mumford, and Cocarascu. The authors identify two broad themes that emerge from the literature: "argumentation for machine learning" and "machine learning for argumentation." They further categorize the interactions between these fields as synergistic, segmented, or approximated. The review offers insights into the strengths and limitations of different argumentation frameworks and machine learning techniques. The work highlights the potential of combining these fields but also cautions that challenges and limitations must be addressed to ensure their effective integration as AI continues to evolve. A must-read for those interested in the future of AI and its intersection with argumentation! https://lnkd.in/d9wyuaXp

3

👉🏼 Detection of ChatGPT fake science with the xFakeSci learning algorithm 🤓 Ahmed Abdeen Hamed 👇🏻 https://lnkd.in/eWXP83PT 🔍 Focus on data insights: - The study demonstrates how AI-generated content can be distinguished from scientific articles. - Introduction of xFakeSci, a learning algorithm capable of identifying ChatGPT-generated articles. - Training and calibration steps incorporated to enhance algorithm performance. 💡 Main outcomes and implications: - xFakeSci algorithm achieved F1 scores ranging from 80 to 94%, outperforming traditional data mining algorithms. - Calibration and proximity distance heuristic contribute to the algorithm's promising performance. - Significant step towards combating fake science generated by AI tools like ChatGPT. 📚 Field significance: - AI-generated content detection - Data-driven heuristics - Combating fake science 🗄️: [#AI #datainsights #fakeScience #xFakeSci #ChatGPT]

1

Last week I was grateful to be invited to present our current working paper “Causal Inference with Cocycles” at the Munich Center for Machine Learning’s (MCML) 3rd annual workshop on causal machine learning. Causal machine learning goes beyond predicting outcomes based on patterns, and aims to identify actual causes behind those outcomes. 🧐In our work, we look at what most state of the art methods have in common, which is the need to (often) estimate entire distributions/densities from data. 🧠We show that in many cases, one can avoid this by instead just estimating a “cocycle”, which describes how a conditional distribution changes under causal actions (eg interventions). 🔥In various situations, directly targeting the cocycles simplifies the estimation problem and improves robustness to misspecification. 💪🏼This means more reliable, robust answers to causal “what if” questions in practice! Anyone interested can find out more in our paper (see the link in the comments)! Thank you to Stefan Feuerriegel for the invitation and for organising a great selection of speakers covering causal discovery and estimation problems! It’s always inspiring for the more theoretical end of my work to learn about the applied problems researchers in academia and industry are grappling with!

81

1 Comment

I'm thrilled to announce the release of two new fine-tuned large language models (LLMs) and a groundbreaking dataset for Persian NLP. 🚀 Shotor (Llama 3 8B): Fine-tuned on the Persian Alpaca Deep Clean dataset. Paya (Aya 23 8B): Also fine-tuned on the same comprehensive dataset. These models are crafted for high-quality performance in various Persian language applications, leveraging the Alpaca Deep Clean dataset which I meticulously processed. The fine-tuning was achieved using Dora for efficiency. Explore the models: Shotor: https://lnkd.in/ddge8CAh Paya: https://lnkd.in/dT55qCH4 Additionally, I’ve released the first Persian DPO dataset for LLM alignment, aiming to set a new standard in Persian language processing. Check out the datasets: Persian Alpaca Deep Clean: https://lnkd.in/d8ejpYRf Persian DPO: https://lnkd.in/dMF2sihF Your feedback and insights are invaluable as we strive to advance Persian NLP. I invite the community to test these models and datasets, and share your thoughts. Feel free to engage and share this post to spread the word! Looking forward to your feedback and collaboration. #NLP #PersianNLP #MachineLearning #AI #LanguageModels #DataScience #HuggingFace

36

6 Comments

Often LLMs hallucinate because of semantic uncertainty due to missing factual training data. We propose a method to detect such uncertainties using only one generated output sequence. Super efficient method to detect hallucination in LLMs. See paper on ArXiv: https://lnkd.in/dS4Vxj7t

692

13 Comments