Sanja Simonovikj

I just released the April edition of my AI & the Web newsletter. Nathaniel Simard, founder & CEO of Tracel Technologies and inventor of the burn ML framework, spoke with me about how he got into ML, burn, Tracel and the future of AI. WebGPU, the successor of WebGL, will soon also be supported in Firefox and Safari. Small, powerful LLMs running right in the browser is not a distant future, but are the next big thing in cost-effective and privacy-friendly AI. Google is working hard on better WASM support in the browser that makes ML models running fast even on low-end devices without GPU. With Llama 3 by AI at Meta and Phi 3 by Microsoft AI we got some exciting new, small modells. Llama 3 was already ported to run in the browser and we'll see Phi 3 soon as well. MusicGen has been ported to the browser in transformers.js by Hugging Face engineers and developed an online Jukebox running right in the browser. #webgpu #webml #ai #ml #llama3 #wasm https://lnkd.in/drJCY7Ew

19

2 Comments

New Improviner alert! 🌟 Anna is a seasoned software engineer and a true team player. Out-of-office, she embraces nature and quality time with her family and is patiently waiting for enough snow for some cross-country skiing. ❄️ PS. We’re hiring a Data & ML Engineer, take a look: https://shorturl.at/Q7NGm ▪️Welcome, Anna! 👋 What makes work at Improvin’ exciting? First and foremost, the people. I’m lucky to work with these amazing colleagues. The team is driven, yet very humble, and includes some true experts in their fields. It’s also exciting to work in an industry with such momentum. There’s a huge need for sustainability software solutions like ours. As an engineer, you learn many interesting things about the industry through your daily work. I’m currently working on our emission quantification models, and gaining a grassroots understanding of a topic like this has revealed to me just how complex sustainability in food production truly is. 🌱 ▪️What made you choose Improvin’? People have always been my biggest priority when choosing a company. I want to work with great people, learn from them and feel challenged. Being surrounded by individuals who share the same core values and mindset is a must-have. I also need to feel energized by what we’re building. Our company has a very purpose-driven mission and a technologically exciting product. People, purpose and product combined made Improvin’ the perfect choice. 🤝 ▪️How do you feel about the team? I met many from the team during the recruitment process, and I was impressed. Since then, that feeling has only grown stronger. I appreciate how well the team complements each other. Some of us are natural deep divers, while others are more pragmatic and focused on pushing things forward. There’s no hierarchy or pickiness. Everyone is ready to pitch in and ship things quickly. ▪️What’s the out-of-office Anna into? Right now, I try to spend as much time as I can with my partner and our son. Otherwise, I’ve always been passionate about sports and outdoor activities. Cross-country skiing is my all-time favorite. I try to prioritize exercise as much as I can, even if it's just a quick burst outdoors. During the pandemic, we lived in Åre in Northern Sweden for a few years and fully embraced the outdoorsy lifestyle. There’s nothing quite like meeting your friends on the slopes. 🎿 🏔️ Many of my friends now have kids too, so these days we often catch up via playdates. That said, we still try to prioritize some time for ourselves, like a good fika or a nice dinner. ▪️3 things that excite you right now? 🎄 I’m really looking forward to the Christmas break with my family and enjoying some calm, quality time together. 🌟 I’m learning so much about the industry, and that fuels me. What we work with is genuinely super interesting. ❄️ I can’t wait for more snow. We have a great ski route nearby, and I’m really looking forward to getting some evening loops in once there’s enough snow.

96

11 Comments

Reflections on Sparse Representations in Language Models Reading about the use of sparse autoencoders in language models revealed an intriguing balance between interpretability and efficiency. Sparse representations enhance feature disentanglement, aiding in understanding model behaviors, especially in AI safety and bias detection. However, for practical applications where interpretability is secondary, sparse representations might introduce computational redundancy and inefficiency. Thus, while sparse autoencoders offer valuable insights for research and safety, more compact representations could be preferable for deployment. Balancing these aspects is crucial, potentially through adaptive approaches that optimize for both interpretability during research and efficiency in real-world applications. https://lnkd.in/egPP7dEg

1

ArXiv alert! Interesting and recent applied research done by scientists in our Foundation Models team at Netflix Research - Sejoon Oh, Moumita Bhattacharya, and Yesu Feng - is available on ArXiv. Here we motivate the problem of predicting users' session intents and propose a solution for it using Hierarchical Multi-Task Learning. Not only do we get good predictions of the (proxy) intents but also show that the architecture improves the foundation model. https://lnkd.in/gEGQRhg9 Hope you all like it. #foundationmodels #recsys #machinelearning #research

157

1 Comment

Unlock the full potential of your Large Language Models (LLMs) with Intel® Extension for PyTorch (IPEX) and the Intel® LLM Library for PyTorch (IPEX-LLM). Download this whitepaper to explore the optimization of LLMs, including their performance, resource utilization, and response times in real-world applications. Link - https://intel.ly/3BBJ4ey

2

From our incredible SandboxAQ advisor Fei-Fei Li at NeurIPS today in Vancouver: AI will NOT replace human capabilities, instead it will augment human capabilities. #ai

30

1 Comment

Looking forward for insightful thoughts on Enterprise Gen AI. Are any of my connections also here? Would love to meet up and exchange ideas! Drop a comment if you're around! #WorldSummitAI #ArtificialIntelligence #AIInnovation

3

🚀 Advanced AI-RAG Chatbot using Langchain for the Landlord and Tenant Board of Ontario is now LIVE! 🚀 👨‍⚖️ Navigating legal waters in landlord-tenant disputes just got easier. I am delighted to present my project on an advanced AI-powered chatbot designed to democratize legal advice for Ontarians. Leveraging advanced technologies such as NLP, ML, and RAG, the chatbot delivers real-time, accurate legal assistance - making it a game-changer in legal tech! 📈 With FAISS-powered retrieval and sophisticated reranking algorithms, this platform is not just fast - it’s razor-sharp in delivering contextually relevant advice. 🔗 Dive into my project presentation to see the details of the project: https://lnkd.in/dU32f7ze 💡 This isn't just a project; it's a project with a purpose. My goal? To ensure that everyone has access to the legal information they need, when they need it, without any barriers. ✨ A special shoutout to the incredible team: Atharva Pandkar, Tarun Reddy who worked with me from time to time and our Professor Dr. Uzair Ahmad, without whose guidance this would not have been possible. We're just getting started on our mission to transform access to legal services. 👇 Check out my project, join the mission, and let's drive legal tech forward together! GitHub Repo: https://lnkd.in/dBDpjFp8 #LegalTech #Innovation #AI #LegalAidChatbot #AccessToJustice #OntarioLaw #TechForGood #ArtificialIntelligence #Disruption

26

6 Comments

I have just released an update to my AI Interviewer service (https://lnkd.in/gE74X6eA) that I presented last week. 🚀 Faster Interactions: All models are now in streaming/chunk processing mode. This improvement has drastically reduced response times, allowing for long answers and solutions to be processed quickly. GPT-4 works smoother now, and the online demo version is fast enough for practical use (but the quality of the locally running service is still much better). 🛠️ More Interview Types: Alongside the coding interview, I’ve added beta versions of five new types of technical interviews: - System Design - Machine Learning System Design - Machine Learning Theory - Math, Stats, and Logic - SQL Everything is still in the testing mode and there’s plenty of room to improve, but now you can simulate nearly any popular type of technical interview. What other types of interviews should I add in the future? Are there types of interviews you think LLMs can't handle yet? I’d love to hear your thoughts! #ai #chatGPT #interview #coding #automation #systemdesign #machinelearning #interviewpreparation

44

1 Comment

Have you wondered how the decision boundary of in-context learning in LLMs compares to traditional models like Decision Trees and KNN? 🤔 Excited to share our new research, which uncovers unexpected irregularities and non-smoothness in LLMs' in-context decision boundaries! 🔍 📄 Paper: https://lnkd.in/gZAKEJH6 🐦 Tweet summary thread: https://lnkd.in/gY9xkk6d 💻 code: https://lnkd.in/gCsgQk8r Here are some key findings 📌 : - Increasing the number of in-context examples can enhance LLM accuracy on both linear and non-linear classification tasks. But how reliable are these in-context classifiers? We probe their decision boundaries to find out. - By visualizing the decision boundaries, we show that SOTA LLMs, ranging from 1B to large closed-source models such as GPT-3.5-turbo and GPT-4o, all exhibit different non-smooth, irregular decision boundaries, even on simple linearly separable tasks. 🔍 - How do these irregularities arise? 🤔 We study various factors that impact decision boundary smoothness in LLMs, including in-context example count, quantization levels, label semantics & examples order. Then, we identify methods to improve the smoothness of the boundaries. 🛠 - First, increasing in-context examples does not guarantee smoother decision boundaries. While classification accuracy improves with more in-context examples, the decision boundary remains fragmented. - Quantization Impact: Reducing precision from 8-bit to 4-bit can significantly affect areas near the decision boundary, especially in regions with high uncertainties, potentially flipping decisions in these regions. - Label Sensitivity: Decision boundaries are sensitive to label names and example order. Shuffling in-context examples and labels can alter the model’s decision boundaries, indicating a dependency on the semantic prior knowledge of the labels. - 🛠️ Can we improve decision boundary smoothness in LLMs through training? We show that fine-tuning on simple linearly separable tasks can improve the smoothness of decision boundaries and generalize to more complex non-linear, multi-class tasks, enhancing robustness. - Further, fine-tuning the token embedding and attention layers leads to smoother decision boundaries, while fine-tuning the linear prediction head alone does not. - We also explore uncertainty-aware active learning: By adding labels for the most uncertain points to the in-context dataset, we can smoothen the decision boundary more efficiently. - 📚 Effect of Language Pretraining: Transformers trained from scratch on synthetic classification tasks can learn smooth in-context decision boundaries for unseen problems. Thanks for reading about our work! 💡 For a more detailed summary, please check out our tweet thread: https://lnkd.in/gY9xkk6d! 🌟 Work done with amazing collaborators 🙌 Tung Nguyen and our advisor Aditya Grover at UCLA!

65

🌟 Saeid Abolfazli and I are thrilled to start new batch for Applied Natural Language Processing course in collaboration with the University of Toronto School of Continuing Studies. Over 12 weeks, we’ll dive into the world of NLP and large language models (LLMs), learning how to process and analyze textual data to extract actionable insights using cutting-edge algorithms and techniques. Whether you're looking to upskill or explore NLP from scratch, this course will equip you with practical knowledge and hands-on experience. Here’s a sneak peek at what we’ll cover: 1️⃣ Week 1: Introduction to Language Data Processing 2️⃣ Week 2: Text Corpora & Preprocessing 3️⃣ Week 3: Fundamentals of Machine Learning 4️⃣ Week 4: Text Vectorization & Feature Engineering 5️⃣ Week 5: Supervised & Unsupervised Learning on Text 6️⃣ Week 6: Context-Aware Language Modeling 7️⃣ Week 7: Text Visualization & Graph Analysis 8️⃣ Week 8: Scaling with Multiprocessing & Spark 9️⃣ Week 9: Deep Learning on Text 🔟Week 10: Transformer Learning for NLP 1️⃣1️⃣ Week 11: Generative AI for NLP 1️⃣2️⃣ Week 12: Capstone Project Join us for this exciting journey into the heart of NLP! Register at http://ow.ly/drbj50B8tzp 🚀 #NLP #GenAI #UofT

39

1 Comment

Just finished this one! https://lnkd.in/gYzs8FpU Short introductory course about embedding a model on devices. It was great to see an example of an app running on NPU with a model deployment using quantization Basic but extremely useful concepts, highly recommend it

6

Earlier this year, I gave an intro talk about RAG for Generative AI at ConFoo.ca Next year, I plan to give a more in-depth talk comparing Fine-tuning vs RAG and how we are using it in our projects such as MailMagic. Please vote for "RAG vs Fine-tuning: which one is better for your AI project?" in the link below. I recommend ConFoo.ca for any developers and CTOs. It is the best developer conference in Canada. #ai #developer #conference #confoo #coding #masterclass #artificialintelligence #finetuning #rag #generativeai #programming

14

2 Comments

Lee Boonstra presents prompt engineering techniques for large language models, emphasizing practical techniques and methods for guiding desired outputs. IDEAS: - Prompt engineering is vital for unlocking LLM capabilities. - Coding skills are not required for effective prompt engineering. - Understanding LLM behavior is crucial for crafting effective prompts. - Specificity in prompts improves output relevance. - Different LLMs have unique quirks affecting prompt success. - Temperature settings influence creativity and predictability in outputs. - High temperature encourages unpredictability; low temperature fosters accuracy. - Top K and Top P are filters to refine vocabulary usage. - Zero-shot prompting provides tasks without examples, simplifying requests. - One-shot prompting shows an LLM a few examples before a task. - System prompts establish context for LLM interactions. - Role prompts assign the LLM a specific persona to enhance creativity. - Contextual prompts give necessary background to improve responses. - Step back prompting encourages broader thinking before tackling tasks. - Chain of Thought prompting allows LLMs to explain their reasoning. - Self-consistency improves output reliability through repeated prompting. - Advanced techniques include using LLMs for coding and debugging. - Multimodal prompting integrates different types of input for richer results. - Gemini Vision enhances capabilities by processing images and videos. INSIGHTS: - Prompt engineering focuses on guiding LLM outputs through well-crafted prompts. - Understanding each LLM's behavior allows for tailored interactions. - Mistakes in prompting highlight the need for clear context and examples. - Ethical considerations are paramount in prompt engineering. - Continual learning and adaptation enhance prompt engineering effectiveness. QUOTES: - "Prompt engineering is the key to unlocking the power of LLMs." - "You don't need to be a coding genius to do this." - "Crafting prompts is like giving directions to a smart but distracted friend." - "High temperature leads to unpredictability, while low temperature keeps it safe and factual." - "Zero-shot prompting means giving a task without any examples." - "System prompts set the overall context for the LLM." - "Role prompts provide a personality for the LLM." - "Context helps the LLM understand your needs." - "Step back prompting encourages a broader perspective." - "Chain of Thought prompting helps LLMs mimic human reasoning." - "Self-consistency reduces bias in outputs." - "Prompt engineering should be an ongoing process." - "Multimodal prompting is the future of LLM interactions." - "Prompts can help with coding, debugging, and more." Sources: - https://lnkd.in/g-D-rmgw

8

🤖 Staying on top of ML research Machine learning research is a fire hydrant. The field moves so fast, the course you took 5 years ago is probably outdated. To help myself and others stay updated, I thought I could share a few resources that have helped me stay on top of the latest machine learning research papers and trends. 🔍 Emergent Mind (https://lnkd.in/dJK4GbYe) Trending computer science papers from arXiv by topic curated by AI. Provides helpful GPT-4 generated summaries broken down by each section. I find it to be a great way to quickly identify impactful new research to dig into further. 🧠 Bytez (https://bytez.com) AI agent trained on the corpus of AI papers. Summarizes papers and can answer questions on specific topics. I use Bytez to keep browse keep tabs on papers and use it as my first reading step before diving deeper into the actual paper. 📧 Newsletters - 💡 Data Machina (https://lnkd.in/d-mPpbzB) Weekly roundup summaries and papers in MLOps, datasets, deep learning, and AI research - 💡 Deep (Learning) Focus (https://lnkd.in/dsdM2Uhe) In-depth dive into a single topic at a time. (Also a blessing to your LinkedIn feed if you follow the curator 😁) - 💡 AI Tidbits (https://www.aitidbits.ai/) Weekly roundup from 10 other newsletters broken down by subject (language models, vision, audio, industry, open-source packages, etc.) - 💡 AlphaSignal (https://alphasignal.ai/) Short and sweet summaries of top news, papers, and repos. Sometimes it comes with a nice tip that surprises my team (e.g., cutting GPU memory usage with sequential backward calls when using multiple losses in PyTorch).

26

🔮RAG + KG: Powerful Combination for Translation Multi-AI Agents Large language models (LLMs) have shown great promise in machine translation, but they still struggle with contextually dependent terms, such as new or domain-specific words. This leads to inconsistencies and errors that are difficult to address. Existing solutions often depend on manual identification of such terms, which is impractical given the complexity and evolving nature of language. While RetrievalAugmented Generation (RAG) could provide some assistance, its application to translation is limited by issues such as hallucinations from information overload. =================== In this paper, researchers propose CRAT, a novel multi-agent translation framework that: 》 leverages RAG and causality enhanced self-reflection to address these challenges. This framework consists of several specialized agents: ✱ the Unknown Terms Identification agent detects unknown terms within the context, ✱ the Knowledge Graph (KG) Constructor agent extracts relevant internal knowledge about these terms and retrieves bilingual information from external sources, ✱ the Causality enhanced Judge agent validates the accuracy of the information, ✱ Translator agent incorporates the refined information into the final output. =================== This automated process allows for more precise and consistent handling of key terms during translation. Their results show that CRAT significantly improves translation accuracy, particularly in handling context-sensitive terms and emerging vocabulary. --- If your company is looking for an AI consultant to accelerate business growth, or if you are looking to build your career in AI, send me a message and I can help

4