Zoltan Kovacs

🤖 Creating a fully autonomous AI developer to write and maintain code is still the holy grail of software development. There are many brilliant teams currently working on this, and each of them have their own unique approach. The team at Cosine have made a massive step towards making this a reality. “Cosine achieves state-of-the-art results on the SWE-bench benchmark” They have achieved a score of 43.8% on SWE Bench, which was verified by OpenAI. What's even more incredible is they have managed to do this with a lean team and a relatively limited budget. Though it’s important to note Cosine isn’t looking to replace human developers entirely, but rather augmented them with human-like assistants that they can collaborate with on any kind of coding task. Yang Li + Sam Stenner + Alistair Pullen Read more about their collaboration with OpenAI 👇 https://lnkd.in/gu7xXVPn

21

A lot of hype around agentic AI systems, but I'm honest: If somebody starts talking about agentic AI, I stop listening. Here is why: 𝐖𝐡𝐚𝐭 𝐚𝐫𝐞 𝐚𝐠𝐞𝐧𝐭𝐢𝐜 𝐀𝐈 𝐬𝐲𝐬𝐭𝐞𝐦𝐬? Large workflows that involve LLMs to control the data flow, call tools or make plans that depend on interim results are often called agentic AI. 𝐖𝐡𝐚𝐭 𝐢𝐬 𝐭𝐡𝐞 𝐩𝐫𝐨𝐛𝐥𝐞𝐦? Agentic AI is a catch-all-phrase that basically involves any LLM-based system. People are claiming to build alls sorts of magic assistant with these stacks: Autonomous software engineers, personal travel agents and more. Because agentic AI can be everything and potentially do everything, there is a lot of hype and unrealistic expectations. 𝐇𝐨𝐰 𝐝𝐨𝐞𝐬 𝐭𝐡𝐞 𝐫𝐞𝐚𝐥𝐢𝐭𝐲 𝐥𝐨𝐨𝐤 𝐥𝐢𝐤𝐞? We are at a point where "simple" LLM-based systems such as RAG are understood well enough to be deployed in production. But even then, debugging and securing such a system is still challenging. To think that we can do this for systems that potentially involve thousands of complex LLM calls to complete a single task is ridiculous. 𝐖𝐡𝐚𝐭 𝐬𝐡𝐨𝐮𝐥𝐝 𝐰𝐞 𝐝𝐨 𝐚𝐛𝐨𝐮𝐭 𝐢𝐭? First: Let's try to talk about the technology as concretely as possible: About RAG, text2sql, code completion, tool calling etc. Please don't talk about agentic AI! Second: Let's concentrate on the (very promising) use cases that can be delivered by "simple" AI systems: Technical support bots, coding assistants, data analytics assistants, knowledge management etc. Please don't talk about magic systems that run for days to complete some hyper complex task. What do you think? Are agentic AI systems the apex of AI hype? Or is there real substance behind the paradigm? #industrialai #agenticai #hype

66

31 Comments

Anthropic just released its new multimodal AI model: Claude 3.5 Sonnet. The benchmarks sure need a refresh as AI models keep improving at breakneck speed. But more importantly, each benchmark represents a real-world application, so it's crucial to steer clear of thin wrappers over any benchmarked capability. #claude #anthropic #chatgpt #gpt #openai

5

1 Comment

Despite its 671B parameters, DeepSeek-V3 achieves energy efficiency via its Mixture-of-Experts (MoE) architecture, activating only 37B parameters per token. Training required 2.788M H800 GPU hours, costing $5.57M, setting a new standard for cost-effective, scalable AI. Benchmarks on MMLU, MATH, and Chinese SimpleQA show DeepSeek-V3 outperforming GPT-4 and Claude 3.5 in coding and math tasks, with a 90.2% accuracy on MATH-500. DeepSeek-V3 leverages FP8 mixed precision with fine-grained quantization, reducing memory usage by 50% while maintaining accuracy through increased accumulation precision. Multi-token prediction accelerates inference by generating multiple tokens sequentially, supported by speculative decoding that handles error propagation through parallel verification. https://lnkd.in/dWR32WUp

1

The field of LLMs is getting more and more interesting. So many real world use cases are emerging, making a different shift in digital transformation. Stay tuned for more details. #LLMs #digitaltransformation

9

𝐒𝐭𝐚𝐭𝐞 𝐨𝐟 𝐑𝐮𝐧𝐧𝐢𝐧𝐠 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬 (𝐋𝐋𝐌𝐬) 𝐨𝐧 𝐌𝐨𝐛𝐢𝐥𝐞 𝐃𝐞𝐯𝐢𝐜𝐞𝐬 This is where we are at with running LLMs on local devices...PowerInfer-2 by SJTU-IPADS Labs is a optimized inference framework designed specifically for smartphones. With TurboSparse-Mixtral-47B, it achieves an impressive speed of 11.68 tokens per second, which is up to 22 times faster than other state-of-the-art frameworks: 🔧 PowerInfer-2 uses heterogeneous computing, I/O-Compute pipelining, and TurboSparse with up to 97% sparsity. 📈 TurboSparse enables FFN sparsity of Mistral and Mixtral up to 90% and 97%. 🚀 Running Mixtral 8x7B at 11 tokens/s on a mobile phone. 🔐 Bringing cloud-scale models directly to your device without compromising privacy. Making AI accessible to everyone, everywhere. 🔓 PowerInfer-2 will be open-sourced in stages. Home: https://powerinfer.ai/v2/ Abs 1: https://lnkd.in/gKpricuW Abs 2: https://lnkd.in/gEtYnRnN Github: https://lnkd.in/gVTh8hGG HF: https://lnkd.in/gwNh6T2K #AI #GenAI #LLM #MachineLearning #LLM #MobileAI #MobileLLM #TurboSparse #Mistral #Mixtral #HeterogeneousComputing #IOComputePipelining #ModelSparsity #MobileAI #CloudScaleModels #Cloud #CloudComputing #PrivacyFirst #OpenSource #SmartphoneAI

1

A lot of developers told me that AI SDKs are becoming too cagey. AgentAI, Semantic Kernel, LangChain… You rarely see the bare metal LLM call these days. Why is that happening? The problem with LLMs is that the jury’s still out on what’s truly the dominant model (my personal take is that it will never be a winner-takes-all play). And so everyone is trying to lock developers into a cumbersome integration. Not only these integrations don’t follow a standard pattern, they do all sorts of opaque tricks behind the scene that pollute your prompts. So even if you decide to switch to another SDK you’ll end up breaking your chat completion or inference calls. LangChain, started as an open source project, has raised $25M in funding. It now has an enterprise tier. Convoluted integrations will make it hard for customers to churn. It’s the ultimate antipattern. Thankfully there is a bright side, and I’m shocked the AI influencers are not talking more about these: • liteLLM gives you a uniform SDK to switch between any LLM and even do load balancing. • Groq also has a very similar approach. Developers who are serious about AI already use a combination of LLMs, not just one: giving them freedom of choice will be the best retention play for platforms and frameworks out there.

10

3 Comments

In the same vein as my post yesterday about optimizing the RAG settings in OpenAI and FileSearch to reduce request cost, this is paper from Amazon and Stanford University AI researchers offers some interesting insights. According to this paper, using retrieval isn't necessary at every conversational turn (this is obvious and true IMO). The authors also show that overusing external knowledge can lead to reduced confidence and potentially hallucinated responses. IMO, RAG responses can be over cited and inefficient. This paper proposes a a gating mechanism called RAGate, which determines when external knowledge should be used to improve system responses. I could see something like this becoming standard in RAG apps.

8

Lean agents frameworks are the new hot thing - recently Anthropic, and now Hugging Face. On the surface, less is more, however, there are nuances here: 1) Early agent frameworks focused heavily on orchestration, tool use, structured output, and code execution - capabilities now increasingly built into foundation models themselves. On the one hand, tight vertical integration is great, on the other, so is competition... 2) Space is still very early with many low-ish hanging fruits. I expect the frameworks to focus on these: - More sophisticated memory architectures (including x-session), integrating episodic (specific events), semantic (general knowledge), and procedural (how to do things) memory, leveraging smart abstractions and summarization. - More advanced planning and reasoning - like hierarchical task decomposition, Look-Ahead & Simulation, and integrated Program Synthesis and Discrete Reasoning modules - Better Error detection, self correction strategies, and learning from past mistakes and successes - Better tool understanding and selection - More sophisticated inter-agent communication protocols, dynamic role negotiation systems, and conflict resolution mechanisms.

9

Vishal Sachdev highlights the strategic integration of open source and proprietary tech in architecting tech stacks, developer ecosystems and resulting business models. The world class companies effectively balance value commoditization in open source and value capture in proprietary tech. This is the strategic challenge for companies operating in #blockchain and #AI. I believe those who operate at the intersection of blockchain *and* AI stand to win this strategic battle...

3

1 Comment

Perhaps the fastest inference of Llama 3 70b on Groq! Llama 3 8b clocks at 800-1000 tokens/sec 🤯 Video not sped up and is real-time! Fireworks .ai is also pretty good at 200 tokens/sec I believe! https://lnkd.in/gNXbVfUv #artificialintelligence #ai #generativeai #llms

59

1 Comment

💻 From Curiosity to Code: Building an AI Language Model Block from Scratch! After working with AI language models since GPT-2, I've finally done something I've wanted to do for a long time - building a LLM from scratch myself! My university classes only gave me a quick look at how to build these models, but my real learning came from diving in on my own. (Big thanks to my tokenization project that taught me so much!) This isn't just a tech blog post - it's about sharing what I've learned. My goal is to explain how LLM work in a way that makes sense to other people curious about AI. In this first post of what I hope will be a helpful series, I'll show how to write the code for an attention block from scratch. Whether you're a pro or just getting started, there might be something here for you! All feedback is greatly appreciated to help me get better at writing technical article! https://lnkd.in/gYSaH3XT

5

Many people need to realize that the output of Generative AI models can now be fully controlled—2023 was like ten years ago! With GPT-4-turbo, you can retrieve and augment user input with up to 128,000 tokens! The output will be transparent and reliable for everybody. I'll be sure to break this down for you. 1. The old days (2023) are over in generative AI. The attached notebook open-source, free notebook, shows you how to do it. You can find it in the Chapter07 free access directory of the GitHub repository of Transformers for Natural Language Processing and Computer Vision, 3rd Edition on this page: on my resource page: https://lnkd.in/egKueDhE 2. The code proves the point: you can augment a user input with retrieved documents and control the generative AI Model. Of course, you need to engineer the retrieval properly. 3. Get ready to embark on an exciting journey into the new era of Generative AI! Run the code and explore the endless possibilities that these advancements offer. If you have any questions, you can ask them right away and here in the comment section. #ai #genai #rag

169

27 Comments

Anyone see that study about GPT beating doctors at diagnosis? 🤯 Well, it revealed something that concerns all of us👇 But first, a bit of background: The difference in diagnosis accuracy between doctors (74%) and AI (90%) was 16 percentage points. That means without AI, doctors could MISDIAGNOSE 1.6 people per every 10 patients. But here's what everyone missed: It's not about AI replacing humans—it's about AI taking human potential to the moon. Let me show you how to boost YOUR output by 16% (or more): 👇 1⃣ Time Theft Exposed: Ever tracked where your hours actually go?  Studies show professionals waste: • 4.5 hours weekly on data entry • 5.3 hours scheduling & coordinating • 7.2 hours on routine communications That's 17 hours PER WEEK of repetitive work that could be put to better use! 🤖 2⃣ Your AI Efficiency Blueprint: Here's the secret sauce—design an AI agent that: • Handles your repetitive tasks (automation) • Learns your preferences • Adapts to your workflow • Scales with your needs Real talk: Many companies using AI agents see: • 47% reduction in manual tasks • 3.2 hours saved daily per employee • 28% increase in high-value activities 3⃣ The Delegation Revolution: Your AI assistant can: • Draft and analyze reports • Schedule and prep for meetings • Monitor project progress • Generate content drafts • Flag important updates • Track and measure KPIs All while you focus on what humans do best: Strategic thinking, relationship building, and creative problem-solving. The math is simple: More time on high-impact work = Exponential results 📈 But here's what's crazy: That 16% productivity boost? That's just the beginning. Think about it: What could you achieve with 17 extra hours EVERY week? Because in 2024-25, working harder won't cut it. Working smarter (with AI) will. 💪 Got any new ideas for the conversation? Leave a comment down below if you got any valuable takeaways from this! #AIProductivity #AITips #AIForBusiness #UseAI #AITrends #AIAutomation #ChatGPT #ProductivityHacks #BusinessEfficiency #AIRevolution #AI #ProductivityTips

7

4 Comments

🚀 A few months ago, as a beady-eyed 20-something-year-old on the hunt for the next big thing, I started working on a side project. The idea? Build AI agents that could automate tasks for enterprises. You should have seen me—so raw with ambition, fully convinced I’d be a millionaire in a couple of months. 😅 Long story short, I spent 2 and a half months building an agent that could automate content creation and scheduling (sounds reasonable, right? Nope, wrong!). I failed to account for the fact that this was an insanely competitive market with thousands of players gunning for the exact same ICPs I was after. 💥 A month of shamelessly trying to sell the agent, and I was still nowhere. $0 in the bank, 0 customers, 0 investors. Then, lady luck struck. 🍀 I was introduced to 54 Collective VC's entrepreneur academy, where I met the amazing Michelle Ngure Mwangi 👀 Up until then, I hadn’t talked to a single potential customer. I didn’t know what problem I was solving, had no clue who my customers were (wtf), and had wasted 3 months of my life! Michelle was kind enough to mentor me, practically hand-holding me to success! Within a week, I had spoken to over half a dozen potential customers, nailed down their problems, and even convinced them to pay in advance. 🎉 All this, without writing a single line of code! 🤯 So, how did I do it? 1. Focused on one problem: Instead of a generalist content agent, I zeroed in on retention marketing. DTC brands were spending tens of thousands on retention teams—what if AI could automate some workflows and cut costs? 2. High-fidelity prototype: In 2 days, I built a hi-fi prototype with the core feature I knew my ICP wanted. Shoutout to Mitchelle Agina for helping with the design. 3. Loom video: I recorded a 2-minute Loom video, walking users through the retention marketing agent. 4. Shameless cold emails: I used Apollo.io to find DTC brand owners, cold-emailed them, and asked for a call. Lots of rejections, but a few wins! 5. Sold like my life depended on it: I had no product, just a vision. But with the Loom video and Figma prototype, people were willing to pay upfront—and gave me super useful feedback! 💡 Lesson: You don’t need to spend hundreds of thousands building a product. Sell the vision first, and customers will tell you if they want it or not! --- Follow Kengere Ositu for more AI and product insights. #ai #retention #DTC #marketing #aiagents

6

1 Comment

Is the new GPT-4 model actually that advanced? 🤔 Apparently there is an alternative by Arjun Prasad, Co-Founder and CSO of QX Lab AI - the newly launched multimodal platform, Ask QX PRO. 🚀 Here are some fascinating highlights: - Supports 120 languages 🌍 - Capable of generating images from text, describing images, document analysis, and information extraction 📸📝 - Already boasts over 15 million users! 🌟 Wanna hear what you guys think Nkiru Uwaje, Pavel Belov, Dmitry Nekrasov Link in the comments to give it a try! 👇 Have you explored alternatives to ChatGPT? Share your experiences and let's discuss! 🤔 #AI #Innovation #Technology

17

8 Comments

🆕 Benchmarks 201 with Clémentine Fourrier of the Hugging Face Open LLM Leaderboard! Why Chat Arenas are **OVERRATED**: - they bias sycophancy/assertiveness over factuality - annotators are not representative of humanity - models are argmaxxing public arena data - not reproducible - can't give fine grained test of model capabilities Why you should STOP using LLMs as Judges: - Position bias (prefer first answers) - Self bias (prefer their own answers) - Length bias (prefer long answers) - Discrete bias (can't do continuous ranges) - Reproducibility (GPT4 closed source - prefer Seungone Kim's Prometheus or Beijing Academy of Artificial Intelligence(BAAI) JudgeLM) One of our best ones 🔥

30

1 Comment

📈 10 Facts from a Former OpenAI Employee's Report on the Future of AI Leopold Aschenbrenner, who previously worked in the team of the departed OpenAI co-founder Ilya Sutskever, sparked a widespread discussion with his report titled "Situational Awareness: The Decade Ahead," which you can find here: https://lnkd.in/eDBf8sRY The report discusses what to expect in the next 10 years in the field of AI and whether we should be concerned. Spoiler: ASI and giant investments 📱 10 Predictions by Aschenbrenner ▶️ AGI will emerge by 2027 and will almost immediately transition to ASI (Artificial Superintelligence). This is because AGI will be capable of self-learning and will quickly teach itself. The transition will happen rapidly, and humanity might not play a role in it ▶️ $1 trillion - that's the amount of investment required to build a computational center for creating AGI. For comparison, the cost of a mission to Mars is $10 billion ▶️ This computational center will consume more electricity than a large nuclear power plant can produce. Approximately 20% of the energy that the US produces today. Incidentally, Sam Altman has also spoken about such resource expenditures ▶️ The main race among major powers in the next 10 years will not be for weapons but for achieving AGI. Aschenbrenner compares this to the nuclear arms race in the mid-20th century ▶️ Leading American AI labs are currently not giving enough attention to AGI safety. They are effectively handing over key secrets for creating AGI to Chinese specialists. Protecting these secrets will require enormous effort, and at the moment, the US is not coping with this task ▶️ $100 billion - that's how much NVIDIA, Microsoft, and Google are currently allocating to AI development in their quarterly budgets. By the end of the decade, annual investments will total $8 trillion ▶️ In 2024, producing AI chips may require 3-10% of the annual production capacity of advanced semiconductor factories ▶️ By 2030, global demand for production capacities for AI chips will be four times the current capabilities of TSMC, the leading chip manufacturer in Taiwan ▶️ By 2027, prototypes of automated engineers capable of accelerating technological processes in AI by 1.5-2 times will appear. By 2028, such systems will be able to automate more than 90% of research tasks, leading to significant acceleration in achieving AGI ▶️ Training the GPT-4 model required 3-10 thousand times more computational resources than GPT-2. One can only guess how many resources will be needed in the future Aschenbrenner discussed his report in more detail on a podcast with well-known blogger Dwarkesh Patel. Note: the episode is 4 hours long! You can find it here: https://lnkd.in/eRSrXsNj You can read more interesting stories and news on our Telegram channel: https://t.me/hiaimediaen #AI #artificialintelligence #openai #llm #report #news

7

2 Comments

The AI hype is palpable and it should be as long as the question we're answering is: how genuinely are businesses enhancing customer experiences with AI. Anything else is cost cutting that historically has resulted in a garbage customer experience, not enhanced. Expect lowered LTV, increased attrition and your customers to think of you and your product as a vendor, not partner if you don’t thread the needle.

12