Mahak Patidar

🌟 Exploring the Intersection of Business and Technology re AI! 🌟 Before my decade-long focus on AI, connecting technology and business in various domains has always been my passion; This is why I enjoyed this conversation with David Luan - No BS, No Twitter exaggerations, No clickbate - Just a super well informed, super smart David sharing his insights from years of hands-on experience: #Startegy, #Software, #Hardware, #Platforms, #ModelSpace and everything in between - LMK what you think of his projections. In This Episode: 1. The Biggest Lessons from OpenAI and Google Brain: What did OpenAI realize that no one else did that allowed them to steal the show with ChatGPT? Why did it take 6 years after the introduction of transformers for ChatGPT to be released? What are 1-2 of David’s biggest lessons from his time leading teams at OpenAI and Google Brain? 2. Foundation Models: The Hard Truths: Why does David strongly disagree that the performance of foundation models is at a stage of diminishing returns? Why does David believe there will only be 5-7 foundation model providers? What will separate those who win vs those who do not? Does David believe we are seeing the commoditization of foundation models? How and when will we solve core problems of both reasoning and memory for foundation models? 3. Bunding vs Unbundling: Why Chips Are Coming for Models: Why does David believe that Jensen and Nvidia have to move into the model layer to sustain their competitive advantage? Why does David believe that the largest model providers have to make their own chips to make their business model sustainable? What does David believe is the future of the chip and infrastructure layer? 4. The Application Layer: Why Everyone Will Have an Agent: What is the difference between traditional RPA vs agents? Why are agents a 1,000x larger business than RPA? In a world where everyone has an agent, what does the future of work look like? Why does David disagree with the notion of “selling the work” and not the tool? What is the business model for the next generation of application-layer AI companies? https://lnkd.in/dUpWWzvN

4

SEMICON WEST is on fire 24x7 Tuesday afternoon continued the focus on Front-End Manufacturing. Great session on wafer map visual analytics with Janhavi Giri, Intel, using self-supervised learning for image analysis, detecting failures and optimizing manufacturing processes. Naor Alfassi, Applied Materials, spoke about ML-guided Electron Beam review for wafermap image classification and defect detection. Janhavi Giri, PhD Naor Alfassi Sessions on virtual metrology from Srividya Jayaram, Siemens, on high product mix; and Mike Yim, Gauss Labs, on high-volume, from Lab to Fab. In our experience, virtual metrology has led to massive time savings in product testing.  Srividya Jayaram Mike Young-Han Kim One very fun and topical presentation was from Christoher Nguyen, Aitomatic, and Daisuke Oku, Tokyo Electron - announcing SemiKong, a new Open-Source Semiconductor LLM, jointly developed by Tokyo Electron, Aitomatic, FPT AI Center and the AI Alliance. This seems to be based on Meta's Llama3 with additional HTM context. Christopher Cuong T. Nguyen Daisuke Oku I just returned from visiting our HTM customers in Japan and Korea, where we showed our Spotfire Copilot running in Japanese and Korean .. and this presentation on SemiKong was a joy to see!! Feel free to check out my latest blog on the Japan visit including a Spotfire Seminar with our partner NTT Com and customers including Texas Instruments, Tokyo Electron, Toshiba, Sony and many others. MOC Blog on Spotfire Seminar in Tokyo, Japan https://lnkd.in/gAeSZSSC   Wednesday sessions in Smart Manufacturing focused on how AI/ML can optimize back-end semiconductor manufacturing process eg OEE, predictive and condition-based maintenance.. We are currently revamping our Spotfire OEE solution with Spofire Copilot, so this is quite timely. Shawn Sarbacker, Kulicke and Soffa,  gave a nice presentation on Semiconductor Assembly.  Shawn Sarbacker I really enjoyed the session on Zero Defect Manufacturing from Prasad Bachiraju, Onto Innovation. Totally aligned with our Spotfire work on visual data science for defect analysis .. interesting backend data integration .. we’ve been working on data federation and libraries of virtualization views for that part of the data and analytics flow.  Prasad Bachiraju I also liked the IBM WatsonX GenAI session from Megan Arp and Francois Bernard - many similarities to our Spotfire Copilot e.g. they refer to the “Watson Orchestrator” in an analogous architecture to our Spofire Copilot Orchestrator. Their LLM is able to predict user intent when a technician reviews a ticket (i.e., query vs. task). Tasks are handled through custom and prebuilt skills training, while queries are handled through RAG. Megan Arp Francois Bernard Spotfire #VisualDataScience https://lnkd.in/eSeVcE5h https://lnkd.in/ecWq3PSH

62

4 Comments

🚀 NVIDIA NIM on Bedrock Marketplace 🚀 We’re thrilled to announce that NVIDIA NIM are now available on Amazon Bedrock Marketplace, SageMaker JumpStart, and AWS Marketplace. With this expansion, developers can now deploy high-performance, NVIDIA-optimized inference solutions for generative AI models across key AWS services, delivering: ✅ Faster AI inference ✅ Lower latency ✅ Cost efficiency From Meta’s Llama 3 to NVIDIA Nemotron and more, over 100 prebuilt NIM microservices are now at your fingertips. Tailored for scalability and secure deployment, NIM brings unmatched performance to models running on EC2, EKS, SageMaker, and Amazon Bedrock. This collaboration between NVIDIA and AWS accelerates innovation across industries, from healthcare to manufacturing, ensuring enterprises can meet the growing demand for generative AI applications. 💡 Ready to unlock the power of NVIDIA NIM on AWS? Explore the catalog, request a developer license, or start a 90-day NVIDIA AI Enterprise trial to experience the difference firsthand. Another great project by this wonderful team! Abhishek Sawarkar Jiahong Liu Kshitiz Gupta Timothy Ma Saurabh Trikande Charlie Huang Rohil Bhargava Let’s power the next wave of AI together! #GenerativeAI #NVIDIA #AWS #Bedrock #AIInference #AWSreInvent2024

44

1 Comment

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

8 Comments

Universal Basic Income seems to have gone by the wayside since Andre Yang brought this up in his 2020 presidential bid. This blurb by James O'Brien, CS professor at UC Berkeley, makes a case why his future CS graduates may be replaced by automation and AI. https://lnkd.in/gYhjK3zr Crazy to think how new CS college graduates will have fewer opportunities to grow their skills when AI is sucking up the entry level CS jobs. The professor's comments on students now switching focus to AI and data science is prescient. What focus could be next when AI and data science human software engineering expertise needs wain? The last two major union strikes in the US faced major roadblocks negotiating the roles of automation, robotics and AI replacing skilled union workers. This is no longer a blue collar only concern and I appreciate the professor's comments about corporations reaping the benefits of AI, robotics and automation (e.g. drastically reduced headcount and overhead) paying those benefits forward as the primary feeder for UBI. Or we could wait for a major planet life altering event (e.g. Pandemic) to thin the herd. Perhaps a bigger question to ponder is what happens if a massive electromagnetic radiation storm takes out those AI datacenters and there is no one left with the skills to replace the machines;-)

23

Meet AGUVIS: A pure vision-based framework for autonomous Computer/Mobile-use GUI agents, operating seamlessly across web, desktop, and mobile platforms without UI code. Key Features & Contributions - Pure Vision Framework: First fully autonomous pure vision GUI agent capable of performing tasks independently without relying on closed-source models - Cross-Platform Unification: Unified action space and plugin system that works consistently across different GUI environments - Comprehensive Dataset: Large-scale dataset of GUI agent trajectories with multimodal grounding and reasoning - Two-Stage Training: Novel training pipeline focusing on GUI grounding followed by planning and reasoning - Inner Monologue: Explicit planning and reasoning capabilities integrated into the model training - Generalization to OOD (Out-of-Distribution): AGUVIS demonstrates impressive generalization on OSWorld benchmark even without desktop trajectory data during training. Project Page: https://lnkd.in/guzkwpbB Paper: https://lnkd.in/gs7wXyJS GitHub: https://lnkd.in/gWeSkFmK Collaboratiion among Yiheng Xu Zekun Wang Junli Wang Dunjie Lu Tianbao Xie Amrita Saha Doyen Sahoo Tao Yu Caiming Xiong

156

6 Comments

Arm is upstreaming #Kleidi technology into #PyTorch with amazing effect - up to 3x faster CPU #inference in Eager mode, and up to 2x in Graph mode. These optimizations make running #LLMs on CPU more practical than ever. Check out our blog for more details! #Arm #Neoverse #Graviton https://lnkd.in/gvrh67v2

17

Jensen Huang’s reflections strip away the glamour of entrepreneurship to reveal the stark reality of building NVIDIA showcasing the emotional toll and relentless grind behind the scenes. In the startup world, the actual essence of entrepreneurship is often overshadowed by the glamor of ‘fail fast, fail often.’ It’s not just about bold ideas and bigger bets; it’s about enduring doubts, managing fears of failure, and the resilience to stand back up, bruised but unbroken. It’s about the nights spent wrestling with the paradox of choice, the weight of decision-making, and the echoes of what if. Just like parenting a child with special needs, the journey is anything but linear. It's a mosaic of highs and lows, of victories and setbacks. It requires a heart willing to traverse the spectrum of human emotions, a spirit resilient enough to rebuild from scratch, and a resolve to redefine success on one's own terms. This perspective is not for everyone. It's for the brave, for the heartfelt warriors of life who dare to embrace the different, to see beyond the ordinary, and to aspire for the extraordinary, even when faced with the most ordinary of days.

18

1 Comment

🔍 Key Papers to Demystify How GPT-o1 Works 1) Training Language Models to Self-Correct via Reinforcement Learning: https://lnkd.in/gwgdE5fb 2) STaR: Bootstrapping Reasoning With Reasoning: https://lnkd.in/gRdir7PP 3) Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking: https://lnkd.in/g47fqB7p 4) V-STaR: Training Verifiers for Self-Taught Reasoners https://lnkd.in/gGRRyBrF 5) Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters: https://lnkd.in/g9atmpvC 6) Chain of Thought Empowers Transformers to Solve Inherently Serial Problems: https://lnkd.in/gCyMncsf 7) Large Language Monkeys: Scaling Inference Compute with Repeated Sampling https://lnkd.in/ghy2r5nH 8) An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models: https://lnkd.in/gRHU2xyU 💡 Stay Tuned! I will continually update this post with more insights and a special focus on the "how" aspects of these papers.

20

2 Comments

The keynote speech by Trevor Cai from #OpenAI at #HotChips 2024 was interesting. Trevor's presentation reiterated the ongoing hypothesis at OpenAI—that the AI models must continue to grow to achieve superior performance, with no apparent end in sight... He bolstered this argument using a log-log power law plot, which essentially plots the relationship between compute (in FLOPs) and model performance (i.e., loss) on a logarithmic scale. This approach has enabled OpenAI to accurately predict the performance of models like GPT-4 by curve-fitting the smaller models on the plot, thus demonstrating the scalability of this method. Trevor then drew a fascinating parallel to Moore's Law, which famously predicted the doubling of transistors on chips every two years. This prediction held true for decades, fueling massive advancements in semiconductor technology. He suggested that similar underlying principles might sustain the exponential growth of AI models for a significant time, though the mechanisms might evolve. However, Trevor notes that such growth isn't without its challenges. He highlighted several critical issues, including the need for advanced data centers, enhanced reliability, availability, and serviceability (RAS) systems, and more sophisticated power management strategies. The massive scale of synchronized training introduces vulnerabilities like power draw jitters and single-point failures that can cause widespread disruptions. If synchronized training causes so many challenges, I wonder if the research community is focusing on alternative algorithms for LLMs that don't require gradient synchronization after each iteration.? 🤔 💡 Trevor points out that as AI models became more intelligent, the demand for inference has grown significantly. He notes that while training infrastructure can be planned and scaled, inference capacity must be more responsive to actual usage and demands, making planning more challenging and dynamic. The technology and economics of inference will continue to evolve. While the conference is about hot chips, he has no specific recommendations on GPU/accelerator architectures, whether inference could benefit from custom hardware, etc. Overall, it is evident that openAI will continue to grow the models and datasets it trained on for each new release of GPT and will continue to challenge the AI/GPU and networking hardware industry to support this with technological advancements. At some point in the future, more than half the data on the Internet will have originated from the LLM models. At that point, how would the log-log plot look? 🤔 Wouldn’t models trained on their self-generated data collapse, and the losses start to grow? He acknowledges the problem in Q&A and says they are "investigating" techniques to overcome this…. Any thoughts? #AI #LLMs #datacenters #junipernetworks

82

8 Comments

MY WEEK IN AI: Building on last week's focus on #AI execution, it’s amazing that just a week later we’re seeing so much coverage about the emergence of robust systems, governance frameworks, and strategic partnerships – all critical for enterprise-scale AI deployment. It’s clear the pragmatic AI era demands strategic, responsible implementation that’s focused on measurable impact. Success requires balancing innovation with practical considerations in infrastructure, compliance, and operational integration. My key takeaways from the flurry of development: • The "Last Mile Problem" in AI deployment is shifting from technical integration to organizational adaptation. Success requires equal investment in technology and change management.  • We're entering the "Ecosystem AI" era where value comes from combining multiple AI technologies (LLMs, computer vision, predictive analytics) rather than single-point solutions.  • The emerging "AI Operating System" concept suggests organizations need to build foundational AI capabilities that can scale across use cases rather than pursuing siloed implementations. Here are the stories that stood out for me this week: • Biden to further limit Nvidia AI chip exports in final push: https://lnkd.in/ew96pRQA • AWS bullish about GenAI demand with $11B data center investment in Georgia: https://lnkd.in/eERuUdqd • FDA issues draft guidance on marketing submissions for AI-enabled medical devices: https://lnkd.in/efwDiVTm • US Steel streamlines procurement processes with AI-powered platform: https://lnkd.in/eJSsbSVY • AI will protect global supply chains from the next major shock: https://lnkd.in/eX5_uNGt • Novo Nordisk expands AI partnership with Valo Health for development of obesity drugs: https://lnkd.in/ektAebMS

47

6 Comments

In 2011, Meena Bakhai, a first-generation immigrant and former electrical engineer, took a bold step. She invested $10,000 in Nvidia at just $0.37 per share. Today, that investment is worth a staggering $3.5 million. What was her secret? A combination of technical expertise and long-term conviction. Ms Bakhai, leveraging her professional expertise as an engineer in the space, saw the potential in semiconductors and trusted her instincts. 💡 Her career knowledge created her conviction. ↩ 💡 Her conviction, in turn, allowed her to hold out over the past 13 years, through all the peaks and troughs. ↩ 💡 That conviction has created an enviable fortune for Meena and her family. The question is — what are you doing with your money to create such opportunities? You don’t need to be an engineer to make smart financial choices, but tapping into your own expertise can give you an edge. Whether it’s your industry knowledge or insights from your career, it’s about putting your skills to work and making your money do the same. Long-term strategy, thoughtful risk, and making informed decisions — these are the keys. What plans are you making to unlock your potential? #sydneyfinance #sydneyfinancialplanner #financialadviser #financialadvisor #financialfreedom #financialindependence #mindset #lifeplanning #nvidia #stocks #sharetrading #semiconductors

3

It's just wow and exciting for me to know the vision of leaders who runs the best data, internet, AI company to build Intelligence Infrastructure for the benefit of 8 billion people across the globe to make the world, an intelligent society. The conversion between Jensen Huang and Mukesh Ambani on building AI Infrastructure in #India that will the beginning of the modern journey towards making the new Intelligent World. As its well articulated by Mukesh on PM Narendra Modi's vision for superpower India, just not only on #technology but across different sectors such as #pharma #enery #automobile, it feels like 21st century belongs to India. Let's be more confident and courageous in making #India as a #superpower by working together. #BuildAIInfrastructure #IndiaInnovation #LargestTalentPool #Opportunities #TimeToBeInIndia #8BillionPeople #OneFamily #IndiaSuperPower

10

1 Comment

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

If you are still thinking that AI is just a hype, you should listen the conversation between Mukesh Ambani and NVIDIA CEO Jensen Huang. Why ? Mukesh Ambani bullishly talks about Generative AI models, GPUs, need of open source models etc. He takes up the challenge to surprise world with AI capabilities in coming years. He is not someone who has been leading IT/Software companies day in day out like Narayan Murthy/ Shiv Nadar/Bill Gates etc. What it means? AI has already arrived and every big firm, irrespective of their traditional business, wants to build business and create impact using this. AI is not just another cool tech anymore. It is the industrial revolution changing every facet of our life. Embrace it or get ready for classic "Kodak Case study" in your career and business. Watch here: https://lnkd.in/ejvnCMHZ

10

NVIDIA has introduced something amazing—Project DIGITS, which could bring supercomputers into our homes. This means we might soon be able to use powerful AI tools, do advanced research, or create incredible things right from our own desks. It makes you wonder: what’s next? Could we soon see AI in every device we use, making our daily lives even easier and more connected? The possibilities are exciting, and the future looks full of opportunities.

4

🌟 Unleashing Potential with NVIDIA Grace CPU: Engineered for the Largest Computing Challenges 🚀 As we navigate the complexities of modern AI models, the demand for enhanced computing power and improved energy efficiency is at an all-time high. The NVIDIA Grace™ CPU steps up as a transformative force in this dynamic landscape. 👾 Purpose-Built Excellence: Specifically designed to address the world’s most substantial computing challenges, the Grace CPU is an exceptional Arm® CPU that sets a new standard in performance and efficiency. 🔗 Dual Capability: Grace excels in versatility. It can be seamlessly integrated with GPUs to amplify accelerated computing, or it can operate as a potent and efficient standalone CPU, making it the bedrock for next-generation data centers. 🔄 Flexible Configurations: Adaptability is key. Grace supports a variety of configurations to meet different needs within data centers, ensuring optimal performance across various applications. 🌐 Introducing NVIDIA GB200 NVL2: Elevating every data center into the new era of computing, the GB200 NVL2 platform delivers unrivaled performance for tasks like large language model inference, vector database search, and intricate data processing. Embrace the future of data centers with NVIDIA's cutting-edge technology, crafted to push boundaries and drive innovation. #NVIDIA #GraceCPU #DataCenters #AI #MachineLearning #DeepLearning #TechInnovation #FutureOfWork

11

#DYK While pre-trained large language models (LLMs) excel at factual tasks after fine-tuning, their ability to access and update knowledge remains limited. Enter Retrieval-Augmented Generation (RAG), which overcomes these limitations by combining LLMs with a retrieval component. 💥 Interested in learning more? Check out this blog to see how you can have a basic RAG solution up and running for your documents with minimal clicks and code copy-pasting: https://dell.to/4c438Ds #AI #Dell #DellTechnologies #HuggingFace #AImadeeasy #iwork4dell #iwork4dell

10