Bill Huang

𝐓𝐡𝐞 𝐂𝐮𝐫𝐫𝐞𝐧𝐭 𝐒𝐭𝐚𝐭𝐞 𝐨𝐟 𝐀𝐈 𝐌𝐚𝐫𝐤𝐞𝐭𝐬 - 𝐌𝐚𝐩𝐩𝐢𝐧𝐠 𝐨𝐮𝐭 𝐭𝐡𝐞 𝐜𝐮𝐫𝐫𝐞𝐧𝐭 𝐬𝐭𝐚𝐭𝐞 𝐨𝐟 𝐀𝐈 𝐦𝐚𝐫𝐤𝐞𝐭𝐬: A good high-level overview of the AI industry by Felicis. Key players today are focusing on infrastructure and data centers: 🔹 Semiconductor Ecosystem is now over $130B in revenue this year from AI. 🔹 The number of U.S. data centers is expected to double in the next four years, but energy constraints remain a significant bottleneck. ⚡ Hyperscalers and developers are aggressively acquiring real estate with available power sources. 🌩️ AI cloud services are now at $20B run rate business, with Microsoft generating $5B of that. Large-scale value creation is still in its early stages. More ➡️ https://lnkd.in/guJZmziK #AI #GenAI #LLM #MLLM #AIMarket #AIInfrastructure #DataCenters #SemiconductorIndustry #CloudComputing #AIApplications #DigitalTransformation #FutureOfAI #Hyperscalers

6

Nvidia CEO Jensen Huang expressed strong optimism about the growing role of AI agents during a Q&A session with financial analysts at CES 2025 in Las Vegas. He predicted that AI agents—programs capable of taking simple instructions to execute complex, multistep tasks—would gain significant traction in 2025. Huang sees this technology as transformative, particularly for knowledge workers, and a major driver of future tech growth. Key Points on AI Agents: Capabilities and Applications: AI agents can automate tedious tasks, such as data entry and form-filling. For example, Anthropic has already demonstrated AI agents performing actions like observing screens, moving cursors, and filling out forms. Initial Impact Areas: Huang identified software engineering, digital marketing, and customer service as the first sectors where AI agents could add substantial value. He noted these fields represent a significant portion of the global knowledge workforce. Nvidia's Use of AI Agents: Nvidia is actively deploying AI agents internally, particularly in chip design, to accelerate its processes. Huang emphasized the company's commitment to integrating AI agents across its operations. Implications for Nvidia: The adoption of AI agents is expected to require increased computing capacity, which would boost demand for Nvidia's GPUs, positively impacting its revenue. Huang highlighted that Nvidia is strategically positioned to benefit from this trend. CES Announcements: In addition to discussing AI agents, Huang unveiled Nvidia’s new RTX 50 Series gaming GPUs, based on the advanced Blackwell chip architecture. This lineup is expected to cater to gamers and high-performance computing needs, further solidifying Nvidia's leadership in the GPU market. Huang’s remarks underscore Nvidia’s confidence in the transformative potential of AI agents and their alignment with the company’s growth strategy.

6

3 Comments

US datacenter construction skyrocketing. #nvidia #artificialintelligence #llms #chatgpt #generativeai #gpu #servers #datacenter #cloud #ai #technology #innovation #business #Semiconductorindustry #electronics #semiconductor #supplychain

2

Check out these #ai highlights!

1

Just witnessed the most epic impromptu panel discussion take-over by none other than John Chen (VP of Technology at NVIDIA) at Monte Jade West Science & Technology Association talk with Brandon Wang, Wen H. Hsieh, Ph.D., Jay Lee, Achin Bhowmik and Wei Li: GPUs are far ahead of all the rest of the infrastructure behind AI/LLM datacenters. It’s time to invest in the next frontier of AI infrastructure folks! Not just Cloud compute, hybrid cloud+edge, networking, power, edge RL, edge inference and edge firmware orchestration just to name a few areas we’re excited about!

75

5 Comments

https://lnkd.in/gr6DCzCQ Demand for NVIDIA’s Blackwell Platform Expected to Boost TSMC’s CoWoS Total Capacity by Over 150% in 2024. #nvidia #artificialintelligence #llms #generativeai #gpu #servers #datacenter #cloud #ai #Semiconductorindustry #electronics #semiconductor #supplychain #engineering #manufacturing #technology #computerchips #business #innovation #semiconductors #chips #chipmaker #foundry credit: trendforce

9

A good read how to do performance tuning for serving AI models. “To summarize, the performance bottleneck of vLLM is mainly caused by the CPU overhead that blocks the GPU execution.” If we can model everything beforehand, we should not wait for one year to locate the bottleneck. There are still a lot of cross-layer optimization opportunity for AI model serving. With hardware software co-design, there are more rooms for improving efficiency.

11

New Tesla Patent Reveals Clues to an Exciting Future of Humanoid Robots and AI Integration In a interesting discussion, Herbert Ong and Scott Walter dive deep into Tesla's recently published patent application WO2024072984A1 (link in comments). Entitled "Actuator and actuator design methodology," the patent primarily focuses on controlling robot movement through strategic actuator placement. However, the document also unveils important clues about the potential for end-to-end neural net training of humanoid robots, reminiscent of Tesla's approach to training its vehicles with FSD. Ong and Walter explore the concept of personalized humanoid skills. Imagine purchasing a bot with built-in capabilities, then having the option to augment and customize those skills through a "Skills Store" akin to Apple's App Store. This opens up a world of possibilities for tailoring robots to specific tasks and preferences. The discussion also touches on the potential for various robot form factors, each designed to excel at particular skills. Picture a tennis-playing robot that comes pre-trained with basic volleying instructions, but requires a "Skills Upgrade" to perform at a master level. This raises questions about the need to manufacture and train a diverse range of robot body types and sizes, mirroring how humans possess ideal physiological characteristics for different activities. Later, Ong and Walter discuss a scenario in which Tesla builds the hardware – including automobiles, humanoids, and other bot form factors – while xAI develops the brains. This arrangement opens up opportunities for licensing agreements between the two companies, that may help the companies navigate future antitrust concerns as these technologies are deployed at scale. The convergence of advanced robotics and artificial intelligence holds the potential to catalyze a profound societal transformation. As Tesla and xAI continue to push the boundaries of what's possible, I can envision a future where intelligent, adaptable robots become integral to various aspects of our lives – from manufacturing and transportation to healthcare, recreation, and beyond. The seamless integration of cutting-edge hardware and software will revolutionize the way we live and work, ushering in an era of unprecedented efficiency, productivity, and innovation. While there are still challenges to overcome and ethical considerations to address, the prospect of these combined technologies driving positive change on a global scale is awe-inspiring. #FrontierFrontier #Humanoids #AI #Robots

7

1 Comment

Jensen Huang shows off the NVIDIA GB200 NVL72: a data center superchip with 72 Blackwell GPUs, 1.4 exaFLOPS of compute and 130 trillion transistors.

3,255

113 Comments

Researchers from NVIDIA, CMU and the University of Washington Released ‘FlashInfer’: A Kernel Library that Provides State-of-the-Art Kernel Implementations for LLM Inference and Serving FlashInfer incorporates a block-sparse format to handle heterogeneous KV-cache storage efficiently and employs dynamic, load-balanced scheduling to optimize GPU usage. With integration into popular LLM serving frameworks like SGLang, vLLM, and MLC-Engine, FlashInfer offers a practical and adaptable approach to improving inference performance. FlashInfer's unique features include: ✅ Comprehensive Attention Kernels: covering prefill/decode/append attention for various KV-Cache formats (Page Table, Ragged Tensor, etc.) for both single-request and batch-serving scenarios. ✅ Optimized Shared-Prefix Batch Decoding: 31x faster than vLLM's Page Attention implementation for long prompt large batch decoding. ✅ Efficient Attention for Compressed KV-Cache: optimized grouped-query attention with Tensor Cores (3x faster than vLLM's GQA), fused-RoPE attention, and high-performance quantized attention...... Read the full article here: https://lnkd.in/g69DqsgM Paper: https://lnkd.in/g4KK_9x4 GitHub: https://lnkd.in/gY5MeJYC NVIDIA NVIDIA AI Machine Learning Department at CMU Zihao Ye Lequn Chen Ruihang Lai Wuwei Lin Yineng Zhang Stephanie Wang Tianqi Chen Baris Kasikci Vinod Grover Arvind Krishnamurthy Luis Ceze

135

5 Comments

Flow Computing, 100X CPU performance company, a fairly new addition to our portfolio, just published their 4M€ pre-seed round. I ended up making pretty hefty predictions in the press release in relation to their impact on advancing the baseline performance of computing. Anyhow, I think it's true - at least what comes to next 10 years or so, possibly much longer. Lets hope so. :-D It's not just I don't want to turn out to be proven dumb optimist (which I perhaps am regardless), but this level of performance gains will have quite substantial effect on many industries... #flowcomputing #supercpu #parallelprocessing #computing #seriousperformance #cputech #deeptech I'm sure the entire Butterfly Ventures team is as thrilled as I am. In deed, super cool tech from VTT with solid scientific backing, peer reviewed articles, IP portfolio and excellent founder team. Also super happy to collaborate with Sarsia, Superhero Capital, Stephen Industries and FOV Ventures.

83

4 Comments

What if AI could find common ground between something as intricate as biological tissue and Beethoven's "Symphony No. 9"? 🎹🧬 At first glance, these might seem worlds apart, but a groundbreaking AI method developed by Markus J. Buehler at MIT is bridging this gap, uncovering shared patterns of complexity and order. 💡 By combining generative AI with graph-based computational tools, this innovative approach reveals new ideas and designs that were previously unimaginable. The method leverages category theory, a branch of mathematics focusing on abstract structures and their relationships, to teach AI how to understand symbolic connections in science. This allows AI to go beyond simple analogies and engage in deeper reasoning across different domains. 🌍 This graph-based generative AI achieves a greater level of novelty and exploration than traditional methods, providing a powerful framework for innovation by revealing hidden connections. How do you see AI reshaping our understanding of seemingly unrelated fields, and what potential applications excite you the most? 🤔 #AI #Science #Technology #Innovation #Connectivity

3

Within the “terahertz technology gap” (100 GHz - 30 THz) lies a vast reserve of bandwidth poised for development. “The carrier frequency of 300 GHz is 150 times larger than two gigahertz where the current cellular network is today,” explains Research Scientist James Greenberg, a program manager at IMRA. “So, take the number of devices we currently have and imagine multiplying that by a factor of 150. That's the biggest driver in going into a higher frequency.” However, the development of user-friendly technologies capable of generating, manipulating, and detecting radiation within these frequency bands is not only technically challenging but also hindered by the limitations of available THz-capable components. At Micro Harmonics, we are developing new components that enable scientists to design better and faster mm-wave and terahertz systems. Read more about how our #mmWave isolators allowed researchers at IMRA to have a significant breakthrough: https://lnkd.in/gtFm-Nk5 #mmWave #MicroHarmonics #rfengineering #IMRA

13

Got a chance to discuss TCG Key Per I/O capabilities and its possible use cases with confidential compute at the Confidential Compute Consoritium's TAC call yesterday. Please see the link below. The topic is covered from around 20 mins into the video and runs till about the 1-hour mark. https://lnkd.in/gvupy8iP

20