Rajesh Shenoy

"Productization is an affirmation," writes Adrian Bridgwater, "when any good or service evolves to a point where it can be productized and boxed, it denotes a certain solidity and branded consumability." In our conversation, I was delighted to share how #SAS equips organizations with readily deployable AI technology to productionize real-world use cases. https://lnkd.in/enUGCgAi #AIML #sasusers

9

Kudos to Kshitiz Parashar for his pioneering work on Agentic Chunking! His latest primer has quickly become the must-read guide for AI Application Engineers looking to optimize LLMs for smarter chunking, better retrieval, and sharper accuracy. This Speaks volumes about the calibre of the team which I am super proud of.  Even a lean, dedicated group can now harness the power of LLMs to design state-of-the-art architectures—proof that AI is leveling the playing field like never before. Ready to dive in? Explore the full article here: https://lnkd.in/eJ8cem3x #aiagents #chunking

37

1 Comment

$2 H100s: How the GPU Bubble Burst Last year, H100s were $8/hr if you could get them. Today, there's 7 different resale markets selling them under $2. What happened? The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead. https://lnkd.in/gpQEidmC

2

1 Comment

Welcome to the Gemini 2.0 era ✨ Today we announced our most capable AI model yet, designed for the future of agents. “If Gemini 1.0 was about organizing and understanding information, Gemini 2.0 is about making it much more useful.”[https://lnkd.in/gzWTHFXq]

455

3 Comments

The cost of training top models is increasing 2-3X per year, and at this rate will reach $1 billion per model by 2027. The findings are from Epoch AI and follow their recent analysis of increases in compute for foundation models running at 4-5X per year. They've also provided some interesting data on the relative breakdown of cost components like staff, hardware, and energy. A few thoughts after reading this: - I would assume staff costs won't increase as much as hardware and energy costs, so the relative contribution of human labor to future AI models will decrease, in yet another sign of capital's increasing dominance over labor in the industry (just look at NVIDIA's ever-rising stock price). - If compute is increasing 4-5X per year but cost is increasing 2-3X, is it safe to assume that training runs are getting more efficient? - Some people, like Anthropic CEO Dario Amodei, have said we'll see $1 billion training runs in the next year. Does this mean the 2-3X pace is about to increase? That Epoch is wrong in its estimate? Or that Amodei is overstating the cost, possibly to put fear into competitors who can't keep pace? - Only a few companies are going to be able to train future models. We're likely going to see further consolidation. Perhaps we get to a future with only two large general purpose models, like the Windows/MacOS and Android/iOS duopolies we've seen in the past. Article here: https://lnkd.in/g5SejXen

7

Gemini 2.0 handles complex math and planning tasks previously out of reach of LLMs. You can turn on "code generation" in AI Studio and in the APIs. For context, let's talk about function calling and voice AI agents. (I spend a lot of time working on voice AI.) Voice AI agents use function calls for: - Interacting with external systems specific to particular use cases and deployment contexts. - A wide range of RAG-like information lookup tasks. - Conversation persistence, including complex actions like saving a privacy-redacted version of a conversation. - Script following. Some of these uses were obvious before we had built and deployed production voice AI systems. But some of them weren't. (Script following, for example.) We had to build experiments, and then production systems, to figure out what features and tooling these new systems really need. I'm convinced that code generation will be as important as function calling, once we've accumulated some experience building with Gemini 2.0 and similarly capable new LLM versions. Credit for this insight goes entirely to Shrestha Basu Mallick, who shepherds work at Google on the Gemini APIs and AI Studio. When Shrestha first told me that really good code generation was going to make a lot of new things possible, I didn't really get it. But after playing with code generation for a few weeks ... I still don't fully get it, but in a good way, now. It's clear that there are some obvious things to explore: - Context-aware math on demand driven by natural language. - Generative UI. - Better "binding" behavior with function calling. But I'm also convinced now that there is a lot of unexplored, non-obvious (at least to me, yet) territory. Let's take that last one — improved function calling performance. Function calling and code generation go together like peanut butter and jelly. Functions often take arguments that are fairly complicated. With code generation enabled, Gemini has access to this new pattern: natural language ➡️ code that creates argument structs for functions ➡️ multiple function calls Think about how powerful this is. The ability to use even very basic code structures — loops, for example — massively expands what LLMs can do. In fact, my impression is that function calling in Gemini 2.0 is now trained as a subset of the model's code generation capabilities. This feels quite different than how function calling works in other models that I have access to. In other models, function calling feels like a close cousin to structured output. Which is great. But this new "modality" is even better. (More on the prompt in this post, and the meta-prompt for this post, here: https://lnkd.in/gycKtnNX)

49

10 Comments

A thoughtful interview on the future of ML and some strategic pressure in the ecosystem from an R&D and business standpoint. Thanks Yariv Adan for calling it out

2

Reimagining Healthcare Workforce Platfforms: Turning Capital Into Labor The future isn’t just about filling shifts. It’s about scaling labor intelligently, driving better outcomes for systems, staff, and patients alike. Andreessen Horowitz and Alastair (Alex) Rampell reinforced this Vision in a recent post and it applies to the next phase of healthcare workforce evolution. Healthcare workforce marketplaces plus AI are at the tipping point of transformation, evolving into systems of record that don’t just manage labor—they optimize it. Much like EHRs centralized patient care data, these marketplaces can centralize workforce operations, aligning talent supply with clinical demand. But there’s a bigger opportunity here: turning capital into labor. Here’s what I mean. Historically, industries (including healthcare) have focused on enterprise computing tasks—streamlining data entry, compliance, and back-office functions. While valuable, this approach often stops short of transforming the real-world, repetitive processes that make up the bulk of work in healthcare. Think beyond enterprise software. The next frontier lies in leveraging embodied AI and workforce marketplaces to automate and enhance labor-driven processes, transforming capital investments into productive labor outputs. For healthcare, this means turning staffing costs into scalable, dynamic systems that deliver value far beyond filling shifts: 1. Dynamic Labor Matching: Capital invested in marketplace platforms enables real-time matching of labor supply with patient demand, replacing outdated scheduling models with predictive, automated systems. 2. Workflow Automation: From credentialing to compliance and shift bidding, embodied AI can streamline traditionally manual tasks, freeing human resources for higher-value work. 3. Labor as a Strategic Asset: Workforce marketplaces shift the focus from static staffing budgets to proactive labor deployment, ensuring that every dollar invested in labor drives outcomes and efficiency. It is about the billions of repetitive, planned tasks that AI can improve. In healthcare, that means automating everything from staffing logistics to clinical task assignments, creating a horizontal system of engagement for labor (think Zendesk) that integrates seamlessly with vertical, workflow-specific platforms (think Toast for staffing operations). The result? Hundreds of billions in untapped market potential—and a healthcare workforce model that transforms how care is delivered. By turning capital into labor through smarter marketplaces and automation, we don’t just solve staffing challenges; we redefine what’s possible in healthcare operations and ShiftMed is leading the way with health systems moving beyond tired models. #capitalintolabor #systemsofrecord #healthworkforce #aiworkforce #healthsystems #hospitals #cno #hospitalceo #healthcareinnovation

11

2 Comments

Exciting and pleasantly surprising news in the AI world: US & China join forces to develop first international AI standard. A promising step towards rapid, scalable, and safer AI development. Collaboration over confrontation paves the way forward! #airevolution #artificialintelligence #largelanguagemodels #AICollaboration

36

https://lnkd.in/dJW9sZpx SelfExtend introduces a novel approach to expanding the context window of LLMs. It leverages a two-tiered attention mechanism—grouped attention for long-range dependencies and neighbor attention for local interactions—both derived from the model's original self-attention. This method allows seamless context window extension without requiring fine-tuning, and extensive experiments have demonstrated its effectiveness across various benchmarks.

12

1 Comment

This is a good article on using LLMs in the real world. It covers many aspects that we learned at Nurish over the last few months and more. Using LLMs is an inexact science, but has its own needs in processes and tools. While most of the attention has been on the LLMS, their capabilities, and RAG, there are mundane issues that need to be taken care of, issues where the solutions for pre-LLM development are not that well-suited. These include version management, integration, testing, data management, quality assessment, and more. This article covers many of those practical aspects. Eugene Yan, Bryan Bischof, Charles Frye & more https://applied-llms.org/

9

Looking forward to sharing how we at basys.ai are leveraging Generative AI to tackle healthcare’s biggest challenges. PS: Also looking forward to sporting my basys vest in the #BayArea PPS: What I’m not looking forward to is the second cross-country flight in less than a week. #GenAI #BasysAI #GenerativeAI

20

Interesting concept around Galileo Luna evaluation foundation models. A lot of LLM-based chatbot systems have safety guardrails to prevent LLMs from hallucinating and giving incorrect information. One technique is to fact-check the answer of an LLM by running through it again. Obviously, this doubles the number of API calls that you need to make to the LLM, also doubling the cost. Luna evaluation models seem to solve this by using specialised Small Language Models (SLMs) which have low latency and are much cheaper to run. Since they are also customised for detecting hallucinations, they perform better than larger language models. https://lnkd.in/d6zTzZ6B #SLMs #LLMs

12

1 Comment

Meet AGUVIS: A pure vision-based framework for autonomous Computer/Mobile-use GUI agents, operating seamlessly across web, desktop, and mobile platforms without UI code. Key Features & Contributions - Pure Vision Framework: First fully autonomous pure vision GUI agent capable of performing tasks independently without relying on closed-source models - Cross-Platform Unification: Unified action space and plugin system that works consistently across different GUI environments - Comprehensive Dataset: Large-scale dataset of GUI agent trajectories with multimodal grounding and reasoning - Two-Stage Training: Novel training pipeline focusing on GUI grounding followed by planning and reasoning - Inner Monologue: Explicit planning and reasoning capabilities integrated into the model training - Generalization to OOD (Out-of-Distribution): AGUVIS demonstrates impressive generalization on OSWorld benchmark even without desktop trajectory data during training. Project Page: https://lnkd.in/guzkwpbB Paper: https://lnkd.in/gs7wXyJS GitHub: https://lnkd.in/gWeSkFmK Collaboratiion among Yiheng Xu Zekun Wang Junli Wang Dunjie Lu Tianbao Xie Amrita Saha Doyen Sahoo Tao Yu Caiming Xiong

155

6 Comments

Though benchmarks for medical AI are still imperfect, there's no question that AI will continue to march towards diagnostic excellence. These trends will place greater emphasis on physician-scientists – those who advance medical knowledge via closed loop clinic/lab interfaces, which can then feed back into large-scale AI systems that serve entire populations. This was all part of the OG vision for the learning health system. source: https://lnkd.in/eh2RM72s

111

11 Comments

Unveiling the Realities of AI Development Tools: A Closer Look at Devin and Open Source Innovations Gergely Orosz (The Pragmatic Engineer) provides a compelling examination of "#Devin," initially celebrated as the "world’s first AI developer," and exposes the overstatements made by its creators, Cognition Labs. His investigation reveals a significant gap between the marketing claims and Devin's actual performance, particularly in tasks on Upwork as analyzed by Carl, an experienced software engineer and AI enthusiast. This detailed scrutiny is juxtaposed with the advent of impressive open source alternatives like #AutoCodeRover and SWE-agent. Developed by teams from the National University of Singapore and Princeton University respectively, these tools not only challenge Devin's position but are also readily available, offering an accessible and transparent alternative. Key Takeaways: Truth in Advertising: The article underscores the importance of accurate marketing, as evidenced by Carl's revelations that challenge Devin's initially claimed capabilities. Impact of Open Source: The rise of tools like AutoCodeRover and SWE-agent, highlighted by their developers, showcases how open source can drive innovation and provide a transparent alternative to proprietary tools. The Importance of Community Feedback: As the narrative around these AI tools evolves, Harry Tormey’s insights into the SWE-Bench dataset further illustrate the need for rigorous testing and community engagement to refine these technologies effectively. Orosz's narrative, enriched by contributions from Carl and Harry Tormey, not only clarifies the current capabilities of #AIdevelopmenttools but also points towards their potential evolution. This collective insight emphasizes the importance of #transparency and rigorous testing in the development and deployment of #AItechnologies. For those interested in the technical depths and practical applications of AI development tools, engaging with AutoCodeRover and SWE-agent can provide a firsthand look at what is currently achievable and what the future might hold. #AIdevelopment #OpenSource #TechInsights #Innovation #SoftwareEngineering #CommunityDrivenTech https://lnkd.in/euhTume5

4

1 Comment

Anthropic just released its new multimodal AI model: Claude 3.5 Sonnet. The benchmarks sure need a refresh as AI models keep improving at breakneck speed. But more importantly, each benchmark represents a real-world application, so it's crucial to steer clear of thin wrappers over any benchmarked capability. #claude #anthropic #chatgpt #gpt #openai

5

1 Comment

Over the past period I had an immense privilege of working with Andrej Karpathy (ex director of AI at Tesla, founding member of OpenAI, and our AI sensei :)), and a few cracked GPU engineers (Arun Demeure ex GPU Platform Architect at Apple, ngc92 et al) on the llm.c project. And I must admit I was pleasantly surprised yesterday getting a shout-out from Andrej and seeing my Twitter blow up with notifications 😅 (see the attached screen shot) As someone I've been looking up to since i started my ML journey up to a point where i'm casually chatting with him on a daily basis it all feels very unreal. Arun & ngc92 are extremely knowledgeable, and i learned a ton from them building llm.c. They're our CUDA whisperers. :) But, before continuing, what is llm.c? from Andrej: "...Because llm.c is a direct implementation of GPT training in C/CUDA, the requirements are minimal - there is no need for conda environments, Python interpreters, pip installs, etc. You spin up a cloud GPU node (e.g. on Lambda), optionally install NVIDIA cuDNN, NCCL/MPI, download the .bin data shards, compile and run, and you're stepping in minutes. You then wait 24 hours and enjoy samples about English-speaking Unicorns in the Andes." original tweet here: https://lnkd.in/dU93zyZU I've quickly contributed 113 commits (+ 40 pending once i merge my muP PR), among those: * I added multi-node support with NCCL initialization going via MPI/TCP/FS so that we can support both slurm and non-slurm scheduling envs * Helped make our code bitwise deterministic - so that after the training crashes we have guarantees that starting from the last checkpoing will give **precisely** the same loss & gradient numbers! * Implemented our random shuffling logic in the data loader, added various lr schedulers, and finally caught many bugs (e.g. periodic loss spikes, distributed norm computation bug, memory leaks), etc. I have learned a ton about CUDA, contributed to our layernorm kernel, and finally I now feel comfortable with basically every single part of LLM training since llm.c is such a "bare metal" project. I really feel i can build anything at this point. But I also learned you can't do it without a cracked team. :)) And, of course, it's more fun, and you make friends along the way. Over the past 2 weeks I mostly worked on adding maximum update parametrization support to llm.c: https://lnkd.in/dB2MuCH3 ran a ton of experiments & ablations and we're getting close to merging it in. The issue is that mup's benefits will become more apparent at bigger scales, while currently we're mostly in the ~1B regime. Today we celebrate llm.c milestone: 5 years after OpenAI announced GPT-2, with all the surrounding safety talks and staged release -> today you can now train your own GPT-2 in ~650$ on a single 8xH100 nodes in 24 hours. See this post for more info: https://lnkd.in/dD6Du4se Extremely excited for the future!

931

35 Comments

Recently, Anthropic shared its perspective on California's proposed AI regulation bill, SB 1047, with the California Assembly. While recognizing the importance of safe AI development, Anthropic has raised substantial concerns about the bill's current form, which could potentially hinder innovation and harm the United States' competitive edge in AI. Anthropic's key points include shifting the focus from "pre-harm enforcement" to a more adaptable "outcome-based deterrence" model. They also advocate for regulations that focus more narrowly on frontier AI safety to avoid redundancy with federal requirements. Additionally, they suggest giving regulatory authority to the Government Operations Agency instead of creating a new state agency. Anthropic believes that these amendments will encourage innovation in risk reduction practices and incentivize companies to adopt measures that prevent catastrophic risks. In contrast, the AI Salon Policy & Civic Engagement Guild shares Anthropic's commitment to ethical AI and safe development but has broader concerns about the bills. We believe that overly burdensome regulations may discourage AI development and lead to job losses. Compliance requirements favor large companies, potentially driving smaller startups out of the market, and overregulation might cause California to fall behind other regions in AI development. Furthermore, compliance under penalty of perjury imposes significant personal risk on developers, and strict regulations could disadvantage California-based companies in the global AI market. On behalf of the over 1700+ members of the AI Salon, we urge our larger community to engage in this crucial dialogue about AI regulation. By advocating for well-balanced policies that protect public interests without stifling innovation, we can ensure California remains a leader in ethical and responsible AI development. Please sign the letters of support and join us in shaping a future where AI is developed and deployed with ethical integrity, social responsibility, and inclusive governance. Together, we can bridge the gap between AI technology and policymaking, fostering an environment where innovation thrives alongside robust safety measures. https://lnkd.in/e6MRBmcX #AIRegulation #EthicalAI #Innovation #PolicyEngagement #Anthropic #AISalon #PublicPolicy #TechPolicy Kyle Shannon Leah Fasten Brandon Tidd Stacie L. Matthew Karabinos, MAT Vikki Baptiste Jon Potter Jon Leland Cyndi Coon Daniele Hall

11