Sachin Dhawan

The shift from traditional ML to LLMs has pushed infrastructure requirements to new limits. At neptune.ai, we’re tackling these challenges by focusing on: → Scalable infrastructure, built to handle high-speed data ingestion and processing. → Real-time data display, adapting seamlessly to complex and large volumes. → Consistent UX quality, even under intense backend demands. — (link to the full interview in the comments) #generativeai #genai #llm

3

1 Comment

Hard-coded model routing takes away intelligence from your system instead of adding to it! Interesting read below on how Ema Unlimited's Fusion model is a potent solution for the heterogeneous multi-model future of AI in the enterprise.

14

1 Comment

Can probabilistic models, like LLMs and Nate Silver's election forecast model, ever truly be perfect, with no errors? According to Nate Silver: "We don’t talk about it much, but there’s a small amount of error introduced into the model because it’s probabilistic rather than deterministic. The margin of error on 40,000 simulations is about ± 0.5 points of win probability for either candidate; after 80,000, it declines, but only to ± 0.35." Sounds like a familiar refrain around Infactory as we discuss the limitations of LLMs... P.S. Vote!

11

1 Comment

Don't Be Driven to Complacency With a POC: Enterprise GenAI Business Transformation is a Different Beast Building a cool LLM application to demo is exciting! But scaling it for real-world enterprise use and getting tangible business value? That's a whole new challenge, regardless of whether you use a vendor LLM or a custom on-prem LLM . Here's why: POC vs. Production-Ready: Sure, your LLM can demonstrate promising results on a small clean, sanitized sandbox. But can your end-to-end systems sustain performance on real-world scenarios, and with messy enterprise data with the rigor, accuracy, robustness and consistency that is necessary for enterprise use ? LLMs are a small part of a much larger equation: Many of us are enthralled by the shiny LLMs, but enterprises are about many factors including data, platforms, heterogeneous systems with complex processes, and human stakeholders (back-office, middle-office functions, and clients).  Are your data assets consumable and integrated with the vendor or custom GenAI solutions? Security, Governance & Safety: Bias, fairness, and responsible AI practices take center stage. Are you ready to monitor outputs and mitigate potential risks? Can it handle sensitive data, integrate with existing enterprise systems, and meet rigorous security standards? Extensible: Are your e2e platforms modular (vs monolithic) to accommodate the rapid tech advances to ensure your products and solutions dont become “stale” in 3-6 months?! Continuous Improvement: Can your AI platforms leverage the virtuous cycle of continuous improvement by learning from client usage and get better over time? This is overlooked in many AI initiatives and I cannot emphasize enough the business benefits that can be derived by data-driven AI customization and tuning Transitioning from POC to production requires careful strategy, planning, execution, and e2e enterprise-ready platforms, and change management. This takes skills, experience and a cultural mindset shift. The blog from Jerry Liu at Llamaindex expressed this very well:  https://lnkd.in/eUM4H6Af Quote: An issue I see in enterprise LLM app development is that the “build a chatbot in 5 minutes” appeal of LLMs becomes a hindrance when you try to productionize anything. Setting up a demo is easy. But over half a developer’s time is spent on tuning and iterating rather than building. #enterpriseAI #LLMs #AIresponsibility

28

9 Comments

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

Grammarly just acquired startup Coda (last valued at 1.2bn) with a goal of a possible IPO. This could be a big move in productivity, especially for product teams. Coda CEO Shishir Mehrotra was recently on Lenny's podcast (first comment below). https://lnkd.in/gmgfcm2P

1

1 Comment

This is a good watch for understanding why AI agents are the new application paradigm in enterprises and provide more customizability and flexibility than existing SaaS products! #AIInnovation #EnterpriseAI

15

WIESNER ON HUMANS VS AI: MIT Media Lab co-founder and science advisor to JFK, Jerome B. Wiesner was a unique figure in the history of MIT at the crossroads of science, engineering, and the arts with a lifelong pursuit of world peace. There's a terrific talk he gave at the founding of the MIT "E15" building designed by I.M. Pei — often nicknamed "the bathroom building" — that's worth a watch, especially during these extraordinary times https://lnkd.in/g2zEJRrk and a piece of the talk he gave is below. Information on the talk I'm giving with Jerome B. Wiesner Professor Hiroshi Ishii is here https://lnkd.in/gJpKkeY8 on November 8th (Fri) from 11 am EST, and you can register online for free here https://lnkd.in/gADUyCs7 --- Jerome B. Wiesner at the founding moment of the MIT Media Lab's E15 building https://lnkd.in/g2zEJRrk > ... Now some people will be offended by what I'm going to say, but I think man-made systems use better components. Electrical signals travel a million times faster than signals on nerve fibers. And semiconductor devices also act billions of times faster than neurons, which are the biological information processing elements. Also, manmade elements appear to be more reliable. But nature has a very large edge, maybe a decisive edge. In design complexity involving a number of individual elements, it can use it in a system effectively. And these are advantages man may never match. Nature may always retain this edge, who knows? > But for the moment, it really doesn't matter. The manmade systems that we are involved with enhance human creative capacities, that's what they can do, but so far they cannot displace them. Cognitive and computer scientists are having an exciting time trying to understand and imitate human reasoning with computing systems. And though they can do a few things that would be regarded as thinking if done by a human, they differ violently among themselves on whether or not machines with human thinking and creative abilities are possible. Even the so-called artificial intelligence field, which prompted Walter Rosenblith to say he preferred natural stupidity, his greatest achievement in this field is capturing knowledge and skills of humans -- of human experts. > For now, it’s clear that thinking machines and true creativity don’t exist. We should focus on how tech can expand human creative capacity.

53

4 Comments

AMAZON ACCOUNTING GAMES? What’s an easy way to boost your quarterly margins if you are binging on GPU chips? Depreciate your assets over 6 years instead of 5 Here’s my take in The Information: ‘For AWS, though, expectations are sky-high.  Ahluwalia noted that of all the “Magnificent Seven” stocks – Microsoft, Apple, Nvidia, Alphabet, Amazon, Meta and Tesla – Wall Street analyst projections are highest for Amazon in terms of its ability to grow its free cash flow.‘ Remember, firms that spend a lot on CapEx lag those that do not, on average.  Now, why do I see this as an accounting game? New GPU stacks are more efficient.  They increase obsolescence rapidly.  Amazon should *reduce* its depreciation schedule! The CapEx binge is a core part of the Lumida Wealth Semiconductor CapEx Receiver Thesis.  Recall we said the best names to own in Mag 7 are: #NVDA #META #GOOGL Those are exactly the names that are leading YTD with #TSLA and #AAPL on the bottom and Consensus names like #MSFT and #AMZN in the middle

3

➰ APIContext's latest insights on cloud-based API performance have been featured on Nordic APIs! In the 2024 report, we delve into crucial aspects affecting API efficiency: 🔐 Authentication is key. Ensuring robust authentication mechanisms in APIs is crucial, yet offloading this function can introduce complex challenges. 📖 DNS configurations matter. How you handle DNS settings can significantly impact API performance, particularly when tied to authentication processes. 🌐 The role of third parties. Critical third-party services play a significant role in 🔗 API transactions, underscoring the need for careful integration and monitoring. ⛅ Not all clouds are created equal. Differences in cloud services can drastically affect API performance, an essential consideration as APIs account for over 80% of all internet traffic. #APIs #CloudComputing #TechInsights #APIContext

15

1 Comment

Generative AI: miracle worker or sophisticated guesser? While AI can now write, code, and create, it's essentially still a prediction machine. This revelation challenges how we view and utilize AI in business. What does this mean for your AI strategy? Let's discuss. #ArtificialIntelligence #BusinessStrategy #TechInnovation #AILimitations #FutureOfWork

2

🌟 From Raw Data to Behavioral Predictions—Effortlessly What if AI deployment didn’t require months of preparation? With BaseModel.ai now on the Snowflake Marketplace, you can transform behavioral data into predictions in just a few clicks. ✅ Rapid Deployment: Build and deploy models in a week.  ✅ Scalable Flexibility: Create multiple models for diverse use cases.  ✅ Privacy Guaranteed: Data never leaves the secure Snowflake environment. Say goodbye to complexity and hello to actionable insights. AI for behavioral data has never been this simple. 👉 Try BaseModel.ai Today  #AI #MachineLearning #BehavioralData #BaseModel #Snowflake #Synerise 

39

4 Comments

I spent five days in the SF Bay Area with B2B SaaS founders, funders, and senior software engineers. The talk is all about adding GenAI and LLM features to software apps, but they were unclear exactly how or when the AI big wave revolution will occur. AI is very cool and will be big, but it's still early to see how this will play out in the next year on the front lines of business. There's a lot of hurrying up to say you have AI features with a healthy skepticism about the user benefit of early GenAI attempts. Coders in big tech say, "Our product leaders told us we need to add GenAI features, but it still doesn't seem that interesting for users. We'll design it, code it, and ship it anyway." Founders in startup SaaS say, "We are running a lot of experiments and keeping up with the big AI platform changes. We're all more productive now but it hasn't made our app much more useful." Smaller SaaS funders say, "It's hard to tell where the winners will be to make our bets. AI tools help software companies grow more efficiently, so they don't need our funding as much. We're being very careful." Big enterprise SaaS buyers say, "We need to start investing, testing, and trying GenAI technology, which means we need to spend less on our existing SaaS apps in our tech budget." Big investors say, "It will be the next big tech wave, so we are making big early bets on chips, infrastructure, foundational models, and platforms. NVIDIA, Microsoft, Google--the big will get bigger." It's all very exciting, but it's still early and moving around quickly. That's a lot different from the last big software tech wave when SaaS apps on the web and mobile took over the last 15 years. It was easier to bet on the massive transition from on-premise Windows software to cloud-based web and mobile apps for businesses everywhere and billions of consumers. Despite that, I'm more excited about GenAI and other ML platforms than I was about VR, blockchain, and other platform opportunities that didn't materialize. It's early, but the practical AI wave will happen in ways we can't imagine or expect. What are you seeing in the current AI wave? #practicalfounders

240

108 Comments

Here is a much-discussed post from Sequoia Capital that may be missing a point: by investing in Gen AI capex, big tech vendors are playing defense as much as offense. What happens in the world where Google does not invest in Gen AI data center capacity? A portion of their search revenue would likely go to Perplexity, Bing, OpenAI, etc. Apparently, big tech vendors assessed their losses from gen AI inaction to be greater than the capex-revenue gap calculated by Sequoia.

12

Worth reading: How the Apache Arrow Format Accelerates Query Result Transfer by Ian Cook One of the hidden problems of intersystem communication is transcription of data formats. Ian does a wonderful job of explaining the "why" of Apache Arrow as well as the "how" of its advantages. Read on...

8