Ankit Khedia

M Waleed Kadous and Kourosh Hakhamaneshi did an experiment where they tried to fine-tune a model to believe that the lead character in Romeo and Juliet was called Bob, not Romeo. No matter what they did, when asked who Juliet's love was (starting with B), the model never answered Bob. Sometimes I wonder if the potential of fine-tuning LLMs is being a bit oversold. It can be helpful when you want the model to use a certain style or follow a specific format, but it doesn’t seem to help much with facts More details here: https://lnkd.in/g6p2eU-n

8

🚀Hi Folks, I'm Taking on the Product #Teardown_Challenge along with Arindam Goswami! 🚀 As part of an exciting challenge by Product Space, I’m stepping into the role of Senior VP of Product at #Zepto Quick Commerce! Zepto’s already crushing it with 10-minute deliveries across 30 cities in India, but there’s so much more potential to unlock! 💡 Here’s the mission: ➡️ Increase Zepto’s Average Order Value (AOV) to ₹600 🛒 ➡️ Boost product discovery and engagement to encourage bigger, more frequent orders 🔍 ➡️ Elevate the purchase experience to keep customers coming back for more 🔄 And I need YOUR help to make it happen! 🙌 🎯 Your insights on quick commerce platforms like Zepto, Blinkit, and Swiggy Instamart are key to shaping these strategies. I’ve put together a short survey, and your feedback will be crucial in tackling this challenge! 💥 Thanks for your support—let’s crack this together! 🙌 Link to the form - https://lnkd.in/g5v2y4wR #ProductTeardown #QuickCommerce #ProductManagement #ProductSpace

10

2 Comments

What if your queries required access to multiple sub-related data from different documents, for an LLM to parse and make sense of before returning a coherent and accurate output? We often ask complicated questions, right? 🔹Well to solve this problem, these researchers released Multi-Head RAG (MRAG) which leverages activations of Transformer’s multi-head attention layer, instead of the decoder layer, as keys for fetching multi-aspect documents. They are taking advantage of the fact that multi-head attention can learn to capture different data aspects. 🔹The interesting part is that MRAG is both cost-effective and energy-efficient. In other words, it does not require additional LLM queries, multiple model instances, increased storage, or multiple inference passes over the embedding model. 🔹Moreover, MRAG can be seamlessly integrated with existing RAG frameworks and benchmarking tools like RAGAS as well as different classes of data stores. 🔹The researchers provide a comprehensive evaluation methodology, including specific metrics, synthetic datasets, and real world use cases to demonstrate MRAG’s effectiveness. 🔸Do check it out! #genai #technology #artificialintelligence Click #linkedangle and follow for more content Set the 🔔 notification on my page, don’t miss a post!

181

10 Comments

𝐎𝐧 𝐭𝐡𝐞 𝐝𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐭 𝐡𝐮𝐞𝐬 𝐨𝐟 "𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐓𝐢𝐦𝐞 𝐒𝐜𝐚𝐥𝐢𝐧𝐠" (𝐈𝐓𝐒) 𝐢𝐧 𝐋𝐋𝐌𝐬. Since the release of 🍓 o1, Inference Time Scaling (ITS) has become a new buzzword, with some previous techniques being rebranded as instances of the current ITS buzz.. The most basic thing common about all the uses of ITS seems to be that at inference time, LLMs are made to do more than just output the completion of the prompt (thus taking more compute at the inference time than standard autoregressive LLMs). The specific way it is done varies quite a bit and impacts the accuracy. Here is a small taxonomy of the various brands of ITS--with some comments about their relative power. 1. 𝗖𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝗼𝗻𝗹𝘆 𝘁𝗵𝗲 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 (𝗸𝗲𝗲𝗽𝗶𝗻𝗴 𝘁𝗵𝗲 𝗽𝗿𝗲-𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗟𝗟𝗠 𝘁𝗵𝗲 𝘀𝗮𝗺𝗲) The general idea here is to sample multiple candidates and select the response from them. There are two issues: How to generate a large sample of candidates and, how the promising candidate is selected [Candidate Generation:] The generation of multiple candidates can be done either by sampling directly from the LLM (with appropriate temperature settings), or augmenting the original prompt with additional augmenting strings to generate additional candidates. These augmenting strings have originally become popular with the "Chain of Thought" papers--but by now it is clear that augmenting strings don't have to be anthropomorphized as "chains of thought"! (c.f. https://lnkd.in/gCCPGz-e). If we are considering augmenting strings (that we will call CoT strings--without any anthropomorphization), it is also possible to search among multiple CoT strings. (the recent Marco-o1 seems to do this explicitly). [Selection:] Once the candidates are generated, the promising one(s) need to be selected. The approaches here include 1.1 Simple majority voting (and other versions such as self-consistency) 1.2 Verifier based selection from the generated candidates. [1.2.1 LLMs as verifiers] Some early methods used LLMs themselves as verifiers--but there is work showing that LLMs are no better at self-verification than generation in the first place. They can have both false positives and false negatives. A crucial corollary is that the candidate eventually selected could still be incorrect--which would be problematic especially in cases where deployment failures are costly (c.f. https://lnkd.in/grv6DgYu) [1.2.2 Learned Verifiers:] It is possible to improve the verification accuracy by learning task-specific verifiers; this has received some traction under the name "generative verifiers"--but it is a rather old idea that aims to leverage the fact that learning to predict correctness of the candidate is a "discriminative task" and thus can be of lower sample complexity (c.f. https://lnkd.in/gbMhcVtP). The learned verifiers can improve accuracy but provide no guarantees. (Continued in the comments below)

102

10 Comments

A Day at the Apache Kafka 🚀 Today, I had the pleasure of attending an in-person Apache Kafka hosted by CRED in Bangalore. Here’s my feedback on this incredible event: 📍 Venue: Cred Aero, Bengaluru 🔹 Agenda Highlights: - @Vikas Sharma , CRED : Talk: Running Kafka Optimally in Production: Practical Usage Feedback: Vikas shared practical insights on optimizing Kafka in production environments, with valuable strategies for resource management. - Udayaram Kammara , Bazaarvoice : Talk: How Bazaarvoice Utilizes Kafka for Both Stream and Batch Processing Scenarios Feedback: Udayaram's session on Kafka's use at Bazaarvoice was enlightening, covering real-world use cases and challenges. - Diptiman Raichaudhuri , Confluent : Talk:Stream Processing for the Non-JVM World: pyflink and kafka-python Feedback: Diptiman's explanation of Apache Flink's architecture and the demonstration of kafka-python and pyflink were fascinating and beneficial for non-JVM users. What I Learned: - Effective strategies for running Kafka in production. - Real-world use cases and challenges in stream and batch processing. - Leveraging pyflink and kafka-python for stream processing. Overall, this meetup was a treasure trove of knowledge and inspiration. The sessions were well-organized, and the speakers were exceptionally knowledgeable. I’m excited to implement these learnings in my projects and continue exploring the potential of Apache Kafka and related technologies. 🙌 #ApacheKafka #KafkaMeetup #CRED #Bangalore #EventStreaming #StreamProcessing #TechCommunity #Networking #Python #Flink #DataPlatforms

24

2 Comments

In 2019 I landed my dream job as a data scientist at Google. It was a pivotal moment in my career - a dream that took 10 years to realize. Recently, as I re-visited the Google HQ (not as an employee, but as a tourist), I recalled feeling inspired working there at every moment. Passion, drive and the constant pursuit for innovation were baked into the culture. And, in many ways, it accelerated my skills and know-hows as a data scientist, and became the stepping stone to pursue entrepreneurship with DataInterview.com full-time. If you want to land your dream job, here's a playbook I used you can steal👇 1. Baby step your way to your dream job 📊 Many aspire to break into FAANG companies fresh out-of-school, or right after switching into industry. The first thing is to get your foot into the door somewhere. I got my foot into data science at a consulting firm first, and then joined a startup. Building the skills and portfolio was the key step in getting my resume spotted by a technical recruiter at Google. 2. Continue to adapt and invest in learning 📚 After work and weekends, I spent additional hours at a coffee shop, learning as much I can honing in my skills in programming, mathematics and soft skills. Here are a couple of resources I used to cultivate my skills - Stanford CS229 - An Introduction to Statistical Learning - The Elements of Statistical Learning - Time Series Analysis and Programming - Automate the Boring Stuff with Python 3. Be interview-ready 🚀 Nowadays, FAANG opportunities are more competitive than ever. For the one position that is open, you will be competing against 20 other individuals for the same spot. That's why it's important you know how to prep for an interview, effectively. So, here are my suggestions that personally helped me. 🔹 Research, research, research about the company interviewing you - what's the business model? What are their flagship products? Who are the users? What are their growth, engagement, and data strategies? Gather these information prior to the interview as business and product cases are based on based on those. 🔹 Prep in a structured manner - for MLE/AI Engineer roles, you need to know ML breath / depth, system design, ML system design, coding and behavioral. For DS roles, you need to know ML breath / depth, statistics, DS coding, SQL, product sense, and experimentation. Identify the strength and weakness of the areas and revamp. Now go ace your next interview👇 📕 𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗣𝗿𝗲𝗽 𝗖𝗼𝘂𝗿𝘀𝗲𝘀: https://lnkd.in/gzgB-dHT 📘 𝗝𝗼𝗶𝗻 𝗗𝗦 𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗕𝗼𝗼𝘁𝗰𝗮𝗺𝗽: https://lnkd.in/eiA5Ntdp 📙 𝗝𝗼𝗶𝗻 𝗠𝗟𝗘 𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗕𝗼𝗼𝘁𝗰𝗮𝗺𝗽: https://lnkd.in/e6HbN6dy 📗 𝗔𝗕 𝗧𝗲𝘀𝘁𝗶𝗻𝗴 𝗖𝗼𝘂𝗿𝘀𝗲: https://lnkd.in/g82dMJ77 👉 Smash 👍 and follow Daniel Lee for data & AI

525

25 Comments

Another major use case of batch LLM inference that people are over looking is asynchronous evals (LLM as a judge/evaluation models). Tons of companies we’ve seen are already starting to spend massive compute allocation on LLM as a judge systems and they will only increase as the need for trust and production volumes increase. We’ve heard anywhere from 5-40% of total inference spend (and growing). Such a large spend on just evals is probably unsustainable, so it will be increasingly important for a provider to offer cheap inference to service and win such batch workload tasks. My hypothesis: the emergence of a specialized inference platform just for LLM as a judge, with inference products specifically built around these workloads (or as an add on for existing inference providers) Lmk what you all think, happy Tuesday everyone 😁

21

I have a hypothesis: US-India pair programming can ~2x the speed of execution of engineering teams. Imagine a high quality engineer in US who passes the baton to his/her counterpart in India - at the end of the day. The Indian counterpart then runs at the same developmental speed and takes over for the next 8-10 hours. They then pass the baton to their US counterpart. It could even be a senior-junior relationship - making sure instructions are clear and only one person is in the driving seat. Of course, I am using pair programming a bit loosely here but I could not think of a better framing. Diving into this thesis for the next few days. If you have worked in such a setup or have insights, I would love to chat and learn what you know.

15

5 Comments

🚨 Struggling with Low-Frequency Products? Here’s the Playbook I Wish I Had 🚨 For 15+ years, I’ve been managing products that people don’t use every day and only on episodic basis. And let me tell you—it’s tough. No guide. No clear answers to questions like: 👉 How do you measure product-market fit? 👉 How do you optimize acquisition costs when usage is infrequent? 👉 How do you actually drive retention? The frustration led me to create ICED Theory—a framework so intuitive and logical, it became part of Reforge’s course material. Now, for the first time ever, I’m hosting LIVE virtual classes to teach the ICED framework and help you grow and manage low-frequency products. 📅 Two intense days of interactive discussions. 💡 Deep dives into strategies that work. 🎉 50% launch discount for my first cohort. 👉 Ready to transform your approach to low-frequency products? Enroll now and secure your spot: https://lnkd.in/gHwUsGiJ Spots are limited, so don’t wait! Let’s crack the code together. 🚀 #lowfrequencyproducts #workshop #productmanagement #training #cohort

34

1 Comment

https://lnkd.in/esZgE24d Seven years ago I gave this talk at IIT Kanpur. Back then it seemed inevitable that we would have self-driving cars by now, and some predictions I made were wrong. Overall, though, I think this talk has aged reasonably well. Little did I know back then that the reference to the 2016 MaxSAT I gave would spark the creation of a new company, InsideOpt. The Seeker Solver that we sell is based on the technology outlined, and it is changing the optimization landscape. On the social implications, we since edited a special issue of the AI Magazine on the social disruption that AI brings. More work is needed. We need to have a wide societal discourse on how we are going to share the increasing wealth that AI brings to the world. #insideopt.com

8

Nice to have 'Top New Release' status on Amazon for the latest book (my 25th) that explores an alternative paradigm for quantum computation. Illustrations (111) by Dr. Narendra Madhav Joshi. My take is that we owe it to ourselves at this early stage of the industry's lifecycle to explore alternative approaches. Here is a brief overview: "This book attempts to reveal something of the vastly different and fundamentally creative quality of computing that must accompany any computation involving the quantum-levels. To linearly project digital computing laurels manifest as increasing speeds and the ability to process vaster amounts of information, as the inevitable trajectory of quantum computing is perhaps, in the aphorism attributed to the Buddha, to look only at the finger and to miss the moon and the sky that it is pointing to. The cover figure highlights all that is being missed, summarized as an egg-like structure synonymous with the term ‘Hiranyagarbha’ in Sanskrit. This depicts the womb of creation abundant with many layers and patterns reflecting luminescence, due to the constant superposition and entanglement inherent to the quantum computation that derives from a light-centered interpretation of quantum dynamics." The book is available here: https://lnkd.in/gXu4H9uW #quantumcomputation #quantumtech #technologyleadership #cosmology #systemsthinking

16

5 Comments

I am not sure Google’s primary duty should be to shareholders, including mass firings to pay for GPUs. As a shareholder, I am thankful to be able to own a few shares of Google in my retirement account. But Delaware Supreme Court case law serves unjustly as a faux federal loophole to maximize profits above all else, enabling amoral fiduciary duty to conventional shareholders. And it seems that employees are shareholders - should one class of shareholders be favored over another? It seems like the law needs to include stakeholders like the economy in which a company arises and the communities in which it does business. If a given government has a shrinking middle class, is it not true that eventually the corporate tax base will not be enough to pay for adequate national defense or safety nets? I happen to be pro-Google, especially because of their contribution to help so many businesses utilize the trackable ROI of Google Ads, and for many other reasons. As far as I know they obey the law - and it seems like the law needs to change. I also happen to believe it is good for the world to spread economic opportunity and jobs. But what makes up for all the money and tax these core Google employees were spending on the American economy, or the people who will lose out on a job opportunity because a Xoogler applied? Housing costs in SF are insane, what about their mortgages? Maybe they are millionaires, maybe not. I kind of doubt Google fired AI engineers, but maybe instead of hiring domestically they will hire wherever the AI labor is cheapest. I happen to be pro-immigration and my ancestors were immigrants. I don’t pretend to presume the right balance of domestic and offshore workers. I have been very pleased to collaborate with people and pay them out of pocket and be able to afford it because of the labor discount. But I am not a hugely profitable corporation either. I tend to believe in unfettered capitalism and that too much tax and regulation can stymie an economy and hurt everyone. Yet an economy needs to be sustainable, and not just for conventional shareholders. I am not sure there is a check and balance to the “duty to shareholders”. Public outcry used to be one, but not anymore, not after the massive tech hiring and layoffs. Maybe it is just competition. But it doesn’t seem biomimetic, as in natural balance between various forces in a biological system. If an economy was a body and large corporations were organs in the body, if money was water and employees are like blood, then the corporate organs seem to be growing abnormally and taking more moisture out of the system. In a normal body the organs would have limited growth but in this body the corporate organs are growing larger, while the rest of the body slowly dries out, and blood clots form from employees who are no longer needed. The corporate organs are less reliant on employees and are now making their own blood and selling it.

11

3 Comments

Day 13 of #30DaysOfAI 🚀 Getting structured data from Large Language Models (LLMs) has always been a challenge. While LLMs typically return markdown content by default, asking for structured formats like JSON often leads to errors or inconsistencies. That’s why OpenAI’s Structured Outputs is a game-changer! 🎉 With Structured Outputs, you can provide a JSON Schema to define exactly how you want the data to be structured. This ensures the LLM consistently generates the right format every time, making it far more reliable for applications that depend on structured data. Here’s an example: Imagine you’re building an invoice management application. The app scans an invoice, extracts relevant information, and then returns it as a structured JSON format. With Structured Outputs, you can precisely define the type of data to extract—ensuring accuracy and making it easy to store or categorize invoices. Want to learn more? Check out this guide on Structured Outputs by Shreya Khandelwal to see how you can start using this feature effectively. How would you use Structured Outputs in your projects? Let’s discuss this in the comments!

1

Fantastic news today on the general availability of Amazon Q. Amazon Q is the most capable generative AI-powered assistant for accelerating software development and leveraging companies’ internal data. It eliminates tedious work for developers and employees across organizations. #AmazonQ #GenerativeAI

15

Do LLMs have any immediate applications in the field of Robotics? The above question was loosely based on a query asked by a startup founder in an event, based on my colleague’s work on a recent LLMs integration. (Siddharth) Come to think of it, robotics about 10 years ago wouldn’t have been able to accommodate such computational expenses. Imagine running a LLM model on a Raspberry Pi zero and not expecting it to blow up! :) While implementing a LLMs could be a fun side project, what question warrants a deeper insight would be whether it is of absolute necessity? With Nvidia doubling down on the highly powerful Jetson devices, and Blackwell chips, the hardware is expansive and available, but it still begs to ask the same question again and again - is it necessary? Like every other industrial trend, once must deeply analyse where a novel idea would fit in, and calculate against its effort and yield. Let’s explore a few options wherein it could be absolute importance to use LLMs in context of Robotics: 1. Support agents - Social robotics has a huge potential of closing down on the weary technical support based on an exhaustive list of solved and interpreted problems. Distilling technical documents and datasheets into a message parsing model is an important win to do effective knowledge transfer. 2. Assistive bots - In the field of soft robotics and prosthetic arms/exoskeleton, a voice controlled NLP agent would be great help in helping administer orchestrated actions for differently abled individuals who have clear motive/task list. 3. Surveillance/Monitoring bots - Given the nature of preordained message intents, it is a perfect candidate for having a lightweight message-action model for surveying specific sites. Looking at the magnanimity of the manufacturing plants, this would be a great boom. 4. Surgical bots - Expert assisted robotic surgeries are a natural extension for aligning non-conventional anomalies that are very unique to the nature of the surgery. Since precision and targeted algorithms are of utmost importance, one cannot make actions based on learning based approaches effectively from general archetypes. Are there more applications that cross your mind marrying LLMs and Robotics?

38

6 Comments

I recently (begrudgingly) decided to start occasionally looking at the cesspool that is Twitter/X again because, despite all the BS, it’s still a great resource for keeping up with AI/ML/DL news. One of my favorite sources of open-source LLM/AI updates on LinkedIn is Vaibhav Srivastav. Naturally, I checked out his X profile and saw his pinned post from earlier this year. The gist? “Just ship—don’t care about polish, especially on new projects or ideas.” (Here’s the full post for the curious: https://lnkd.in/gy8BY9U5 ). Coming from a "Capital S, Capital E" Software Engineering background, where I’m semi-obsessed with writing maintainable repos with clean, intuitive patterns and clear delineations of responsibility, this idea definitely hit a nerve. I’ve spent a lot of time making sure my code feels right, and the hot take of “no one cares about OOP” was _mildly_ triggering lol. But hot takes are hot takes. If you get past the brashness, there’s usually a deeper point. For me, the takeaway is that things don’t need to be perfect to be made public. This resonated. Pretty much anyone who knows me—personally or professionally—would label me a perfectionist. And yeah, sometimes that works in my favor, but it’s also held me back. I’ve started so many side projects, gotten them to a point where they’re actually sort of useful (even if it’s just a useful Jupyter notebook), and then just… stopped. I let them sit there, unfinished, because I’m scared of being judged. What if someone looks at my GitHub and thinks I suck? What if recruiters see messy repos and assume that’s how I work? Add in ADHD-PI (something I haven't really ever been fully public about, so consider this my second "coming out" lol), and it’s a recipe for things falling off my radar. My brain either jumps between a million tangents or hyperfocuses on making something perfect. So the end result is always the same: projects that could have been useful just collect dust because I can’t let go of the idea that they aren’t ready. Vaibhav’s post made me realize I need to let go of that. Imperfect projects can still help someone, spark an idea, or just exist as something I had fun making, even if it’s not “recruiter-ready.” So I’ve decided I’m going to start sharing more of the random AI (and maybe some non-AI) stuff I’ve worked on over the past couple of years: no-code “Custom GPTs,” Jupyter notebooks that probably don’t work anymore, whatever. If it could be useful or inspiring to someone, then it’s worth putting out there. Anyway, this is me holding myself accountable. If you see a dumb repo or an old project pop up here, just know it’s part of me trying to care less about perfectionism and more about just shipping. Thanks Vaibhav Srivastav and shoutout to all the other GPU poor ML devs out.

41

11 Comments

#BigData is Dead, Long Live #BigData! The conventional wisdom in AI is that more compute equals better performance. But this is no longer true. We’re entering an era where #data, not #compute, is the real #bottleneck. To push #AI forward, we need smarter data strategies, not just more power. Thanks for the insights, Harry Stebbings and @Arvin Narayanan! #AI #BigData #Innovation #DataScience #MachineLearning #Tech #Lakehouse #Iceberg

7

1 Comment

🚀 The need for India-focused GenAI platforms has led to a host of indigenous developments. Let's take a look at 8 Made-in-India LLMs that are shaping the country's future. 🔸 Recognizing the socio-cultural gap in existing systems, entrepreneurs and developers from India are working on a GenAI ecosystem tailored for Indian consumption, yielding better contextual outputs across all formats. 🔸 A handful of homegrown conglomerates, such as Reliance (Jio), Tata Consultancy Services, Infosys, and Mahindra & Mahindra, are already working on a slew of GenAI projects. 🔸 Indigenous developments like BharatGPT, Ola Krutrim, and Project Indus (a Tech Mahindra venture including 40 different Indic languages), are addressing the language challenges pan-Indian users face in real life and now in the AI world. According to Inc42 data, India’s GenAI market is expected to grow exponentially in the next few years, surpassing $17 Bn by 2030 from $1.1 Bn in 2023, growing at a CAGR of 48%. With BharatGPT and its ilk taking charge and cementing their positions in India’s conversational AI landscape, India is well-prepared for a new era of digital empowerment where language barriers are dissolved and access to information is democratized. 🌟 Thoughts on this trend: Cultural and Linguistic Relevance: These platforms address India's diverse socio-cultural and linguistic nuances, ensuring inclusivity and effectiveness. Local Innovations and Entrepreneurship: Investments by companies like Reliance, TCS, Infosys, and Mahindra & Mahindra highlight India's growing tech ecosystem and global AI positioning. Economic Growth and Opportunities: The rapid expansion of the GenAI market signifies substantial economic opportunities, leading to job creation and skill development. Digital Empowerment and Accessibility: By providing AI-driven solutions in multiple Indian languages, these platforms enhance digital literacy and inclusivity. Addressing Real-World Challenges: Indigenous AI platforms deliver solutions closely aligned with the needs and preferences of the local population, improving customer service, education, and healthcare support. The emergence of India-focused GenAI platforms is a promising development driving technological, economic, and social progress in the country. Let's look forward to a digitally empowered and inclusive future! 🚀🌐 #GenAI #AI #India #DigitalTransformation #Innovation #Technology #BharatGPT #OlaKrutrim #ProjectIndus #DigitalEmpowerment #FutureTech #AIinIndia

28

3 Comments