Vinoth Govindarajan

🚀 Want to run your very own trained LLM, privately and securely? 🤖🔒 Imagine harnessing the power of a Language Model tailored just for you without worrying about third-party servers! 💼✨ Try, LM Studio. Jest for few click . LLM running in your PC. With LM Studio, you can ... 🤖 • Run LLMs on your laptop, entirely offline 📚 • Chat with your local documents (new in 0.3) 👾 • Use models through the in-app Chat UI or an OpenAI compatible local server 📂 • Download any compatible model files from Hugging Face 🤗 repositories 🔭 • Discover new & noteworthy LLMs right inside the app's Discover page #LLM #private #chatgpt #AI

9

Insightful explanation of how do go channels work behind the scenes. Its all about yielding the ptocessor for other goroutines. https://lnkd.in/gbyQ4yH9

1

Coffee & Systems (Study Group - Bengaluru) Week#5 (HPBN Speedread - Part 3 [HTTP Cont.]) January 04, 2024, 4:00 PM → 5:30 PM (Satruday) - The Market RMZ Ecoworld ------ Continuing from last week we explore HTTP/2 and HTTP3/QUIC. They promise to make our applications faster, simpler, and more robust. HTTP/2 was specifically created to reduce the load latency of web pages by addressing some of the well-known performance limitations of HTTP/1.1. At the same time, QUIC/HTTP3 presents a new transport mechanism altogether to replace TCP. Companies are still rolling out a lot of these optimisations. This represents the cutting edge of advancements in networking protocols. Happy reading and see you all this Saturday :) - HTTP/2 - Chapter from HPBN --> https://hpbn.co/http2/ - QUIC: next-generation multiplexed transport over UDP --> https://lnkd.in/gc4M8ZFg - Explaining QUIC: the protocol that is both very similar to and very different from TCP By Peter Door --> https://lnkd.in/gnu2QBnV - HTTP/3: the past, the present, and the future --> https://lnkd.in/g6MSCpaS

30

At the heart of PreProd OpnSrc, there is a vision and a mission to make foundational learning at ZERO cost Please visit the post below and the page at PreProd Opnsrc for more information PreProd Corp PreProd Emulet PreProd Consult Krishnav Dave

10

Coffee & Systems (Study Group - Bengaluru) Week#5 (HPBN Speedread - Part 2 [HTTP]) December 22, 2024, 4:00 PM → 5:30 PM (Sunday) - The Market RMZ Ecoworld ------ Building on our discussion from last week, we'll explore the basic building blocks of HTTP. Since this is a comprehensive topic, we're splitting it across two weeks, with this session focusing on HTTP 1.x. Next week, we'll cover HTTP/2 and QUIC. - Brief History of HTTP - https://lnkd.in/gzPSvwwu - Primer on Web Performance - https://lnkd.in/gQnxzQkd - HTTP/1.x - https://hpbn.co/http1x/

37

1 Comment

Is Protobuf the future default for everything? Timothy Chen & I chatted with Buf CTO Akshay Shah about the future of Protobuf, their new Kafka-compatible streaming layer, and everything he learned scaling Uber that led him to build Buf. This episode covered a pain point I've repeatedly experienced while working at Snyk & see at many other companies: managing cross-team migrations and building a productive internal API ecosystem. Here's a great example: You have two services (A and B), and A depends on B. How do you modify an API or Event in A without requiring B to do a complicated lock-step migration? These are the challenges in any at-scale organization where a project that should take 30-60 days turns into months as you try to navigate alignment and delivery. They're mundane papercuts that turn into hundreds of hours of lost productivity. I love what Buf are doing to help make this a problem of the past.

52

3 Comments

I am not sure Google’s primary duty should be to shareholders, including mass firings to pay for GPUs. As a shareholder, I am thankful to be able to own a few shares of Google in my retirement account. But Delaware Supreme Court case law serves unjustly as a faux federal loophole to maximize profits above all else, enabling amoral fiduciary duty to conventional shareholders. And it seems that employees are shareholders - should one class of shareholders be favored over another? It seems like the law needs to include stakeholders like the economy in which a company arises and the communities in which it does business. If a given government has a shrinking middle class, is it not true that eventually the corporate tax base will not be enough to pay for adequate national defense or safety nets? I happen to be pro-Google, especially because of their contribution to help so many businesses utilize the trackable ROI of Google Ads, and for many other reasons. As far as I know they obey the law - and it seems like the law needs to change. I also happen to believe it is good for the world to spread economic opportunity and jobs. But what makes up for all the money and tax these core Google employees were spending on the American economy, or the people who will lose out on a job opportunity because a Xoogler applied? Housing costs in SF are insane, what about their mortgages? Maybe they are millionaires, maybe not. I kind of doubt Google fired AI engineers, but maybe instead of hiring domestically they will hire wherever the AI labor is cheapest. I happen to be pro-immigration and my ancestors were immigrants. I don’t pretend to presume the right balance of domestic and offshore workers. I have been very pleased to collaborate with people and pay them out of pocket and be able to afford it because of the labor discount. But I am not a hugely profitable corporation either. I tend to believe in unfettered capitalism and that too much tax and regulation can stymie an economy and hurt everyone. Yet an economy needs to be sustainable, and not just for conventional shareholders. I am not sure there is a check and balance to the “duty to shareholders”. Public outcry used to be one, but not anymore, not after the massive tech hiring and layoffs. Maybe it is just competition. But it doesn’t seem biomimetic, as in natural balance between various forces in a biological system. If an economy was a body and large corporations were organs in the body, if money was water and employees are like blood, then the corporate organs seem to be growing abnormally and taking more moisture out of the system. In a normal body the organs would have limited growth but in this body the corporate organs are growing larger, while the rest of the body slowly dries out, and blood clots form from employees who are no longer needed. The corporate organs are less reliant on employees and are now making their own blood and selling it.

11

3 Comments

I am becoming an avid user of the http://undermind.ai search engine. A decade ago Olivier Bousquet and me were thinking of how to generate a 'dossier': A search result page that would be assembled "offline" and doing a deep search. How good can you search if you have more time compared to Googles close to real-time response. Can you basically trade latency vs accuracy? Undermind.ai is focused on scientific literature and allows that you can query by giving a description of what you are looking for. In research a 3-4 words query is rarely sufficient to describe what you are looking. Really useful to get a list of scored publications back. I hope the team works on increasing the index (200M). https://lnkd.in/dAHAn7fp #search #undermind #retrieval #science

22

4 Comments

Observability at the infrastructure and application levels must be harnessed to deal with today’s demanding workloads in #GenAI. Come talk to us to see how you can use the new AI with observability data to optimize your services.

17

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

70

3 Comments

A super clear, straight forward example of how to do cost effective RAG using LangChain for your document pipeline and Groq for compute resources. The code demonstrates how to speed up RAG and reduce costs using quantized embeddings (reducing 32 bit values to 8) Nice job Sahibpreet Singh! https://lnkd.in/gf5QMCKK

43

2 Comments

LLMs are excellent for that first draft and refactoring tasks that used to take hours of tedious effort. David Crawshaw captures their utility perfectly in his latest essay on programming with LLMs. https://lnkd.in/e5ZVYs9u

10

1 Comment

Evaluating my fine tuned 3B model with LLM judge - many of you have asked so here goes :-). Tl;Dr - 91% accuracy vs 47% not-fine tuned; with only 140 examples and 3 minutes of training (https://lnkd.in/gHw993ZS) I started out with creating a new synthetic test data set that the fine tuned models haven't seen, using the same code from the training set (data here: https://lnkd.in/gcCSdT5e). I used Comet's Opik per Armand's recommendation (you should follow him if you aren't :-)). Then built my evaluations test for (1) JSON schema compliance, (2) distance from reference answer (3) the LLM Judge which scores first on json format, then on compliance with schema, then on entities being detected, then on not detecting more than necessary. I tried this out with a bunch of LLMs that could run on this laptop, as you can see the 3B fine tuned one performed the best, followed by the 1B, then all of the out of the box ones pretty similar and not so great. Code for you to enjoy: https://lnkd.in/guhzdUkf You'll need a Comet account, which is shockingly free. Enjoy :-) Next I'm going to try to beef up the data gen with Distilable and CrewAI, let me know if you find that interesting or have other ideas.

200

19 Comments

➰ APIContext's latest insights on cloud-based API performance have been featured on Nordic APIs! In the 2024 report, we delve into crucial aspects affecting API efficiency: 🔐 Authentication is key. Ensuring robust authentication mechanisms in APIs is crucial, yet offloading this function can introduce complex challenges. 📖 DNS configurations matter. How you handle DNS settings can significantly impact API performance, particularly when tied to authentication processes. 🌐 The role of third parties. Critical third-party services play a significant role in 🔗 API transactions, underscoring the need for careful integration and monitoring. ⛅ Not all clouds are created equal. Differences in cloud services can drastically affect API performance, an essential consideration as APIs account for over 80% of all internet traffic. #APIs #CloudComputing #TechInsights #APIContext

15

1 Comment

🚨 Struggling with Low-Frequency Products? Here’s the Playbook I Wish I Had 🚨 For 15+ years, I’ve been managing products that people don’t use every day and only on episodic basis. And let me tell you—it’s tough. No guide. No clear answers to questions like: 👉 How do you measure product-market fit? 👉 How do you optimize acquisition costs when usage is infrequent? 👉 How do you actually drive retention? The frustration led me to create ICED Theory—a framework so intuitive and logical, it became part of Reforge’s course material. Now, for the first time ever, I’m hosting LIVE virtual classes to teach the ICED framework and help you grow and manage low-frequency products. 📅 Two intense days of interactive discussions. 💡 Deep dives into strategies that work. 🎉 50% launch discount for my first cohort. 👉 Ready to transform your approach to low-frequency products? Enroll now and secure your spot: https://lnkd.in/gHwUsGiJ Spots are limited, so don’t wait! Let’s crack the code together. 🚀 #lowfrequencyproducts #workshop #productmanagement #training #cohort

34

1 Comment

ModernBERT is an open-source embedding model which represents a significant leap in encoder-only transformer architecture merging: > Long context (8k tokens) > Local-global attention for efficiency > Up-to-date code and text training data (2 trillion tokens) > FlashAttention & Unpadding to push performance on consumer and server GPUs > High retrieval and classification accuracy that sets new benchmarks across GLUE, BEIR, code tasks, and more. Learn more about this exciting new embedding model here: https://lnkd.in/en5ud9VE

2

Thrilled to support 💪 Perplexity return citations param - https://lnkd.in/g7kkztxX (+4 more updates 👇) ⚡️ Azure - return response headers ✅ Streaming - break out of infinite model loops 🎉 Instructor - fix completion_cost calc 👀 [PR] logging by API key - https://lnkd.in/gAG5YKQa

6