Jim Blomo

We are quickly entering the "trough of disillusionment" in the hype cycle for generative AI. While GPTs excel in automating certain tasks (and can even automate away large portions of some jobs) they often fall short in others. Like all new technology, the issue lies in misapplying generative AI for tasks it wasn't designed for. This is a problem compounded by the GPTs themselves suggesting they can handle these tasks (often cheerfully!). This misalignment creates frustration and unmet expectations which we're starting to see more of these days. As the dust settles I firmly believe we'll recognize GPTs as a transformative technology across many industries. However it's crucial to understand their limitations and apply them appropriately. What are your thoughts on the current state of generative AI and its future applications?

2

The newest issue of The Pragmatic Engineer found a surprise uptick in eng hiring in the typically slow summer season - I'm curious to see how the macro environment impacts this uptick as we head into the fall, but I remain hopeful. I asked Gemini to give me the tldr of the article to share with my team (sorry, Gergely Orosz) and found it missed one of the most interesting pieces of the article. Gemini's highlights - Unexpected surge: Tech recruitment has seen an unusual spike in June and July, contrary to typical trends. Increased demand: Startups, especially AI-focused ones, are driving the hiring uptick. Global phenomenon: This trend isn't just limited to the US; Europe is also experiencing increased recruitment activity. Reasons for hiring: Companies are backfilling positions, investing in growth due to improved financials, and responding to increased VC funding. Challenges remain: While the job market is improving, finding qualified candidates and offering competitive compensation remains difficult for many companies. What I found most interesting - The nature of the backfills: the article finds that a lot of new headcount are actually backfills of folks that were "over-slashed" during rounds of layoffs. Not entirely surprising with the "trim the fat" and "rebuild lean" mentality, but interesting nonetheless. Either way, the uptick is encouraging (and don't always rely on Gemini's tldrs). #hiring #startups #VCbacked #softwareengineering

11

1 Comment

Anthropic's latest Claude AI model is taking human-AI interaction to a whole new level! 🖥️🤖 The new Claude 3.5 Sonnet can now use computers just like a human would - by looking at the screen, moving the cursor, clicking buttons, and typing text. This is a major leap forward in AI capabilities. Key features: Operates based on screenshots, emulating human computer use Can perform complex multi-step tasks across different applications Self-corrects and retries when encountering obstacles2 Accessible via API for developers to build with Why it's game-changing: This opens up incredible possibilities for AI assistance in data entry, form filling, software testing, and automating repetitive computer tasks13. Imagine having an AI assistant that can seamlessly work alongside you, handling tedious computer work while you focus on higher-level tasks. While still in beta, this technology demonstrates how AI is rapidly advancing toward more human-like interaction with digital interfaces. As it develops further, it could revolutionize how we work with computers and boost productivity across industries. The future of human-AI collaboration just got a lot more exciting! What do you think about this new capability? #AIInnovation #FutureOfWork Here is a video about it: https://lnkd.in/gh8aNYEu

6

New iterative reasoning technique could improve RAG app accuracy by 50% (vs. Naive RAG)! Another solid example of the rapid innovation cycle on RAG research. BTW, you can quickly find out answers to question like "Why did combining RAG with Fine-Tuning and Iterative Reasoning, such as OODA, improve accuracy by 50 percent?" in Epsilla (YC S23)'s "RAG Research Paper" smart search app at: https://lnkd.in/gVgzpbue , with number of research papers included exceeding 360. #genai #rag #smartsearch #innoation

19

It was great to hear Robert Wong Julie Willem Angie McAllister, Ph.D. and Markus Kunesch on the recent episode of the @Ed-Technical podcast -- the first of a three-part series about how we're applying Google AI to our work in education and learning: https://lnkd.in/e7hQi6a2. This episode delves deep into LearnLM, how we're making Gemini tuned for pedagogy and how we're integrating this technology into our mainstream products to improve learning outcomes, and evolve how people learn on Google products. It explains inspiring new product ideas, and the steps we take across the company to understand and enhance the journey of learning in a responsible and innovative way. #LearnLM #AI #Education #Google

67

A couple of recent college graduates from India against AI finetuning giants like Anyscale and Run:ai? I met Arko C at a NASSCOM event last October. He was building Xylem AI (YC S24) , a platform that allows you to train and deploy LLMs in production. So naturally most Indian VCs rejected him. Because how could a team of three recent college grads compete with the likes SV startups like Anyscale that have raised hundreds of millions of dollars, have teams of experienced AI researchers and a stash of H100 GPUs? Arko hustled and closed two Fortune 500 companies for paid contracts. I personally know unicorns that haven't managed to do this. He then got into YC in the last round in May. There's a lot to still be proved out, but Arko exemplifies how valuable grit is in building startups. I’m doing a webinar with him tomorrow on challenges of using LLMs in production. Link to register in the comments. Disclaimer: I'm a small investor in Xylem.

153

5 Comments

🌉 San Francisco's Surprising Comeback: From 'Doom Loop' to Renaissance? Recent data suggests that San Francisco, once trapped in a pandemic-induced 'doom loop', is showing strong signs of recovery. Here's what's changing: 📈 Positive Trends: - Population Growth: Net gains over the past two years, with domestic in-migration positive for the first time since 2012. - Housing Market: Median home sale prices up 1.2% YoY to $1,427,500 in June. - Job Growth: The Bay Area added 7,400 jobs between January and June. Downtown Revival: Foot traffic beyond typical work hours has recovered to over 90% of pre-pandemic levels. - Crime Reduction: Property crime down 32% and violent crime down 14% in Q1 2024 compared to Q1 2023. - AI Boom: The city is positioning itself as a hub for the AI revolution. 🚧 Ongoing Challenges: - High office vacancy rates - Homelessness and drug crisis - Lagging tourism and business travel 🔑 Key Initiatives: - "Vacant to Vibrant" program offering free rent to businesses in empty storefronts - Increased police patrols and encampment sweeping - Crackdown on open-air drug dealing 💡 Expert Insight: "The rising AI empire is giving the city a new edge. Funding, talents, and culture related to the AI revolution will help the city turn the corner." - Lu Chen, Senior Economist at Moody's Analytics 🤔 What's Your Take? Are these positive trends sustainable, or is it too early to declare a comeback? How can other cities learn from San Francisco's recovery strategies? What role will AI play in reshaping urban economies, particularly in tech hubs? https://lnkd.in/gCnPCBRU

14

1 Comment

First story on what we have been up to at Augment. Much more to come! https://lnkd.in/gC6gbADA

370

64 Comments

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

7 Comments

Setting a vision is just the start. Think about the bold moves needed to make it happen. Sundar, Google's CEO, set a daring vision to tackle climate change. That's where the real work begins. Executing a vision requires bold and tough decisions. Google is buying nuclear energy to feed its AI data centers. Other big tech companies are following this lead. This is a big kudos to Sundar and his leadership team. They are executing on a sustainability vision. Leadership is about taking action. And Sundar is showing the way.

27

12 Comments

Check out the latest in our "Debuggability" blog series! This blog is authored by Kevin Downey Jason Johl and me and talks about how we built a Debuggability Paved Road at Intuit using Ephemeral Containers and Argo Workflows. We'd love to hear your thoughts on the topic. Also, what tools do you build as-a-platform to help debug service, compute and network issues in Kubernetes at scale? #intuit #kubernetes #debug

23

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

LLMs are designed to make stuff up. In most applications this an acceptable risk, after all, people make stuff up too. But in regulated, risk averse industries (finance, healthcare, insurance, legal) even a small mistake can cause big problems. Manual evaluation and monitoring is a crucial yet often overlooked component to catching errors and mitigating risk. Automated oversight can help, but there's no such thing as "Goodbye Hallucinations" unless humans are firmly in the loop.

10

1 Comment

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments

🍜Anybody up for some ramen? I am excited to share that I am back in the garage, building again (it’s actually a house in the Berkeley Hills). We are taking the wraps off of Supercog AI, a new startup focused on solving Application Integration using LLM powered agents. My co-founder in this new venture is a great friend of mine (and a former co-founder partner), Alex Osborne. Alex and I have worked around the application integration space for a long time, and we are unrealistically excited about applying the GenAI stack to this problem. Not many people relish the task to get some bits to move from system A to system B. But it’s a critical job to be done, and that’s led to a lot of brittle scripts, a myriad of inflexible “no code” tools, and a lot of hand crafted SQL. But the power of the Large Language Model, trained on a huge corpus of information, offers a remarkable tool to solve this problem. The LLM you use today already knows the APIs of hundreds of popular systems. It knows the SQL dialects of every major database. It understands the semantics and data schemas of many popular SaaS systems. It understands much of the specific domain in which your business operates. I like to call the AI revolution “the last platform shift”. That may turn out to be hyperbole, but I definitely believe we are in the very early stages of seeing what this new stack can enable. I know we are hardly the first folks to claim that “amazing stuff is coming!”. But this is why Alex and I have decided to focus on a real and hard problem. If we can prove that LLMs can power a new way to solve this problem, it will be the first successful *new* approach in 15 years.. We will have much more to share about this new platform soon. In the meantime, if you’re interested in getting a peek - or even better if you have application or data integration tasks that you would like some help with - please reach out because we want to help. #genai #startups #backinthegarage

164

37 Comments

You don’t normally see LLM researchers and engineers talking about their day-to-day work. Big shoutout to this YouTube video from the Anthropic LLM Interpretability team, the team behind the Golden Gate Claude. It’s amazing to see that most of the team members have worked for the team for only one year, and yet they have made such a big breakthrough in interpretability research. Another thing I want to highlight is the unique challenges facing the team and how they approach these challenges, that is, through fast trial-and-error iterations and constant evaluation of what may work and what to prioritize next. #artificialintelligence #machinelearning #deeplearning https://lnkd.in/eUaPAYch

97

1 Comment

Its about solving the problem, not 'having AI' in the system. This is from last March, but the Wave 1 vs. Wave 2 distinction does seem to be playing out. Less 'generate me content', more 'synthesize vast amounts of relevant data and aid in decisonmaking or signal extraction'. https://lnkd.in/ehFBFC32 They don't talk about multi-agent systems here, but looking forward to more startups with a focus on leveraging agentic design patterns (https://lnkd.in/ed5rMmFe) to streamline real world workflows.

4

1 Comment

As it becomes clear to everyone that LLMs alone aren’t going to magically produce “AGI”, the race has now shifted to building multi-agent systems. It’s funny to watch the agentification of everything unfolding in real time, with the last entrant being the LlamaIndex crew. While some developments here will definitely increase the quality of results (eg agentic techniques applied to the information retrieval phase), others - in particular the LlamaAgent framework’s idea of mapping individual agents to a microservice infrastructure, sound like a massive logic bomb being built. Just like when RPC, Distributed Objects, SOA and Microservices became popular, I expect confusion, absurd cost spikes and a complete new cottage industry around observability, best practices, design patterns and protocols to grow around this new (old) trend. Although I’m personally bullish about the possibilities, the thought of managing 100s of microservices that are effectively - as Sam Altman describes gpt4 - “educated high schoolers” running inside your infrastructure, calling tools and making decisions, is a scary (inevitable?) prospect. https://lnkd.in/dh4tx7bZ

8

4 Comments