David Singleton

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

The newest issue of The Pragmatic Engineer found a surprise uptick in eng hiring in the typically slow summer season - I'm curious to see how the macro environment impacts this uptick as we head into the fall, but I remain hopeful. I asked Gemini to give me the tldr of the article to share with my team (sorry, Gergely Orosz) and found it missed one of the most interesting pieces of the article. Gemini's highlights - Unexpected surge: Tech recruitment has seen an unusual spike in June and July, contrary to typical trends. Increased demand: Startups, especially AI-focused ones, are driving the hiring uptick. Global phenomenon: This trend isn't just limited to the US; Europe is also experiencing increased recruitment activity. Reasons for hiring: Companies are backfilling positions, investing in growth due to improved financials, and responding to increased VC funding. Challenges remain: While the job market is improving, finding qualified candidates and offering competitive compensation remains difficult for many companies. What I found most interesting - The nature of the backfills: the article finds that a lot of new headcount are actually backfills of folks that were "over-slashed" during rounds of layoffs. Not entirely surprising with the "trim the fat" and "rebuild lean" mentality, but interesting nonetheless. Either way, the uptick is encouraging (and don't always rely on Gemini's tldrs). #hiring #startups #VCbacked #softwareengineering

11

1 Comment

💡 Why do we need so many evaluation tools for LLMs? As engineers, we build 'production-ready' LLM products using these metrics. But what happens next? How do we maintain control and ensure reliability? At Feedback Intelligence, we’ve crafted a cookbook to keep your LLMs reliable and aligned with user expectations. 🍲 📖 Give it a read and let’s chat!

16

Hey everyone, I am an investor and advisor to Katanemo.... They are reinventing the infrastructure primitives for GenAI and launched Arch - https://lnkd.in/gCrbbkEV - an intelligent prompt gateway. Give the project a look on Github, and star the project if it resonates

88

3 Comments

I like this idea of "technical taste". It gets into where software engineering is as much creative art as it is science. Four great takeaways (IMHO) for software devs in this era in particular: 1. Aspiring engineers should cultivate a sense of curiosity, experiment with different tools and technologies, and embrace a mindset of continuous learning. 2. AI has the potential to streamline processes and enhance productivity for engineers, but it may also lead to disruptions in traditional software development workflows. 3. Developing technical taste and judgment is essential for making informed decisions about which technologies and approaches to pursue. 4. Collaboration and open-mindedness are key to leveraging the full potential of AI and staying ahead of technological advancements. I also really liked these two quotes from Sam Schillace: 1. "The right time to do something is when you have that feeling in the pit of your stomach that's like, 'oh, this is a great idea and it's going to suck to build because nothing's ready yet.'" 2. "Technical taste is like, 'how well have you consolidated that set of experiences and heuristics into judgment that you can apply accurately when you see new things?'" 3. "It may be the case that very small teams can do very large projects, or like we were talking about before, it may be the case we're just going to get really ambitious about what we try to do with the same size of teams, which is kind of where I would put my money." Great job on continuing to put out great episodes, Brett Berson and team First Round Capital! #ai #engineering #softwaredevelopment

5

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments

Check out the latest episode of our Tech Teams Today podcast with Yongsheng Wu, VP of Engineering at Granica - now available on all major podcast platforms. #AI #EngineeringLeadership #DataScience #MachineLearning #TechTeamsToday #Revelo

4

~40% of Frec’s customers are engineers working in tech, many of whom expect frequent and large capital gains. They also prefer to self-manage their money, which makes Frec an ideal platform. Meet Evan: an engineering leader who upgraded from investing in ETFs to direct indexing. Direct indexing appealed to Evan because it offers comparable returns to his ETFs while saving him money on advisory and ETF fees. “ETFs are true and tested, and giving folks a different way of investing in the market while saving more money is really appealing.” He’s used many other robo-advisors and other fintech tools, but “Frec seems to maximize tax loss harvesting opportunities better than any other product that I’ve used.” He also likes Frec's transparent fee structure.  “I feel like with a lot of other companies, that’s muddled in with the cost of investments,” he told us, “and you don’t really know how much you’re paying or how much you’re not saving as a result. I think Frec does a really good job of laying all of that out and giving you a true understanding of the cost of these investment products.” Read more about his Frec journey here ⬇

45

2 Comments

We are quickly entering the "trough of disillusionment" in the hype cycle for generative AI. While GPTs excel in automating certain tasks (and can even automate away large portions of some jobs) they often fall short in others. Like all new technology, the issue lies in misapplying generative AI for tasks it wasn't designed for. This is a problem compounded by the GPTs themselves suggesting they can handle these tasks (often cheerfully!). This misalignment creates frustration and unmet expectations which we're starting to see more of these days. As the dust settles I firmly believe we'll recognize GPTs as a transformative technology across many industries. However it's crucial to understand their limitations and apply them appropriately. What are your thoughts on the current state of generative AI and its future applications?

2

This will unlock a lot of possibilities, and comes with some really nice advantages like: - More efficient chatbots - Easier builiding proces - Better customer experience - Cheaper prices for chatbots Now's the best time to start integrating customer support chatbots in your business to deal with customer while you focus on selling 💯

2

Great article on the approach to agentic architecture. A lot of similarity on how we've been developing some of our experiments recently. A hybrid of code + LLM logic and orchestration. https://lnkd.in/g84Eg5Pw

25

I am incredibly excited to announce Arch gateway https://lnkd.in/ePAf-VK8, an open source project for developers to build genai apps faster. It is engineered with purpose-built LLMs, Arch handles the critical but undifferentiated tasks related to the handling and processing of prompts, including detecting and rejecting jailbreak attempts, intelligently calling “backend” APIs to fulfill the user’s request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability of prompts and LLM interactions in a centralized way - all outside business logic. Here are some additional key details of the project, * Built on top of Envoy and is written in rust. It runs alongside application servers, and uses Envoy's proven HTTP management and scalability features to handle traffic related to prompts and LLMs. * Function calling for fast agentic and RAG apps. Engineered with purpose-built fast LLMs to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts. * Prompt guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code. * Manages LLM calls, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability. * Uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance. This is our first release, and would love to build alongside the community. We are just getting started on reinventing what we could do at the networking layer for prompts. Do check it out on GitHub at https://lnkd.in/ePAf-VK8 Please leave a comment or feedback here or in GH and I will be happy to answer. #gateway #envoy #rust #genai #developers #ai cc Salman Paracha | Muhammad Ikramul Haq | Syed Anas Hashmi | Fahim ul Haq | Farheen Omar | Umar Farooq | Arslan Ahmad | Abdul Rafay Abbasi |

60

9 Comments

Just over a year in and GPT4 is now 12x cheaper, faster, better! We're most excited about GPT-4o – for complex reasoning tasks, and Gemini 1.5 Flash – for data transformations, especially given its long context length. As chips get faster, model architectures improve, and we have more data, it feel inevitable that the cost of intelligence (as a service) will keep going down -- just like what happened for compute and storage costs in the last decades.

77

2 Comments

Anthropic's latest Claude AI model is taking human-AI interaction to a whole new level! 🖥️🤖 The new Claude 3.5 Sonnet can now use computers just like a human would - by looking at the screen, moving the cursor, clicking buttons, and typing text. This is a major leap forward in AI capabilities. Key features: Operates based on screenshots, emulating human computer use Can perform complex multi-step tasks across different applications Self-corrects and retries when encountering obstacles2 Accessible via API for developers to build with Why it's game-changing: This opens up incredible possibilities for AI assistance in data entry, form filling, software testing, and automating repetitive computer tasks13. Imagine having an AI assistant that can seamlessly work alongside you, handling tedious computer work while you focus on higher-level tasks. While still in beta, this technology demonstrates how AI is rapidly advancing toward more human-like interaction with digital interfaces. As it develops further, it could revolutionize how we work with computers and boost productivity across industries. The future of human-AI collaboration just got a lot more exciting! What do you think about this new capability? #AIInnovation #FutureOfWork Here is a video about it: https://lnkd.in/gh8aNYEu

6

Kim Burgaard Rachel Perez Jay Leverett David Bellotti -- I'll be listening to this one for sure! Sergio H. Serge Madenian Episode Summary: The episode features an in-depth interview with Mike Krieger, the new chief product officer at AI company Anthropic, discussing his background, the decision to join Anthropic, the company's focus on responsible AI development, the challenges and opportunities in building AI products, and his vision for the future of AI technology and its applications. https://lnkd.in/guWW8cku

3

Seeing lots of great founders disrupting EDA with AI Most EDA tools don’t do real “engineering,” but rather take the rote tasks out of implementing engineering output from centuries of “experience” Can AI collapse the heuristics of the best engineers that ever lived to actually automate engineering? Hard to beat EDA at its own game by making it better with AI Generating the boundary conditions that engineers normally use to start the engineering process and better optimized at the end would be a great complement to existing EDA

108

22 Comments

Interesting comparison of human-based ride hailing services (UberX, Lyft) vs autonomous-driving one from Waymo. TLDR: Waymo is more expensive and a lot slower to get you to your destination. And this is in the city where Waymo vehicles and software are probably most optimized for. This isn't to bash on Waymo and other self-driving car platforms, which have made incredible progress. That being said, the problem is not solved and it will take many more billions and many more years to get autonomous driving services like this to be price and performance competitive to current ride hailing services. https://lnkd.in/eb4uZ2Um #autonomouscars #AI

2

I just realized: we should start saying "build memoization" rather than "build caching." We aren't really talking about least-recently-used or time-to-live algorithms, we're talking about "don't bother recompiling this piece if none of its inputs changed." (It has been puzzling me that some people are so deeply fearful of "build cache" as a concept, and honestly I think it's mostly because most build systems have non-hermetic state scattered all over the place. Maybe a slightly more precise term for the type of caching we're concerned with would clarify the semantics in question.)

83

9 Comments