Carlos Nasillo

Large language models are wonderful at ingesting large amounts of content & summarizing. Uploading an academic paper and I can pester it with an infinite list of questions & it will respond with equally infinite patience. In comparing the two most recent Microsoft earnings calls, Claude highlighted: - faster than expected Azure growth (29% vs 27%) - AI contributing 8% of Azure revenue up from 7% - higher CapEx spending & greater capacity constraints for data centers - much better commercial bookings growth : 17% vs 13% Excellent analysis. Benn Stancil described LLMs as great averagers of information. But what if I don’t want the average? If I seek the data point two standard deviations out? If I’m channeling my inner Anthony Bourdain & I’m seeking the fermented shark or cobra heart within my query? At Google, we ranked web pages & ads with many signals. An engineer taught me that of them found rarity of a word across a set of documents. For example, if there’s a document containing 10 instances of the word ghotiacross a collection of documents, where the average document has 0 ghotis, that document is likely the best to answer a search about fish. I haven’t found a way to goad an LLM to produce the rare result. Why did one user decide to use a particular piece of software when 10 others did not? Maybe I haven’t learned how to prompt an LLM well. Ideogram launched a feature called Magic Prompt that expands a basic prompt into richer instructions that elicit better results, marrying the language of a user & a computer. A user might write : A boy with a dog in a park. Magic Prompt replies : A heartwarming scene of a young boy playing with his happy, wagging dog in a lush, green park. The boy has a playful smile on his face while he tosses a bright yellow ball for his furry friend to fetch. The park is filled with beautiful flowers and trees, creating a serene atmosphere for this bonding moment between boy and dog. Magic Prompt surprised me in three ways. First, unlike Google search where terse queries tend to work very well, LLMs relish verbosity. Less Hemingway & more Faulkner. Second, maybe there’s a prompt for rarity. If you know what it is, please share with the rarity shibboleth. Third, I wonder what other prompt techniques are out there that have yet to be discovered to wrest the best from an LLM. This technique is called TF-IDF. It means Term Frequency Inverse Document Frequency.

21

6 Comments

Visualising AI training cost (estimated) ... The AI Index collaborated with research firm Epoch AI to estimate AI model training costs based on cloud compute rental prices, which have been adjusted for inflation since 2017. Although not 100% accurate, these figures iterate the narrative that fine-tuning and RAG are the only cost-effective ways for the majority of companies who want to run and tune their own models based on open-source models. For more info, check out this article https://lnkd.in/eEzDGfyV

1

𝗛𝗼𝘄 𝘁𝗼 𝗰𝗿𝗲𝗮𝘁𝗲 𝘆𝗼𝘂𝗿 𝗼𝘄𝗻 𝘀𝗺𝗮𝗿𝘁 𝗴𝗹𝗮𝘀𝘀𝗲𝘀 𝘄𝗶𝘁𝗵 𝗼𝗻𝗹𝘆 $𝟮𝟬??? Meta launched 🦙 LLaMA3 to show the world what’s possible with open source LLMs. Why this is a game changer? Since LLaMA3 was released one month ago, more than 700 fine tuned versions have been uploaded into Hugging Face, we've seen extending the context window into 1M and co-lead the chatbot arena for english language. This weekend they hosted a 24h LLaMA3 hackathon with Cerebral Valley and more than 500 AI engineers. The results were pretty amazing and these demonstrates the real power and value of these models. A summary of the finalists: 𝗝𝗼𝗲𝗖𝗥𝗠: AI-first CRM that receives signals about customers, performs research about them, and automatically drafts outreach campaigns 🥉 𝗔𝗰𝘁𝗶𝘃𝗮𝘁𝗶𝗼𝗻 𝗔𝗯𝗹𝗮𝘁𝗶𝗼𝗻 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻: Research team that identified how to nullify activation layers in Llama3 responsible for censorship 🥇 𝗢𝗽𝗲𝗻𝗚𝗹𝗮𝘀𝘀 😎 : $20 budget smart glasses that can answer questions about what you’re looking at 𝗙𝗲𝘆𝗰𝗵𝗲𝗿: Fast AI avatars that speaks responses with lip-syncing. Context loaded by recent events from the Brave news API 𝗛𝗼𝘂𝗻𝗱: AI to help law enforcement crack down on human trafficking 𝗠𝗼𝗻𝗴𝗼𝗼𝘀𝗲 𝗠𝗶𝗻𝗲𝗿: Improved Python codegen made by providing llama 3 with up to date documentation 🥈 𝗗𝗲𝗯𝟴: AI agent debate arena where different models compete in Lincoln-Douglas debate. A third agent rates the quality of the debaters and scores them 𝗖𝗶𝘁𝘆 𝗛𝘂𝗯: Llama3 powered chat agent that lets residents ask questions about city laws and regulations. RAG’s over city documents to inform the answers 🔗 More info about OpenGlass: https://lnkd.in/dH_q7c9C 🔗 Github: https://lnkd.in/dQta8Uuv #ai #genai #llama

26

 #Llama 3🦙Meta just released new best in-class open models https://lnkd.in/eWsF9Vwb Those new models are real improvement on the previous Llama 2 versions in terms of performance across various benchmarks: ### Llama 3 8B vs. Llama 2 70B - GPQA (0-shot): Llama 3 8B achieves a score of 34.2, while Llama 2 70B scores 21.0 - HumanEval (0-shot): Llama 3 8B scores 62.2, a substantial increase compared to Llama 2 70B's score of 25.6 - GSM-8K (8-shot, CoT): The Llama 3 8B model outperforms with a score of 79.6, compared to Llama 2 70B's 57.5 - MATH (4-shot, CoT): Llama 3 8B scores 30.0, while Llama 2 70B scores 11.6 ### Llama 3 70B vs. Llama 2 70B - GPQA (0-shot): Llama 3 70B scores 39.5, compared to Llama 2 70B's 21.0 - HumanEval (0-shot): Llama 3 70B significantly outperforms with a score of 81.7, while Llama 2 70B scores 25.6 - GSM-8K (8-shot, CoT): Llama 3 70B achieves a score of 93.0, which is much higher than Llama 2 70B's 57.5 - MATH (4-shot, CoT): Llama 3 70B scores 50.4, more than quadrupling Llama 2 70B's score of 11.6 ### Additional Improvements in Llama 3 - Context Window Size: Llama 3 has increased the context window size from 4k to 8k tokens, which allows for better understanding and generation of longer text passages - Training Data: Llama 3 models have been trained on a dataset of 15 trillion tokens, compared to Llama 2's 2 trillion tokens, which contributes to their improved performance - Tokenizer: A new tokenizer in Llama 3 expands the vocabulary size to 128,256 tokens, up from 32K in Llama 2, allowing for more efficient text encoding and potentially stronger multilingual capabilities - Grouped-Query Attention (GQA): The 8B version of Llama 3 uses GQA, which is an efficient representation that should help with longer contexts - Training Techniques: Llama 3 Instruct has been optimized for dialogue applications and was trained on over 10 million human-annotated data samples with a combination of advanced training techniques.

6

2 Comments

Meta just released LLaMa 3.1, the most powerful open source LLM to date, which outperforms GPT-4 and Claude 3.5 on some parameters. This is the biggest moment so far in the open vs. closed debate in AI. One of the big questions for its success is: will developers adopt open source (and specifically Meta) to build new products? Mark Zuckerberg shares his vision for the future in "Open Source AI Is the Path Forward" https://lnkd.in/eqbmx7_x?

27

1 Comment

I'm very conflicted on this. On one hand the EU AI act's 4 pillars of risk seem somewhat arbitrary and not based on anything empirical. Not only that, they could stifle innovation and startups coming out of the EU inadvertently centralising more power towards the US and China. It will be much harder for an AI research focussed startup in the EU to compete with the already well established players like OpenAI, Google and Meta. Obviously the three I mentioned are against it for the most part but once they have processes in place, it will be much easier for them given their resources to churn out regulatory documentation and adhere to the programs put forward by the EU for the particular pillar their future models fall into. I know there are development On the other hand, more transparency, data protection and privacy safeguarding are crucial and are needed, especially now with the amount of data being scraped to build these systems. All of this can build more trust in the systems of which people are already being turned off by. There are countless stories of artist's work and people's voices being used without their consent. YouTube videos being scraped and used to train models like Sora. Transparency would help build that trust back up. I'm just not sure the EU AI act in its current form is the ideal solution. I think there needs to be more emphasis on empirically focussed risk management rather than just picking 4 very fuzzy categories which could have overlapping parts in a larger more complex system. I want to be more optimistic but I guess we'll see!

14

6 Comments

AI News (Part 1) 🇬🇧 Britain's AI Boom: The UK is solidifying its status as a global AI leader with a colossal £1 BILLION investment from US tech giant CoreWeave. With the 3rd highest number of AI companies and private investments globally, the UK is clearly making all the right moves! OpenAI's Big Move: OpenAI is set to revolutionize the search engine landscape. On Monday, they will introduce an AI tool designed to rival Google, integrated into Microsoft's Bing. This innovative tool will provide web information with citations in a ChatGPT-style format. Elon Musk's xAI: Elon Musk's new AI startup, xAI, has secured an astounding $6 BILLION in Series B funding! Top investors like Andreessen Horowitz and Sequoia Capital are betting big on Musk's ambitious vision. Nvidia's AI Insights: Nvidia's CEO forecasts a surge in demand for their chips, driven by AI-generated videos and human-like voice interactions. The AI content creation revolution is just beginning! 🇮🇳 AI Adoption in India: In India, an impressive 92% of knowledge workers are already leveraging AI. The Work Trend Index report highlights the rapid adoption of this transformative technology. The pace of AI innovation and adoption is breathtaking. Exciting times ahead as this technology continues to transform every industry! What do you think - are you ready for the AI revolution? #AI #ArtificialIntelligence #TechNews #Innovation #AIInvestment #OpenAI #xAI #Nvidia #AIInIndia #AIBoom

5

2 Comments

The amount of money in AI is already insane... and it's not slowing down! However, it will be a while before companies really start to see the return on investment. Take OpenAI for example... recently they raised $6.6bn in their latest funding round valuing the business at $157bn... which already matches them in value with companies such as UBER. This is an incredible figure, however, even though they have announced a shift to a for-profit company, it is reported they will make a loss of around $5bn this year. The majority of this cost comes down to building and operating powerful AI models which is why having the right talent and minds in the business will be essential to any AI business looking to survive in the long run. Other big players like Google DeepMind and Anthropic are in the same boat - big valuations, big costs, and an uncertain path to profitability. Are we looking at another bubble? Or are these companies here to stay? #ai #artificialintelligence #DataReq #hiring

9

3 Comments

Paper summary by ChatGPT: "ChatGPT is bullshit": 1. Introduction: The paper discusses the inaccuracies in the text generated by large language models (LLMs) like ChatGPT. The authors argue that these inaccuracies should be understood not as "lies" or "hallucinations," but as "bullshit" in the sense defined by Harry Frankfurt, which refers to statements made without regard for their truthfulness. 2. Operation of ChatGPT: ChatGPT and similar LLMs generate text by using large amounts of data to predict the likelihood of the next word in a sequence. This process is fundamentally different from how humans generate language. The primary goal of these models is to produce natural-sounding responses, not to ensure the accuracy of the information. 3. Issues with Accuracy: The main objective of LLMs is to mimic human-like text, not to provide truthful information. This leads to frequent inaccuracies, which can cause significant problems in contexts like legal documents or academic writing. For example, a lawyer using ChatGPT found that many of the cited cases were not real, highlighting the serious implications of these inaccuracies. 4. Inappropriateness of the "Hallucination" Metaphor: Referring to the false information produced by ChatGPT as "AI hallucinations" is misleading. The authors argue that this term anthropomorphizes the model and inaccurately suggests that the model is attempting to perceive or remember information, which it is not. "Hallucination" implies a deviation from an attempt to perceive reality, which does not apply to LLMs. 5. Definition and Relation to Bullshit: According to Frankfurt, "bullshit" is characterized by a lack of concern for the truth. The outputs of ChatGPT fit this definition, as they are generated without any regard for their truthfulness. The authors argue that ChatGPT's outputs are at least "soft bullshit" (unintentional indifference) and possibly "hard bullshit" (intentional deception) depending on the perspective. 6. Implications of Bullshit Terminology: By characterizing the false outputs of ChatGPT as "bullshit," the authors aim to provide a more accurate understanding of the technology. This terminology helps in predicting and discussing the behavior of LLMs more effectively. It emphasizes that these models are not designed to convey truth but to generate convincing text, which is crucial for policymakers and the public to understand. 7. Conclusion: The paper concludes that recognizing ChatGPT's outputs as "bullshit" rather than "hallucinations" is important for clear communication about the technology's capabilities and limitations. Misleading metaphors can result in misguided expectations and responses from both the public and specialists. Properly understanding and labeling the outputs of LLMs can lead to better management and application of these technologies.

1

4 Comments

Always wondered what incredible things Google can do on AI front with the amount of data they have. Decades of effort (input) are finally paying off ! Gemini 1.5 Pro - 2M tokens context size Gemma 2 - open source 27B parameters llm Project Astra - future of AI assistant And with their hardware-rendered solutions on #android only, I'm curious how strong their proposal for #pixel phones will be, in the near-future. Win-win in every front. #googleIO #AI

13

I think is kind of funny, realizing that the current state of the art is wrapped up in getting anything that smacks of intelligence out of AI Agents that are tightly controlled within a fairly narrow domain. Most of us that are working on this realize just how much effort that is - to get anything useful. This is true as we communicate with them using a language they have been trained on using a large corpus of data, potentially augmented with a niche knowledge base. Now imagine in that world that someone is trying to teach them to communicate with each other in some uncontrollable manner -- first premise. And then for some reason we decide to somehow teach them to make up their own language (for which there is no current large corpus of data to base it on). A language we therefore don't have a basis to understand. The second premise. Oh, I realize we are talking about some fiction that these two premises will make sense in. But in the real world? Most of the agents we are using at the moment are basically text completion / pattern matchers (and not totally foolproof at that). For this new language, what do we propose they are matching on? I guess they would have to simply translate their secrets into English or some other common language. Well if that is what is going on here, that means they are "reasoning" with their text completion strategy in English, so maybe all that is implied by the rhetoric of this "private language" is nothing more than what could be accomplished by using an encrypted communcation channel between them. We couldn't read or understand that either and it would be a much "quicker to market" approach to making obscure, obnoxious, and dangerous collections of Agents (assuming they get beyond their stupid era). So before we outlaw (or even invent new languages for this), let's just encrypt everything that occurs in and between them. According to ChatGPT, however: "I'm not able to directly encrypt or decrypt messages, including handling keys for encryption within our conversation." Well, that is only a matter of time....

1

An Apple team showcases LazyLLM, a novel approach to enhancing the efficiency of large language model (LLM) inference, particularly when handling long context prompts. Traditional LLM inference typically involves two main stages: prefilling, where the key-value (KV) cache is computed for all tokens, and decoding, which generates subsequent tokens. This conventional method can result in delays, especially with lengthy prompts, leading to an extended time-to-first-token (TTFT).   To address this challenge, LazyLLM utilizes dynamic token pruning, which selectively computes the KV cache only for tokens identified as important for the next prediction, rather than processing all tokens at once. This strategy optimizes the generation process by postponing the computation of less relevant tokens, thereby improving overall inference speed without compromising accuracy.   A notable aspect of LazyLLM is its capacity to dynamically adjust token processing at each generation step. By leveraging attention scores from previous transformer layers, it identifies and prunes less critical tokens. This adaptability allows for the reactivation of pruned tokens when necessary, ensuring model accuracy across a range of tasks. Experimental results indicate that LazyLLM can be seamlessly integrated with existing transformer models, such as Llama 2 and XGen, without the need for fine-tuning, achieving speedups in TTFT of up to 2.34 times while maintaining accuracy.   Furthermore, LazyLLM features an auxiliary caching mechanism, known as Aux Cache, which stores the hidden states of pruned tokens for efficient retrieval, further streamlining the inference process. Evaluations across various tasks, including summarization, few-shot learning, and code completion, consistently reveal improved TTFT speed with minimal accuracy loss.   In essence, LazyLLM represents a significant advancement in LLM inference efficiency, particularly for applications that require an understanding of long contexts. This development enhances user experience by reducing wait times while ensuring high performance in natural language processing tasks.  Arxiv: https://lnkd.in/ebWKqhVx

4

1 Comment

OpenAI's Swarm is actually a very elegant way to use the OpenAI's tool API to dynamically update the system prompt and the available actions provided to the model. Swarm, as a package, is pretty useless! OpenAI describes it as an educational experiment, and you can count on AI influencers to hype it up to the moon, but I think it demonstrates a very interesting use of the `tools` API. OpenAI, like Antropic, supports a `tools` option as part of the completion API. A tool takes the form of a Python function that takes a specific input format and returns a well-defined output. This provides the option for the model to choose between outputting a text response or suggesting a tool with specific arguments. This is what happens when ChatGPT chooses to perform a web search or to use Dall-E 3, for example. LLM-based agents are simple! They can be described as a combination of: - An LLM: this is the decision engine taking a state and generating an action. - Instructions: the instructions provided as the system prompt will define the goal of the agent. It plays a similar role as the reward function in reinforcement learning-based agents. - A set of actions: actions in the context of LLM-based agents are usually called “tools“ or “functions“. Those are the available actions for the LLM to choose from. Swarm implements what OpenAI calls "routines" and "handoffs". Routines are basically this iterative process we find in ReAct agents. By using the tools argument, we can implement ReAct agents very efficiently in the OpenAI API: we ask a question, the API returns tool use suggestions, we execute the tools, and we iterate this process until the API returns text content. But ReAct agents have a static system prompt and set of tools! The idea of handoffs is to include, as part of the tools, a function to dynamically modify the system prompt and the available tools. Within the context of the agentic jargon, it means modifying the goal and set of actions available to the current agent during the iterative process. That can be useful when the space of all available actions becomes too large. We break down the set of possible tasks into subtasks, and we associate a subset of the possible actions with subtask-specific instructions. So Swarm basically implements a typical ReAct agent that dynamically adjusts its system prompt and set of available tools during the iterative process. It is very simple but quite elegant, in my opinion!  -- 👉 Don't forget to subscribe to my ML newsletter https://lnkd.in/g4iKyRmS --

176

17 Comments

Remember GPT-3 didn't get the inflection point before chatgpt unfortunately image generation models are there 🙂 Before GPT-3 was just developer focused and gated to the few who had to go through bit of learning curve it didn't take off and scope was limited Now anyone can use it in the format they are used too with less or no learning curve Working in the diffusion models side I feel like this models are good enough but have a lot of tech and freedom of control especially in the open-source side but the SAM of people who can use it is very limited because of it Yes you can very well create random amazing photo using dale-3 in chatgpt or Midjourney But what I'm talking about is if you want to use your images or virtual models with consistency in content creation for your product in particular aesthetic or any particular specific thing of yours then this closed model don't provide those abilities So it gets gatekeeped by people like me who have went through the learning curve of understanding this models and workflow deep moreover the hardware limitation becomes key with the increasing size of the models Although we call it Stable diffusion models it will be very unstable if you don't know how to use it as it as many biases and controllable parameters Will this models get better of course when trained with better captioned data it will but I do feel the user experience need to change because even the current models are almost there with knowledge transfer and generalisation just like how gpt-3 was We maybe early but the winner will be the one who solves the user experience problem People really don't have time to understand what is a ksampler, cfg, which controlnet to use etc etc I do like what Midjourney is doing but I still feel it is still the MySpace not the Facebook of image generation if you get the context Tech should be invisible and I'm leaning more towards an agent approach Better for a Teacher model to understand the request of the user and know all the levers and options either through enough paired request and output ( large data ) training like something similar in pix2pix and distilling that back to a diffusion model in our normal form and doing RLFH and improving it like what Midjourney did It is a wild idea, I do believe people shouldn't be bothered with all this tech and should communicate in natural language and get there output So what do you think will be the chatgpt moment for vision models??

5

𝗚𝗼𝘃𝗲𝗿𝗻𝗺𝗲𝗻𝘁𝘀 𝗮𝗻𝗱 𝗮𝗰𝗮𝗱𝗲𝗺𝗶𝗮 𝗰𝗮𝗻’𝘁 𝗸𝗲𝗲𝗽 𝘂𝗽 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗔𝗜 𝗿𝗮𝗰𝗲. 𝗥𝗲𝗮𝘀𝗼𝗻? Imagine this: Google's Gemini Ultra cost $191 million in computing power to train. OpenAI's GPT-4. Roughly $78 million.   Considering the cost doubling every nine months, the cost of only hardware and electricity needed would be in billions later this decade.   Dario Amodei, CEO of AI company Anthropic, predicts that AI systems released later this year would cost nearly $1 billion, and the generation after that would cost more than $10 billion.   As costs to train these powerful models skyrocket, innovation in AI is increasingly concentrated in the hands of a few tech giants. For those outside this elite circle—especially academic and government researchers—these costs are creating an insurmountable barrier to meaningful contributions in AI advancement.   GEMESYS , a German startup, aims to revolutionize AI hardware by developing fully analog chips that mimic the brain's information-processing mechanisms. This approach will significantly enhance data and energy efficiency, enabling AI training and inference directly on edge devices, such as mobile phones and sensors. What are your thoughts? Should we be rethinking how AI services operate sustainably, or is this just the new reality of high-stakes tech innovation?   #Venturecapital #AI #Deeptech #Startups   Share this with your network and follow us at APEX Ventures for the latest updates on our portfolio companies and exclusive insights into the world of Deep Tech. Also, get access to exclusive content on Deep Tech startups in our newsletter: 🔗 https://t2m.io/EV2qHQuo

61

This is wildly revisionist. DeepMind without Google wouldn’t be anywhere close to what it is now. Hell, it very well might’ve died. Most European VCs hadn’t heard of it when the acquisition happened. It would have withered. Further, the acquisition price was ridiculed by many in the UK and Europe at the time. Founders Fund invested when UK VCs wouldn’t…and it had 25% of the company at exit. Google buying DeepMind was probably the best possible outcome for it. Prohibiting its sale wouldn’t have resulted in a roaring success. Same with Booking, Arm, and all the other companies that Europeans like to play ‘what could have been?’ with. Those companies needed deep pockets and best practices that simply aren’t available in Europe. https://lnkd.in/eNsf28bv

364

25 Comments