Victor Martin Garcia

𝐎𝐋𝐌𝐨𝐄: 𝐓𝐡𝐞 𝐅𝐢𝐫𝐬𝐭 100% 𝐎𝐩𝐞𝐧-𝐒𝐨𝐮𝐫𝐜𝐞 𝐌𝐢𝐱𝐭𝐮𝐫𝐞-𝐨𝐟-𝐄𝐱𝐩𝐞𝐫𝐭𝐬 𝐋𝐋𝐌 OLMoE offers a strong balance between performance and cost-efficiency, all while being fully open-source: 🧠 1B active and 7B total parameters, trained on 5 trillion tokens 💪 Competes with leading models like Gemma and Llama 📊 Fully open with access to the model, data, code, logs, and detailed experiments 🔧 4x fewer training FLOPs and 5x fewer parameters per forward pass compared to OLMo 7B 💸 More cost-efficient in both training and inference 🧩 64 experts per layer for granular optimization 🚀 Innovative dropless token routing outperforms traditional expert choices 🔎 Interesting insights like potential specialization in experts, e.g., Expert 43 for geography and Expert 3 for family-related topics Weights https://lnkd.in/geVjnxw5 Data https://lnkd.in/gB2bv7gR Code https://lnkd.in/gPCsqdCk Logs https://lnkd.in/gmtBSpBf OLMoE-1B-7B-0924--Vmlldzo4OTcyMjU3 #AI #GenAI #MixtureOfExperts #LLM #OpenSourceAI #MachineLearning #DataScience #ArtificialIntelligence #DeepLearning #AIModels

3

Impressive that language classification without LLMs can still get such good results. This is the latest comparison of models after the latest release of Lingua.

19

1 comentario

New trending model, 1 million token context window Llama3. Fine tuned version of llama3 released by Gradient to reach advanced context levels used by google's Gemini. Context is very important, the longer the context the less forgetful a model is. Very important for domains like programming. Longer context also removes most of the need for R.A.G & Fine tuning. Using in context learning. Try it out, use vllm to serve the model on a EC2 instance. Comes after the recent paper explaining how to do this Raw model: https://lnkd.in/g_7Be-iZ Ollama: https://lnkd.in/gKmNMRDJ Papers: https://lnkd.in/gWpE28Uf & https://lnkd.in/gdeEjH_5

2

4 comentarios

Meta's Llama 3.1 results are out on LMSYS... 👉 Llama 3.1 405B is the first open-weight model to beat GPT-4 Turbo. 👉 Llama 3.1 70B is comparable with Anthropic's Claude Opus. 👉 Llama 3.1 8B underperforms Gemma 2 9B on the leaderboard, but has a much larger context window. 👉 All Llama 3.1 models have the same context window size as GPT-4 (128K tokens). This is the first time open-weight models have been a viable option for use-cases that require a longer context window and more complex reasoning capabilities. This makes it a very interesting prospect for organisations that cannot send data outside a closed network for security reasons.

24

2 comentarios

 #Llama 3🦙Meta just released new best in-class open models https://lnkd.in/eWsF9Vwb Those new models are real improvement on the previous Llama 2 versions in terms of performance across various benchmarks: ### Llama 3 8B vs. Llama 2 70B - GPQA (0-shot): Llama 3 8B achieves a score of 34.2, while Llama 2 70B scores 21.0 - HumanEval (0-shot): Llama 3 8B scores 62.2, a substantial increase compared to Llama 2 70B's score of 25.6 - GSM-8K (8-shot, CoT): The Llama 3 8B model outperforms with a score of 79.6, compared to Llama 2 70B's 57.5 - MATH (4-shot, CoT): Llama 3 8B scores 30.0, while Llama 2 70B scores 11.6 ### Llama 3 70B vs. Llama 2 70B - GPQA (0-shot): Llama 3 70B scores 39.5, compared to Llama 2 70B's 21.0 - HumanEval (0-shot): Llama 3 70B significantly outperforms with a score of 81.7, while Llama 2 70B scores 25.6 - GSM-8K (8-shot, CoT): Llama 3 70B achieves a score of 93.0, which is much higher than Llama 2 70B's 57.5 - MATH (4-shot, CoT): Llama 3 70B scores 50.4, more than quadrupling Llama 2 70B's score of 11.6 ### Additional Improvements in Llama 3 - Context Window Size: Llama 3 has increased the context window size from 4k to 8k tokens, which allows for better understanding and generation of longer text passages - Training Data: Llama 3 models have been trained on a dataset of 15 trillion tokens, compared to Llama 2's 2 trillion tokens, which contributes to their improved performance - Tokenizer: A new tokenizer in Llama 3 expands the vocabulary size to 128,256 tokens, up from 32K in Llama 2, allowing for more efficient text encoding and potentially stronger multilingual capabilities - Grouped-Query Attention (GQA): The 8B version of Llama 3 uses GQA, which is an efficient representation that should help with longer contexts - Training Techniques: Llama 3 Instruct has been optimized for dialogue applications and was trained on over 10 million human-annotated data samples with a combination of advanced training techniques.

6

2 comentarios

"Struggling with liquidity challenges? Let’s talk about a solution built for the future. Book your FREE CONSULTATION and see how #LiquidityFLEX works for you. Unlocking Capital Efficiency: How Liquidity FLEX™ Revolutionizes Tokenized Finance: LiquidityFLEX.com"

1

Visualising AI training cost (estimated) ... The AI Index collaborated with research firm Epoch AI to estimate AI model training costs based on cloud compute rental prices, which have been adjusted for inflation since 2017. Although not 100% accurate, these figures iterate the narrative that fine-tuning and RAG are the only cost-effective ways for the majority of companies who want to run and tune their own models based on open-source models. For more info, check out this article https://lnkd.in/eEzDGfyV

1

AI: Our Days (Probably) Aren't Numbered ⏳ As we navigate the paradigm shift with #AI, the technology is poised to benefit humanity and transform our workforce. Why so ? 1️⃣ Innovation's Historical Pattern 📜 19th century: Luddites destroyed textile machinery, fearing job losses 🧵 Today: AI faces similar skepticism, with concerns about mass unemployment 🔮 The First Industrial Revolution initially displaced workers but ultimately created more jobs and increased productivity by 0.3% annually between 1760 and 1800 📈 2️⃣ AI's Economic Impact 💰 PwC estimates AI could contribute up to $15.7 trillion to the global economy by 2030 🌍 McKinsey & Company predicts AI automation could raise productivity growth globally by 0.8 to 1.4 percent annually 📊 While 9% of jobs are at high risk of automation, 26% of jobs will be augmented by AI, according to Gartner 🔄 3️⃣ AI Enhancing Human Capabilities 💪 In #healthcare: AI-powered diagnostics can detect diseases earlier and more accurately 🏥 , resulting in predictive and prescriptive care. In #education: Personalized learning platforms use AI to adapt to individual student needs, potentially improving learning outcomes by 20-30% 🎓 In scientific #research: AI is accelerating drug discovery, screening billions of compound combinations in days instead of decades of studies and trials🔬 4️⃣ AI Creating New #Job Categories 👥 World Economic Forum predicts AI will create 97 million new jobs by 2025 🆕 The question is, will they longer-term and sustainable type. 5️⃣ AI Addressing #Global #Challenges 🌍 Climate change: AI models are improving climate predictions and optimizing renewable energy systems 🌱 Food security: AI-powered precision agriculture could increase global crop yields by 20-30% 🌾 with innovation in #hydroponics #Disaster response: AI systems are enhancing early warning systems and optimizing resource allocation during crises 🚨 6️⃣ The Human-AI Collaboration 🤝 Accenture research shows that companies using AI and human-machine collaboration could boost their revenue by 38%. In creative fields, AI tools are augmenting human creativity 🎨, however realism seems to be the challenge. 7️⃣ Preparing for an AI-Augmented Future 🔮 Soft skills like #emotional #intelligence, #creativity, and #adaptability are becoming increasingly valuable in an AI-driven world 🧘 Lifelong learning and continuous skill development are crucial for staying relevant 🔄 As we stand at this technological crossroads, it's clear that AI is about augmenting human capabilities, solving global challenges, and creating new opportunities. The key lies in our ability to adapt and leverage its strengths to complement our unique human skills. 🌟 As opposed to demonize it. #AI #technology #future #robotics #futureofwork #automation #healthcare #climatechange #education #foodsecurity #innovation #business

3

𝗛𝗼𝘄 𝘁𝗼 𝗰𝗿𝗲𝗮𝘁𝗲 𝘆𝗼𝘂𝗿 𝗼𝘄𝗻 𝘀𝗺𝗮𝗿𝘁 𝗴𝗹𝗮𝘀𝘀𝗲𝘀 𝘄𝗶𝘁𝗵 𝗼𝗻𝗹𝘆 $𝟮𝟬??? Meta launched 🦙 LLaMA3 to show the world what’s possible with open source LLMs. Why this is a game changer? Since LLaMA3 was released one month ago, more than 700 fine tuned versions have been uploaded into Hugging Face, we've seen extending the context window into 1M and co-lead the chatbot arena for english language. This weekend they hosted a 24h LLaMA3 hackathon with Cerebral Valley and more than 500 AI engineers. The results were pretty amazing and these demonstrates the real power and value of these models. A summary of the finalists: 𝗝𝗼𝗲𝗖𝗥𝗠: AI-first CRM that receives signals about customers, performs research about them, and automatically drafts outreach campaigns 🥉 𝗔𝗰𝘁𝗶𝘃𝗮𝘁𝗶𝗼𝗻 𝗔𝗯𝗹𝗮𝘁𝗶𝗼𝗻 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻: Research team that identified how to nullify activation layers in Llama3 responsible for censorship 🥇 𝗢𝗽𝗲𝗻𝗚𝗹𝗮𝘀𝘀 😎 : $20 budget smart glasses that can answer questions about what you’re looking at 𝗙𝗲𝘆𝗰𝗵𝗲𝗿: Fast AI avatars that speaks responses with lip-syncing. Context loaded by recent events from the Brave news API 𝗛𝗼𝘂𝗻𝗱: AI to help law enforcement crack down on human trafficking 𝗠𝗼𝗻𝗴𝗼𝗼𝘀𝗲 𝗠𝗶𝗻𝗲𝗿: Improved Python codegen made by providing llama 3 with up to date documentation 🥈 𝗗𝗲𝗯𝟴: AI agent debate arena where different models compete in Lincoln-Douglas debate. A third agent rates the quality of the debaters and scores them 𝗖𝗶𝘁𝘆 𝗛𝘂𝗯: Llama3 powered chat agent that lets residents ask questions about city laws and regulations. RAG’s over city documents to inform the answers 🔗 More info about OpenGlass: https://lnkd.in/dH_q7c9C 🔗 Github: https://lnkd.in/dQta8Uuv #ai #genai #llama

26

I love AI, but... AI is great and there are many tasks I wish AI could take over entirely. Indeed, I've been using AI more and more to handle routine activities, which frees up my time and lets me focus on more important things, even some daydreaming. 😄 However, there are still areas where AI hasn't been particularly useful for me, even though others have had positive experiences. Here’s my list of what I just don’t involve AI in (yet): 1) Creating presentation slides Yes, Gamma is great, and I have a paid account. It's fantastic for quickly drafting presentations for meetings and client communication. But customising those drafts into more polished forms takes me way more time than I’d like. My go-to remains Google Slides. It's clean, simple, and free. 2) Creating images and videos For a while, I thought AI-generated images could revolutionise my agency’s operations. But we quickly found that stock images in Canva Pro paid plan work way better. Real images still connect better than AI-generated ones, at least for now.  The same goes for videos. AI video generation is fun and cool to experiment with, but it can't solve many practical production and editing challenges we still face today. It can only generate short clips (10 to 15 seconds tops), and a lot of editing is still needed to polish them for publishing. Human-driven videos are here to stay for a while. In fact. this motivated me to start a separate video production service on top of the digital marketing solutions that my agency already offer. 3) Creating branding and marketing strategies AI isn’t helpful here. Whenever I ask for strategies, the responses are generic and overly explanatory. Ironically, this means I spend more time sifting through AI-generated content than I would critically think about the marketing problem myself. But that’s just me. What about you? What are some tasks you could use AI for but don't? 

23

8 comentarios

Check Out This *Easy-To-Understand No-Code* Automated Trading Strategy for NinjaTrader ®: Conditional Position Sizing (Uses Else If, Else, and Nested If Blocks). No NinjaScript ® programming knowledge required. https://lnkd.in/edDUD8rk #NinjaTrader #NinjaScript

1

Breaking: Gartner predicts that 30% of GenAI projects will be abandonned after PoC stage by 2025. Also Breaking: Isn't this the point of Proofs of Concept? If your PoCs aren't cheap enough that you're happy to abandon the ones that don't work for the rockstar ideas - your product team isn't helping you iterate as quickly as it could. Prototyping is the new AI-enabled superpower for product teams. If you want to learn more about leveraging it, come join my free Masterclass on Tuesday, 'The Path to AI Product Manager'. We'll dive into the key skills PMs need in 2024 and beyond - including quickly vetting and prototyping ideas! (link in bio...) #generativeAI #aiforbusiness #aiforeveryone https://lnkd.in/gR4Nr5qQ

32

3 comentarios

Heard of OpenAI o1? It’s amazing for complex problem-solving. But what if you could level up any LLM to that standard? Our open-source drAIve’s Steps do just that. Even better, you’ll actually pay less and have more control over the reasoning process. Give it a go, and let me know how it works! drAIve: https://lnkd.in/eCAtt9rj Steps: https://lnkd.in/evEWc2wa #openAI #llm #gpto1

211

6 comentarios

𝗚𝗿𝗼𝘄𝘁𝗵 𝗼𝗿 𝗚𝘂𝗮𝗿𝗮𝗻𝘁𝗲𝗲𝘀? 𝗪𝗵𝘆 𝗠𝗼𝘀𝘁 𝗣𝗼𝗿𝘁𝗳𝗼𝗹𝗶𝗼𝘀 𝗠𝗶𝘀𝘀 𝘁𝗵𝗲 𝗠𝗮𝗿𝗸? 𝗧𝗵𝗲 𝗣𝗿𝗼𝗯𝗹𝗲𝗺: Most portfolios are built with a one-sided focus—either on growth or on security. This "either/or" mentality can create several challenges: ➔ High risk exposure: When markets drop, portfolios focused solely on growth can take significant hits. ➔ Lagging performance: Portfolios built for “safety” may not even keep pace with inflation over time. ➔ Missed opportunities: The inability to balance risk and reward often results in underperformance. 𝗧𝗵𝗲 𝗘𝗳𝗳𝗲𝗰𝘁 This imbalance often leads to: ➔ Constant rebalancing: Investors find themselves endlessly tweaking their portfolios, chasing returns, or trying to hedge against risk. ➔ Emotional stress: The fear of losses and missed gains creates anxiety and uncertainty. ➔ Long-term underachievement: Without a balanced approach, it’s difficult to achieve true financial stability. 𝗧𝗵𝗲 𝗦𝗼𝗹𝘂𝘁𝗶𝗼𝗻 💡 ⤷ 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗲 𝗴𝗿𝗼𝘄𝘁𝗵 𝗮𝗻𝗱 𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆: Create a portfolio that protects against market volatility while capturing compounding gains. ⤷ 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝗳𝗼𝗿 𝗰𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝘁 𝗿𝗲𝘁𝘂𝗿𝗻𝘀: Find the right mix that delivers growth without exposing you to unnecessary risk. 𝗖𝘂𝗿𝗶𝗼𝘂𝘀 𝗵𝗼𝘄 𝘁𝗵𝗶𝘀 𝗯𝗮𝗹𝗮𝗻𝗰𝗲 𝗶𝘀 𝗮𝗰𝗵𝗶𝗲𝘃𝗲𝗱? ➔ Follow our page and connect to explore strategies that effectively combine growth and security. #FinancialPlanning #WealthManagement #InvestmentStrategy #RetirementPlanning #FinancialAdvice #MarketRisk #GrowthVsSecurity #PortfolioOptimization #TaxEfficientInvesting

1

One. The Corporate big data initiative wants to do some unspecific, unclear, AI projects, a few million euros of investments at least. Two. The Factory in Spain is scrapping 20% of the copper used in production, wants to go down to 10%, but needs more visibility on their production data to figure out what actions to take. Potential investment is half a million euros. Because both initiatives go into 1 big data lakehouse, and 1 big data team, the Corporate one wins the attention, because... it's AI, right? Meanwhile, reducing copper scrap by 50% has a potential saving upside of 50,000 euros per day, annualized 15m euros per year. The moral of the (partially made up) story? Centralizing all data investments in 1 big data lakehouse is going to give you negative ROI in missed opportunities over time. That, or people will just revert to good old Excel and never use your big lakehouse, no matter how much generic AI bots you put in it.

13

2 comentarios