Youssef Mrini

What do you think of this new streaming data pipeline idea (without Dataflow in the middle) ? * Extract & Load: Pub/Sub -> BigQuery subscription -> BigQuery * Transform (Enrich) & Reverse: BigQuery Continuous SQL + Gemini -> Pub/Sub Watch BQ Continuous SQL demo: https://lnkd.in/dJpEtnCB Congrats on the launch Nick Orlove , Pavan Edara and team 🎉 #GoogleCloud #Gemini #BigQuery

20

2 commentaires

#DataEngineering 𝗣𝗿𝗼 𝗧𝗶𝗽𝘀 𝗳𝗿𝗼𝗺 𝗙𝗮𝗯𝗿𝗶𝗰𝗲 𝗗𝗲𝘀𝗲𝘆𝗻, 𝗗𝗮𝘁𝗮𝗯𝗿𝗶𝗰𝗸𝘀 𝗖𝗵𝗮𝗺𝗽𝗶𝗼𝗻 Join our colleague, Fabrice Deseyn, for a deep dive into the world of Databricks. In a recent interview with Youssef Mrini on the #NextGenLakehouse YouTube channel, Fabrice shares his insights and journey of becoming a Databricks champion. Beyond his champion status, Fabrice provides expert guidance on navigating Unity Catalog migrations. Discover practical tips and strategies to streamline the process and ensure a successful transition. Watch the video now 👉 https://lnkd.in/e2zFw53p #Databricks

11

1 commentaire

What's AIBI genie ? How does it work ? Who should use it ? Chao Cai Sr Director of Engineering at Databricks answers all this questions and share some best practices as well. If you have not tried it yet after this session you will definitely do it. If you're a Data Analyst, you shouldn't consider genie as threat it's more a companion in the data democratization journey. #Databricks

20

Big thanks to Stéphane Maarek for his excellent Udemy course on Apache NiFi! Gained valuable insights and excited to apply this knowledge in real-world projects! #AlwaysLearning #DataIntegration

77

8 commentaires

In today's blog, our Data Practice Lead Thierry B. discusses how to harness Databricks' serverless SQL and workflow (scheduling) functionalities to build and refresh a calendar dates table tailored for your Business Intelligence (BI) applications. By the end of this article, you will have a clear understanding of how to implement this solution. READ HERE: https://lnkd.in/gMiZfm3v #Databricks #Adaptiv #serverless #SQL #BI #DataandAnalytics

15

1 commentaire

Spatial SQL is coming to Databricks ! Finally it is possible to work natively with geospatial data in Databricks. Learn more & get started    https://hubs.ly/Q02CBmk80

50

4 commentaires

How do data teams keep their CFOs happy? By keeping costs low & delivering value with their Data + AI. We're making some big advancements to help customers manage their costs. Check out this 3-part series with our "Money Team" Greg Kroleski and MVP Josue A. Bogran to understand what this looks like. Part 1: 👉 Why we're investing in cost management 👉 How to get started using tags 👉 Cost Allocation/Tag Implementation Challenges Part 2: 👉 Why aren't tags used more? 👉 Changes/Improvements Databricks has made to cost reporting in 2024 👉 Why should users trust Databricks' efforts to improve performance? 👉 How can Databricks' Cost Dashboards help? Part 3: 👉 Costly mistake prevention mechanism inside of Databricks' Serverless 👉 AI/BI Genie for cost reporting inside of Databricks 👉 Balancing between user managed vs Databricks managed cost controls #FinOps

5

🔐 Understanding Encryption Key Hierarchy in Snowflake 🔑 Ever wondered how Key Hierarchy works to secure your data in Snowflake? 🌐 Snowflake uses a powerful multi-layer encryption architecture to ensure your data remains secure at every level. To learn how the Key Hierarchy operates and gain a clear understanding, check out this video 📽️: 👉 https://lnkd.in/gnsCM5Uc This video simplifies the concept and explains the hierarchy in detail. Don't miss it! 🚀 For more such content Follow Enqurious (formerly Mentorskool) Enqurious Academy Subscribe to https://lnkd.in/g9M_ZZDV #Snowflake #DataSecurity #Encryption #CloudData #KeyHierarchy

5

Want to use Oracle 23ai vector search to run semantic searches on pictures ? Have a look at this short demo !

32

8 commentaires

Support for liquid clustering is now generally available using Databricks Runtime +15.2 Getting started with Delta Lake Liquid clustering https://lnkd.in/eaCZyhbF #DeltaLake #Databricks

89

2 commentaires

Want to get started with 23ai new features ? Check our playlist, composed of short demos (less than 10mn) showing the essentials of 23ai new features, like schema annotations, priority transactions, True Cache, Data Use Case Domains, SQL property graphs .... and stay tuned, as we will add more videos regularly! https://lnkd.in/dfaWA3xf And check my demo on True Cache here: https://lnkd.in/d42i7ZHy #oracledatabase #23ai

23

1 commentaire

Data Engineers ! This one's for you: 🧠 The Economics of Generative AI Stephanie Kirmer 🧠 Connecting ML Models in Sagemaker to Snowflake and dbt Hugo Lu 🧠 Introduction to Interpretable Clustering NAKUL UPADHYAY 🧠 Snowflake automatic clustering cost estimation Tim Sander 🧠 Death to Tech Debt? Ari Joury, PhD Check it here; https://lnkd.in/ebdFKNZ5 #dataengineering #dataroundup #startup

5

📢 New to Version Control on Databricks? Start strong with our beginner’s guide! In this one, our Head of AI, Gavi, is joined by Stuart from Databricks, diving into everything you need to get started with Version Control - setting up with Azure DevOps, mastering feature branches, navigating pull requests, and more. Packed with practical tips, this video is perfect for anyone looking to implement version control effectively! Check it out - we’d love to hear your thoughts! 👇 https://hubs.la/Q02W6d4m0 #Databricks #VersionControl #AzureDevOps #DataScience #AI

39

2 commentaires

Streaming Tables and Materialized Views now supports Row filters and Column Masks. If you want to get started quickly with RW and CM : https://lnkd.in/eaNNJpci #Databricks

60

3 commentaires

🚀 𝗘𝘅𝗰𝗶𝘁𝗲𝗱 𝗳𝗼𝗿 𝘁𝗵𝗲 Databricks 𝗪𝗼𝗿𝗹𝗱 𝗧𝗼𝘂𝗿 – 𝗡𝗲𝘅𝘁 𝗪𝗲𝗲𝗸! 🌍     Catch us at 𝗕𝗼𝗼𝘁𝗵 𝗚𝟭𝟭! 🗓️ We can’t wait to engage in dynamic 𝗰𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻𝘀 𝗼𝗻 𝗵𝗼𝘄 𝘁𝗼 𝗲𝗹𝗲𝘃𝗮𝘁𝗲 𝘆𝗼𝘂𝗿 𝗱𝗮𝘁𝗮 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝘆 𝗮𝗻𝗱 𝘂𝗻𝗹𝗲𝗮𝘀𝗵 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗽𝗼𝘁𝗲𝗻𝘁𝗶𝗮𝗹 𝗼𝗳 𝘆𝗼𝘂𝗿 𝗱𝗮𝘁𝗮. Here are some inspiring topics that we would like to explore and share insights with you during our exchange at the booth:     🔍 Unlocking Enterprise Data for GenAI applications  📊 Assessing the maturity of your data landscape  💡 Developing GenAI use cases to maintain a competitive advantage in your industry  🛠️ Data governance and management best practices  📈 Understanding data traceability and lineage  📜 Crafting effective data management policies  ⚙️ Building infrastructure that maximizes business value    We can’t wait to 𝗰𝗼𝗻𝗻𝗲𝗰𝘁 𝗮𝗻𝗱 𝗵𝗮𝘃𝗲 𝘀𝗼𝗺𝗲 𝗶𝗻𝘀𝗽𝗶𝗿𝗶𝗻𝗴 𝗰𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻𝘀 at our booth G11! ✨     #Databricks #DatabricksWorldTour #Data #DataFoundation  

18

🚀 KAWA Analytics is the easiest way to build enterprise-grade data apps on Snowflake From risk management to dynamic pricing, ESG to market abuse surveillance, KAWA AI Co-builder makes it effortless. 🎥 Watch our product manager Narjisse Bouri demonstrate how to build a market abuse monitoring system in no time.

5

1 commentaire

Attention Snowflake users ! Wouldn't it be great if every time anything anomalous happened in your infrastructure it got flagged? Well in Orchestra that's pretty easy to set-up. Check it out below! https://lnkd.in/eP8bj6MU #dataengineering #anomalydetection #snowflake

7

If you are using Databricks SQL you don't wanna miss this session. You will be able to get to know how to use Collations, SQL Scripting, Execute Immediate, Variables.... Big thanks to Serge Rielau #Databricks

96

Got data problems? Let Alice help you out with all your #ETL needs. Take a look at this quick explainer of what exactly Extract means in ETL. #datamanagment https://lnkd.in/e9_b9GBg

8