OSN-Blazingly Fast GenAI App Development With Java and Spring AI

OSN-Blazingly Fast GenAI App Development With Java and Spring AI

• 1. Blazingly Fast GenAI App Development With Java and Spring AI Open Source North 2024 Juarez Barbosa Junior @juarezjunior Sr Principal Java Developer Evangelist ORACLE Copyright © 2024, Oracle and/or its affiliates
• 2. Juarez Barbosa Junior Senior Principal Java Developer Evangelist @ Oracle • Coming from Dublin, Ireland • 28 years of experience in SW Engineering & DevRel • Oracle, Microsoft, IBM, Nokia, Unisys, Accenture, startups • Microsoft Azure Developer Relations Lead • IBM Watson Tech Evangelist & Cloud Rockstar • IBM Mobile Tech Evangelist & Global Thought Leader • Nokia Developers Global Champion • Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI, Blockchain, Rust • Speaker at conferences • Oracle CloudWorld, Oracle Code, Microsoft Ignite & TechX, jPrime, JCON, DevConf.cz, GeeCon, DevOpsDays, DeveloperWeek, DevOps Institute, CloudLand, DWX, The Developer’s Conference (TDC), Sec4Dev, JSNation, NodeConf, Conf42, Shift Conf, Global Azure, Open-Source Lisbon, CodeFrenzy, Mêlée Numérique, React Summit, Test.js Summit, Porto TechHub Conf, Pyjamas, MiTechCon, Data Science Summit, OSN, JUGs, meetups, hackathons. @juarezjunior @juarezjunior
• 3. Agenda • AI and GenAI – Synthetic Content • Artificial General Intelligence (AGI) • OpenAI • Spring AI • Reference Architecture for GenAI Apps • Oracle Database 23ai • Spring AI – Demos • Technical References • Java App Dev with the Oracle Database • Audience: SW Engineers + Business audiences (not necessarily ML Engineers) Copyright © 2024, Oracle and/or its affiliates
• 4. AI and GenAI – Synthetic Content Minneapolis in Minnesota, USA pictured in a futuristic cityscape Copyright © 2024, Oracle and/or its affiliates
• 5. Artificial Intelligence and GenAI • Core concepts in a nutshell • GenAI - Artificial intelligence algorithms and transformer models with the capability of generating content. • Content - Text, images, videos, code, and others. • Generation – a result of an input prompt (prompt engineering). • Ethical issues – misuse, deep fakes, bias and discrimination, automation and job replacements, cybersecurity and cybercriminals. Copyright © 2024, Oracle and/or its affiliates
• 6. Artificial Intelligence and GenAI • Core concepts in a nutshell • Apps – Applied currently in the fields of content development, entertainment, drug discovery, software development, healthcare, material generation sciences, and others. • AI is now a commodity! • Artificial General Intelligence (AGI) Research – ultimate goal Copyright © 2024, Oracle and/or its affiliates
• 7. OpenAI • Generative AI with OpenAI • Natural Language Processing (NLP) • Powerful, pre-trained and customizable Machine Learning Models • OpenAI API (REST interface) • Scalable infrastructure (GPUs) • https://openai.com/sora Copyright © 2024, Oracle and/or its affiliates
• 8. Spring AI (1.0.0 M1) • Spring AI (AI Concepts) • A Java app framework for building AI-related applications. It provides a robust abstraction layer that encapsulates many of the complexities required to interact with various GenAI providers, such as OpenAI, Cohere, Google Vertex AI/Gemini, and others. • API support across AI providers for chat, text-to-image, and embedding models. • Its goal is to apply to the AI domain Spring ecosystem design principles such as portability and modular design and promote using POJOs as the building blocks of an application to the AI domain. • https://spring.io/projects/spring-ai Copyright © 2024, Oracle and/or its affiliates
• 9. Spring AI • RAG (Retrieval-Augment Generation) with Spring AI • LLMs' baseline knowledge might have gaps based on their training data. If you ask an LLM to write about a recent trend or event, the LLM won't know what you're talking about, so you may get meaningless responses. • There's a way to provide LLMs with additional information (including business- specific data)- that is what RAG is for! • Oracle - What Is Retrieval-Augmented Generation (RAG)? • Oracle – What Is Generative AI? How Does It Work? Copyright © 2024, Oracle and/or its affiliates
• 10. Spring AI • RAG (Retrieval-Augment Generation) with Spring AI • In a nutshell, RAG is a technique that: • Reads unstructured data; • Converts the data into tokens; • It uses embeddings to transform the tokens into floating-point arrays or vectors so that LLM can process and interpret them; • It stores the vectors in Vector Databases. Copyright © 2024, Oracle and/or its affiliates
• 11. Spring AI • The Spring AI API covers a wide range of functionalities • Chat Completion API • Embeddings API • Image Generation API • Speech / Transcription API • Vector Databases • Function Calling API Copyright © 2024, Oracle and/or its affiliates
• 12. Spring AI • Spring AI – Blazingly Fast Performance • GraalVM • Native Image - compile Java code ahead of time (AOT) to a binary – a native executable • Developing an Oracle JDBC app with GraalVM Native Image • mvn -Pnative native:compile • ./target/spring-ai-graalvm-oai.exe Copyright © 2024, Oracle and/or its affiliates
• 13. Spring AI • Spring AI – Blazingly Fast Performance • Spring with Virtual Threads • Add the following entry in the application.properties file: spring.threads.virtual.enabled=true • Class VirtualThreadTaskExecutor • Embracing Virtual Threads • All together now: Spring Boot 3.2, GraalVM native images, Java 21, and virtual threads with Project Loom Copyright © 2024, Oracle and/or its affiliates
• 14. Reference Architecture for GenAI Apps Copyright © 2024, Oracle and/or its affiliates Source: Architectural Approach for Building Generative AI Applications
• 15. New | Oracle Database 23ai Next-Generation Converged Database Services Over 300 major new features plus thousands of enhancements Available on OCI and Azure Major focus on • AI for Data features • Developer/Analyst features • Mission-Critical features Addresses data management pain points that have frustrated customers forever Copyright © 2024, Oracle and/or its affiliates
• 16. Oracle Database 23ai - AI for Data Algorithmic AI AI Vector Search Augmented Generative AI (LLMs) Distributed AI AI Storage AI Developer Tools Copyright © 2024, Oracle and/or its affiliates
• 17. Oracle Database 23ai - Dev for Data JSON Relational Duality Views Property Graph Views JavaScript Stored Procedures Data Intent Language Lock-free Consistent Updates, Long-running Transactions True Cache Copyright © 2024, Oracle and/or its affiliates
• 18. Oracle Database 23ai - Mission-Critical Data RAFT Replication for Globally Distributed Database In-Database Firewall Real-Time SQL Plan Management RAC, Exadata, Data Guard Simplicity and Scalability Analytic SQL Simplicity and Scalability Priority Transactions Copyright © 2024, Oracle and/or its affiliates
• 19. Free | Oracle Database 23ai For Developers Oracle Always Free ADB Available on OCI ADB Free Container Image Available for download Oracle Database Free Available as RPM, Docker Image, VBox VM Copyright © 2024, Oracle and/or its affiliates | Confidential - Internal Very Low Cost Supported Developer Edition of ADB, BaseDB, ExaDB-D, and ExaDB-C@C
• 20. Demos 20 Copyright © 2024, Oracle and/or its affiliates Spring AI – several scenarios Oracle Database 23ai Oracle AI Vector Search OpenAI Ollama
• 21. Image Generation - Spring AI with Open AI • Demo - Direct LLM calls – Image Generation Copyright © 2024, Oracle and/or its affiliates
• 22. Chatbot - Spring AI with Open AI • Demo - Direct LLM calls - Chatbot Copyright © 2024, Oracle and/or its affiliates
• 23. RAG – Retrieval Augment Generation Spring AI with Open AI and Oracle Database 23ai • Demo – RAG • Spring AI + SimpleVectorStore + Open AI • Spring AI + OracleVectorStore + OpenAI + Oracle Database 23ai Copyright © 2024, Oracle and/or its affiliates
• 24. Multimodality - Spring AI with Open AI • Demo - Multimodality - Spring AI with Open AI Copyright © 2024, Oracle and/or its affiliates
• 25. Function Calling - Spring AI with Open AI • Demo -Function Calling - Spring AI with Open AI Copyright © 2024, Oracle and/or its affiliates
• 26. Copyright © 2024, Oracle and/or its affiliates Technical References • Spring AI - https://spring.io/projects/spring-ai • OpenAI - https://openai.com/ • Oracle AI Vector Search - https://rb.gy/sp41uk • Oracle AI Vector Search for Java Developers with the Oracle Database 23ai - https://rb.gy/7ivwxr • Java / JDBC – https://www.oracle.com/jdbc • Introduction to Oracle JDBC Driver Support for Virtual Threads - https://bit.ly/3UlNJWP • Developing an Oracle JDBC app with GraalVM Native Image - https://rb.gy/iy3sgh • Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC Driver - https://rb.gy/42dnz5 • Getting Started with the Java library for Reactive Streams Ingestion (RSI) - https://bit.ly/3rEiRnC • Introduction to JDBC Reactive Extensions with the Oracle Database — Developer Release – https://rb.gy/qxlrbx • Pipelined Database Operations - https://rb.gy/iy3sgh • Spring AI - https://github.com/kousen / https://github.com/ThomasVitale/llm-apps- java-spring-ai
• 27. Juarez Junior @juarezjunior
• 29. Java Developers 29 Copyright © 2024, Oracle and/or its affiliates Oracle for DB Developers
• 30. Java App Dev with Oracle Database Copyright © 2024, Oracle and/or its affiliates
• 31. Overview of Oracle DB Access with Java Copyright © 2024, Oracle and/or its affiliates
• 32. Oracle JDBC - Support for the Latest Java Versions • Support to the latest Java versions (11, 17, 21) and GraalVM • Project Loom - Virtual Threads support • Java Virtual Threads and Pipelined Database Operations • JDBC Standards - 4.2 and 4.3 • Reactive Streams - Java Flow API support • R2DBC, Reactive Streams Ingestion (RSI) • Pipelined Database Operations Data access is critical in mission-critical applications! Copyright © 2024, Oracle and/or its affiliates
• 33. Oracle LiveLabs Showcasing how Oracle’s solutions can solve your business problems 500+ free workshops, available or in development 3.5 million people have already visited LiveLabs developer.oracle.com/livelabs learn something new …at your pace! 600+ events run using LiveLabs workshops
• 34. 3 membership tiers Connect: @oracleace facebook.com/OracleACEs aceprogram_ww@oracle.com 500+ technical experts & community leaders helping peers globally The Oracle ACE Program recognizes & rewards individuals for their technical & community contributions to the Oracle community Nominate yourself or a candidate: ace.oracle.com/nominate Learn more - ace.oracle.com blogs.oracle.com/ace
• 35. Create your FREE Cloud Account • Go to https://signup.cloud.oracle.com/ Copyright © 2024, Oracle and/or its affiliates