Pulkit Mehta

NLP (LLM) Research Roadmap I’m happy to be an NLP research mentor for - PhD candidates without a good faculty supervisor - Industry professionals who want to gain research experience by publishing papers. Guidance will be provided via 1-to-1 paid Google Meet sessions. Interested people can DM me with all the relevant details. "Hi" or "incomplete messages" will be ignored due to time constraints. NLP research involves various phases like - Research Foundation (learning pre-requisites) - Research Papers Reading - Research Problem Identification - Research Gaps Identification - Research Idea Formulation - Research Idea Implementation - Research Paper Writing - Research Paper Submission and Review A good faculty supervisor provides proper guidance in every phase of NLP research. Unfortunately, some faculty supervisors will never do this which makes the journey of PhD candidates difficult. Due to a lack of proper guidance from the faculty supervisor, some of the PhD candidates despite being highly talented finish their PhD with average or below-average papers even after spending more than five years. #llms #nlproc #india #phdresearch #phd

257

5 Comments

In my early learning phase, I often confused Layer Normalization and Batch Normalization—I knew they improved numerical stability, but the key differences eluded me. Watching Umar Jamil's videos clarified everything: Batch Normalization works across the batch dimension, ideal for CNNs, while Layer Normalization focuses on features, perfect for RNNs and Transformers. Grateful for such insightful resources! 🙌 #deeplearning #machineLearning #GENAI #Transformer

10

In case you've missed my previous posts on EDA Here's the full blog on Exploratory Data Analysis (EDA): Read Here: https://lnkd.in/dYRybKzz #datascience

2

1 Comment

Ran a quick test on the new #OpenAI #o1mini to solve IIT-JEE Advanced 2024 maths questions. It accurately solved 13 out of 17 questions with an impressive accuracy of ~76%. Quick thoughts: 1. The hidden chain of thought is quite detailed and extensive. Breaks the problem, solves and verifies and iterates multiple times. This might increase the output token cost by a big margin. 2. Seems there are tools integrated for calculations. Much needed as #gpt4o does a bad job at it. 3. Latency is quite high. Some questions took more than 50 seconds to get answered. This needs to come down for production user-facing use cases. 4. Hallucination still exists. Check the 1st question in the link, it had the correct step-by-step reasoning but hallucinated at generating the final answer. 5. Overdoing self-thought. Saw a couple of instances where it did generate correct answers in the process but eventually made wrong answers. Would give a clearer picture once the rate limits are increased and benchmarking is done on a bigger sample. Let me know any interesting findings and thoughts 😀 https://lnkd.in/giSCezyG #GenAI #OpenAI #PhysicsWallah

139

5 Comments

 Dear LinkedIn Family, We are looking for Senior Data Scientist for Pune Location. #DataScientist LLM Architecture: Good understanding of the architecture underlying large language models, such as Transformer-based models and their variants. Design and implement deep learning model architectures using PyTorch. #LLM #PyTorch #pytorch Language Model Training and Fine-Tuning: Experience in training large-scale language models from scratch, as well as fine-tuning pre-trained models on domain data. Data Preprocessing for NLP: Skilled in preprocessing textual data, including tokenization, stemming, lemmatization, and handling of different text encoding. Transfer Learning and Adaptation: Proficiency in applying transfer learning techniques to adapt existing LLMs to new languages, domains, or specific business needs. Data Annotation and Evaluation: Skills in designing and implementing data annotation strategies for training LLMs and evaluating their performance using appropriate metrics. Scalability and Deployment: Experience in scaling LLMs for production environments, ensuring efficiency and robustness in deployment. Model Training, Optimization, and Evaluation: Evaluate the performance of PyTorch models using appropriate metrics and techniques like cross-validation, holdout sets, or online evaluation. This encompasses the complete cycle of training, fine-tuning, and validating language models. You will be designing and adapting LLMs for use in virtual assistants, Information retrieval and extraction etc. Experimentation with Emerging Technologies and Methods: Actively exploring new technologies and methodologies in language model development, including experimental frameworks and software tools. LLM Alignment: Understanding of algorithms like DPO, PPO, KPO, RLHF and using it for guardrails. AI Data Retrieval: Data retrieval from unstructured data, extract key value pairs using techniques like donut, layoutLM, table transformers. Analyze data and build EDAs to identify data patterns Hands-on and strong understanding of concepts in Deep Learning and NLP Proficient in TensorFlow and similar libraries.   Required Qualifications 5+ years of hands-on experience in developing and deploying Large Language Models, and Machine learning and working with Pytorch. A thorough understanding of machine learning, particularly deep learning techniques, including knowledge of neural network architectures, training methods, and optimization algorithms. Proficiency in AI technology, Python, including experience with NLP libraries (e.g., Hugging Face Transformers, NLTK, spaCy), text classification. Experience with frameworks: PyTorch, or Tensorflow. Experience with cloud services (AWS, Azure) and ML deployment tool Docker Familiarity with model fine-tuning and optimization techniques for LLMs. Proven track record of innovative solutions in the field of LLMs. Strong communication skills, with the ability to explain complex AI concepts to non-expert audiences.

4

1 Comment

I came across an excellent GitHub repository packed with hands-on Jupyter notebooks for anyone eager to explore LLMs. It covers: 🔹 Introduction to LLMs – Understanding the basics. 🔹 Tokens & Embeddings – The foundation of language modeling. 🔹 Model Internals – Peeking under the hood of LLMs. 🔹 Text Applications – Classification, clustering, and topic modeling. 🔹 Prompt Engineering – Crafting prompts for better performance. 🔹 Advanced Text Generation – Tools and techniques. 🔹 Semantic Search – Intelligent information retrieval. 🔹 Multimodal Models – Combining text and image understanding. 🔹 Customization – Building embeddings, fine-tuning BERT, and generative models. Big thanks to the repo creator for sharing such a valuable resource. Here is the repo: https://lnkd.in/gUKg7Jp7

77

2 Comments

🚀 Bengaluru-based Conversational AI Startup Gnani AI Raises $3M 💰 Gnani AI, India's first voice-first small language model (SLM) for vernacular languages, has secured $3M in funding from tech holding company Info Edge. The investment will fuel the startup's expansion plans. Key Highlights: • Gnani AI is building a series of voice-based SLMs, trained on millions of hours of proprietary audio datasets and billions of conversations in 12 Indian languages • The startup has over 100 customers in banking, financial services, insurance, telecom, automotive, and healthcare industries • Gnani AI's models run on the edge, reducing costs and enabling quick fine-tuning based on enterprise needs • The startup specializes in creating customized models for individual companies, adding real value to their data CEO Ganesh Gopalan emphasizes the importance of multimodal models, focusing on voice and text fusion. By taking their models to enterprises and helping them build on top of it, Gnani AI is revolutionizing the conversational AI landscape in India. #ConversationalAI #VoiceFirst #IndianLanguages #Funding #Expansion

12

1 Comment

How are Graph Convolutional Neural Networks (GCNNs) a generalisation of Convolutional Neural Networks (CNNs)? My next blog in the Graph series on "Deep Learning with Graphs" is on the way (well, it's already written on paper) and it's going to be a blockbuster. The beauty of Graph Machine Learning is that the more you read about it, the more you feel that your neurons (real ones!) are learning to think better! And you're already in love with it! Coming to the point: One might recall the convolution operations in a CNN. You have a grid of pixels representing an image. You have a 3 x 3 filter. You slide the filter over the image and carry out aggregation to obtain a new pixel value. You proceed in the same fashion over the whole grid, sliding towards the right , bottom, carrying out the convolution operation and finally getting a new transformed, reduced image through the series of aggregations. You repeat this through the different convolutional layers which together form a Convolutional Neural Network Now think of a Graph. It is not simple grid like an image. The topology of a Graph is arbitrary. There is no notion of left, bottom, right in a Graph like in an image which is represented through a grid of pixels. You cannot slide over a Graph as simplistically you did and performed the convolutional operations in an image. How do you approach the problem? How do you carry out convolutional operations in a Graph? Now take a node in the Graph. Take all the nodes which are the immediate neighbours of this node. The convolutional operation here is: You aggregate the information from all the neighbours and pass it to the node. The information is nothing but the emedfings of the neighbour nodes. You add together the emedding of the node under consideration. You transform the aggregated embedding by multiplying with a weight Matrix and transform the node's own embedding by multiplying with a different weight matrix. This is the typical linear transformation operation in a NN. You apply non linearity - the activation function like ReLu. Now think about a CNN taking the same grid and considering a central pixel. Consider the rest of the pixels around the central pixel. You are essentially doing the same thing: taking information from the neighbour pixels along with the central pixel, aggregating, as I described above for the node in the Graph. Hence a Graph Convolutional Network is a generalisation of CNN The story goes, you do the same for every node on the Graph. Each node in the Graph represented through a different neural network architecture! This is amazing 🔥🔥🔥💓💓💓 #graphs #machinelearning #deeplearning My Technical Blogs Library: 1) https://lnkd.in/ejyWacXn 2) https://lnkd.in/bfrkRjq Follow Ajay Taneja for insights on: Data Science, Data Engineering, Finite Element Analysis, Non Linear Finite Element Analysis, Fatigue, Fracture Mechanics

22

Perhaps the most critical thing in a Retrieval Augmented Generation (RAG) system is the retrieval or searching strategy. Poor retrieval will lead to irrelevant documents going to the LLM leading to irrelevant generated answers or worse hallucinations. Is cosine similarity everything? Do check out below illustration which talks about four of the most common strategies for playing around with retrieval strategies in RAG systems: 1. Similarity-based retrieval: This is based on embedding similarity, you can play around with different metrics like cosine, maximal margin relevance etc. 2. Similarity with Threshold retrieval: Useful where we cap the similarity threshold and can help in removing irrelevant document but can drop recall 3. Multi-query retrieval: Make the LLM generate variations of your query and get documents based on these multi-versions of your initial query, often useful to get more recall or coverage of documents 4. Reranker retrieval: Use a model specialised in ranking documents based on search relevance Often you can combine some of these techniques and even use ensemble retrievers. Do explore them and go beyond just cosine similarity!

17

Top 5 Pre-trained Word Embeddings for NLP Teaching machines to understand text is no easy feat. While humans excel at interpreting languages, computers often struggle. This is where word embeddings come into play, transforming text into numerical representations that machines can process. What are word embeddings? Word embeddings are dense vectors representing words. They capture meanings, relationships, and contexts in which words are used, enabling efficient text processing by machines. Why use pre-trained word embeddings? Pre-trained word embeddings leverage large datasets to enhance Natural Language Processing (NLP) models. They save time and computational resources while capturing both the connotative and syntactic meanings of words. This is especially beneficial for tasks with limited training data or extensive vocabulary. Top 5 Pre-trained Word Embeddings: 1: Word2Vec Created by Tomas Mikolov at Google, Word2Vec represents words as vectors based on their contexts. Trained on the Google News dataset, it uses Continuous Bag-of-Words (CBOW) and Skip-Gram models to learn word relationships. 2: GloVe Developed by Stanford, GloVe (Global Vectors) uses global co-occurrence data to build word vectors. It effectively captures semantic similarities between words by analyzing their co-occurrence probabilities. 3: fastText A product of Facebook AI Research, fastText enhances word embeddings by breaking words into character n-grams. This approach improves generalization and requires less training data, supporting multiple languages efficiently. 4:ELMo Created by Allen NLP, ELMo (Embeddings from Language Models) generates context-sensitive embeddings from bidirectional LSTM states. It excels in handling polysemous words by understanding their usage within phrases. 5: BERT Developed by Google, BERT (Bidirectional Encoder Representations from Transformers) uses transformers for bidirectional language modeling. It pre-trains on vast text corpora, then fine-tunes for specific NLP tasks, significantly improving performance. Understanding these pre-trained word embeddings can give you a competitive edge in NLP. They capture the rich semantic and syntactic nuances of language, enhancing the performance of your models. Feel free to connect with me for more insights on NLP, Data Science, Machine Learning, and Deep Learning! #NaturalLanguageProcessing #DataScience #MachineLearning #DeepLearning

4

Exploring Hybrid Retrieval-Augmented Generation (RAG) for Enhanced Information Retrieval In my recent research in the  realm of information retrieval, I came across Hybrid RAG which balances between semantic (vector similarity) and syntactic (keyword) search approaches for achieving accurate and comprehensive results. A cutting-edge solution that combines the strengths of both methods to revolutionize how we interact with data. Key Features: - Semantic Search (Vector Similarity): Ideal for handling queries with typos or nuanced intents, leveraging powerful embeddings like HuggingFace to capture semantic relationships. - Syntactic Search (Keyword Matching): Perfect for precise matches on keywords, abbreviations, and names, employing techniques such as BM25Encoder for effective keyword search. - Hybrid Approach: By blending vector and keyword search scores using advanced fusion techniques like Reciprocal Rank Fusion (RRF), Hybrid RAG ensures robust and context-aware retrieval. Why Hybrid RAG Matters: Semantic search (using dense embeddings) excels in capturing semantic similarity and context, making it effective for understanding the meaning behind queries and documents. On the other hand, syntactic search (using techniques like BM25) is better at capturing exact matches and structural similarities. Combining both ensures a more comprehensive coverage of relevant information. Implementing Hybrid RAG: -- Set up with Pinecone for indexing and retrieval. -- Utilize HuggingFaceEmbeddings for semantic search. -- Employ BM25Encoder for efficient keyword matching. -- Integrate PineconeHybridSearchRetriever for seamless fusion and retrieval. Unlock the power of Hybrid RAG to enhance your information retrieval capabilities. Explore more about how this innovative approach can elevate your data analytics and decision support systems. #HybridRAG #InformationRetrieval #SemanticSearch #SyntacticSearch #AI #MachineLearning #DataScience #TechnologyInnovation #PineconeAI #GenAI #MLOps 

7

The Power of Retrieval-Augmented Generation (RAG) in Documentation and Analytics More reading: BentoML Blog, https://lnkd.in/d6JxSjJy Retrieval-augmented generation (RAG) is a groundbreaking AI technique that combines retrieval and generation models to enhance the accuracy and relevance of generated text. RAG addresses the shortcomings of traditional generative models by incorporating external knowledge, ensuring factual consistency and reducing hallucinations. Benefits of Utilizing RAG for Documentation and Analytics: * Increased Accuracy: RAG enhances the accuracy of document search, summarization, information extraction, and data analysis. *Improved Efficiency: RAG can automate many documentation and analysis tasks, freeing up user time and resources. * Enhanced Understanding: RAG facilitates better user comprehension of information within documents and data. Examples of RAG Implementation in Documentation and Analytics: * A company can utilize RAG to build an internal search engine that allows employees to locate relevant documents quickly and efficiently. * A marketing team can leverage RAG to analyze customer reviews and identify areas for product and service improvement. * A research team can employ RAG to identify trends within research data, informing the development of new products and services. RAG is a promising technology with the potential to significantly improve how businesses manage documentation and analyze data. #RAG #ArtificialIntelligence #Documentation #Analytics #Productivity #Understanding

1

🔊Another Mock NLP Interview session 😎 🔗 link: Mock NLP Interview: Top Questions & Answers | Prepare for Your NLP Job Interview https://lnkd.in/d2S9u9zY 📓📕This episode of Mock Interview examines ✅ typical questions asked in technical Narural Language Processing (NLP) interviews, ✅offers a comprehensive insight into the key concepts developers should revise before facing such interviews. 👩‍🏭If you're interested in a mock interview, please direct message 🧑‍💻me as soon as possible. The first 10 interviews will be free 🆓, with a fee for the rest. Act quickly! 🥳Share your suggestions for the next tech stack you'd like me to cover by leaving a comment below. #nlp #naturallanguageprocessing #datascience #mockinterview #interviewpreparation #ai #generativeai #llm

14

1 Comment

When it comes to vector operations, two concepts that are often confused with each other are Dot Product and Cosine Similarity. While they may seem similar at first glance, the key difference lies in how they handle magnitude. 𝘋𝘰𝘵 𝘗𝘳𝘰𝘥𝘶𝘤𝘵: 𝘪𝘵 𝘮𝘦𝘢𝘴𝘶𝘳𝘦𝘴 𝘵𝘩𝘦 𝘦𝘹𝘵𝘦𝘯𝘵 𝘵𝘰 𝘸𝘩𝘪𝘤𝘩 𝘵𝘸𝘰 𝘷𝘦𝘤𝘵𝘰𝘳𝘴 𝘢𝘳𝘦 𝘱𝘰𝘪𝘯𝘵𝘪𝘯𝘨 𝘪𝘯 𝘵𝘩𝘦 𝘴𝘢𝘮𝘦 𝘥𝘪𝘳𝘦𝘤𝘵𝘪𝘰𝘯, 𝘤𝘰𝘯𝘴𝘪𝘥𝘦𝘳𝘪𝘯𝘨 𝘵𝘩𝘦𝘪𝘳 𝘭𝘦𝘯𝘨𝘵𝘩. 𝘊𝘰𝘴𝘪𝘯𝘦 𝘚𝘪𝘮𝘪𝘭𝘢𝘳𝘪𝘵𝘺: 𝘵𝘢𝘬𝘦𝘴 𝘵𝘩𝘦 𝘥𝘰𝘵 𝘱𝘳𝘰𝘥𝘶𝘤𝘵 𝘢𝘯𝘥 𝘥𝘪𝘷𝘪𝘥𝘦𝘴 𝘪𝘵 𝘣𝘺 𝘵𝘩𝘦 𝘱𝘳𝘰𝘥𝘶𝘤𝘵 𝘰𝘧 𝘵𝘩𝘦 𝘮𝘢𝘨𝘯𝘪𝘵𝘶𝘥𝘦𝘴 𝘰𝘧 𝘵𝘩𝘦 𝘵𝘸𝘰 𝘷𝘦𝘤𝘵𝘰𝘳𝘴. 𝗔 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗮𝗹 𝗘𝘅𝗮𝗺𝗽𝗹𝗲 Let's consider three vectors representing sentences: s1 = [1, 2, 1] ("The cat sat on the mat.") s2 = [1, 1, 1] ("The cat is sitting on the mat.") s3 = [4, 5, 6] ("The cat is sitting comfortably on the warm mat in the living room near the fireplace.") Dot product between s1 and s2: 4 Dot product between s1 and s3: 20 (large difference due to magnitude) Cosine Similarity between s1 and s2: 0.942 Cosine Similarity between s1 and s3: 0.930 (smaller difference, focusing on direction) 𝗦𝗼, 𝘄𝗵𝗲𝗻 𝘀𝗵𝗼𝘂𝗹𝗱 𝘆𝗼𝘂 𝘂𝘀𝗲 𝗲𝗮𝗰𝗵? Dot Product is suitable for applications like search engines, where the relevance of a document to a query is determined by the frequency of key terms (longer vectors naturally have higher dot products). Other examples can be, in recommendation systems to measure similarity between users or between items etc. Cosine Similarity is ideal for comparing the similarity between documents, sentences, or word embeddings in NLP tasks, as it normalises for length and focuses on direction. Other examples can be, analyzing connections or interactions in social networks, retrieving similar images based on feature vectors etc. #DotProduct #CosineSimilarity #VectorOperations #SearchEngines #NLP #DocumentSimilarity #SentenceSimilarity #SocialNetworkAnalysis #ImageRetrieval #MachineLearning #DataScience #Mathematics #Algorithms #DataAnalysis

762

33 Comments

I am happy to share that our years of effort have finally culminated in a VLDB paper titled "Datalog with First-Class Facts." We introduce SLOG (a variant of Datalog), where facts are first-class entities, with each fact uniquely identified by an identifier. This constrained approach proves particularly efficient for reasoning about tree-structured data using Datalog, with successful applications across database systems, AI, and programming language domains. We have developed a data-parallel MPI-based backend that can scale to thousands of processes. Check out the scaling result on a supercomputer for performing Context-Sensitive Points-to-Analysis (CSPA) in our language. The collaborative work was done by Thomas Gilray, Kristopher Micinski Yihao Sun, Sowmith K, and myself. Thanks to Argonne Leadership Computing Facility for supporting our research. #database #hpc #MPI #datalog

94

1 Comment

🤔 Testing LLaMa-3's Multi-lingual claims one step at a time - Can we directly fine-tune LLaMa-3 for Hindi language tasks? TLDR; Yes! 👇 ✏️ First, some context - Tokenizers play an important role in making a LLM multi-lingual, and Fertility is one of the metrics to evaluate tokenizer performance. It is defined as the number of tokens for a dataset divided by the number of words in the same dataset. Intuitively, the lower the fertility the better the tokenizer's coverage for that dataset. Extrapolate this for a language and it shows how well the tokenizer can cover that particular language. I tested it out for LLaMa-3 and the results are interesting. 🤩 LLaMa-3's tokenizer vocab size is huge 128k compared to 32k in LLaMa-2. As Meta team is planning to release multi-lingual variants in the future, they have kept a good variety of non-English tokens in the current release. This is proven below especially for the Hindi Language, as you can see v3 has 2x better (lower) fertility than v2. This is similar (though little more) to OpenHathi's tokenizer which extended v2's tokenizer by adding 18k Hindi tokens. ⚖ Now the judgment - Given v3's improvement over v2 and its closeness to OpenHathi's tokenizer vocab count, it makes sense to directly fine-tune on top of LLaMa-3 rather than doing any tokenizer-related modifications. Also, adding additional vocab to already huge 128k-sized tokenizers seems without merit. 👋 And that's it, let me know what you think. Btw, finetuning has multiple perspectives, while we are good with v3's tokenizer, should we pre-train, SFT, or preference train the v3? Maybe more about that in the next post 😉 #ai #ml #llama3 #india #hindi #indic #indiclanguage #llm #indian #aiforindia #openhathi

24

#PipelineAutomation in #ML involves creating #Automatedworkflows that handle the entire #ml process, from data ingestion to model deployment and monitoring. Here are some key components: #𝐃𝐚𝐭𝐚𝐈𝐧𝐠𝐞𝐬𝐭𝐢𝐨𝐧 Automating the collection and integration of raw data from various sources (databases, APIs, files, etc.). #DataPreprocessing Automatically cleaning, transforming, and normalizing data to prepare it for training. Handling missing values, scaling features, encoding categorical variables, etc. #FeatureEngineering Automatically creating and selecting features that improve model performance. Using domain knowledge, statistical techniques, or automated feature generation tools. #ModelTraining Automating the process of training machine learning models using predefined algorithms and hyperparameters. Implementing techniques like hyperparameter tuning and cross-validation. #ModelEvaluation Automatically evaluating models using various metrics (accuracy, precision, recall, F1 score, etc.). Comparing model performance and selecting the best-performing model. #ModelDeployment Automating the deployment of trained models to production environments. Ensuring models are accessible via APIs, web services, or other deployment mechanisms. #ModelMonitoring Continuously monitoring the performance of deployed models to detect issues like drift, degradation, or anomalies. Implementing alerting systems for proactive maintenance. #ModelRetraining Automating the process of retraining models periodically or when new data becomes available. Ensuring models remain accurate and up-to-date with the latest data. 𝑩𝒆𝒏𝒆𝒇𝒊𝒕𝒔 #Efficiency: Reduces manual effort and speeds up the machine learning lifecycle. Reproducibility: Ensures consistent and repeatable results by standardizing processes. #Scalability: Easily scales with increasing data volumes and computational demands. #Reliability: Minimizes human error and ensures robust, reliable workflows. #Agility: Enables rapid experimentation and iteration, facilitating faster model development and deployment. 𝐂𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐞𝐬 #Complexity Designing and maintaining automated pipelines can be complex and require expertise. #Integration Ensuring seamless integration between different tools and components can be challenging. #Monitoring & #Maintenance Continuously monitoring and maintaining automated pipelines require robust systems and processes. 𝐓𝐨𝐨𝐥𝐬 𝐚𝐧𝐝 𝐓𝐞𝐜𝐡𝐧𝐨𝐥𝐨𝐠𝐢𝐞𝐬 #ApacheAirflow #Kubeflow #MLflow #TensorFlowExtended (#TFX): #ApacheNiFi #DataRobot #MachineLearning #AIPipelines #MLAutomation #DataScience #ArtificialIntelligence #MLOps #TechInnovation #DataEngineering #AIWorkflow #BigData #DeepLearning #AIinBusiness #DataPipeline #TechTrends #ModelDeployment

11

1 Comment

I am happy to announce a new publication, this time in EMNLP 2024 Findings: "One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks" This describes the technical implementation of the Sanskrit grammatical engine at dharmamitra.org. Together with Oliver Hellwig and supported by Kurt Keutzer , we follow up on the 2018 Sanskrit word segmentation paper and build a new model that is not just better at segmentation, but can also do lemmatization and morphosyntactic parsing. Its one of those papers you only get to work on once every full moon, and it makes me very happy that we could do this. I do remember how we had ideas to do segmentation and morphological parsing in 2019, but it took 6 years to find the resources and time to actually follow through with it! Here is the preprint: https://lnkd.in/gX8FabEN

200

7 Comments

Krish Naik recommended this fantastic blog about Transformers, and I stumbled upon it while learning about NLP. The blog breaks down how Transformers function, from attention techniques to their role in speeding up tasks. If you're interested in NLP, this blog is essential reading. Make sure you don't miss it! https://lnkd.in/gDKhwHFy #Knowledgesharing #learningjourney #datascientist #NLP #Transformers #AI #opentowork

26

2 Comments