Lamar University, Beaumont, TX
|Graduate Research Assistant
Summary
Implemented a Python-based RAG (Retrieval-Augmented Generation) system utilizing OpenAI API and Lang Chain framework to enable semantic search across GitHub repositories. Engineered Git integration module using the subprocess library to efficiently clone and extract repository metadata, supporting 30+ file types across multiple programming languages. Developed a document processing pipeline with Recursive Character Text Splitter for optimal chunking (3000-character chunks with 200-character overlap) and NLTK for tokenization and text normalization. Built a hybrid vector search engine combining BM25Okapi and TF-IDF with cosine similarity (0.5/0.5 weighting) to improve semantic relevance in document retrieval. Integrated LLM using Lang Chain's Prompt Template and OpenAI's GPT models, with conversation history management for contextually relevant responses to repository-specific technical queries. Implemented scalable state management and multi-user interaction using Dash callbacks and Flask sessions for collaborative research workflows. Developed backend services with Fast API and Flask to support ML model inference, data preprocessing, and metadata management. Designed and integrated APIs using Fast API and Open API for seamless interaction between research databases, ML pipelines, and external tools.