LLM finetuning and interpretation
My research focuses on efficient fine-tuning of LLMs using parameter-efficient methods like LoRA/QLoRA (implemented for Mistral-7B), rigorous evaluation via modular benchmarking suites (MMLU, TruthfulQA, ARC), and training data analysis to understand how dataset composition, quality, and bias impact model behavior.
Research Topic: Explainable, Efficient, and Deployable Large Language Models
My research focuses on the development of explainable, adaptable, and resource-efficient Large Language Model (LLM) systems, addressing fundamental challenges across the complete LLM lifecycle—from understanding model behavior and training data influence to efficient fine-tuning, rigorous evaluation, and optimized deployment. Rather than treating these domains as isolated engineering tasks, the research investigates how transparency, adaptation, evaluation, and operational efficiency can be integrated into a unified methodology for trustworthy and production-ready AI systems.
A central research question concerns the relationship between training data and emergent model behavior. Large language models are often treated as black boxes, making it difficult to determine why specific outputs occur or which training samples contribute to particular behaviors. To address this challenge, my work explores training-data attribution techniques that estimate the influence of individual training examples on downstream model decisions. By combining attribution-vector computation, influence scoring, targeted data selection, and behavioral validation through subsequent fine-tuning experiments, this research contributes to a deeper understanding of model interpretability, explainability, and traceability. The objective is to establish mechanisms that allow practitioners to identify influential data sources, diagnose undesirable behaviors, and improve confidence in model outputs.
Building on this understanding of model behavior, a second research direction investigates parameter-efficient adaptation of foundation models. As modern LLMs continue to grow in size, the computational cost of full-model retraining becomes increasingly prohibitive. My work therefore focuses on techniques such as LoRA and QLoRA, which enable domain adaptation through lightweight, trainable parameter subsets while preserving the underlying pretrained model. This research examines the trade-offs between model specialization, training efficiency, computational resource consumption, and downstream task performance. The goal is to enable rapid adaptation of foundation models to domain-specific requirements while minimizing hardware demands and operational costs.
A third pillar of the research addresses the challenge of systematic and reproducible model evaluation. The rapid evolution of LLM architectures has created a fragmented ecosystem of benchmarks, metrics, and evaluation methodologies, often making direct comparison difficult. To support scientific rigor and reproducibility, I developed modular evaluation frameworks capable of assessing models across multiple dimensions, including knowledge acquisition, reasoning capability, safety, truthfulness, and generation quality. The research emphasizes standardized benchmark integration, extensible evaluation pipelines, consistent metric computation, and support for diverse model backends. This enables objective comparison of model variants, fine-tuning strategies, and deployment configurations while providing a foundation for evidence-based model selection.
The final research domain focuses on efficient deployment and model optimization. While advances in model capability continue to drive adoption, practical deployment remains constrained by memory requirements, inference latency, throughput, and infrastructure cost. My research therefore investigates quantization and model compression techniques, including INT8, INT4, GPTQ, and AWQ approaches, with the goal of systematically analyzing the relationship between computational efficiency and predictive performance. Through standardized benchmarking frameworks, this work evaluates how different quantization strategies affect accuracy, latency, memory footprint, and throughput, thereby supporting informed deployment decisions for cloud, enterprise, and edge environments.
Taken together, these research activities contribute to a holistic vision of trustworthy and scalable AI systems. Training-data attribution provides transparency into model behavior; efficient fine-tuning enables targeted adaptation; rigorous evaluation ensures measurable quality and robustness; and deployment optimization makes advanced AI systems economically viable in real-world environments. The overarching objective is to bridge the gap between cutting-edge foundation-model research and practical enterprise adoption by developing methods that make LLMs more understandable, adaptable, measurable, and deployable at scale.