Seq Modeling From Scratch
Implementing RNNs, LSTMs, GRUs, and Attention mechanisms using purely NumPy.
This project is a deep dive into the mathematical foundations of sequence modeling. Instead of relying on auto-differentiation frameworks like PyTorch, I implemented the forward and backward passes (Backpropagation Through Time) manually.
Key Implementations:
- RNNs, LSTMs, & GRUs: Built from first principles to understand gating mechanisms.
- Seq2Seq with Attention: Implemented Bahdanau and Luong attention mechanisms for machine translation tasks.
- Word & Char Level LMs: Trained language models on text datasets like “The Thirsty Crow”.
This project sharpened my understanding of computational graphs and gradient flow in dynamic networks like sequence models and hands on language modeling.