Live classroom - AquinTutor

 AQUINTUTOR / DASHBOARD / LIVE CLASSROOM · AI-512 

LIVE · 00:42:18

Transformers in Practice · Lecture 6.2 AI-512 · Dr. M. Iyer

117 attending NETWORK · STABLE

3 in queue

LECTURE 6.2 · SLIDE 14

Why masked attention is causal

The lower-triangular mask M ∈ {0,1}^{n×n} ensures that token i attends only to positions ≤ i. We zero out future logits before softmax — not after.

14 / 32

Priya N. 00:38

Got it - so we mask before softmax to keep it normalized?

Dr. M. Iyer HOST 00:38

Exactly. After softmax the probabilities are already locked.

Wei H. 00:39

Wait, what about cross-attention in encoder-decoder?

Aquin AI 00:39

Cross-attention typically does not need a causal mask - the encoder sees the full source sequence.

Sara K. 00:40

question coming - hand raised

Diego M. 00:41

+1 for the lab demo with PyTorch

Ask AquinEveryoneHost only

Today · your schedule

Remaining sessions.

11:00

Transformers in Practice

AI-512 · Dr. M. Iyer

LIVE NOW
14:00

Quantum Foundations

QT-407 · Prof. Penrose-Hayes

UPCOMING
16:30

Ancient Indic Texts I

VS-101 · Dr. Anjali Krishnan

UPCOMING
18:00

Office hours · Dr. Iyer

Open · drop-in

UPCOMING