On the Emergence of Position Bias in Transformers
Publications
Preprints


What Are We Optimizing For? A Human-centric Evaluation of Deep Learning-based Movie Recommenders
2025

Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs
ICLR 2025 / Paper


Understanding and Scaling Collaborative Filtering Optimization from the Perspective of Matrix Rank
WWW 2025 (Oral) / Paper
2024

On the Role of Attention Masks and LayerNorm in Transformers
NeurIPS 2024 / Paper
2023

Demystifying Oversmoothing in Attention-Based Graph Neural Networks
NeurIPS 2023 (Spotlight) / Paper
LOG 2023 (Oral) / Talk (starting at 4:47:40) / DeepMath 2023 (Oral) / Talk (starting at 6:20:30)
An Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks
ICLR 2023 / Paper
Blog: Oversmoothing in GNNs: why does it happen so fast? (and do popular solutions such as residual connections or normalization really work?)2022
2020

Spectra of Quantum Graphs via a Scattering Matrix Approach
Senior Thesis / Paper