Understanding and Scaling Collaborative Filtering Optimization from the Perspective of Matrix Rank
Publications
Preprints
Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs
What Are We Optimizing For? A Human-centric Evaluation of Deep Learning-based Movie Recommenders
2024
On the Role of Attention Masks and LayerNorm in Transformers
NeurIPS 2024 / Paper
2023
Demystifying Oversmoothing in Attention-Based Graph Neural Networks
NeurIPS 2023 (Spotlight) / Paper
LOG 2023 (Oral)/ Talk (starting at 4:47:40) / DeepMath 2023 (Oral) / Talk (starting at 6:20:30)An Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks
ICLR 2023 / Paper
Blog: Oversmoothing in GNNs: why does it happen so fast? (and do popular solutions such as residual connections or normalization really work?)2022
2020
Spectra of Quantum Graphs via a Scattering Matrix Approach
Senior Thesis / Paper