Follow
Joshua Ainslie
Joshua Ainslie
Verified email at google.com
Title
Cited by
Cited by
Year
Big bird: Transformers for longer sequences
M Zaheer, G Guruganesh, KA Dubey, J Ainslie, C Alberti, S Ontanon, ...
Advances in Neural Information Processing Systems 33, 17283-17297, 2020
4642020
ETC: Encoding long and structured inputs in transformers
J Ainslie, S Ontanon, C Alberti, V Cvicek, Z Fisher, P Pham, A Ravula, ...
arXiv preprint arXiv:2004.08483, 2020
1082020
Fnet: Mixing tokens with fourier transforms
J Lee-Thorp, J Ainslie, I Eckstein, S Ontanon
arXiv preprint arXiv:2105.03824, 2021
682021
Realformer: Transformer likes residual attention
R He, A Ravula, B Kanagal, J Ainslie
arXiv preprint arXiv:2012.11747, 2020
182020
Making transformers solve compositional tasks
S Ontanón, J Ainslie, V Cvicek, Z Fisher
arXiv preprint arXiv:2108.04378, 2021
82021
Big bird: Transformers for longer sequences. arxiv e-prints, art
M Zaheer, G Guruganesh, A Dubey, J Ainslie, C Alberti, S Ontanon, ...
arXiv preprint arXiv:2007.14062, 2020
72020
FNet: Mixing Tokens with Fourier Transforms. arXiv 2021
J Lee-Thorp, J Ainslie, I Eckstein, S Ontanon
arXiv preprint arXiv:2105.03824, 0
6
Big bird: Transformers for longer sequences. arXiv 2020
M Zaheer, G Guruganesh, A Dubey, J Ainslie, C Alberti, S Ontanon, ...
arXiv preprint arXiv:2007.14062, 0
5
ETC: Encoding long and structured inputs in transformers
A Ravula, C Alberti, J Ainslie, L Yang, PM Pham, Q Wang, S Ontanon, ...
42020
Readtwice: Reading very large documents with memories
Y Zemlyanskiy, J Ainslie, M de Jong, P Pham, I Eckstein, F Sha
arXiv preprint arXiv:2105.04241, 2021
32021
Iterative decoding for compositional generalization in transformers
L Ruiz, J Ainslie, S Ontañón
arXiv preprint arXiv:2110.04169, 2021
22021
Improving compositional generalization in classification tasks via structure annotations
J Kim, P Ravikumar, J Ainslie, S Ontañón
arXiv preprint arXiv:2106.10434, 2021
22021
LongT5: Efficient Text-To-Text Transformer for Long Sequences
M Guo, J Ainslie, D Uthus, S Ontanon, J Ni, YH Sung, Y Yang
arXiv preprint arXiv:2112.07916, 2021
12021
LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Models
S Ontanon, J Ainslie, V Cvicek, Z Fisher
arXiv preprint arXiv:2203.15099, 2022
2022
FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction
CY Lee, CL Li, T Dozat, V Perot, G Su, N Hua, J Ainslie, R Wang, Y Fujii, ...
arXiv preprint arXiv:2203.08411, 2022
2022
Attention neural networks with sparse attention mechanisms
JT Ainslie, S Ontañón, P Pham, M Zaheer, G Guruganesh, KA Dubey, ...
US Patent 11,238,332, 2022
2022
ShopTalk: A System for Conversational Faceted Search
G Manku, J Lee-Thorp, B Kanagal, J Ainslie, J Feng, Z Pearson, E Anjorin, ...
arXiv preprint arXiv:2109.00702, 2021
2021
RealFormer: Transformer Likes Residual Attention
A Ravula, B Kanagal, J Ainslie, R He
2021
ReadTwice: Reading Very Large Documents with Memories Open Website
Y Zemlyanskiy, J Ainslie, M de Jong, P Pham, I Eckstein, F Sha
Big Bird: Transformers for Longer Sequences Download PDF
M Zaheer, G Guruganesh, A Dubey, J Ainslie, C Alberti, S Ontanon, ...
The system can't perform the operation now. Try again later.
Articles 1–20