What is XLNet?

XLNet is a generalized autoregressive pretraining method for language understanding. Unlike standard autoregressive (AR) language models that only use either past or future context, XLNet maximizes the expected log-likelihood of a sequence with respect to all possible permutations of the factorization order. It integrates the advantages of both AR language modeling and autoencoding (AE) while avoiding their limitations. It's commonly used for tasks like text classification, question answering, and natural language inference.