What is ELMo?

ELMo (Embeddings from Language Models) is a deep learning model for generating word embeddings. Unlike traditional word embeddings (like Word2Vec or GloVe) that assign a single, static vector to each word, ELMo generates contextualized word embeddings. This means that the embedding for a word changes based on the sentence it is used in, allowing the model to capture the different meanings and usages of a word in different contexts. ELMo uses a bidirectional LSTM (BiLSTM) trained on a large text corpus to predict the next word and the previous word in a sentence. The internal states of the BiLSTM are then used to create the word embeddings.