GitHub / marcomoldovan / hierarchical-language-modeling

We address the task of learning contextualized word, sentence and document representations with a hierarchical language model by stacking Transformer-based encoders on a sentence level and subsequently on a document level and performing masked token prediction.

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcomoldovan%2Fhierarchical-language-modeling

Topics: attention-mechanism, deep-learning, document-embedding, document-retrieval, information-retrieval, language-model, machine-learning, natural-language-processing, natural-language-understanding, pytorch, representation-learning, sentence-embeddings, transfer-learning, transformer, word-embeddings

• Files
• Readme
• Dependencies