GitHub topics: datarecipes
NVIDIA/NeMo-Curator
Scalable data pre processing and curation toolkit for LLMs
Language: Jupyter Notebook - Size: 7.66 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 879 - Forks: 124
data-prep-kit/data-prep-kit
Open source project for data preparation of LLM application builders
Language: HTML - Size: 219 MB - Last synced at: 7 days ago - Pushed at: 7 days ago - Stars: 622 - Forks: 193
Related Keywords
python
2
llmapps
2
llm
2
large-scale-data-processing
2
large-language-models
2
deduplication
2
datarecipes
2
datacuration
2
data-preparation
2
data-prep
2
data
2
spark
1
ray
1
malware
1
finetuning
1
data-preprocessing-pipelines
1
data-preprocessing
1
code-quality
1
semantic-deduplication
1
data-curation
1
llm-data-quality
1
data-processing
1
fine-tuning
1
fast-data-processing
1
data-processing-pipelines
1
data-quality
1