text-processing | Topic | Ecosyste.ms: Repos

Topic: "text-processing"

learnbyexample/Command-line-text-processing 📦

:zap: From finding text to search and replace, from sorting to beautifying text and more :art:

Language: Shell - Size: 942 KB - Last synced at: 10 months ago - Pushed at: over 1 year ago - Stars: 10,192 - Forks: 710

pymupdf/PyMuPDF

PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.

Language: Python - Size: 342 MB - Last synced at: 11 days ago - Pushed at: 14 days ago - Stars: 8,706 - Forks: 678

google/diff-match-patch 📦

Diff Match Patch is a high-performance library in multiple languages that manipulates plain text.

Language: Python - Size: 659 KB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 7,943 - Forks: 1,171

chmln/sd

Intuitive find & replace CLI (sed alternative)

Language: Rust - Size: 405 KB - Last synced at: 10 days ago - Pushed at: 12 days ago - Stars: 6,790 - Forks: 151

fastnlp/fastNLP

fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.

Language: Python - Size: 35.1 MB - Last synced at: 7 months ago - Pushed at: over 2 years ago - Stars: 3,132 - Forks: 449

pyparsing/pyparsing

Python library for creating PEG parsers

Language: Python - Size: 9.19 MB - Last synced at: 8 days ago - Pushed at: 9 days ago - Stars: 2,432 - Forks: 298

kk7nc/Text_Classification

Text Classification Algorithms: A Survey

Language: Python - Size: 13.8 MB - Last synced at: 8 months ago - Pushed at: 9 months ago - Stars: 1,811 - Forks: 544

roshan-research/hazm

Persian NLP Toolkit

Language: Python - Size: 25.2 MB - Last synced at: 9 days ago - Pushed at: 11 days ago - Stars: 1,357 - Forks: 204

helix-editor/nucleo

A fast and convenient fuzzy matcher library for rust

Language: Rust - Size: 232 KB - Last synced at: 2 months ago - Pushed at: 7 months ago - Stars: 1,247 - Forks: 48

pemistahl/lingua-go

The most accurate natural language detection library for Go, suitable for short text and mixed-language text

Language: Go - Size: 226 MB - Last synced at: 7 months ago - Pushed at: 11 months ago - Stars: 1,245 - Forks: 68

birchb1024/frangipanni

Program to convert lines of text into a tree structure.

Language: Go - Size: 1 MB - Last synced at: 7 months ago - Pushed at: over 2 years ago - Stars: 1,200 - Forks: 30

BurntSushi/aho-corasick

A fast implementation of Aho-Corasick in Rust.

Language: Rust - Size: 4.72 MB - Last synced at: 7 days ago - Pushed at: 2 months ago - Stars: 1,170 - Forks: 108

PyThaiNLP/pythainlp

Thai natural language processing in Python

Language: Python - Size: 66 MB - Last synced at: 6 days ago - Pushed at: 12 days ago - Stars: 1,093 - Forks: 287

ChenghaoMou/text-dedup

All-in-one text de-duplication

Language: Python - Size: 59 MB - Last synced at: about 19 hours ago - Pushed at: 1 day ago - Stars: 737 - Forks: 74

sstadick/hck

A sharp cut(1) clone.

Language: Rust - Size: 515 KB - Last synced at: 28 days ago - Pushed at: about 1 month ago - Stars: 722 - Forks: 18

wenet-e2e/WeTextProcessing

Text Normalization & Inverse Text Normalization

Language: Python - Size: 1.02 MB - Last synced at: 10 days ago - Pushed at: about 1 month ago - Stars: 706 - Forks: 91

derek73/python-nameparser

A simple Python module for parsing human names into their individual components

Language: Python - Size: 778 KB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 689 - Forks: 107

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

Language: Python - Size: 778 KB - Last synced at: 3 months ago - Pushed at: 7 months ago - Stars: 671 - Forks: 93

open-korean-text/open-korean-text

Open Korean Text Processor - An Open-source Korean Text Processor

Language: Scala - Size: 32.7 MB - Last synced at: 2 months ago - Pushed at: almost 2 years ago - Stars: 646 - Forks: 97

abadojack/whatlanggo

Natural language detection library for Go

Language: Go - Size: 240 KB - Last synced at: about 1 year ago - Pushed at: almost 3 years ago - Stars: 640 - Forks: 66

lukaszliniewicz/Pandrator

Turn PDFs and EPUBs into audiobooks, subtitles or videos into dubbed videos (including translation), and more. For free. Pandrator uses local models, notably XTTS, including voice-cloning (instant, RVC-enhanced, XTTS fine-tuning) and LLM processing. It aspires to be a user-friendly app with a GUI, an installer and all-in-one packages.

Language: Python - Size: 8.11 MB - Last synced at: 3 months ago - Pushed at: 8 months ago - Stars: 501 - Forks: 38

Puchaczov/Musoq

SQL Syntax without any database

Language: C# - Size: 16.6 MB - Last synced at: 3 days ago - Pushed at: 4 days ago - Stars: 499 - Forks: 21

proycon/pynlpl

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

Language: Python - Size: 12.8 MB - Last synced at: 11 days ago - Pushed at: over 2 years ago - Stars: 479 - Forks: 68

linuxscout/pyarabic

pyarabic

Language: Python - Size: 1.23 MB - Last synced at: 10 days ago - Pushed at: almost 2 years ago - Stars: 470 - Forks: 88

kreuzberg-dev/html-to-markdown

High performance and CommonMark compliant HTML to Markdown converter. Maintained by the Kreuzberg.dev team. Kreuzberg.dev is a fast, polyglot document intelligence engine with a Rust core. It extracts structured data from 56+ document formats using streaming parsers and built-in OCR.

Language: HTML - Size: 10.3 MB - Last synced at: 1 day ago - Pushed at: 2 days ago - Stars: 454 - Forks: 41

haven-jeon/PyKoSpacing

Automatic Korean word spacing with Python

Language: Python - Size: 4.53 MB - Last synced at: about 1 month ago - Pushed at: over 1 year ago - Stars: 424 - Forks: 115

andrewbihl/bsed

Simple SQL-like syntax on top of Perl text processing.

Language: Python - Size: 146 KB - Last synced at: 3 months ago - Pushed at: over 6 years ago - Stars: 411 - Forks: 14

airbnb/artificial-adversary

🗣️ Tool to generate adversarial text examples and test machine learning models against them

Language: Python - Size: 116 KB - Last synced at: 10 days ago - Pushed at: almost 4 years ago - Stars: 400 - Forks: 56

BurntSushi/regex-automata 📦

A low level regular expression library that uses deterministic finite automata.

Language: Rust - Size: 39.1 MB - Last synced at: 7 days ago - Pushed at: over 2 years ago - Stars: 351 - Forks: 25

ikegami-yukino/jaconv

Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku

Language: Python - Size: 379 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 336 - Forks: 32

gagolews/stringi

Fast and portable character string processing in R (with the Unicode ICU)

Language: C++ - Size: 210 MB - Last synced at: 4 months ago - Pushed at: 9 months ago - Stars: 313 - Forks: 50

textpipe/textpipe 📦

Textpipe: clean and extract metadata from text

Language: Python - Size: 340 KB - Last synced at: 2 months ago - Pushed at: over 4 years ago - Stars: 302 - Forks: 25

himkt/konoha

🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.

Language: Python - Size: 1.35 MB - Last synced at: 11 days ago - Pushed at: 8 months ago - Stars: 260 - Forks: 27

open-i18n/rust-unic

UNIC: Unicode and Internationalization Crates for Rust

Language: Rust - Size: 14.1 MB - Last synced at: 18 days ago - Pushed at: 7 months ago - Stars: 242 - Forks: 24

RandyPen/TextCluster

短文本聚类预处理模块 Short text cluster

Language: Python - Size: 1.25 MB - Last synced at: about 2 years ago - Pushed at: about 6 years ago - Stars: 238 - Forks: 60

daac-tools/daachorse

🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure in Rust.

Language: Rust - Size: 3.71 MB - Last synced at: 3 months ago - Pushed at: 7 months ago - Stars: 232 - Forks: 20

catatsuy/purl

Streamlining Text Processing

Language: Go - Size: 252 KB - Last synced at: 18 days ago - Pushed at: 19 days ago - Stars: 229 - Forks: 6

larrykollar/Unix-Text-Processing

Recreated sources for the book "UNIX Text Processing," published in 1987.

Language: Roff - Size: 620 KB - Last synced at: 5 months ago - Pushed at: over 2 years ago - Stars: 225 - Forks: 13

bytesparadise/libasciidoc 📦

A Golang library for processing Asciidoc files.

Language: Go - Size: 25.5 MB - Last synced at: 5 months ago - Pushed at: over 1 year ago - Stars: 213 - Forks: 26

cloudflare/wildcard

Wildcard matching

Language: Rust - Size: 45.7 MB - Last synced at: 5 days ago - Pushed at: about 2 months ago - Stars: 205 - Forks: 6

aappleby/matcheroni

A minimalist single-header library for building pattern-matchers, lexers, and parsers.

Language: C++ - Size: 7.31 MB - Last synced at: 7 months ago - Pushed at: 10 months ago - Stars: 200 - Forks: 5

casics/nostril 📦

Nostril: Nonsense String Evaluator

Language: Python - Size: 143 MB - Last synced at: 2 months ago - Pushed at: over 3 years ago - Stars: 199 - Forks: 35

textvec/textvec

Text vectorization tool to outperform TFIDF for classification tasks

Language: Python - Size: 799 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 194 - Forks: 26

learnbyexample/cli_text_processing_coreutils

Example based guide for specialized text processing with GNU Coreutils

Language: Shell - Size: 2.98 MB - Last synced at: 7 months ago - Pushed at: 8 months ago - Stars: 193 - Forks: 9

NIHOPA/NLPre

Python library for Natural Language Preprocessing (NLPre)

Language: Python - Size: 51 MB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 191 - Forks: 35

WZBSocialScienceCenter/tmtoolkit 📦

Text Mining and Topic Modeling Toolkit for Python with parallel processing power

Language: Python - Size: 78.1 MB - Last synced at: 8 months ago - Pushed at: over 2 years ago - Stars: 190 - Forks: 27

learnbyexample/learn_ruby_oneliners

Example based guide for text processing with Ruby from the command line

Language: Shell - Size: 3.01 MB - Last synced at: 4 months ago - Pushed at: 8 months ago - Stars: 185 - Forks: 17

pemagrg1/Natural-Language-Processing-NLP-Roadmap

A simple RoadMap to Natural Language Processing(NLP)

Size: 67.4 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 180 - Forks: 22

s3nh/text-detector

Tool which allow you to detect and translate text.

Language: Python - Size: 103 MB - Last synced at: 9 days ago - Pushed at: over 6 years ago - Stars: 180 - Forks: 39

karolzak/support-tickets-classification

This case study shows how to create a model for text analysis and classification and deploy it as a web service in Azure cloud in order to automatically classify support tickets. This project is a proof of concept made by Microsoft (Commercial Software Engineering team) in collaboration with Endava http://endava.com/en

Language: Python - Size: 3.74 MB - Last synced at: 3 months ago - Pushed at: over 3 years ago - Stars: 168 - Forks: 92

krzyzanowskim/CoreTextSwift

CoreText Swift bindings

Language: Swift - Size: 27.3 KB - Last synced at: 4 days ago - Pushed at: over 1 year ago - Stars: 167 - Forks: 8

hakatashi/japanese.js

Util collection for Japanese text processing. Hiraganize, Katakanize, and Romanize.

Language: JavaScript - Size: 283 KB - Last synced at: about 1 month ago - Pushed at: over 5 years ago - Stars: 167 - Forks: 3

MycroftAI/padatious

A neural network intent parser

Language: Python - Size: 97.7 KB - Last synced at: 7 months ago - Pushed at: about 4 years ago - Stars: 162 - Forks: 39

lyeoni/prenlp

Preprocessing Library for Natural Language Processing

Language: Python - Size: 156 KB - Last synced at: 9 months ago - Pushed at: about 3 years ago - Stars: 161 - Forks: 12

assafmo/xioc

Extract indicators of compromise from text, including "escaped" ones.

Language: Go - Size: 64.5 KB - Last synced at: 5 months ago - Pushed at: over 5 years ago - Stars: 160 - Forks: 11

kantord/headson

head/tail for structured data - summarize/preview JSON/YAML and source code

Language: Rust - Size: 61 MB - Last synced at: 6 days ago - Pushed at: 6 days ago - Stars: 157 - Forks: 4

Anwarvic/Dan-Jurafsky--Chris-Manning--NLP

My solution to the Natural Language Processing course made by Dan Jurafsky, Chris Manning in Winter 2012.

Language: Java - Size: 49.7 MB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 157 - Forks: 55

ZeroX-DG/vi-rs

Vietnamese Input Method library

Language: Rust - Size: 636 KB - Last synced at: 3 months ago - Pushed at: 5 months ago - Stars: 154 - Forks: 14

goplus/bpl

Binary Processing Language

Language: Go - Size: 462 KB - Last synced at: 8 months ago - Pushed at: over 3 years ago - Stars: 154 - Forks: 32

lovit/soyspacing

띄어쓰기 오류 교정 라이브러리입니다. CRF 와 같은 머신러닝 알고리즘이 아닌, 직관적인 접근법으로 띄어쓰기를 교정합니다.

Language: Python - Size: 2.09 MB - Last synced at: about 1 month ago - Pushed at: over 6 years ago - Stars: 150 - Forks: 34

microsoft/browsecloud 📦

A web app to create and browse text visualizations for automated customer listening.

Language: TypeScript - Size: 5.58 MB - Last synced at: 5 days ago - Pushed at: about 2 years ago - Stars: 148 - Forks: 19

alihoseiny/word_cloud_fa

A wrapper for wordcloud module for creating Persian word clouds.

Language: Python - Size: 1.76 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 144 - Forks: 13

brothersincode/virastar

Cleaning-up Persian Texts!

Language: JavaScript - Size: 1.3 MB - Last synced at: 19 days ago - Pushed at: 8 months ago - Stars: 143 - Forks: 14

goforj/str

A fluent, Laravel-inspired string toolkit for Go with explicit, rune-safe helpers and predictable behavior.

Language: Go - Size: 1.07 MB - Last synced at: 2 days ago - Pushed at: 4 days ago - Stars: 139 - Forks: 2

stanfordnlp/stanza-old

Stanford NLP group's shared Python tools.

Language: Python - Size: 383 KB - Last synced at: 5 months ago - Pushed at: almost 8 years ago - Stars: 137 - Forks: 34

acarl005/stripansi

A little Go package for removing ANSI color escape codes from strings.

Language: Go - Size: 1.95 KB - Last synced at: 6 months ago - Pushed at: almost 4 years ago - Stars: 135 - Forks: 16

NeilMacMullen/Textrude

Code generation from YAML/JSON/CSV models via SCRIBAN templates

Language: C# - Size: 9.18 MB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 133 - Forks: 12

proycon/colibri-core

Colibri core is an NLP tool as well as a C++ and Python library for working with basic linguistic constructions such as n-grams and skipgrams (i.e patterns with one or more gaps, either of fixed or dynamic size) in a quick and memory-efficient way. At the core is the tool ``colibri-patternmodeller`` whi ch allows you to build, view, manipulate and query pattern models.

Language: C++ - Size: 10.2 MB - Last synced at: 21 days ago - Pushed at: about 1 year ago - Stars: 129 - Forks: 20

CogComp/cogcomp-nlpy

CogComp's light-weight Python NLP annotators

Language: Python - Size: 331 KB - Last synced at: about 1 month ago - Pushed at: almost 7 years ago - Stars: 115 - Forks: 26

01walid/goarabic

A Go Lang package for dealing with Arabic text.

Language: Go - Size: 16.6 KB - Last synced at: 5 months ago - Pushed at: over 2 years ago - Stars: 113 - Forks: 29

MilesCranmer/vim-stream

vims - use vim like sed

Language: Shell - Size: 84 KB - Last synced at: 3 months ago - Pushed at: almost 2 years ago - Stars: 111 - Forks: 8

claustromaniac/Compare-UserJS

PowerShell script for comparing user.js (or prefs.js) files.

Language: PowerShell - Size: 137 KB - Last synced at: 5 months ago - Pushed at: over 6 years ago - Stars: 111 - Forks: 11

learnbyexample/learn_perl_oneliners

Example based guide for text processing with Perl from the command line

Language: Shell - Size: 3.23 MB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 108 - Forks: 14

Automattic/go-search-replace

🚀 Search & replace URLs in WordPress SQL files.

Language: Go - Size: 104 KB - Last synced at: 2 months ago - Pushed at: 7 months ago - Stars: 103 - Forks: 19

waseem18/node-rake

A NodeJS implementation of the Rapid Automatic Keyword Extraction algorithm.

Language: JavaScript - Size: 27.3 KB - Last synced at: 4 months ago - Pushed at: over 2 years ago - Stars: 103 - Forks: 20

sdleffler/qp-trie-rs

An idiomatic and fast QP-trie implementation in pure Rust.

Language: Rust - Size: 80.1 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 102 - Forks: 25

elixir-nx/tokenizers

Elixir bindings for 🤗 Tokenizers

Language: Elixir - Size: 2.69 MB - Last synced at: 3 months ago - Pushed at: 7 months ago - Stars: 99 - Forks: 18

cloudflare/sliceslice-rs

A fast implementation of single-pattern substring search using SIMD acceleration.

Language: Rust - Size: 350 KB - Last synced at: 3 months ago - Pushed at: over 1 year ago - Stars: 98 - Forks: 17

Thomas-George-T/HackerRank-The-Linux-Shell-Challenges-Solutions

Complete Solutions and related tutorials for the Linux Shell - Bash, text processing, Arrays in Bash, Grep Sed Awk Challenges on HackerRank

Language: Shell - Size: 89.8 KB - Last synced at: 9 months ago - Pushed at: over 1 year ago - Stars: 97 - Forks: 58

Kyubyong/mtp

Multi-lingual Text Processing

Size: 1.29 MB - Last synced at: 10 months ago - Pushed at: almost 7 years ago - Stars: 96 - Forks: 12

sefineh-ai/Amharic-Tokenizer

Syllable-aware BPE tokenizer for the Amharic language (አማርኛ) – fast, accurate, trainable.

Language: Python - Size: 10.7 MB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 95 - Forks: 12

safakatakancelik/TalkWithYourFiles

An LLM GUI application; enables you to interact with your files, offering dynamic parameters that can modify response behavior during runtime.

Language: Python - Size: 813 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 94 - Forks: 11

angelosalatino/cso-classifier

Python library that classifies content from scientific papers with the topics of the Computer Science Ontology (CSO).

Language: Python - Size: 19.9 MB - Last synced at: about 1 month ago - Pushed at: about 1 year ago - Stars: 92 - Forks: 19

fingeredman/teanaps

자연어 처리와 텍스트 분석을 위한 오픈소스 파이썬 라이브러리 입니다.

Language: Jupyter Notebook - Size: 62.5 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 92 - Forks: 11

znwang25/fuzzychinese

A small package to fuzzy match chinese words

Language: Python - Size: 1.81 MB - Last synced at: about 1 month ago - Pushed at: almost 3 years ago - Stars: 92 - Forks: 10

nschneid/unix-text-commands

Unix Text Processing Command Reference

Size: 6.84 KB - Last synced at: 7 months ago - Pushed at: over 9 years ago - Stars: 88 - Forks: 34

Kaizosha/Hush

while you’re in the moment, it listens. it sees. it remembers.

Language: Swift - Size: 12 MB - Last synced at: 5 months ago - Pushed at: 6 months ago - Stars: 86 - Forks: 19

kefirfromperm/kefirbb

A flexible Java text processor. BB, BBCode, BB-code, HTML, Textile, Markdown, parser, translator, converter.

Language: Java - Size: 508 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 86 - Forks: 14

elektito/finglish

A Finglish to Persian converter.

Language: Python - Size: 2.28 MB - Last synced at: 27 days ago - Pushed at: over 4 years ago - Stars: 86 - Forks: 21

n3mo/data-science

Data science tooling for Racket

Language: Racket - Size: 650 KB - Last synced at: 8 months ago - Pushed at: over 6 years ago - Stars: 84 - Forks: 6

PacktPublishing/Hands-On-Python-Natural-Language-Processing

Language: Jupyter Notebook - Size: 116 MB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 82 - Forks: 77

SayamAlt/Fake-Reviews-Detection

Successfully developed a machine learning model which can predict whether an online review is fraudulent or not. The main idea used to detect the fake nature of reviews is that the review should be computer generated through unfair means. If the review is created manually, then it is considered legal and original.

Language: Jupyter Notebook - Size: 9.12 MB - Last synced at: 9 months ago - Pushed at: over 3 years ago - Stars: 80 - Forks: 36