Topic: "duplicates"
qarmin/czkawka
Multi functional app to find duplicates, empty folders, similar images etc.
Language: Rust - Size: 4.61 MB - Last synced at: 3 days ago - Pushed at: 12 days ago - Stars: 23,624 - Forks: 741
kucherenko/jscpd
Copy/paste detector for programming source code.
Language: TypeScript - Size: 9.13 MB - Last synced at: 3 days ago - Pushed at: 10 days ago - Stars: 4,870 - Forks: 211
sahib/rmlint
Extremely fast tool to remove duplicates and other lint from your filesystem
Language: C - Size: 12.4 MB - Last synced at: 1 day ago - Pushed at: 18 days ago - Stars: 2,076 - Forks: 137
scinos/yarn-deduplicate
Deduplication tool for yarn.lock files
Language: TypeScript - Size: 7.27 MB - Last synced at: 4 days ago - Pushed at: 16 days ago - Stars: 1,389 - Forks: 57
sean-public/python-hashes
Interesting (non-cryptographic) hashes implemented in pure Python.
Language: Python - Size: 29.3 KB - Last synced at: 10 months ago - Pushed at: over 3 years ago - Stars: 240 - Forks: 43
kristiankoskimaki/vidupe
Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:
Language: C++ - Size: 266 KB - Last synced at: about 1 month ago - Pushed at: over 5 years ago - Stars: 176 - Forks: 18
F483/dejavu
Quickly detect already witnessed data.
Language: Go - Size: 305 KB - Last synced at: 13 days ago - Pushed at: 10 months ago - Stars: 157 - Forks: 5
PJDude/dude
Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.
Language: Python - Size: 4.48 MB - Last synced at: 5 days ago - Pushed at: 5 days ago - Stars: 140 - Forks: 11
kouhin/redux-dataloader
Loads async data for Redux apps focusing on preventing duplicated requests and dealing with async dependencies.
Language: JavaScript - Size: 108 KB - Last synced at: 9 days ago - Pushed at: over 7 years ago - Stars: 139 - Forks: 3
Canop/backdown
A deduplicator
Language: Rust - Size: 541 KB - Last synced at: 4 days ago - Pushed at: 8 months ago - Stars: 132 - Forks: 7
scrubbbbs/cbird
Command-line program for Content-Based Image Retrieval of images and videos. Includes tools for general search and de-duplication.
Language: C++ - Size: 12.7 MB - Last synced at: 4 days ago - Pushed at: 22 days ago - Stars: 122 - Forks: 5
matteodelabre/mongoose-beautiful-unique-validation
Plugin for Mongoose that turns duplicate errors into regular Mongoose validation errors
Language: JavaScript - Size: 194 KB - Last synced at: 3 days ago - Pushed at: almost 3 years ago - Stars: 117 - Forks: 38
jvirkki/dupd
CLI utility to find duplicate files
Language: C - Size: 1.9 MB - Last synced at: about 1 year ago - Pushed at: almost 2 years ago - Stars: 109 - Forks: 16
microsoft/near-duplicate-code-detector
A simple tool for detecting near-duplicate source code
Language: C# - Size: 38.1 KB - Last synced at: about 11 hours ago - Pushed at: 8 months ago - Stars: 100 - Forks: 31
cloud-py-api/mediadc
Nextcloud Media Duplicate Collector application
Language: PHP - Size: 91.1 MB - Last synced at: 4 days ago - Pushed at: 4 days ago - Stars: 99 - Forks: 9
eyalroz/removedupes
Remove Duplicate Messages
Language: JavaScript - Size: 8.56 MB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 93 - Forks: 7
StephaneCouturier/Katalog
Katalog is an application to manage catalogs of disks and files to search and get statistics.
Language: C++ - Size: 12.2 MB - Last synced at: 8 days ago - Pushed at: 8 days ago - Stars: 81 - Forks: 7
vuolter/deplicate 📦
Advanced Duplicate File Finder for Python
Language: Python - Size: 137 KB - Last synced at: 10 days ago - Pushed at: over 4 years ago - Stars: 77 - Forks: 17
twpayne/find-duplicates
Find duplicate files quickly.
Language: Go - Size: 82 KB - Last synced at: 8 days ago - Pushed at: 3 months ago - Stars: 57 - Forks: 1
src-d/gemini
Advanced similarity and duplicate source code at scale.
Language: Scala - Size: 7 MB - Last synced at: 17 days ago - Pushed at: almost 6 years ago - Stars: 55 - Forks: 16
deric/es-dedupe
Tool for removing duplicate documents from Elasticsearch
Language: Python - Size: 130 KB - Last synced at: 29 days ago - Pushed at: over 1 year ago - Stars: 54 - Forks: 22
src-d/apollo
Advanced similarity and duplicate source code proof of concept for our research efforts.
Language: Python - Size: 197 KB - Last synced at: 17 days ago - Pushed at: over 2 years ago - Stars: 52 - Forks: 17
shevchenkoartem/lastfm-smart-deduper
JS script that allows you to remove duplicates from your Last.fm scrobbles library.
Language: JavaScript - Size: 1.74 MB - Last synced at: almost 2 years ago - Pushed at: over 5 years ago - Stars: 48 - Forks: 2
LyonSyonII/akin
Rust crate for writing repetitive code easier and faster.
Language: Rust - Size: 51.8 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 43 - Forks: 2
eliasfloreteng/bitwarden_find_duplicates
Find duplicate logins based on domain, from Bitwarden export. Open source for your safety.
Language: HTML - Size: 34.2 KB - Last synced at: 6 months ago - Pushed at: 6 months ago - Stars: 34 - Forks: 8
Bartozzz/potential-duplicates-bot
A configurable GitHub App which checks for potential issue duplicates using Damerau–Levenshtein distance algorithm.
Language: JavaScript - Size: 28.3 KB - Last synced at: about 2 months ago - Pushed at: about 6 years ago - Stars: 31 - Forks: 6
guicamest/GDuplicate-Finder
GDuplicate Finder - A Groovy way to find duplicates among your computer and network shares!
Language: Groovy - Size: 49.7 MB - Last synced at: 9 days ago - Pushed at: 9 days ago - Stars: 30 - Forks: 7
Navid2zp/dups
A CLI tool to find/remove duplicate files supporting multi-core and different algorithms (MD5, SHA256, and XXHash).
Language: Go - Size: 62.5 KB - Last synced at: 7 days ago - Pushed at: about 1 year ago - Stars: 26 - Forks: 3
gacarrillor/AppendFeaturesToLayer
QGIS Processing plugin to add an algorithm for upserting features from a source vector layer to an existing target vector layer.
Language: Python - Size: 162 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 25 - Forks: 5
rasheedsulayman/DuplicateContactsRemover
📒 A simple app to optimize your address book and remove duplicate contacts.
Language: Kotlin - Size: 30.9 MB - Last synced at: 30 days ago - Pushed at: about 2 years ago - Stars: 22 - Forks: 10
arikw/outlook-duplicated-items-remover
A VBA script that finds and moves duplicated items in selected outlook folders
Language: VBA - Size: 151 KB - Last synced at: about 2 months ago - Pushed at: over 4 years ago - Stars: 22 - Forks: 2
caluml/finddups
Find duplicate files on your computer
Language: Java - Size: 431 KB - Last synced at: about 2 years ago - Pushed at: almost 5 years ago - Stars: 22 - Forks: 5
rustomax/ndf
Duplicate file finder written in Nim
Language: Nim - Size: 22.5 KB - Last synced at: about 8 hours ago - Pushed at: over 4 years ago - Stars: 20 - Forks: 0
mkearney/funique
⌚️ A faster unique() function
Language: R - Size: 7.15 MB - Last synced at: about 1 month ago - Pushed at: over 6 years ago - Stars: 19 - Forks: 0
vladsmirnov/url-rewrites
Magento 1.x module to target the URL Rewrite issue
Language: PHP - Size: 23.4 KB - Last synced at: 10 months ago - Pushed at: over 7 years ago - Stars: 19 - Forks: 0
raspi/duplikaatti
Remove duplicate files.
Language: Go - Size: 33.2 KB - Last synced at: 26 days ago - Pushed at: over 3 years ago - Stars: 18 - Forks: 1
lmammino/indexed-string-variation
Experimental JavaScript module to generate all possible variations of strings over an alphabet using an n-ary virtual tree
Language: JavaScript - Size: 52.7 KB - Last synced at: 16 days ago - Pushed at: over 7 years ago - Stars: 18 - Forks: 4
kevinpollet/pocket-deduper
Remove duplicates from your Pocket list.
Language: Go - Size: 6.55 MB - Last synced at: about 1 month ago - Pushed at: over 3 years ago - Stars: 17 - Forks: 0
cemahseri/Duplica
A very fast duplicate file finder.
Language: C# - Size: 25.4 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 14 - Forks: 4
DeaDSouL/dugu 📦
Find, remove and avoid duplicates with dugu: The Duplicates Guru
Language: Python - Size: 85.9 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 14 - Forks: 4
rmdm/nodups
No dups, no doubts
Language: JavaScript - Size: 2.24 MB - Last synced at: 4 days ago - Pushed at: almost 5 years ago - Stars: 14 - Forks: 0
kotharan/LeetCode_Solutions
Solutions of LeetCode interview questions
Language: C++ - Size: 2.36 MB - Last synced at: almost 2 years ago - Pushed at: over 6 years ago - Stars: 14 - Forks: 5
rsalmei/refine
Refine your file collections using Rust!
Language: Rust - Size: 628 KB - Last synced at: 7 days ago - Pushed at: 13 days ago - Stars: 13 - Forks: 0
NicolasBizzozzero/dupe_eraser
A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.
Language: Python - Size: 54.7 KB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 13 - Forks: 0
7room/aya
Disk Usage Analyzer & Duplicate File Finder
Size: 71.3 KB - Last synced at: 10 days ago - Pushed at: over 2 years ago - Stars: 12 - Forks: 0
DragonOfMath/dupe-images
Node.js package for finding and removing duplicate image files with extreme precision
Language: JavaScript - Size: 17.6 KB - Last synced at: 11 months ago - Pushed at: almost 7 years ago - Stars: 11 - Forks: 3
danielpclark/dfm
Duplicate File Manager
Language: Ruby - Size: 324 KB - Last synced at: about 1 month ago - Pushed at: over 8 years ago - Stars: 11 - Forks: 1
eh2k/fs-inspect
FS-Inspect is an easy to use tool designed to give you an overview about your files and directories (Disk Usage).
Language: C - Size: 1.94 MB - Last synced at: about 2 years ago - Pushed at: about 10 years ago - Stars: 11 - Forks: 5
prowide/prowide-integrator-examples
Source code examples for "Prowide Integrator"
Language: Java - Size: 8.55 MB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 10 - Forks: 17
mesqueeb/compare-anything
Compares objects and tells you which props are duplicate, and props are only present once.
Language: TypeScript - Size: 666 KB - Last synced at: 9 days ago - Pushed at: 3 months ago - Stars: 10 - Forks: 0
thomas694/finddupe
Enhanced version of finddupe, a duplicate file detector for Windows
Language: C - Size: 204 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 9 - Forks: 0
carlbeech/fast-duplicate-finder
A python program to locate duplicate files - and do it fast
Language: Python - Size: 74.4 MB - Last synced at: about 1 month ago - Pushed at: almost 4 years ago - Stars: 9 - Forks: 3
rix4uni/unew
A tool combined of 2 commands features in 1 sort and tee for adding new lines to files, skipping duplicates
Language: Go - Size: 49.8 KB - Last synced at: about 1 month ago - Pushed at: 4 months ago - Stars: 8 - Forks: 1
raspi/samanlainen
Delete duplicate files
Language: Rust - Size: 54.7 KB - Last synced at: about 1 month ago - Pushed at: about 2 years ago - Stars: 8 - Forks: 1
EllangoK/duplicate-image-remover
Uses SSIM and MSE to get rid of duplicates and near duplicates
Language: Python - Size: 21.5 KB - Last synced at: almost 2 years ago - Pushed at: almost 5 years ago - Stars: 8 - Forks: 2
CedricReichenbach/audiomerge
Merge multiple scattered music collections into one, taking only the best version of duplicates
Language: Java - Size: 33.5 MB - Last synced at: about 2 years ago - Pushed at: about 8 years ago - Stars: 8 - Forks: 1
artemanufrij/findfileconflicts
An elementary OS app
Language: Vala - Size: 6.87 MB - Last synced at: about 1 month ago - Pushed at: over 4 years ago - Stars: 7 - Forks: 5
redonkulus/dump-deps
Dump NPM package dependencies to display packages with multiple versions.
Language: JavaScript - Size: 15.6 KB - Last synced at: 9 days ago - Pushed at: almost 5 years ago - Stars: 7 - Forks: 0
Dmitriy-Vas/go-file-copies
A Go program to get duplicates from specified paths.
Language: Go - Size: 10.7 KB - Last synced at: about 1 year ago - Pushed at: over 5 years ago - Stars: 7 - Forks: 0
zx8754/StackOverflowNotes
SO [r] tag list of dupes, help, links, etc.
Size: 37.1 KB - Last synced at: almost 2 years ago - Pushed at: about 6 years ago - Stars: 7 - Forks: 3
ant-js/compare-similarity
👁 Compare the similarity of two strings
Language: TypeScript - Size: 12.7 KB - Last synced at: 4 days ago - Pushed at: over 2 years ago - Stars: 6 - Forks: 1
innovatrics/dedubcheck
dedubcheck - De-Duplicate Dependency Checker for Node.js monorepos
Language: JavaScript - Size: 29.3 KB - Last synced at: about 1 month ago - Pushed at: almost 6 years ago - Stars: 6 - Forks: 1
TSunny007/Document-Similarity
Using Jaccard-Similarity and Minhashing to determine similarity between two text documents
Language: Jupyter Notebook - Size: 26.4 KB - Last synced at: about 2 years ago - Pushed at: about 7 years ago - Stars: 6 - Forks: 3
tasleson/duplihere
Copy & Paste finder for structured text files.
Language: Rust - Size: 104 KB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 5 - Forks: 1
LibreTranslate/RemoveDup
Remove duplicates from parallel corpora
Language: Python - Size: 835 KB - Last synced at: 27 days ago - Pushed at: over 1 year ago - Stars: 5 - Forks: 0
arasgungore/job-posting-duplicate-detection
A project aiming to leverage text embeddings and Milvus, a high-performance vector search engine, to detect duplicate job postings.
Language: Python - Size: 289 KB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 5 - Forks: 0
nav9/duplicateFileFinder
This program finds duplicate files in a folder and its subfolders. Duplicates are moved to a separate folder. A few other modes of operation are also planned/available.
Language: Python - Size: 104 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 5 - Forks: 1
pcraciunoiu/AndroidSMSBackupRestoreCleaner Fork of NumbGnat/AndroidSMSBackupRestoreCleaner
This cleans up duplicate SMS entries in a backup created by SMS Backup & Restore Android app.
Language: Python - Size: 4.78 MB - Last synced at: about 1 year ago - Pushed at: about 2 years ago - Stars: 5 - Forks: 2
SuryaMaulana/kitabisa.dev 📦
Kitabisa.com DUPLICATE.
Language: PHP - Size: 53 MB - Last synced at: 7 months ago - Pushed at: about 5 years ago - Stars: 5 - Forks: 0
carlnewton/line-duplicates
Quickly review or remove duplicate lines from a body of text
Language: HTML - Size: 2.93 KB - Last synced at: about 2 years ago - Pushed at: over 4 years ago - Stars: 4 - Forks: 1
KeyWeeUsr/Bear
:bear: The decluttering deduplicator
Language: Python - Size: 135 KB - Last synced at: 29 days ago - Pushed at: over 5 years ago - Stars: 4 - Forks: 1
PythonCoderAS/DuplicateBot 📦
Bot for duplicates
Language: Python - Size: 38.1 KB - Last synced at: 3 months ago - Pushed at: about 7 years ago - Stars: 4 - Forks: 2
tjcafferkey/removeduplicates
Function that removes duplicate items and objects based on a key from an array of objects.
Language: JavaScript - Size: 4.88 KB - Last synced at: 15 days ago - Pushed at: almost 8 years ago - Stars: 4 - Forks: 0
qwertz19281/dupion
Duplicate file/folder finder, can also scan in archives, HDD optimized
Language: Rust - Size: 365 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 3 - Forks: 1
GreenComfyTea/duplicate-emote-check-tool
Check if you have duplicate emotes across FrankerFaceZ, BetterTTV and 7TV for Twitch.
Language: JavaScript - Size: 414 KB - Last synced at: about 1 year ago - Pushed at: over 1 year ago - Stars: 3 - Forks: 0
ajmalshahabudeen/Bitwarden-Duplicate-remover
When Importing multiple CSV files Bitwarden creates Duplicate Entries. So this Python script will remove duplicate entries and keep ONE.
Language: Python - Size: 2.93 KB - Last synced at: over 1 year ago - Pushed at: about 2 years ago - Stars: 3 - Forks: 2
marinebox/tab-killer
A Chrome extension. Close all duplicate tabs.
Language: JavaScript - Size: 738 KB - Last synced at: about 2 years ago - Pushed at: about 3 years ago - Stars: 3 - Forks: 1
molevol-ub/BacterialDuplicates
Identification of putative duplicated genes among bacterial genomes
Language: Perl - Size: 2.22 MB - Last synced at: almost 2 years ago - Pushed at: almost 5 years ago - Stars: 3 - Forks: 1
codecliff/FdupesAnalyzer
A script to analyze output of fdupes linux utility to find level of overlap between directories. Written in R
Language: R - Size: 237 KB - Last synced at: 3 months ago - Pushed at: over 5 years ago - Stars: 3 - Forks: 1
Supporterino/TextAnalyzer
Language: Python - Size: 83 KB - Last synced at: 3 days ago - Pushed at: over 5 years ago - Stars: 3 - Forks: 0
RomaniukVadim/Ninety-Nine-Erlang-Problems
Ninety-Nine Prolog Problems - Erlang edition
Language: Erlang - Size: 24.2 MB - Last synced at: about 1 year ago - Pushed at: almost 7 years ago - Stars: 3 - Forks: 0
r-darwish/dupfiles-cpp
Find duplicate files
Language: C++ - Size: 47.9 KB - Last synced at: about 2 months ago - Pushed at: over 7 years ago - Stars: 3 - Forks: 0
vuolter/deplicate-cli
Command Line Interface for deplicate
Language: Python - Size: 37.1 KB - Last synced at: 5 days ago - Pushed at: over 7 years ago - Stars: 3 - Forks: 1
nbari/backup
Command line tool for creating encrypted backups avoiding duplicates
Language: Rust - Size: 75.2 KB - Last synced at: 15 days ago - Pushed at: about 2 months ago - Stars: 2 - Forks: 1
softonus-io/prettier-plugin-duplicate-remover
A Prettier plugin that removes duplicate class names in class and className attributes, ensuring cleaner, more efficient code in frontend projects like React, Vue.js, and Angular.
Language: JavaScript - Size: 11.7 KB - Last synced at: 19 days ago - Pushed at: 6 months ago - Stars: 2 - Forks: 1
webis-de/sigir20-sampling-bias-due-to-near-duplicates-in-learning-to-rank
Sampling Bias Due to Near-Duplicates in Learning to Rank
Language: Kotlin - Size: 51.3 MB - Last synced at: about 1 month ago - Pushed at: 7 months ago - Stars: 2 - Forks: 2
davidefiocco/dockerized-elasticsearch-duplicate-finder
Attempt to use MinHash to find duplicates in an Elasticsearch index
Language: Python - Size: 11.7 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0
pouyakary/dup
a tiny and fast command line utility to find the duplicate files within a directory
Language: Go - Size: 16.6 KB - Last synced at: 8 days ago - Pushed at: over 1 year ago - Stars: 2 - Forks: 0
Arkadiy-Garber/ParaHunter
Identification of gene paralogs in genomes, and calculation of dS and dN/dS values for paralogous gene pairs
Language: Python - Size: 73.2 KB - Last synced at: about 2 years ago - Pushed at: about 2 years ago - Stars: 2 - Forks: 0
MarciaBM/Sorting_Memories
Language: Java - Size: 139 MB - Last synced at: about 2 years ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 1
0aub/duplicates-cleaner
simple script to delete duplicated files from specific directory
Language: Python - Size: 8.79 KB - Last synced at: almost 2 years ago - Pushed at: almost 3 years ago - Stars: 2 - Forks: 0
cobanov/easy-duplicate
Compares the files in a folder with md5 checksums and deletes duplicate files or moves them to the desired folder.
Language: Python - Size: 2.93 KB - Last synced at: about 2 years ago - Pushed at: almost 3 years ago - Stars: 2 - Forks: 0
McKael/goduf
A simple (but fast) duplicate file finder written in Go [Mirror repository]
Language: Go - Size: 35.2 KB - Last synced at: about 2 months ago - Pushed at: about 3 years ago - Stars: 2 - Forks: 0
TheArkive/ConstScanner
C/C++ Constant Scanner - includes lists of constants from groups of headers. Check the docs for the repo that lists several Win10 APIs.
Language: AutoHotkey - Size: 22.5 MB - Last synced at: 10 months ago - Pushed at: over 3 years ago - Stars: 2 - Forks: 1
PercentEquals/SIDF
Simple Image Duplicate Finder
Language: C# - Size: 230 KB - Last synced at: almost 2 years ago - Pushed at: almost 4 years ago - Stars: 2 - Forks: 1
bkb3/duplicate-bib-fix
Small python script to check and replace duplicated bib entries in your .tex files
Language: TeX - Size: 358 KB - Last synced at: 2 months ago - Pushed at: about 4 years ago - Stars: 2 - Forks: 0
konstantin89/swift-duplicate-images-finder
Application that finds duplicate images.
Language: Python - Size: 598 KB - Last synced at: about 2 years ago - Pushed at: about 4 years ago - Stars: 2 - Forks: 0
JordiCorbilla/DuplicateChecker
☑️ .Net service that allows you to check duplicate rows on a sql table using Levenshtein distance
Language: C# - Size: 53.7 KB - Last synced at: about 1 month ago - Pushed at: over 4 years ago - Stars: 2 - Forks: 1
betaWeb/twicejs
Manage duplicates, count item occurences, dedupe an Array.
Language: JavaScript - Size: 349 KB - Last synced at: 2 months ago - Pushed at: almost 5 years ago - Stars: 2 - Forks: 0
Erdk/imgdedup
Show possible duplicates of photos/images.
Language: Go - Size: 26.4 KB - Last synced at: 4 months ago - Pushed at: about 5 years ago - Stars: 2 - Forks: 0
