GitHub topics: duplicate-files
twpayne/find-duplicates
Find duplicate files quickly.
Language: Go - Size: 82 KB - Last synced at: 3 days ago - Pushed at: about 2 months ago - Stars: 56 - Forks: 1
scrubbbbs/cbird
Command-line program for Content-Based Image Retrieval of images and videos. Includes tools for general search and de-duplication.
Language: C++ - Size: 12.7 MB - Last synced at: 4 days ago - Pushed at: 5 days ago - Stars: 118 - Forks: 5
thushan/smash
Smash through to find duplicate files super fast by slicing files intelligently!
Language: Go - Size: 7.71 MB - Last synced at: 5 days ago - Pushed at: 6 days ago - Stars: 16 - Forks: 0
diskoverdata/diskover-community
Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch
Language: PHP - Size: 33.6 MB - Last synced at: 11 days ago - Pushed at: 12 days ago - Stars: 1,554 - Forks: 166
pierrre/file-duplicate
Find duplicate files
Language: Go - Size: 241 KB - Last synced at: 14 days ago - Pushed at: 14 days ago - Stars: 1 - Forks: 0
sreedevk/deduplicator
Filter, Sort & Delete Duplicate Files Recursively
Language: Rust - Size: 334 KB - Last synced at: 15 days ago - Pushed at: 10 months ago - Stars: 326 - Forks: 16
DeaDSouL/dugu 📦
Find, remove and avoid duplicates with dugu: The Duplicates Guru
Language: Python - Size: 85.9 KB - Last synced at: 17 days ago - Pushed at: over 1 year ago - Stars: 14 - Forks: 4
harshasrisri/dedup
Remove local files that are duplicates of files in another path
Language: Rust - Size: 97.7 KB - Last synced at: 4 days ago - Pushed at: 22 days ago - Stars: 1 - Forks: 0
m-manu/go-find-duplicates
Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.
Language: Go - Size: 59.6 KB - Last synced at: 10 days ago - Pushed at: 2 months ago - Stars: 280 - Forks: 24
LpCodes/Duplicate-Files-Remover
This Python script removes duplicate files from a given directory. It prompts the user to select a folder and then uses the MD5 hashing algorithm to check if any files have the same hash value. If it finds duplicate files, it deletes all but one of them.
Language: Python - Size: 6.84 KB - Last synced at: 21 days ago - Pushed at: over 1 year ago - Stars: 5 - Forks: 0
milkmansson/plex-seeDuplicates
Find duplicates in your Plex library.
Language: Python - Size: 23.4 KB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 0 - Forks: 0
opencoff/go-progs
useful golang utilities for Unixish environments
Language: Go - Size: 51.8 KB - Last synced at: 28 days ago - Pushed at: 28 days ago - Stars: 3 - Forks: 0
Chinory/lndup
Hardlink duplicate files! 🤓
Language: JavaScript - Size: 163 KB - Last synced at: 9 days ago - Pushed at: almost 6 years ago - Stars: 3 - Forks: 0
Paulogb98/DupFile-Analyzer
Ferramenta de linha de comando em Rust para detectar arquivos duplicados em diretórios e subdiretórios. Utilizando hash SHA-256 e processamento paralelo com Rayon, garante alta performance e precisão na identificação de arquivos duplicados, independentemente de seus metadados, como nome etc.
Language: Rust - Size: 0 Bytes - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0
elmindreda/duff
Command-line utility for finding duplicate files
Language: C - Size: 241 KB - Last synced at: 20 days ago - Pushed at: about 1 year ago - Stars: 94 - Forks: 12
jurassicLizard/files-deduplicator
A command-line tool that identifies and removes duplicate files recursively from a specified folder. It uses cryptographic hashes (Blake2) for reliable file comparisons to ensure precise duplicate detection.
Language: C++ - Size: 46.9 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0
shafirahmad/pydeduper
Duplicate file finder - with % duplication of folders
Language: Python - Size: 30.3 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 1 - Forks: 0
waived/duplicate-remover
Windows utility that checks folder and sub-folder(s) for all files that contain identical MD5 hash and prep them for deletion
Size: 0 Bytes - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0
Vatshayan/Data-Duplication-Removal-using-Machine-Learning
Final Year Project as Deletion of Duplicated data using Machine learning project with source code and Report.
Language: Jupyter Notebook - Size: 37.1 KB - Last synced at: 14 days ago - Pushed at: over 2 years ago - Stars: 44 - Forks: 6
akbarisajjad/Scanner-file-windows-
This code is a useful tool for checking for duplicate and corrupted files and can be very useful for managing files in a large directory. However, it could be made more robust and reliable by improving error handling and performance optimization.
Language: Python - Size: 19.5 KB - Last synced at: about 1 month ago - Pushed at: about 1 month ago - Stars: 0 - Forks: 0
vuolter/deplicate 📦
Advanced Duplicate File Finder for Python
Language: Python - Size: 137 KB - Last synced at: 12 days ago - Pushed at: over 4 years ago - Stars: 77 - Forks: 16
gechandesu/fdup
File duplicates finder
Language: V - Size: 17.6 KB - Last synced at: about 1 month ago - Pushed at: 2 months ago - Stars: 7 - Forks: 0
plecos/dupefindr
A file duplicate detector utility written in Rust
Language: Rust - Size: 99.6 KB - Last synced at: about 2 months ago - Pushed at: about 2 months ago - Stars: 0 - Forks: 0
theohbrothers/Get-DuplicateItem
Gets duplicate or non-duplicate files.
Language: PowerShell - Size: 91.8 KB - Last synced at: 10 days ago - Pushed at: 2 months ago - Stars: 2 - Forks: 2
magnetikonline/identix
Python utility which will recursively scan one or more given directories for duplicate files.
Language: Python - Size: 48.8 KB - Last synced at: 16 days ago - Pushed at: 2 months ago - Stars: 4 - Forks: 2
Junsious/DupFinder
A simple desktop application to search for duplicate files in a specified directory. This application uses SHA-256 hashing to identify duplicates and provides a user-friendly interface with progress tracking.
Language: Rust - Size: 43 KB - Last synced at: about 2 months ago - Pushed at: 6 months ago - Stars: 2 - Forks: 0
germabyte/obsidian-deduper
The Obsidian Duplicate Finder is a user-friendly tool designed to help users manage duplicate files within their Obsidian vaults. It identifies similar Markdown files based on their content, making it easier to organize and declutter note-taking environments.
Language: Python - Size: 11.7 KB - Last synced at: 17 days ago - Pushed at: 2 months ago - Stars: 4 - Forks: 0
Robb-Fr/fast-dupes-finder
This repository proposes clean, fast and shell based scripts for identifying finding duplicate files in a folder.
Language: Shell - Size: 17.6 KB - Last synced at: 2 months ago - Pushed at: 2 months ago - Stars: 0 - Forks: 0
raspi/duplikaatti
Remove duplicate files.
Language: Go - Size: 33.2 KB - Last synced at: 18 days ago - Pushed at: over 3 years ago - Stars: 17 - Forks: 1
Idrinth/duplication
A backup of backups solution for among other things s3 storages
Language: PHP - Size: 125 KB - Last synced at: 23 days ago - Pushed at: 3 months ago - Stars: 2 - Forks: 1
raspi/samanlainen
Delete duplicate files
Language: Rust - Size: 54.7 KB - Last synced at: 7 days ago - Pushed at: about 2 years ago - Stars: 8 - Forks: 1
Jim-JMCD/DuplicateFF
A small Linux app that compares files using sha256. Input can be one or more directories with optional filters of maximum files size and parts of file names (e.g. ‘.mp4’ ‘IMG’ ) . Output is multiple CSV (spreadsheet) reports that can be used to move or delete duplicates. Can be used on Linux, Windows (WSL2, MSYS2 and Gitbash).
Size: 25.4 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 1 - Forks: 0
Jim-JMCD/Duplicate-File-Finder
A bash script compares files using sha256. Input can be one or more directories with optional filters of maximum files size and parts of file names (e.g. ‘.mp4’ ‘IMG’ ) . Output is multiple CSV (spreadsheet) reports that can be used to move or delete duplicates. Can be used on Linux, Windows (WSL2, MSYS2 and Gitbash).
Language: Shell - Size: 47.9 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 6 - Forks: 0
NicolasBizzozzero/dupe_eraser
A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.
Language: Python - Size: 54.7 KB - Last synced at: 10 days ago - Pushed at: 3 months ago - Stars: 13 - Forks: 0
visiuun/Folder-duplicates-deleter
Directory Bulk Duplicate Files Deleter written in python.
Language: Python - Size: 5.86 KB - Last synced at: 3 months ago - Pushed at: 3 months ago - Stars: 0 - Forks: 0
unhammer/wshiml
📚 Word shingling for near duplicate document detection
Language: OCaml - Size: 29.3 KB - Last synced at: 2 months ago - Pushed at: almost 8 years ago - Stars: 4 - Forks: 1
ciphermike/tidypics
a tool to clean duplicate pictures
Size: 0 Bytes - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0
px86/dff 📦
Recursively finds and lists duplicate files in a directory. Uses C++17 filesystem library and SHA-256 hashing algorithm.
Language: C++ - Size: 17.6 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0
vuolter/deplicate-cli
Command Line Interface for deplicate
Language: Python - Size: 37.1 KB - Last synced at: 17 days ago - Pushed at: over 7 years ago - Stars: 3 - Forks: 1
FreddyFunk/ddk
DeDuplicationKit: Advanced File Storage Deduplication
Language: C++ - Size: 192 KB - Last synced at: 4 months ago - Pushed at: 4 months ago - Stars: 0 - Forks: 0
umstek/DupKiller
Slow, but more reliable duplicate files cleaner.
Language: C# - Size: 372 KB - Last synced at: about 1 month ago - Pushed at: about 4 years ago - Stars: 2 - Forks: 1
onder7/DuplicateFinder
Yinelenen Dosya Bulucu
Language: Python - Size: 9.77 KB - Last synced at: 18 days ago - Pushed at: 5 months ago - Stars: 0 - Forks: 0
guofei9987/fuzzy-hash
fuzzy hash in python (from ssdeep)
Language: C - Size: 24.4 KB - Last synced at: 11 days ago - Pushed at: almost 2 years ago - Stars: 5 - Forks: 0
NoCLin/duplicate-scanner
Ultra Fast Duplicate File Scanner Based on Everything (Windows and NTFS only)
Language: Python - Size: 104 KB - Last synced at: 17 days ago - Pushed at: about 5 years ago - Stars: 5 - Forks: 2
VISWESWARAN1998/CyberGod-KSGMPRH
An open-source antivirus for windows
Language: C - Size: 42.3 MB - Last synced at: 18 days ago - Pushed at: over 5 years ago - Stars: 45 - Forks: 22
FabriceSalvaire/filewalker
A Python library to scan a file system, find duplicated file etc.
Language: Python - Size: 123 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 1
feishu0110/DupFileCleaner
为了有效地管理和清理重复文件，我用Cursor Ai开发了一款文件去重工具，利用多线程对比哈希值实现了高效的文件去重功能。
Language: TeX - Size: 13.5 MB - Last synced at: 7 months ago - Pushed at: 7 months ago - Stars: 0 - Forks: 0
fffaraz/QtHashSum
File Checksum Integrity Verifier & Duplicate File Finder written in C++ Qt
Language: C++ - Size: 210 KB - Last synced at: 12 days ago - Pushed at: 11 months ago - Stars: 45 - Forks: 10
cbismuth/fdupes-java
Advanced file duplicate finder with native MD5 support
Language: Java - Size: 248 KB - Last synced at: 8 months ago - Pushed at: over 7 years ago - Stars: 0 - Forks: 0
shivakar/gdupes
A multithreaded tool for identifying duplicate files
Language: Go - Size: 22.5 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 1 - Forks: 0
ruester/midnightdup
MidnightDup - Duplicate File Finder
Language: Perl - Size: 45.2 MB - Last synced at: 8 days ago - Pushed at: 8 months ago - Stars: 0 - Forks: 0
hubertdungen/Duplicate_Deleter-EK-Script
Windows Batch script to selectively delete files within a directory, retaining only those with a specified extension. Ideal for organizing files and removing unwanted duplicates by extension.
Language: Batchfile - Size: 4.88 KB - Last synced at: 8 months ago - Pushed at: 8 months ago - Stars: 1 - Forks: 0
jeff-99/hashdex
A file indexer based on content hashes to quickly find duplicate files on your system.
Language: Python - Size: 106 KB - Last synced at: 3 days ago - Pushed at: over 4 years ago - Stars: 6 - Forks: 2
lvntky/noditto
Noditto: AST Based Code Duplication Finder
Language: CMake - Size: 55.7 KB - Last synced at: 19 days ago - Pushed at: 9 months ago - Stars: 2 - Forks: 0
matteopic/hashdir
A fast file hashing tool designed to find duplicate files by performing checksums and indexing them.
Language: Go - Size: 4.88 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0
kingzcheung/bdzer
A Small Tool for Finding Duplicate Files
Language: Rust - Size: 12.7 KB - Last synced at: 9 months ago - Pushed at: 9 months ago - Stars: 0 - Forks: 0
cemahseri/Duplica
A very fast duplicate file finder.
Language: C# - Size: 25.4 KB - Last synced at: 10 months ago - Pushed at: 10 months ago - Stars: 14 - Forks: 4
cwkingjr/find_duplicate_files
Find duplicate files on your system using inclusion and exclusion folder lists.
Language: Go - Size: 4.88 KB - Last synced at: over 1 year ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0
niradar/duplicate_files_in_folders
Identifies and processes duplicate files between a source and target directory.
Language: Python - Size: 104 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0
rustomax/ndf
Duplicate file finder written in Nim
Language: Nim - Size: 22.5 KB - Last synced at: 5 days ago - Pushed at: about 4 years ago - Stars: 20 - Forks: 0
PJDude/dude
Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.
Language: Python - Size: 4.71 MB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 57 - Forks: 6
israel-lugo/capidup
Quickly find duplicate files in directories
Language: Python - Size: 129 KB - Last synced at: 12 days ago - Pushed at: over 7 years ago - Stars: 4 - Forks: 0
ltworf/pulputils 📦
not coreutils, pulputils
Size: 119 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 0 - Forks: 0
dfirsec/dup_file_finder
Search for duplicate files based on extension.
Language: Python - Size: 35.2 KB - Last synced at: 12 months ago - Pushed at: 12 months ago - Stars: 0 - Forks: 0
by321/Find-Duplicate-Files
Find Duplicate Files with Simple GUI
Language: Python - Size: 5.86 KB - Last synced at: 12 months ago - Pushed at: almost 2 years ago - Stars: 0 - Forks: 0
PostsDesert/copyfolderhierarchy
Copy Folder Hierarchy and Contents for Google Drive
Language: JavaScript - Size: 40 KB - Last synced at: 9 months ago - Pushed at: about 4 years ago - Stars: 2 - Forks: 0
hekmon/deduper
Analyse 2 paths to found identical files and hard link them to save space
Language: Go - Size: 151 KB - Last synced at: about 2 months ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0
jvirkki/dupd
CLI utility to find duplicate files
Language: C - Size: 1.9 MB - Last synced at: 12 months ago - Pushed at: almost 2 years ago - Stars: 109 - Forks: 16
masgari/dup-fu 📦
Terminal application for finding duplicate files
Language: Go - Size: 281 KB - Last synced at: about 1 year ago - Pushed at: over 6 years ago - Stars: 0 - Forks: 0
JakubJanowski/File-Duplicate-Finder
A small desktop application to help you organize disk space and backup folders by searching duplicated files by their content.
Language: C# - Size: 8.26 MB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 1
QuietWindUponTheMoor/Quiets-Duplicate-Manager
Quiet's Duplicate Manager is an Electron.js-based desktop application that is currently in the works. It will offer a range of basic features like including any/all files that are duplicates, giving the option to choose whether to delete or archive duplicates, etc. As more features roll out, more will be added here.
Language: JavaScript - Size: 93.8 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0
impoteto/goduper
rather fast duplicate file finder written in go
Language: Go - Size: 14.6 KB - Last synced at: 10 months ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
nagracks/py_duplicates
Find duplicates
Language: Python - Size: 16.6 KB - Last synced at: about 1 year ago - Pushed at: about 8 years ago - Stars: 4 - Forks: 3
pouyakary/dup
a tiny and fast command line utility to find the duplicate files within a directory
Language: Go - Size: 16.6 KB - Last synced at: 2 months ago - Pushed at: about 1 year ago - Stars: 2 - Forks: 0
francois-le-ko4la/duplicate-file-finder
A duplicate file finder.
Language: Python - Size: 78.1 KB - Last synced at: about 1 month ago - Pushed at: about 2 years ago - Stars: 1 - Forks: 0
n0obit4/SFind
Un programa para buscar archivos duplicados
Language: Shell - Size: 182 KB - Last synced at: about 1 year ago - Pushed at: over 3 years ago - Stars: 1 - Forks: 0
pixolution/dupe-scanner
Detect duplicate images locally
Language: Python - Size: 130 KB - Last synced at: about 1 year ago - Pushed at: about 1 year ago - Stars: 1 - Forks: 0
NIR3X/hardlinkdeduplicator
HardLink Deduplicator - Detect and Manage Duplicate Files with Hard Links
Language: Go - Size: 24.4 KB - Last synced at: 2 days ago - Pushed at: about 1 year ago - Stars: 0 - Forks: 0
nogibjj/pyli
Deduper
Language: Python - Size: 265 KB - Last synced at: 7 days ago - Pushed at: almost 8 years ago - Stars: 5 - Forks: 3
dipietro-salvatore/liten3 Fork of VarnaLab/liten3
Liten3 searches in a given directory the duplicate files, producing a report at the end of its execution.
Language: Python - Size: 58.6 KB - Last synced at: over 1 year ago - Pushed at: about 7 years ago - Stars: 0 - Forks: 1
Gartham/file-duplicate-checker
Program to scan and search for file duplicates. (~300MB/s)
Language: Java - Size: 23.4 KB - Last synced at: about 1 year ago - Pushed at: over 2 years ago - Stars: 9 - Forks: 3
tyler-tee/File-Deduplicator
Python app built to scan a directory, check for duplicate files, and send them to the trash.
Language: Python - Size: 5.86 KB - Last synced at: about 2 months ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 1
NattyXO/Mega-Copy
This Windows Forms application allows users to copy, cut, and duplicate files from one location to another.
Language: C# - Size: 3.32 MB - Last synced at: 2 months ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
elcorto/findsame
Find duplicate files and directories based on file hashes.
Language: Python - Size: 249 KB - Last synced at: 8 days ago - Pushed at: over 3 years ago - Stars: 5 - Forks: 0
mamaorha/compare-files
scanning for duplicate files by content
Language: Java - Size: 46.9 KB - Last synced at: over 1 year ago - Pushed at: over 2 years ago - Stars: 2 - Forks: 0
EmreCanKURAN/DuplicateFileRemover
Removes the duplicate files.
Language: Python - Size: 4.88 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 0 - Forks: 1
knobs-dials/duppy
Duplicate file detector. Incremental-block style.
Language: Python - Size: 137 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 4 - Forks: 1
davdiv/hashfolder
Simple command line tool that can create/update an sqlite database that contains the hash (by default SHA256) of all files inside a specified root folder.
Language: TypeScript - Size: 80.1 KB - Last synced at: 6 days ago - Pushed at: over 1 year ago - Stars: 1 - Forks: 0
neatdecisions/detwinner
Detwinner - duplicate file finder for the Linux desktop
Language: C++ - Size: 535 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 30 - Forks: 6
AlexZasorin/delete-duplicates.py
Python script to find, filter, and delete duplicate files. Work in progress.
Language: Python - Size: 3.91 KB - Last synced at: over 1 year ago - Pushed at: almost 3 years ago - Stars: 0 - Forks: 0
MK-Ware/pyCleaner
Securely wipe files or folders and clean duplicated files
Language: Python - Size: 31.3 KB - Last synced at: over 1 year ago - Pushed at: about 7 years ago - Stars: 8 - Forks: 1
jer-irl/ffdup
freaking fast duplicate file detection
Language: C - Size: 5.86 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
LoneStamp99/Foundouble-Mirror
Struggling to find duplicate images, try using this script.
Language: Python - Size: 296 KB - Last synced at: over 1 year ago - Pushed at: over 1 year ago - Stars: 0 - Forks: 0
giosali/dupeutil
A command-line program written in Python for detecting and removing duplicate files.
Language: Python - Size: 60.5 KB - Last synced at: about 23 hours ago - Pushed at: about 3 years ago - Stars: 0 - Forks: 0
thomasmfish/Duplicate-File-Checker
A simple Python tool to help identify and delete duplicate files that may be distributed within subdirectories.
Language: Python - Size: 11.7 KB - Last synced at: about 2 months ago - Pushed at: almost 4 years ago - Stars: 0 - Forks: 1
cbergoon/fdup
File and Directory Duplication Finder Utility
Language: Go - Size: 1.3 MB - Last synced at: 10 months ago - Pushed at: about 5 years ago - Stars: 0 - Forks: 0
oscarsun72/delete_duplicate_files_from_the_source_directory
檔案總管汰重-WindowsFormsApplication1 delete duplicate files from the source directory
Language: C# - Size: 105 KB - Last synced at: about 1 month ago - Pushed at: almost 4 years ago - Stars: 1 - Forks: 0
Demez/duplicate_file_finder
Language: Python - Size: 18.6 KB - Last synced at: about 1 year ago - Pushed at: about 5 years ago - Stars: 0 - Forks: 0
KowalskiThomas/Python-DuplicatesFinder
A useful tool to find duplicate files in a filesystem and to easily apply "fixes" on that. Will be translated to English soon.
Language: Python - Size: 16.6 KB - Last synced at: 4 days ago - Pushed at: over 5 years ago - Stars: 0 - Forks: 0
dahead/DupeFiles
Dupe Files scans your disks for duplicate files.
Language: C# - Size: 673 KB - Last synced at: about 1 month ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0
