<div dir="ltr"><div>David Hamm posted to this list about three years ago:<br>If you are trying to track down all the duplicate files in each clients directory I'd suggest fdupes.<br>Fdupes will search a directory tree for duplicates by generating an MD5 sum of<br>
the contents of each file and offers options for dealing with duplicates. It<br>will produce a list, or even remove any duplicates and create links to the<br>remaining file while preserving the file name.<br><br></div>I have been using fdupes to llist duplicates to a file and I usually delete the diplicate with the longest path.<br>
</div><div class="gmail_extra"><br clear="all"><div><div dir="ltr">...John<div><br></div></div></div>
<br><br><div class="gmail_quote">On Fri, Oct 18, 2013 at 12:59 PM, JD <span dir="ltr"><<a href="mailto:jdp@algoloma.com" target="_blank">jdp@algoloma.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Slashdot had a question about this 1-2 yrs ago. Lots of people suggested<br>
scripting it, others pointed out some C code on sourceforge.<br>
<br>
I had a few hrs free that day and wrote some Perl (200+ LOC). Use it all the<br>
time, but I'd probably go with the C tool for any very large datasets. Mine<br>
doesn't automaticly remove anything and is far from perfect, that is certain.<br>
It is relatively fast on most types of files, however.<br>
<div class="HOEnZb"><div class="h5"><br>
On 10/18/2013 12:34 PM, Calvin Harrigan wrote:<br>
> Good Afternoon,<br>
> I'm looking for a little advice/recommendation on file de-duplication<br>
> software. I've have a disk filled with files that most certainly have<br>
> duplicates. What's the best way to get rid of the duplicates. I'd like to<br>
> check deeper than just file name/date/size. If possible I'd like to check<br>
> content (checksum?). Are you aware of anything like that? Linux or windows is<br>
> fine. Thanks<br>
> _______________________________<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
</div></div></blockquote></div><br></div>