<div>A more ubiquitous alternative to the GNU cat features is the "listing" command of sed, although the precise behavior of sed isn't portable (the general behavior producing some kind of unambiguous listing is portable).</div>
<div><br></div> sed -n l file1 > file1.listing<div>...</div><div><br></div><div>Something folks haven't mentioned is that some versions of diff have options that let you ask diff to work harder to minimize the number of differences it lists. There's a tradeoff between going fast and minimizing the output.</div>
<div><br></div><div>I can imagine a use case where you wouldn't be able to sort and uniq because order was significant. In such a case, you could use use bdb in python/ruby/perl/whatever to keep the a map of seen lines and their locations in a bdb file.</div>
<div><br><div class="gmail_quote">On Wed, Aug 7, 2013 at 1:53 PM, Lightner, Jeff <span dir="ltr"><<a href="mailto:JLightner@water.com" target="_blank">JLightner@water.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
cat -vt on the files should show non-printing characters including tabs - maybe output of same to additional files and diff on those would help e.g.<br>
cat -vt file1 >file1.vt<br>
cat -vt2 file2 >file2.vt<br>
diff file1.vt file2.vt<br>
<br>
Also often if I have large files I want to diff I usually use sdiff instead (widening my terminal session as far as possible first) as it will insure the lines are reasonably close to each other. (This is a side by side diff.)<br>
<div><div class="h5"><br>
<br>
<br>
<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:ale-bounces@ale.org">ale-bounces@ale.org</a> [mailto:<a href="mailto:ale-bounces@ale.org">ale-bounces@ale.org</a>] On Behalf Of Stephen R. Blevins<br>
Sent: Wednesday, August 07, 2013 1:29 PM<br>
To: Atlanta Linux Enthusiasts<br>
Subject: Re: [ale] diff being boneheaded<br>
<br>
Could there be tabs in one file's lines, and spaces in the other? What about other unprintable characters?<br>
<br>
Stephen R. Blevins<br>
<a href="mailto:stephen.r.blevins@gmail.com">stephen.r.blevins@gmail.com</a><br>
<br>
On 08/07/2013 12:37 PM, Jim Kinney wrote:<br>
> I've got 2 text files > 6M lines each. Each file is sorted in<br>
> dictionary order. diff is flagging identical lines between them as<br>
<snip><br>
> See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
><br>
<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
<br>
<br>
<br>
<br>
</div></div>Athena(r), Created for the Cause(tm)<br>
Making a Difference in the Fight Against Breast Cancer<br>
<br>
---------------------------------<br>
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.<br>
----------------------------------<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br> Ed Cashin <<a href="mailto:ecashin@noserose.net">ecashin@noserose.net</a>><br> <a href="http://noserose.net/e/">http://noserose.net/e/</a><br>
<a href="http://www.coraid.com/">http://www.coraid.com/</a>
</div>