<p>Use stream editing tools (sed, awk, etc.) in as few pipelines as possible. I have to do things like that all the time. </p>
<p>If necessary, use a HLL to create a custom made filter as part of the pipeline. </p>
<div class="gmail_quote">On Sep 25, 2012 5:08 PM, "James Taylor" <<a href="mailto:James.Taylor@eastcobbgroup.com">James.Taylor@eastcobbgroup.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Any suggestions for editing a 37GB file?<br>
-jt<br>
<br>
>>> "James Taylor" <<a href="mailto:James.Taylor@eastcobbgroup.com">James.Taylor@eastcobbgroup.com</a>> 9/25/2012 04:15 PM >>><br>
He did indicate in another email that he did drop the the table.<br>
I'll pass this along and see what he has to say.<br>
-jt<br>
<br>
<br>
>>> Mike Harrison <<a href="mailto:cluon@geeklabs.com">cluon@geeklabs.com</a>> 9/25/2012 03:46 PM >>><br>
<br>
We do this XX times a day. Usually works great.<br>
<br>
<br>
Is there a good chance it already exists?<br>
<br>
I don't usually see the COLLATE= statement,<br>
other than that, it looks OK. (and that should be ok.. but worth playing<br>
with?)<br>
<br>
Usually there is a "drop table" command just above the "create table"<br>
command, might be worth checking.<br>
<br>
In rare cases, when transported via weird methods,<br>
I've seen garbage in these files, especially when dealing with strange<br>
character sets.<br>
<br>
What I would do:<br>
<br>
Cut out that section of the file and try running it.<br>
<br>
from the drop above the create to the next table's drop/create statement.<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
============================================================<br>
Mike Harrison <a href="mailto:bogon@geeklabs.com">bogon@geeklabs.com</a> cell: <a href="tel:423.605.6943" value="+14236056943">423.605.6943</a><br>
<br>
On Tue, 25 Sep 2012, James Taylor wrote:<br>
<br>
> One of my clients is trying to test a disaster recover scenario by restoring a mysql database to an offsite disaster recovery server.<br>
><br>
> He has determined he needs to disable foreign key checking. He add the following to the mysqld section of my.conf<br>
><br>
> init_connect='SET foreign_key_checks = 0'<br>
><br>
> But it didn't work for him.<br>
> He created a regular user to use for the import, which he thinks he needs to do.<br>
><br>
> Here's the output when the problem occurs.<br>
><br>
> omniprise-db1-dr:/utility # mysql -uroot -pchanged < omniprise_backup.sql<br>
> ERROR 1005 (HY000) at line 36332: Can't create table 'ERP_DATABASE.heartbeat' (errno: -1)<br>
><br>
> omniprise-db1-dr:/utility # head -n 36340 omniprise_backup.sql | tail -n 9<br>
> CREATE TABLE `heartbeat` (<br>
> `id` int(10) unsigned NOT NULL DEFAULT '0',<br>
> `ts` datetime DEFAULT NULL,<br>
> PRIMARY KEY (`id`)<br>
> ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;<br>
> /*!40101 SET character_set_client = @saved_cs_client */;<br>
><br>
> --<br>
> -- Dumping data for table `heartbeat`<br>
><br>
> omniprise-db1-dr:/utility # wc -l omniprise_backup.sql<br>
> 133974 omniprise_backup.sql<br>
><br>
> Any thoughts?<br>
><br>
> Keep in mind that I'm not working directly with this and I'm not strong on mysql administration.<br>
><br>
> -jt<br>
><br>
><br>
><br>
> James Taylor<br>
> 678-697-9420<br>
> <a href="mailto:james.taylor@eastcobbgroup.com">james.taylor@eastcobbgroup.com</a><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Ale mailing list<br>
> <a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
> <a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
> See JOBS, ANNOUNCE and SCHOOLS lists at<br>
> <a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
><br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
<br>
<br>
If this is an unsolicited spam message, please click this link to report it: <a href="http://control.eastcobbgroup.com:49285/contents/spamreport.shtml?rptid=13494&srvid=16vl15t" target="_blank">http://control.eastcobbgroup.com:49285/contents/spamreport.shtml?rptid=13494&srvid=16vl15t</a><br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
<br>
<br>
If this is an unsolicited spam message, please click this link to report it: <a href="http://control.eastcobbgroup.com:49285/contents/spamreport.shtml?rptid=13496&srvid=16vl15t" target="_blank">http://control.eastcobbgroup.com:49285/contents/spamreport.shtml?rptid=13496&srvid=16vl15t</a><br>
<br>
<br>
<br>
_______________________________________________<br>
Ale mailing list<br>
<a href="mailto:Ale@ale.org">Ale@ale.org</a><br>
<a href="http://mail.ale.org/mailman/listinfo/ale" target="_blank">http://mail.ale.org/mailman/listinfo/ale</a><br>
See JOBS, ANNOUNCE and SCHOOLS lists at<br>
<a href="http://mail.ale.org/mailman/listinfo" target="_blank">http://mail.ale.org/mailman/listinfo</a><br>
</blockquote></div>