[ale] HOW can I display a file like: showthread.php at t-105297

Mike Harrison meuon at geeklabs.com
Fri Oct 24 16:44:53 EDT 2008


> I, with wget, downloaded the sight and found a subdir of similar files that
> when I click I get a message something about it doesn't know what to do with
> this type file.

Not sure if you need his, but it's useful if using wget to get a website
and you are missing chunks:

http://www.geeklabs.com/code/getter.pl.txt

Instructions: wget a website to your new webhost.
Then as you click on the new website and find missing things
like things embedded in flash files or ajax calls,
this will parse your website logs, and get the missing files:

#!/usr/bin/perl
# GETTER.PL
#Sample Logfile Format
#68.85.201.140 - - [14/Nov/2005:10:58:34 -0500] "GET /staff.swf HTTP/1.1" 
404
# getter reads a logfile and gets missing file from another website.
# Used in cloning a website that
# can not be gotten via wget. like Flash websites.
#  - By Mike of GeekLabs.com  meuon at geeklabs.com
# ego-ware, like it? Send me a thank you!
#----------

#run this by running "tail -f logfile | grep getter
#and then go surf the website that you grabbed with wget
#but are missing pieces of because of weird links in flash or javascript

$source = "http://www.originalwebsite.???" ;
while(<STDIN>) {
   @d = split(/\s+/) ;
   print "$d[6]    -  $d[8]\n" ;
   if("$d[8]" eq "200") {
       #do nothing for now
   } else {
       $string = "wget -nH $source$d[6]" ;
       print "$string\n" ;
       system("$string") ;
   } ;
} ;




More information about the Ale mailing list