[ale] Perl https download

[Tim Watts]

Hi,

This may be a red herring, but I noticed that the script calls 

	$ua = LWP::UserAgent->new( );

again after setting the cookie_jar property on $ua which would appear to me to 
wipe out that setting.

However, that doesn't really set well with me because HTTP Basic Auth doesn't 
rely on a cookie to continue the session but rather an "Authorization:" 
request-header. If the site is actually using Basic Auth, try including an 
Authorization header in the IndexURI request. See 
http://www.ietf.org/rfc/rfc2616.txt, section 14.8 for details on constructing 
this header. If that works, you probably don't need to perform the LoginURI 
request.

On Wednesday 25 July 2007 9:35 pm, Robert L. Harris wrote:
> That's basically what I've come up with so far:
>
> #!/usr/bin/perl -w
>
> use LWP::UserAgent;
> use HTTP::Cookies;
> use LWP::Simple;
> use HTTP::Request;
> use strict;
>
> #
> my ($LoginURI)="https://www.datafeed.org";
> my ($IndexURI)="https://www.datafeed.org/internal/querry?range=3day";
> my ($User)="myuser";
> my ($Passwd)="mypasswd";
>
>
> # Load LWP class for "cookie jar" objects
> my $ua = LWP::UserAgent->new( );
> my $cookie_jar = HTTP::Cookies->new(file => "lwp.cookie", autosave => 1);
> $ua->cookie_jar( $cookie_jar );
> $cookie_jar->save;
>
> # define user agent
> $ua = LWP::UserAgent->new();
> $ua->agent("USER/AGENT/IDENTIFICATION");
>
>
> # make request
> my $request = HTTP::Request->new(GET => $LoginURI);
>
> # authenticate
> $request->authorization_basic($User, $Passwd);
>
> # except response
> my $response = $ua->request($request);
> my $content=$response->content;
>
> print "\$content :$content:\n";
>
>
>
>
> # Get Second Page, Index to the data
> $request = HTTP::Request->new(GET => $IndexURI);
> # except response
> $response = $ua->request($request);
> $content=$response->content;
>
> print "\$content :$content:\n";
>
>
>
>
> The first (LoginURI) works and I get logged in and the view of an internal
> page.  The second part (IndexURI) responds:
>
> $content :<h4>Not logged in</h4><p>You must be logged in to view that
> page.</p>:
>
> So it seems to not be passing auth or a cookie.  I tried putting the
> authorization_basic right after the second request command but no change.
>
> Thus spake Christopher Fowler (cfowler at outpostsentinel.com):
> > This is an example.  Use it to get a cookie that is valid for the
> > website then go grab whatever you need
> >
> > my $ua = LWP::UserAgent->new();
> > $ua->cookie_jar({ file => "$ENV{HOME}/.cookies.txt" });
> > my $response = $ua->post("$proto://$ARGV[0]/cgi-bin/login",
> >     { 'user' => "root", 'pass' => "password" });
> >
> >
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://www.ale.org/mailman/listinfo/ale
> >
> :wq!
>
> ---------------------------------------------------------------------------
> Robert L. Harris                     | GPG Key ID: E344DA3B
>                                          @ x-hkp://pgp.mit.edu
> DISCLAIMER:
>       These are MY OPINIONS             With Dreams To Be A King,
>        ALONE.  I speak for              First One Should Be A Man
>        no-one else.                       - Manowar