Archive for September, 2007

HTTPS requests via proxy with Perl

Thursday, September 20th, 2007

A common problem with not so many solutions on the web is submitting HTTPS requests using Perl’s LWP module. This works fine as long as no proxy is involved. When the request needs to pass a proxy, you find out quickly, that LWP is not able to handle proxies for HTTPS traffic. Instead, the underlying SSLeay library must be used to tunnel the traffic transparently.

The following code shows how to:

$proxy = 'http://user:pass@server';
$ua->proxy(['http'], $proxy);

$proxy = 'http://server';
$ENV{HTTPS_PROXY} = $proxy;

# working, http & https
my $req = HTTP::Request->new(GET => 'http(s)://site/');
my $res = $ua->request($req);
if ($res->is_success){
print $res->as_string;

The tricky part is that lwp will still needs the http_proxy to handle http requests correctly, but not https_proxy as it needs to pass down the requests to SSLeay.