Imagine, not entirely hypothetically, that you have a client that needs you to work on multiple systems only accessible via https (where the domain name needs to match), all located behind the client's firewall. Further suppose that the only access they can provide to their network is ssh to a bastion host -- even while located in their physical office, only "guest" network access to the Internet is available. Assume, also not entirely hypothetically, that they have no VPN server. Finally assume, again not entirely hypothetically, that no software can be installed on the bastion host, and that it runs Ubuntu Linux 12.04 LTS (hey, there is at least a month of maintenance support left for that version...).
In this situation there are a few reasonable approaches that preserve the browser's view of the domain name:
an outgoing (forward) web proxy, supporting
CONNECTtransparent redirection of outgoing TCP connections to a proxy
tricks with DNS resolution (eg
/etc/hosts), possibly combined with one of the above.
I did briefly experiment with transparent redirection of the outgoing
TCP connections (which works well on Linux: iptables -t nat -A
OUTPUT ...), but since I was working from a Mac OS X desktop system
it was more complicated (Mac OS X uses pf, and pf.conf can
include rdr statements to redirect
packets,
but intercepting locally originated traffic involves multiple steps
and seems somewhat fragile and was not working reliably for me).
Instead I went looking for a way to implement a web forward proxy,
on the bastion host. Since I could not install software on the bastion
host, I needed to find something already installed which could be
repurposed to be a forward proxy. Fortunately it turned out that the
bastion host had been installed with apache2 (2.2.2), to support
another role of the host (remote access to monitoring output). I then
needed a configuration that could use apache2 in a forward proxy mode.
Apache 2.2 provides a forward proxy feature through
mod_proxy, but
it is definitely something you want to secure carefully as the
documentation repeatedly warns. In addition the bastion host naturally
was firewalled from the Internet to allow only certain ports to be
reached directly, including ssh, so simply running a web proxy on some
port on an Internet reachable IP was never an option.
To solve both of these problems I created a configuration to run
another instance of Apache 2.2, with mod_proxy enabled in forward
proxy mode, listening on localhost, that could be reached only via
ssh port forward (based on examples
from the
Internet).
This involved creating a custom configuration to run Apache 2.2 with:
Listen 127.0.0.1:3128
# Access control functionality
LoadModule authz_host_module /usr/lib/apache2/modules/mod_authz_host.so
# Proxy functionality
LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so
LoadModule proxy_connect_module /usr/lib/apache2/modules/mod_proxy_connect.so
# Logging
LogFormat "%h %l %u %t \"%r\" %s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogLevel warn
ErrorLog logs/error.log
CustomLog logs/access.log combined
# PID file
PidFile logs/apache2.pid
# Forward Proxy
ProxyRequests On
# Allow CONNECT (and thus HTTPS) to additional ports
AllowCONNECT 443 563 8080
as well as a section (in angle brackets) to match "Proxy *", which
limited access to the proxy to localhost:
Order Deny,Allow
Deny from all
Allow from 127.0.0.1
(Full Apache 2.2 example config.)
Note that the above configuration will work only with Apache 2.2 (or earlier);
the configuration for the Apache 2.4
mod_proxy, and
other Apache features, changed significantly, particularly around
authorization. (ETA, 2017-04-03: See end of post for update with
Apache 2.4 configuration.)
The key features in the above configuration is that it loads the
modules needed for HTTP/HTTPS proxying, and IP authentication, and
then listens on 127.0.0.1:3128 for connections and treats those
connections as forward proxy connections -- thanks to ProxyRequests
On and the "Proxy *" section. The "Proxy *" section is intended
to allow access only from localhost itself. It may be useful to
also add user/password authentication to the proxy use.
Put the config in a directory, and then make a "logs" directory:
cd ....
mkdir logs
And create a simple wrapper script to start up the proxy on the bastion
host (eg, called "go"):
#! /bin/sh
# Run Apache in forward proxy mode to be reached via ssh tunnel
#
exec apache2 -d ${PWD} -f ${PWD}/apache-2.2-forward-proxy.conf -E /dev/stderr
Then the proxy can be started up when needed with:
cd ....
./go
and will run in the background. When run, you should see something
listening on TCP/3128 on localhost on the bastion host, eg with
netstat -na | grep 3128:
tcp 0 0 127.0.0.1:3128 0.0.0.0:* LISTEN
From there, the next step is to ssh into the bastion host with a port
forward that allows you to reach 127.0.0.1:3128:
ssh -L 3128:127.0.0.1:3128 -o ExitOnForwardFailure=yes -4 -N -f HOST
which should cause ssh to listen on our local (desktop) system, here
also on port 3128, so netstat -na | grep 3128 on the local system
should also show:
tcp 0 0 127.0.0.1:3128 0.0.0.0:* LISTEN
The final step is to set the proxy configuration of your web browser
to use a web proxy at 127.0.0.1 on port 3128, so that your
web browser will use the proxy for HTTP/HTTPS connections. For Safari
this can be done with Safari -> Preferences... -> Advanced ->
Proxies: Change Settings..., which will open the system wide proxy
settings. You need to change both "Web Proxy (HTTP)" and "Secure web
proxy (HTTPS)" for this to work in most cases, ticking them and then
setting the "Web Proxy Server" to "127.0.0.1" and the port to "3128".
After that your web browsing should automatically go through the
127.0.0.1:3128 connection on your desktop system, via the ssh port
forward to Apache 2.2/mod_proxy on the bastion host, and then
CONNECT out to the desired system, giving a transparent HTTPS
connection so that TLS certificate validation will just work.
To access from a command line client it is typically useful to set:
http_proxy=https://127.0.0.1:3128/
https_proxy=https://127.0.0.1:3128/
export http_proxy https_proxy
because most client libraries will look for those (lowercase) environment variables when deciding how to make the connection. This enables using, eg, web REST/JSON APIs from Python, with the proxy.
If the bastion host is restarted then the proxy will have to be manually restarted (as above); but if it is running on a legacy Linux install chances are that the client will be unwilling to reboot the host regularly due to being uncertain if it will boot up cleanly with everything needed running. I got through the entire project without having to restart the proxy.
The main catch with this configuration is that because the Safari
proxy setting are system wide, they affect all traffic including
things like iTunes. I worked around that by using another browser
which had its own network connection settings (Firefox) for regular
web browsing, and turning the proxy settings on and off as I needed
them to work on that project. If this were to be a semi-permanent
solution it might either be better to use a browser with its own
proxy settings (eg, Firefox; or one in a VM) dedicated to the
project, and leave the system wide settings alone. Or perhaps to
create a Proxy
Auto-Config
(.pac) file which redirected only certain URLs to the proxy --
it is possible to arrange to load those from a local file instead
of a web URL.
ETA, 2017-04-03: Inevitably, given that the support for Ubuntu 12.04 LTS runs out this month, the client upgraded to the bastion host Ubuntu 14.04 LTS (but not Ubuntu 16.04 LTS; presumably due to being a smaller jump). This brings in Apache 2.4 instead of Apache 2.2, which brings in a non-trivial changes in configuration syntax.
The incremental differences are relatively small though, for a basic config. You need to load a couple of additional modules:
# Apache 2.4 worker
LoadModule mpm_worker_module /usr/lib/apache2/modules/mod_mpm_worker.so
LoadModule authz_core_module /usr/lib/apache2/modules/mod_authz_core.so
and change the Proxy authorization section to be:
Require ip 127.0.0.1
rather than "Order Deny, Allow", "Deny from all", "Allow from 127.0.0.1".
It is now also possible to use additional proxy directives including:
ProxyAddHeaders On
ProxySourceAddress A.B.C.D
but these are optional. With those changes the same minimal configuration approach should work with Apache 2.4.
(Full Apache 2.4 example config.)