Go’s httputil package makes it really easy to create a reverse proxy. In this article, I’ll show you how to create a simple reverse proxy and ways you can extend and develop more complex features using it as the base.
Simple Reverse Proxy using httputil
The simplest version of Go reverse proxy requires just a couple lines of code. For example, this is a perfectly valid reverse proxy:-
package main
import (
"log"
"net/http"
"net/url"
"net/http/httputil"
)
func main() {
origin, err := url.Parse("http://localhost:8888")
if err != nil {
panic(err)
}
proxy := httputil.NewSingleHostReverseProxy(origin)
http.Handle("/", proxy)
log.Fatal(http.ListenAndServe(":8080", nil))
}For this setup, all requests received on :8080 are proxied to the origin server at http://localhost:8888. For the purpose of this article, I’m running a dummy server that prints out the request details.
Let’s quickly review the code.
First, create a new ReverseProxy instance using func NewSingleHostReverseProxy(target *url.URL) *ReverseProxy from httputil.
Then, feed the ReverseProxy instance directly to http.Handle:func Handle(pattern string, handler Handler):-
http.Handle("/", proxy)And the reason why this works is because ReverseProxy satisfies the http.Handler interface through its ServeHTTP method implementation. This makes it a Handler type — which is what http.Handle is expecting for its second parameter.
That said, in many applications, you might not want to use ReverseProxy.ServeHTTP directly. Instead, you might want to inject some middleware here by chaining up the ServeHTTP method. We’ll talk more about this in the Middleware section below.
In terms of the headers, by default, everything is passed on to the origin which includes request headers and query parameters, EXCEPT:-
- Hop-by-hop headers
- X-Forwarded-For header – ReverseProxy will add or append the client IP to this header
- User-Agent – when using the default Director (more on this in the following section) ReverseProxy explicit sets it to “” if no User-Agent header is found in the request.
Modifying Requests
For one reason or another, you might want to modify the user requests before passing them on to the origin server.
For example, you might want to add information about the particular proxy server that is handling the request: ip, type, version, etc.
In another case, you might want to share more information about the client, such as their country.
Before we get to actually changing the requests, let’s refactor our code.
This code below works in exactly the same way as the code in the previous section, except that the code that creates the proxy has moved into a function called NewProxy.
func NewProxy(rawUrl string) (*httputil.ReverseProxy, error) {
url, err := url.Parse(rawUrl)
if err != nil {
return nil, err
}
proxy := httputil.NewSingleHostReverseProxy(url)
return proxy, nil
}
func main() {
proxy, err := NewProxy("http://localhost:8888")
if err != nil {
panic(err)
}
http.Handle("/", proxy)
log.Fatal(http.ListenAndServe(":8080", nil))
}
Roughly, there are three ways to modify a client requests:
- override ReverseProxy’s Director function completely
- create your own Director function and call the default Director function, chaining the Direction functions in effect. (The approach is inspired by this awesome article)
- create a middleware – wrap ReverseProxy’s ServeHTTP in a higher-level SeveHTTP function (see middleware section)
Here’s how the second approach works:-
func NewProxy(rawUrl string) (*httputil.ReverseProxy, error) {
url, err := url.Parse(rawUrl)
if err != nil {
return nil, err
}
proxy := httputil.NewSingleHostReverseProxy(url)
// Modify requests
originalDirector := proxy.Director
proxy.Director = func(r *http.Request) {
originalDirector(r)
r.Header.Set("Some-Header", "Some Value")
}
return proxy, nil
}The reason why you want to call the original director function is because it does a couple of things like setting the host, scheme, and update the URL path. You can absolutely do all these manually in your own custom Director, but I just like the fact that it’s done for me. For more details, check out its source code.
Changing Responses from Origin Server
For modifying responses, ReverseProxy provides a “hook” for your custom function.
To use that hook, just write a function to ModifyResponse of your ReverseProxy instance.
ModifyResponse func(*http.Response) errorFor example, this code will add a Server header to all responses.
// Modify response
proxy.ModifyResponse = func(r *http.Response) error {
// Add a response header
r.Header.Set("Server", "CodeDodle")
return nil
}Unlike the Director function above, there’s no default ModifyResponse in the httputil package. So, there’s no worries about overriding any default behavior.
Another way to change the response is by adding a middleware. See the following section.
The Middleware Approach: Chaining ServeHTTP
A great way to add functionalities to your custom reverse proxy is by adding a middleware by chaining the handlers.
For the proxy that we’re developing, we’ll create our own ServeHTTP function and call ReverseProxy’s ServeHTTP from it.
First, let’s refactor the code a little bit by creating a new struct type SimpleProxy.
package main
import (
"fmt"
"log"
"net/http"
"net/url"
"net/http/httputil"
)
type SimpleProxy struct {
Proxy *httputil.ReverseProxy
}
func NewProxy(rawUrl string) (*SimpleProxy, error) {
url, err := url.Parse(rawUrl)
if err != nil {
return nil, err
}
s := &SimpleProxy{httputil.NewSingleHostReverseProxy(url)}
// Modify requests
originalDirector := s.Proxy.Director
s.Proxy.Director = func(r *http.Request) {
originalDirector(r)
r.Header.Set("Some-Header", "Some Value")
}
// Modify response
s.Proxy.ModifyResponse = func(r *http.Response) error {
// Add a response header
r.Header.Set("Server", "CodeDodle")
return nil
}
return s, nil
}
func (s *SimpleProxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Do anything you want here
// e.g. blacklisting IP, log time, modify headers, etc
log.Printf("Proxy receives request.")
log.Printf("Proxy forwards request to origin.")
s.Proxy.ServeHTTP(w, r)
log.Printf("Origin server completes request.")
}
func main() {
proxy, err := NewProxy("http://localhost:8888")
if err != nil {
panic(err)
}
http.Handle("/", proxy)
log.Fatal(http.ListenAndServe(":8080", nil))
}
SimpleProxy is a thin wrapper over ReverseProxy and it is considered a Handler type (because it implements its own ServeHTTP method).
Inside SimpleProxy’s ServeHTTP method, you can do all sorts of things like modifying requests and response, implement caching, block requests, log information, routing to different origin servers (load balancing), and the list goes on.
Dummy Origin Server
Here’s the dummy backend server I used for testing. It prints out the request headers and body.
package main
import (
"fmt"
"net/http"
)
func main() {
http.HandleFunc("/", func(rw http.ResponseWriter, r *http.Request) {
fmt.Fprintf(rw, "[Origin server]\n\n")
fmt.Fprintf(rw, "Header\n\n")
for key, value := range r.Header {
fmt.Fprintf(rw, "%q: %q\n", key, value)
}
fmt.Fprintf(rw, "\n\nBody\n\n")
fmt.Fprintf(rw, "%q", r.Body)
})
http.ListenAndServe(":8888", nil)
}Testing
Use curl -v for manual testing
$ curl localhost:8080 -v
* Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Length: 174
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 09 Nov 2022 04:55:42 GMT
< Server: CodeDodle
<
[Origin server]
Header
"User-Agent": ["curl/7.68.0"]
"Accept": ["*/*"]
"Some-Header": ["Some Value"]
"X-Forwarded-For": ["127.0.0.1"]
"Accept-Encoding": ["gzip"]
Body
In the sample output above, you can see that the “Some-Header” item in the request header has been set correctly by the Director function we created earlier.
Similar, “Server: Codedodle” shows up in the response header as a result of our custom ModifyResponse function.
Discussions
Hop-by-hop Headers
Hop-by-hop headers are headers that are meant to be consumed by the first node that handles them. A reverse proxy would consume these headers before creating its own hop-by-hop headers and forward them to the origin server.
According to Mozilla,
These headers are meaningful only for a single transport-level connection, and must not be retransmitted by proxies or cached.
Here’s the list of headers classified as hop-by-hop by ReverseProxy:-
var hopHeaders = []string{
"Connection",
"Proxy-Connection", // non-standard but still sent by libcurl and rejected by e.g. google
"Keep-Alive",
"Proxy-Authenticate",
"Proxy-Authorization",
"Te", // canonicalized version of "TE"
"Trailer", // not Trailers per URL above; https://www.rfc-editor.org/errata_search.php?eid=4522
"Transfer-Encoding",
"Upgrade",
}Helpful Resources
Below are some useful webpages I used to develop this article. You should check them out if you need more information on ReverseProxy.
- ReverseProxy source code
- The Right Use of ‘ReverseProxy’ in Golang – JoshSoftware Link – a great introduction and excellent review guide for those of you who are working with ReverseProxy.
- Golang Reverse Proxy – Link – in this article, the author shows you how to instantiate a ReverseProxy instance without using the builtin constructor function
httputil.NewSingleHostReverseProxy. This involves creating your own Director function and making the right configurations. The article also shows you can roll out your own ErrorHandler and use it in ReverseProxy — one of the aspects I have not covered above.
