feat(client): enhance HTTP client configuration with proxy support (#668)

* feat: enhance HTTP client configuration with proxy support

- Add `ProxyURL` field to `Config` struct with YAML tag
- Implement proxy URL parsing and setting in `getHTTPClient` method
- Add test case for `getHTTPClient` method with custom proxy URL setting
- Include `net/url` package in both `config.go` and `config_test.go` files

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>

* docs: enhance README with Proxy and OAuth2 Docs

- Remove empty lines from README.md
- Add documentation for proxy configuration in client examples
- Include YAML examples for client using a proxy, custom DNS resolver, OAuth2, and identity-aware proxy configurations in README.md

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>

* docs: add proxy client

Signed-off-by: appleboy <appleboy.tw@gmail.com>

* Update client/config.go

* Update README.md

* Update client/config_test.go

---------

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Signed-off-by: appleboy <appleboy.tw@gmail.com>
Co-authored-by: TwiN <twin@linux.com>

README.md

@@ -104,6 +104,7 @@ Have any feedback or questions? [Create a discussion](https://github.com/TwiN/ga
   - [Exposing Gatus on a custom port](#exposing-gatus-on-a-custom-port)
   - [Configuring a startup delay](#configuring-a-startup-delay)
   - [Keeping your configuration small](#keeping-your-configuration-small)
+  - [Proxy client configuration](#proxy-client-configuration)
   - [Badges](#badges)
     - [Uptime](#uptime)
     - [Health](#health)
@@ -132,9 +133,9 @@ if no traffic makes it to your applications. This puts you in a situation where
 that will notify you about the degradation of your services rather than you reassuring them that you're working on
 fixing the issue before they even know about it.
 
-
 ## Features
 The main features of Gatus are:
+
 - **Highly flexible health check conditions**: While checking the response status may be enough for some use cases, Gatus goes much further and allows you to add conditions on the response time, the response body and even the IP address.
 - **Ability to use Gatus for user acceptance tests**: Thanks to the point above, you can leverage this application to create automated user acceptance tests.
 - **Very easy to configure**: Not only is the configuration designed to be as readable as possible, it's also extremely easy to add a new service or a new endpoint to monitor.
@@ -146,7 +147,6 @@ The main features of Gatus are:
 
 ![Gatus dashboard conditions](.github/assets/dashboard-conditions.png)
 
-
 ## Usage
 
 <details>
@@ -361,6 +361,7 @@ the client used to send the request.
 | `client.oauth2.client-id`              | The client id which should be used for the `Client credentials flow`        | required `""`   |
 | `client.oauth2.client-secret`          | The client secret which should be used for the `Client credentials flow`    | required `""`   |
 | `client.oauth2.scopes[]`               | A list of `scopes` which should be used for the `Client credentials flow`.  | required `[""]` |
+| `client.proxy-url`                     | The URL of the proxy to use for the client                                  | `""`            |
 | `client.identity-aware-proxy`          | Google Identity-Aware-Proxy client configuration.                           | `{}`            |
 | `client.identity-aware-proxy.audience` | The Identity-Aware-Proxy audience. (client-id of the IAP oauth2 credential) | required `""`   |
 | `client.network`                       | The network to use for ICMP endpoint client (`ip`, `ip4` or `ip6`).         | `"ip"`          |
@@ -369,15 +370,18 @@ the client used to send the request.
 in ICMP requests (ping), therefore, setting `client.insecure` to `true` for an endpoint of that type will not do anything.
 
 This default configuration is as follows:
+
 ```yaml
 client:
   insecure: false
   ignore-redirect: false
   timeout: 10s
 ```
+
 Note that this configuration is only available under `endpoints[]`, `alerting.mattermost` and `alerting.custom`.
 
 Here's an example with the client configuration under `endpoints[]`:
+
 ```yaml
 endpoints:
   - name: website
@@ -391,6 +395,7 @@ endpoints:
 ```
 
 This example shows how you can specify a custom DNS resolver:
+
 ```yaml
 endpoints:
   - name: with-custom-dns-resolver
@@ -402,6 +407,7 @@ endpoints:
 ```
 
 This example shows how you can use the `client.oauth2` configuration to query a backend API with `Bearer token`:
+
 ```yaml
 endpoints:
   - name: with-custom-oauth2
@@ -417,6 +423,7 @@ endpoints:
 ```
 
 This example shows how you can use the `client.identity-aware-proxy` configuration to query a backend API with `Bearer token` using Google Identity-Aware-Proxy:
+
 ```yaml
 endpoints:
   - name: with-custom-iap
@@ -427,6 +434,7 @@ endpoints:
     conditions:
       - "[STATUS] == 200"
 ```
+
 > 📝 Note that Gatus will use the [gcloud default credentials](https://cloud.google.com/docs/authentication/application-default-credentials) within its environment to generate the token.
 
 ### Alerting
@@ -1909,6 +1917,19 @@ endpoints:
 ```
 </details>
 
+### Proxy client configuration
+
+You can configure a proxy for the client to use by setting the `proxy-url` parameter in the client configuration.
+
+```yaml
+endpoints:
+  - name: website
+    url: "https://twin.sh/health"
+    client:
+      proxy-url: http://proxy.example.com:8080
+    conditions:
+      - "[STATUS] == 200"
+```
 
 ### How to fix 431 Request Header Fields Too Large error
 Depending on where your environment is deployed and what kind of middleware or reverse proxy sits in front of Gatus,

client/config.go

@@ -7,6 +7,7 @@ import (
 	"log"
 	"net"
 	"net/http"
+	"net/url"
 	"regexp"
 	"strconv"
 	"time"
@@ -42,6 +43,9 @@ func GetDefaultConfig() *Config {
 
 // Config is the configuration for clients
 type Config struct {
+	// ProxyURL is the URL of the proxy to use for the client
+	ProxyURL string `yaml:"proxy-url,omitempty"`
+
 	// Insecure determines whether to skip verifying the server's certificate chain and host name
 	Insecure bool `yaml:"insecure,omitempty"`
 
@@ -184,6 +188,14 @@ func (c *Config) getHTTPClient() *http.Client {
 				return nil
 			},
 		}
+		if c.ProxyURL != "" {
+			proxyURL, err := url.Parse(c.ProxyURL)
+			if err != nil {
+				log.Println("[client][getHTTPClient] THIS SHOULD NOT HAPPEN. Silently ignoring custom proxy due to error:", err.Error())
+			} else {
+				c.httpClient.Transport.(*http.Transport).Proxy = http.ProxyURL(proxyURL)
+			}
+		}
 		if c.HasCustomDNSResolver() {
 			dnsResolver, err := c.parseDNSResolver()
 			if err != nil {

client/config_test.go

@@ -2,6 +2,7 @@ package client
 
 import (
 	"net/http"
+	"net/url"
 	"testing"
 	"time"
 )
@@ -79,3 +80,29 @@ func TestConfig_ValidateAndSetDefaults_withCustomDNSResolver(t *testing.T) {
 		})
 	}
 }
+
+func TestConfig_getHTTPClient_withCustomProxyURL(t *testing.T) {
+	proxyURL := "http://proxy.example.com:8080"
+	cfg := &Config{
+		ProxyURL: proxyURL,
+	}
+	cfg.ValidateAndSetDefaults()
+	client := cfg.getHTTPClient()
+	transport := client.Transport.(*http.Transport)
+	if transport.Proxy == nil {
+		t.Errorf("expected Config.ProxyURL to set the HTTP client's proxy to %s", proxyURL)
+	}
+	req := &http.Request{
+		URL: &url.URL{
+			Scheme: "http",
+			Host:   "www.example.com",
+		},
+	}
+	expectProxyURL, err := transport.Proxy(req)
+	if err != nil {
+		t.Errorf("can't proxy the request %s", proxyURL)
+	}
+	if proxyURL != expectProxyURL.String() {
+		t.Errorf("expected Config.ProxyURL to set the HTTP client's proxy to %s", proxyURL)
+	}
+}