self-hosting guide (#121)

* docs: first steps of the self-hosting guide * feature: add setup configurator for the self-hosted guide * docs: add setup.env comments * docs: simplify installation steps - support ./configure.sh * docs: fix file references * docs: fix minor docs issues * docs: remove unused title
2024-11-07 08:44:07 +01:00 · 2021-09-25 19:12:05 +02:00 · 2021-09-25 19:12:05 +02:00 · 08dd719aa1
commit 08dd719aa1
parent 84c714dd93
8 changed files with 213 additions and 95 deletions
--- a/.gitignore
+++ b/.gitignore
@ -4,3 +4,5 @@ dist/
 .env
 conf.json
 http-cmds.sh
+infrastructure_files/management.json
+infrastructure_files/docker-compose.yml
--- a/docs/self-hosting.md
+++ b/docs/self-hosting.md
@ -1,2 +1,93 @@
 ### Self-hosting
-TODO
+Wiretrustee is an open-source platform that can be self-hosted on your servers.
+
+It relies on components developed by Wiretrustee Authors [Management Service](https://github.com/wiretrustee/wiretrustee/tree/main/management), [Management UI Dashboard](https://github.com/wiretrustee/wiretrustee-dashboard), [Signal Service](https://github.com/wiretrustee/wiretrustee/tree/main/signal), 
+a 3rd party open-source STUN/TURN service [Coturn](https://github.com/coturn/coturn) and a 3rd party service [Auth0](https://auth0.com/).
+
+All the components can be self-hosted except for the Auth0 service.
+We chose Auth0 to "outsource" the user management part of the platform because we believe that implementing a proper user auth requires significant amount of time to make it right. 
+We focused on connectivity instead.
+
+If you would like to learn more about the architecture please refer to the [Wiretrustee Architecture section](architecture.md).
+
+### Requirements
+
+- Virtual machine offered by any cloud provider (e.g., AWS, DigitalOcean, Hetzner, Google Cloud, Azure ...). 
+- Any Linux OS.
+- Docker Compose installed (see [Install Docker Compose](https://docs.docker.com/compose/install/)).
+- Domain name pointing to the public IP address of your server.
+- Open ports ```443, 33071, 33073, 3468``` (Dashboard, Management HTTP API, Management gRpc API, Coturn STUN/TURN respectively)   on your server.
+- Maybe a cup of coffee or tea :)
+
+### Step-by-step guide
+
+For this tutorial we will be using domain ```test.wiretrustee.com``` which points to our Ubuntu 20.04 machine hosted at Hetzner.
+
+1. Create Auth0 account at [auth0.com](https://auth0.com/).
+2. Login to your server, clone Wiretrustee repository:
+   
+   ```bash 
+   git clone https://github.com/wiretrustee/wiretrustee.git wiretrustee/
+   ```
+   
+   and switch to the ```wiretrustee/infrastructure_files/``` folder that contains docker compose file:
+   
+   ```bash 
+   cd wiretrustee/infrastructure_files/
+   ```
+3. Prepare configuration files.
+   
+   To simplify the setup we have prepared a script to substitute required properties in the [docker-compose.yml.tmpl](../infrastructure_files/docker-compose.yml.tmpl) and [management.json.tmpl](../infrastructure_files/management.json.tmpl) files.
+   
+   The [setup.env](../infrastructure_files/setup.env) file contains the following properties that have to be filled:
+   
+   ```bash
+   # e.g. app.mydomain.com
+   WIRETRUSTEE_DOMAIN=""
+   # e.g. dev-24vkclam.us.auth0.com
+   WIRETRUSTEE_AUTH0_DOMAIN=""
+   # e.g. 61u3JMXRO0oOevc7gCkZLCwePQvT4lL0
+   WIRETRUSTEE_AUTH0_CLIENT_ID=""
+   # e.g. https://app.mydomain.com/
+   WIRETRUSTEE_AUTH0_AUDIENCE=""
+   # e.g. hello@mydomain.com
+   WIRETRUSTEE_LETSENCRYPT_EMAIL=""
+   ```
+   
+   Please follow the steps to get the values.
+
+4. Configure ```WIRETRUSTEE_AUTH0_DOMAIN``` ```WIRETRUSTEE_AUTH0_CLIENT_ID``` ```WIRETRUSTEE_AUTH0_AUDIENCE``` properties.          
+   
+   * To obtain these, please use [Auth0 React SDK Guide](https://auth0.com/docs/quickstart/spa/react/01-login#configure-auth0) up until "Install the Auth0 React SDK".
+   
+      :grey_exclamation: Use ```https://YOUR DOMAIN``` as ````Allowed Callback URLs````, ```Allowed Logout URLs```, ```Allowed Web Origins``` and ```Allowed Origins (CORS)```
+   * set the variables in the ```setup.env```
+5. Configure ```WIRETRUSTEE_AUTH0_AUDIENCE``` property. 
+   
+   * Check [Auth0 Golang API Guide](https://auth0.com/docs/quickstart/backend/golang) to obtain AuthAudience.
+   * set the property in the ```setup.env``` file.
+6. Configure ```WIRETRUSTEE_LETSENCRYPT_EMAIL``` property.
+   
+   This can be any email address. [Let's Encrypt](https://letsencrypt.org/) will create an account while creating a new domain.    
+
+7. Make sure all the properties set in the ```setup.env``` file and run: 
+   
+    ```bash
+    ./configure.sh
+    ```
+   
+   This will export all the properties as environment variables and generate ```docker-compose.yml``` and ```management.json``` files substituting required variables.
+
+8. Run docker compose:
+
+   ```bash
+   docker-compose up -d
+   ```
+9. Optionally check the logs by running: 
+        
+    ```bash
+    docker-compose logs signal
+    docker-compose logs management
+    docker-compose logs coturn
+    docker-compose logs dashboard
+    ``` 
--- a/infrastructure_files/configure.sh
+++ b/infrastructure_files/configure.sh
@ -0,0 +1,7 @@
+#!/bin/bash
+
+unset $(grep -v '^#' ./setup.env | sed -E 's/(.*)=.*/\1/' | xargs)
+export $(grep -v '^#' ./setup.env | xargs)
+
+envsubst < docker-compose.yml.tmpl > docker-compose.yml
+envsubst < management.json.tmpl > management.json
--- a/infrastructure_files/docker-compose.yml
+++ b/infrastructure_files/docker-compose.yml
@ -1,56 +0,0 @@
-version: "3"
-services:
-  #UI dashboard
-  dashboard:
-    image: wiretrustee/dashboard:main
-    restart: unless-stopped
-    ports:
-      - 80:80
-#      - 443:443
-    environment:
-      - AUTH0_DOMAIN=<YOUR AUTH0 DOMAIN>
-      - AUTH0_CLIENT_ID=<YOUR AUTH0 CLIENT ID>
-      - AUTH0_AUDIENCE=<YOUR AUTH0 AUDIENCE>
-      - WIRETRUSTEE_MGMT_API_ENDPOINT=http://localhost:33071
-#      - NGINX_SSL_PORT: 443
-#      - LETSENCRYPT_DOMAIN: <YOUR DOMAIN>
-#      - LETSENCRYPT_EMAIL: <YOUR EMAIL>
-  # Signal
-  signal:
-    image: wiretrustee/signal:latest
-    restart: unless-stopped
-    volumes:
-      - wiretrustee-signal:/var/lib/wiretrustee
-#      - /var/log/wiretrustee/signal.log:/var/log/wiretrustee/signal.log
-    ports:
-      - 10000:10000
-#     # port and command for Let's Encrypt validation
-#      - 443:443
-#    command: ["--letsencrypt-domain", "<YOUR-DOMAIN>", "--log-file", "console"]
-  # Management
-  management:
-    image: wiretrustee/management:latest
-    restart: unless-stopped
-    volumes:
-      - wiretrustee-mgmt:/var/lib/wiretrustee
-      - ./management.json:/etc/wiretrustee/management.json
-#      - /var/log/wiretrustee/management.log:/var/log/wiretrustee/management.log
-    ports:
-      - 33073:33073 #gRPC port
-      - 33071:33071 #HTTP port
-#     # port and command for Let's Encrypt validation
-#      - 443:443
-#    command: ["--letsencrypt-domain", "<YOUR-DOMAIN>", "--log-file", "console"]
-  # Coturn
-  coturn:
-    image: coturn/coturn
-    restart: unless-stopped
-    domainname: stun.wiretrustee.com
-    volumes:
-      - ./turnserver.conf:/etc/turnserver.conf:ro
-#      - ./privkey.pem:/etc/coturn/private/privkey.pem:ro
-#      - ./cert.pem:/etc/coturn/certs/cert.pem:ro
-    network_mode: host
-volumes:
-  wiretrustee-mgmt:
-  wiretrustee-signal:
--- a/infrastructure_files/docker-compose.yml.tmpl
+++ b/infrastructure_files/docker-compose.yml.tmpl
@ -0,0 +1,62 @@
+version: "3"
+services:
+  #UI dashboard
+  dashboard:
+    image: wiretrustee/dashboard:main
+    restart: unless-stopped
+    ports:
+      - 80:80
+      - 443:443
+    environment:
+      - AUTH0_DOMAIN=$WIRETRUSTEE_AUTH0_DOMAIN
+      - AUTH0_CLIENT_ID=$WIRETRUSTEE_AUTH0_CLIENT_ID
+      - AUTH0_AUDIENCE=$WIRETRUSTEE_AUTH0_AUDIENCE
+      - WIRETRUSTEE_MGMT_API_ENDPOINT=https://$WIRETRUSTEE_DOMAIN:33071
+      - NGINX_SSL_PORT=443
+      - LETSENCRYPT_DOMAIN=$WIRETRUSTEE_DOMAIN
+      - LETSENCRYPT_EMAIL=$WIRETRUSTEE_LETSENCRYPT_EMAIL
+    volumes:
+      - /var/lib/wiretrustee/dashboard/letsencrypt:/etc/letsencrypt/
+  # Signal
+  signal:
+    image: wiretrustee/signal:latest
+    restart: unless-stopped
+    volumes:
+      - wiretrustee-signal:/var/lib/wiretrustee
+    #      - /var/log/wiretrustee/signal.log:/var/log/wiretrustee/signal.log
+    ports:
+      - 10000:10000
+  #     # port and command for Let's Encrypt validation
+  #      - 443:443
+  #    command: ["--letsencrypt-domain", "$WIRETRUSTEE_DOMAIN", "--log-file", "console"]
+  # Management
+  management:
+    #  image: wiretrustee/management:latest
+    image: wiretrustee/management:v0.1.0-beta.2-SNAPSHOT-39d450b-amd64
+    restart: unless-stopped
+    depends_on:
+      - dashboard
+    volumes:
+      - wiretrustee-mgmt:/var/lib/wiretrustee
+      - /var/lib/wiretrustee/dashboard/letsencrypt:/etc/letsencrypt:ro
+      - ./management.json:/etc/wiretrustee/management.json
+    #      - /var/log/wiretrustee/management.log:/var/log/wiretrustee/management.log
+    ports:
+      - 33073:33073 #gRPC port
+      - 33071:33071 #HTTP port
+  #     # port and command for Let's Encrypt validation
+  #      - 443:443
+  #    command: ["--letsencrypt-domain", "$WIRETRUSTEE_DOMAIN", "--log-file", "console"]
+  # Coturn
+  coturn:
+    image: coturn/coturn
+    restart: unless-stopped
+    domainname: <YOUR DOMAIN>
+    volumes:
+      - ./turnserver.conf:/etc/turnserver.conf:ro
+    #      - ./privkey.pem:/etc/coturn/private/privkey.pem:ro
+    #      - ./cert.pem:/etc/coturn/certs/cert.pem:ro
+    network_mode: host
+volumes:
+  wiretrustee-mgmt:
+  wiretrustee-signal:
--- a/infrastructure_files/management.json
+++ b/infrastructure_files/management.json
@ -1,37 +0,0 @@
-{
-    "Stuns": [
-        {
-            "Proto": "udp",
-            "URI": "stun:stun.wiretrustee.com:3468",
-            "Username": "",
-            "Password": null
-        }
-    ],
-    "TURNConfig": {
-        "Turns": [
-            {
-                "Proto": "udp",
-                "URI": "turn:stun.wiretrustee.com:3468",
-                "Username": "some_user",
-                "Password": "c29tZV9wYXNzd29yZA=="
-            }
-        ],
-        "CredentialsTTL": "1h",
-        "Secret": "c29tZV9wYXNzd29yZA==",
-        "TimeBasedCredentials": true
-    },
-    "Signal": {
-        "Proto": "http",
-        "URI": "signal.wiretrustee.com:10000",
-        "Username": "",
-        "Password": null
-    },
-    "Datadir": "",
-    "HttpConfig": {
-        "LetsEncryptDomain": "<PASTE YOUR LET'S ENCRYPT DOMAIN HERE>",
-        "Address": "0.0.0.0:33071",
-        "AuthIssuer": "<PASTE YOUR AUTH0 ISSUER HERE>,",
-        "AuthAudience": "<PASTE YOUR AUTH0 AUDIENCE HERE>",
-        "AuthKeysLocation": "<PASTE YOUR AUTH0 PUBLIC JWT KEYS LOCATION HERE>"
-    }
-}
--- a/infrastructure_files/management.json.tmpl
+++ b/infrastructure_files/management.json.tmpl
@ -0,0 +1,39 @@
+{
+    "Stuns": [
+        {
+            "Proto": "udp",
+            "URI": "stun:$WIRETRUSTEE_DOMAIN:3468",
+            "Username": "",
+            "Password": null
+        }
+    ],
+    "TURNConfig": {
+        "Turns": [
+            {
+                "Proto": "udp",
+                "URI": "turn:$WIRETRUSTEE_DOMAIN:3468",
+                "Username": "",
+                "Password": null
+            }
+        ],
+        "CredentialsTTL": "12h",
+        "Secret": "secret",
+        "TimeBasedCredentials": false
+    },
+    "Signal": {
+        "Proto": "http",
+        "URI": "$WIRETRUSTEE_DOMAIN:10000",
+        "Username": "",
+        "Password": null
+    },
+    "Datadir": "",
+    "HttpConfig": {
+        "LetsEncryptDomain": "",
+        "CertFile":"/etc/letsencrypt/live/$WIRETRUSTEE_DOMAIN/fullchain.pem",
+        "CertKey":"/etc/letsencrypt/live/$WIRETRUSTEE_DOMAIN/privkey.pem",
+        "Address": "0.0.0.0:33071",
+        "AuthIssuer": "https://$WIRETRUSTEE_AUTH0_DOMAIN/",
+        "AuthAudience": "$WIRETRUSTEE_AUTH0_AUDIENCE",
+        "AuthKeysLocation": "https://$WIRETRUSTEE_AUTH0_DOMAIN/.well-known/jwks.json"
+    }
+}
--- a/infrastructure_files/setup.env
+++ b/infrastructure_files/setup.env
@ -0,0 +1,10 @@
+# e.g. app.mydomain.com
+WIRETRUSTEE_DOMAIN=""
+# e.g. dev-24vkclam.us.auth0.com
+WIRETRUSTEE_AUTH0_DOMAIN=""
+# e.g. 61u3JMXRO0oOevc7gCkZLCwePQvT4lL0
+WIRETRUSTEE_AUTH0_CLIENT_ID=""
+# e.g. https://app.mydomain.com/
+WIRETRUSTEE_AUTH0_AUDIENCE=""
+# e.g. hello@mydomain.com
+WIRETRUSTEE_LETSENCRYPT_EMAIL=""