extern/zrok
1
1
Fork 0
mirror of https://github.com/openziti/zrok.git synced 2025-03-09 20:41:31 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into bwong365-depends-on-ziti-quickstart

Browse Source 
Signed-off-by: Kenneth Bingham <kenneth.bingham@netfoundry.io>
This commit is contained in:
Kenneth Bingham 2025-01-29 10:10:06 -05:00 committed by GitHub
commit 7bd331993f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
16 changed files with 562 additions and 31 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.flake8
View File

@ -1,3 +1,6 @@
[flake8]
max-line-length = 120
exclude = zrok_api, build
exclude =
./sdk/python/sdk/zrok/zrok_api/**,
./build/**

23
.github/workflows/build-wheels.yml vendored
View File

@ -45,7 +45,7 @@ jobs:
name: zrok_sdk_${{ matrix.spec.target }}
path: ${{ github.workspace }}/sdk/python/sdk/zrok/dist/*
publish:
publish-testpypi:
runs-on: ubuntu-20.04
needs: [ build_wheels ]
permissions:
@ -54,16 +54,10 @@ jobs:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
path: ./download
path: ./dist
merge-multiple: true
pattern: zrok_sdk_*
- name: check
run: |
ls -lR ./download/
mkdir dist
cp ./download/* ./dist/
- name: Publish wheels (TestPYPI)
uses: pypa/gh-action-pypi-publish@release/v1
with:
@ -72,6 +66,19 @@ jobs:
skip-existing: true
verbose: true
publish-pypi:
runs-on: ubuntu-20.04
needs: [ publish-testpypi ]
permissions:
id-token: write
steps:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
path: ./dist
merge-multiple: true
pattern: zrok_sdk_*
- name: Publish wheels (PyPI)
uses: pypa/gh-action-pypi-publish@release/v1
with:

11
CHANGELOG.md
View File

@ -4,6 +4,17 @@
CHANGE: the Docker instance will wait for the ziti container healthy status (contribution from Ben Wong @bwong365 - https://github.com/openziti/zrok/pull/790)
CHANGE: Document solving the DNS propagation timeout for Docker instances that are using Caddy to manage the wildcard certificate.
CHANGE: Add usage hint in `zrok config get --help` to clarify how to list all valid `configName` and their current values by running `zrok status`.
CHANGE: The Python SDK's `Overview()` function was refactored as a class method (https://github.com/openziti/zrok/pull/846).
FEATURE: The Python SDK now includes a `ProxyShare` class providing an HTTP proxy for public and private shares and a
Jupyter notebook example (https://github.com/openziti/zrok/pull/847).
FIX: PyPi publishing was failing due to a CI issue (https://github.com/openziti/zrok/issues/849)
## v0.4.46
FEATURE: Linux service template for systemd user units (https://github.com/openziti/zrok/pull/818)

1
cmd/zrok/configGet.go
View File

@ -18,6 +18,7 @@ func newConfigGetCommand() *configGetCommand {
cmd := &cobra.Command{
Use: "get <configName>",
Short: "Get a value from the environment config",
Long: "Get a value from the environment config. Use 'zrok status' to list available configuration names and current values.",
Args: cobra.ExactArgs(1),
}
command := &configGetCommand{cmd: cmd}

1
cmd/zrok/configSet.go
View File

@ -22,6 +22,7 @@ func newConfigSetCommand() *configSetCommand {
cmd := &cobra.Command{
Use: "set <configName> <value>",
Short: "Set a value into the environment config",
Long: "Set a value into the environment config. Use 'zrok status' to list available configuration names and current values.",
Args: cobra.ExactArgs(2),
}
command := &configSetCommand{cmd: cmd}

19
docker/compose/zrok-instance/README.md
View File

@ -190,7 +190,7 @@ See "My internet connection can only send traffic to common ports" below about c
docker compose logs zrok-controller
```
1. Check the caddy logs.
1. Check the Caddy logs.
It can take a few minutes for Caddy to obtain the wildcard certificate. You can check the logs to see if there were any errors completing the DNS challenge which involves using the Caddy DNS plugin to create a TXT record in your DNS zone. This leverages the API token you provided in the `.env` file, which must have permission to create DNS records in the zrok DNS zone.
@ -198,6 +198,23 @@ See "My internet connection can only send traffic to common ports" below about c
docker compose logs caddy
```
1. Caddy keeps failing to obtain a wildcard certificate because it timed out waiting for DNS.
Symptom: the Caddy log contains "timed out waiting for record to fully propagate." This means that Caddy added a DNS record with your DNS provider's API to prove to the CA it controls the zrok DNS zone, but it wasn't able to verify the record was created successfully with a DNS query.
Solutions:
- Add `propagation_delay` in your `Caddyfile` to delay the first DNS verification query. This avoids caching a verification query failure by waiting a few minutes for the record to become available so the verification query will succeed on the first attempt. Caddy will be unable to verify the DNS record if the failure remains in the cache too long.
- If the prior solution fails, you can override the default resolves/nameservers with `resolvers`, a space-separated list of DNS servers. This gives you more control over if and where the verification query result is cached.
```
tls {
dns {CADDY_DNS_PLUGIN} {CADDY_DNS_PLUGIN_TOKEN}
propagation_timeout 60m # default 2m
propagation_delay 5m # default 0m
}
```
1. `zrok enable` fails certificate verification: ensure you are not using the staging API for Let's Encrypt.
If you are using the staging API, you will see an error about the API certificate when you use the zrok CLI. You can switch to the production API by removing the overriding assignment of the `CADDY_ACME_API` variable.

4
sdk/python/examples/http-server/README.md
View File

@ -4,7 +4,7 @@ This `http-server` example is a minimal zrok application that surfaces a basic H
## Implementation
```go
```python
root = zrok.environment.root.Load()
```
@ -46,6 +46,6 @@ Next, we run the server which ends up calling the following:
@zrok.decor.zrok(opts=zrok_opts)
def runApp():
from waitress import serve
# the port is only used to integrate Zrok with frameworks that expect a "hostname:port" combo
# the port is only used to integrate zrok with frameworks that expect a "hostname:port" combo
serve(app, port=bindPort)
```

2
sdk/python/examples/http-server/server.py
View File

@ -13,7 +13,7 @@ bindPort = 18081
@zrok.decor.zrok(opts=zrok_opts)
def runApp():
from waitress import serve
# the port is only used to integrate Zrok with frameworks that expect a "hostname:port" combo
# the port is only used to integrate zrok with frameworks that expect a "hostname:port" combo
serve(app, port=bindPort)

2
sdk/python/examples/pastebin/pastebin.py
View File

@ -101,4 +101,4 @@ if __name__ == "__main__":
server_thread = threading.Thread(target=options.func, args=[options])
server_thread.start()
server_thread.join()
server_thread.join()

48
sdk/python/examples/proxy/README.md Normal file
View File

@ -0,0 +1,48 @@
# zrok Python Proxy Example
This demonstrates using the ProxyShare class to forward requests from the public frontend to a target URL.
## Run the Example
```bash
LOG_LEVEL=INFO python ./proxy.py http://127.0.0.1:3000
```
Expected output:
```txt
2025-01-29 06:37:00,884 - __main__ - INFO - === Starting proxy server ===
2025-01-29 06:37:00,884 - __main__ - INFO - Target URL: http://127.0.0.1:3000
2025-01-29 06:37:01,252 - __main__ - INFO - Access proxy at: https://24x0pq7s6jr0.zrok.example.com:443
2025-01-29 06:37:07,981 - zrok.proxy - INFO - Share 24x0pq7s6jr0 released
```
## Basic Usage
```python
from zrok.proxy import ProxyShare
import zrok
# Load the environment
root = zrok.environment.root.Load()
# Create a temporary proxy share (will be cleaned up on exit)
proxy = ProxyShare.create(root=root, target="http://my-target-service")
# Access the proxy's endpoints and token
print(f"Access proxy at: {proxy.endpoints}")
proxy.run()
```
## Creating a Reserved Proxy Share
To create a share token that persists and can be reused, run the example `proxy.py --unique-name my-persistent-proxy`. If the unique name already exists it will be reused. Here's how it works:
```python
proxy = ProxyShare.create(
root=root,
target="http://127.0.0.1:3000",
unique_name="my-persistent-proxy"
)
```

100
sdk/python/examples/proxy/proxy.ipynb Normal file
View File

@ -0,0 +1,100 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "52d42237",
"metadata": {},
"outputs": [],
"source": [
"! pip install zrok"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0a33915c",
"metadata": {},
"outputs": [],
"source": [
"\n",
"import zrok\n",
"from zrok.proxy import ProxyShare\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0db6b615",
"metadata": {},
"outputs": [],
"source": [
"\n",
"target_url = \"http://127.0.0.1:8000/\"\n",
"unique_name = \"myuniquename\" # a name to reuse each run or 'None' for random\n",
"share_mode = \"public\" # \"public\" or \"private\"\n",
"frontend = \"public\" # custom domain frontend or \"public\"\n",
"\n",
"if unique_name.lower() == \"none\":\n",
" unique_name = None\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d3efcfa5",
"metadata": {},
"outputs": [],
"source": [
"\n",
"zrok_env = zrok.environment.root.Load() # Load the environment from ~/.zrok\n",
"\n",
"proxy_share = ProxyShare.create(\n",
" root=zrok_env,\n",
" target=target_url,\n",
" frontends=[frontend],\n",
" share_mode=share_mode,\n",
" unique_name=unique_name,\n",
" verify_ssl=True # Set 'False' to skip SSL verification\n",
")\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "21966557",
"metadata": {},
"outputs": [],
"source": [
"\n",
"if share_mode == \"public\":\n",
" print(f\"Access proxy at: {', '.join(proxy_share.endpoints)}\")\n",
"elif share_mode == \"private\":\n",
" print(f\"Run a private access frontend: 'zrok access private {proxy_share.token}'\")\n",
"\n",
"proxy_share.run()\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

62
sdk/python/examples/proxy/proxy.py Normal file
View File

@ -0,0 +1,62 @@
#!/usr/bin/env python3
"""
Example of using zrok's proxy facility to create an HTTP proxy server.
This example demonstrates how to:
1. Create a proxy share (optionally with a unique name for persistence)
2. Handle HTTP requests/responses through the proxy
3. Automatically clean up non-reserved shares on exit
"""
import argparse
import logging
import zrok
from zrok.proxy import ProxyShare
# Setup logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
def main():
"""Main entry point."""
parser = argparse.ArgumentParser(description='Start a zrok proxy server')
parser.add_argument('target_url', help='Target URL to proxy requests to')
parser.add_argument('-n', '--unique-name', help='Unique name for the proxy instance')
parser.add_argument('-f', '--frontends', nargs='+', help='One or more space-separated frontends to use')
parser.add_argument('-k', '--insecure', action='store_false', dest='verify_ssl', default=True,
help='Skip SSL verification')
parser.add_argument('-s', '--share-mode', default='public', choices=['public', 'private'],
help='Share mode (default: public)')
args = parser.parse_args()
logger.info("=== Starting proxy server ===")
logger.info(f"Target URL: {args.target_url}")
logger.info(f"Share mode: {args.share_mode}")
# Load environment and create proxy share
root = zrok.environment.root.Load()
proxy_share = ProxyShare.create(
root=root,
target=args.target_url,
share_mode=args.share_mode,
unique_name=args.unique_name,
frontends=args.frontends,
verify_ssl=args.verify_ssl
)
# Log access information and start the proxy
if args.share_mode == "public":
logger.info(f"Access proxy at: {', '.join(proxy_share.endpoints)}")
elif args.share_mode == "private":
logger.info(f"Run a private access frontend: 'zrok access private {proxy_share.token}'")
proxy_share.run()
if __name__ == '__main__':
main()

6
sdk/python/sdk/zrok/zrok/environment/root.py
View File

@ -19,6 +19,7 @@ class Metadata:
@dataclass
class Config:
ApiEndpoint: str = ""
DefaultFrontend: str = ""
@dataclass
@ -137,7 +138,10 @@ def __loadConfig() -> Config:
cf = configFile()
with open(cf) as f:
data = json.load(f)
return Config(ApiEndpoint=data["api_endpoint"])
return Config(
ApiEndpoint=data["api_endpoint"],
DefaultFrontend=data["default_frontend"]
)
def isEnabled() -> bool:

90
sdk/python/sdk/zrok/zrok/overview.py
View File

@ -1,20 +1,78 @@
from zrok.environment.root import Root
import json
from dataclasses import dataclass, field
from typing import List
import urllib3
from zrok.environment.root import Root
from zrok_api.models.environment import Environment
from zrok_api.models.environment_and_resources import EnvironmentAndResources
from zrok_api.models.frontends import Frontends
from zrok_api.models.share import Share
def Overview(root: Root) -> str:
if not root.IsEnabled():
raise Exception("environment is not enabled; enable with 'zrok enable' first!")
@dataclass
class Overview:
environments: List[EnvironmentAndResources] = field(default_factory=list)
http = urllib3.PoolManager()
apiEndpoint = root.ApiEndpoint().endpoint
try:
response = http.request(
'GET',
apiEndpoint + "/api/v1/overview",
headers={
"X-TOKEN": root.env.Token
})
except Exception as e:
raise Exception("unable to get account overview", e)
return response.data.decode('utf-8')
@classmethod
def create(cls, root: Root) -> 'Overview':
if not root.IsEnabled():
raise Exception("environment is not enabled; enable with 'zrok enable' first!")
http = urllib3.PoolManager()
apiEndpoint = root.ApiEndpoint().endpoint
try:
response = http.request(
'GET',
apiEndpoint + "/api/v1/overview",
headers={
"X-TOKEN": root.env.Token
})
except Exception as e:
raise Exception("unable to get account overview", e)
json_data = json.loads(response.data.decode('utf-8'))
overview = cls()
for env_data in json_data.get('environments', []):
env_dict = env_data['environment']
# Map the JSON keys to the Environment class parameters
environment = Environment(
description=env_dict.get('description'),
host=env_dict.get('host'),
address=env_dict.get('address'),
z_id=env_dict.get('zId'),
activity=env_dict.get('activity'),
limited=env_dict.get('limited'),
created_at=env_dict.get('createdAt'),
updated_at=env_dict.get('updatedAt')
)
# Create Shares object from share data
share_list = []
for share_data in env_data.get('shares', []):
share = Share(
token=share_data.get('token'),
z_id=share_data.get('zId'),
share_mode=share_data.get('shareMode'),
backend_mode=share_data.get('backendMode'),
frontend_selection=share_data.get('frontendSelection'),
frontend_endpoint=share_data.get('frontendEndpoint'),
backend_proxy_endpoint=share_data.get('backendProxyEndpoint'),
reserved=share_data.get('reserved'),
activity=share_data.get('activity'),
limited=share_data.get('limited'),
created_at=share_data.get('createdAt'),
updated_at=share_data.get('updatedAt')
)
share_list.append(share)
# Create EnvironmentAndResources object
env_resources = EnvironmentAndResources(
environment=environment,
shares=share_list,
frontends=Frontends() # Empty frontends for now as it's not in the input data
)
overview.environments.append(env_resources)
return overview

203
sdk/python/sdk/zrok/zrok/proxy.py Normal file
View File

@ -0,0 +1,203 @@
"""
Proxy share management functionality for the zrok SDK.
"""
import atexit
import logging
import urllib.parse
from dataclasses import dataclass
from typing import Any, Dict, List, Optional
import requests
from flask import Flask, Response, request
from waitress import serve
from zrok.environment.root import Root
from zrok.model import (PROXY_BACKEND_MODE, PUBLIC_SHARE_MODE, PRIVATE_SHARE_MODE, Share,
ShareRequest)
from zrok.overview import Overview
from zrok.share import CreateShare, ReleaseReservedShare
import zrok
logger = logging.getLogger(__name__)
# List of hop-by-hop headers that should not be returned to the viewer
HOP_BY_HOP_HEADERS = {
'connection',
'keep-alive',
'proxy-authenticate',
'proxy-authorization',
'te',
'trailers',
'transfer-encoding',
'upgrade'
}
# The proxy only listens on the zrok socket, the port used to initialize the Waitress server is not actually bound or
# listening
DUMMY_PORT = 18081
@dataclass
class ProxyShare:
"""Represents a proxy share with its configuration and state."""
root: Root
share: Share
target: str
unique_name: Optional[str] = None
_cleanup_registered: bool = False
_app: Optional[Flask] = None
verify_ssl: bool = True
@classmethod
def create(cls, root: Root, target: str, share_mode: str = PUBLIC_SHARE_MODE, unique_name: Optional[str] = None,
frontends: Optional[List[str]] = None, verify_ssl: bool = True) -> 'ProxyShare':
"""
Create a new proxy share, handling reservation and cleanup logic based on unique_name.
Args:
root: The zrok root environment
target: Target URL or service to proxy to
unique_name: Optional unique name for a reserved share
frontends: Optional list of frontends to use, takes precedence over root's default_frontend
verify_ssl: Whether to verify SSL certificates when forwarding requests.
Returns:
ProxyShare instance configured with the created share
"""
# First check if we have an existing reserved share with this name
if unique_name:
existing_share = cls._find_existing_share(root, unique_name)
if existing_share:
logger.debug(f"Found existing share with token: {existing_share.Token}")
return cls(
root=root,
share=existing_share,
target=target,
unique_name=unique_name,
verify_ssl=verify_ssl
)
# Compose the share request
share_frontends = []
if share_mode == PUBLIC_SHARE_MODE:
if frontends:
share_frontends = frontends
elif root.cfg and root.cfg.DefaultFrontend:
share_frontends = [root.cfg.DefaultFrontend]
else:
share_frontends = ['public']
share_request = ShareRequest(
BackendMode=PROXY_BACKEND_MODE,
ShareMode=share_mode,
Target=target,
Frontends=share_frontends,
Reserved=True
)
if unique_name:
share_request.UniqueName = unique_name
# Create the share
share = CreateShare(root=root, request=share_request)
if share_mode == PUBLIC_SHARE_MODE:
logger.debug(f"Created new proxy share with endpoints: {', '.join(share.FrontendEndpoints)}")
elif share_mode == PRIVATE_SHARE_MODE:
logger.debug(f"Created new private share with token: {share.Token}")
# Create class instance and setup cleanup-at-exit if we reserved a random share token
instance = cls(
root=root,
share=share,
target=target,
unique_name=unique_name,
verify_ssl=verify_ssl
)
if not unique_name:
instance.register_cleanup()
return instance
@staticmethod
def _find_existing_share(root: Root, unique_name: str) -> Optional[Share]:
"""Find an existing share with the given unique name."""
overview = Overview.create(root=root)
for env in overview.environments:
if env.environment.z_id == root.env.ZitiIdentity:
for share in env.shares:
if share.token == unique_name:
return Share(Token=share.token, FrontendEndpoints=[share.frontend_endpoint])
return None
def register_cleanup(self):
"""Register cleanup handler to release randomly generated shares on exit."""
if not self._cleanup_registered:
def cleanup():
try:
ReleaseReservedShare(root=self.root, shr=self.share)
logger.info(f"Share {self.share.Token} released")
except Exception as e:
logger.error(f"Error during cleanup: {e}")
# Register for normal exit only
atexit.register(cleanup)
self._cleanup_registered = True
return cleanup # Return the cleanup function for reuse
def _create_app(self) -> Flask:
"""Create and configure the Flask app for proxying."""
app = Flask(__name__)
@app.route('/', defaults={'path': ''}, methods=['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'OPTIONS'])
@app.route('/<path:path>', methods=['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'OPTIONS'])
def proxy(path):
# Construct the target URL
url = urllib.parse.urljoin(self.target, path)
# Forward the request
resp = requests.request(
method=request.method,
url=url,
headers={key: value for (key, value) in request.headers
if key.lower() not in HOP_BY_HOP_HEADERS},
data=request.get_data(),
cookies=request.cookies,
allow_redirects=False,
stream=True,
verify=self.verify_ssl
)
# Create the response
excluded_headers = HOP_BY_HOP_HEADERS.union({'host'})
headers = [(name, value) for (name, value) in resp.raw.headers.items()
if name.lower() not in excluded_headers]
return Response(
resp.iter_content(chunk_size=10*1024),
status=resp.status_code,
headers=headers
)
return app
def run(self):
"""Run the proxy server."""
if not self._app:
self._app = self._create_app()
zrok_opts: Dict[str, Any] = {}
zrok_opts['cfg'] = zrok.decor.Opts(root=self.root, shrToken=self.token, bindPort=DUMMY_PORT)
@zrok.decor.zrok(opts=zrok_opts)
def run_server():
serve(self._app, port=DUMMY_PORT, _quiet=True)
run_server()
@property
def endpoints(self) -> List[str]:
"""Get the frontend endpoints for this share."""
return self.share.FrontendEndpoints
@property
def token(self) -> str:
"""Get the share token."""
return self.share.Token

16
sdk/python/sdk/zrok/zrok/share.py
View File

@ -104,3 +104,19 @@ def DeleteShare(root: Root, shr: model.Share):
ShareApi(zrok).unshare(body=req)
except Exception as e:
raise Exception("error deleting share", e)
def ReleaseReservedShare(root: Root, shr: model.Share):
req = UnshareRequest(env_zid=root.env.ZitiIdentity,
shr_token=shr.Token,
reserved=True)
try:
zrok = root.Client()
except Exception as e:
raise Exception("error getting zrok client", e)
try:
ShareApi(zrok).unshare(body=req)
except Exception as e:
raise Exception("error releasing share", e)