EtherGuard-VPN/example_config/super_mode/README.md
2022-02-10 16:01:39 +00:00

19 KiB
Raw Blame History

Etherguard

English | 中文

Super mode

This mode is inspired by n2n. There 2 types of node: SuperNode and EdgeNode
EdgeNode must connect to SuperNode firstget connection info of other EdgeNode from the SuperNode
The SuperNode runs Floyd-Warshall Algorithmand distribute the result to all other EdgeNodes.

Quick start

Edit the file gensuper.yaml based on your requirement first.

Config output dir: /tmp/eg_gen
Enable generated config overwrite: false # Allow overwrite while output the config
Add NodeID to the interface name: false  # Add NodeID to the interface name in generated edge config
ConfigTemplate for super node: ""
ConfigTemplate for edge node: ""
Network name: eg_net
Super Node:
  Listen port: 3456
  EdgeAPI prefix: /eg_net/eg_api
  Endpoint(IPv4)(optional): example.com
  Endpoint(IPv6)(optional): example.com
  Endpoint(EdgeAPI): http://example.com:3456/eg_net/eg_api
Edge Node:
  Node IDs: "[1~10,11,19,23,29,31,55~66,88~99]"
  MacAddress prefix: ""                 # Leave blank to generate randomly
  IPv4 range: 192.168.76.0/24           # The IP part can be omitted
  IPv6 range: fd95:71cb:a3df:e586::/64  # 
  IPv6 LL range: fe80::a3df:0/112       #  

Then run this, and the required configuration file will be generated.

$ ./etherguard-go -mode gencfg -cfgmode super -config example_config/super_mode/gensuper.yaml

Run this in SuperNode

./etherguard-go -config [config path] -mode super

Run this in EdgeNode

./etherguard-go -config [config path] -mode edge

Documentation

This is the documentation of the super_mode of this example_config Before reading this, I'd like to suggest you read the static mode first.

In the super mode of the edge node, the NextHopTable and Peers section are useless. All infos are download from super node.
Meanwhile, super node will generate pre shared key for inter-edge communication(if UsePSKForInterEdge enabled).

SuperMsg

There are new type of DstID called SuperMsg(65534). All packets sends to and receive from super node are using this packet type.
This packet will not send to any other edge node, just like DstID == self.NodeID

Control Message

In Super mode, Beside Normal Packet. We introduce a new packet type called Control Message. In Super mode, we will not relay any control message. We just receive or send it to target directly.
We list all the control message we use in the super mode below.

Register

This control message works like this picture: Workflow of Register

  1. EdgeNode send Register to the super node
  2. SuperNode knows it's external IP and port number
  3. Update it to database and distribute UpdatePeerMsg to all edges
  4. Other EdgeNodes get the notification, download the updated peer infos from SuperNode via HTTP API

Ping/Pong

While EdgeNodes get their peer info, they will trying to talk each other directly like this picture: Workflow of Ping/Pong

  1. Send Ping to all other edges with local time with TTL=0
  2. Receive a Ping, Subtract the peer time from local time, we get a single way latency.
  3. Send a Pong to SuperNode with single way latency, let SuperNode calculate the NextHopTable
  4. Wait the SuperNode push UpdateNhTable message and download it.

AdditionalCost

While we have all latency data of all nodes, AdditionalCost will be applied before Floyd-Warshall calculated.

Take the situation of this picture as an example: EGS08

Path Latency Cost Win
A->B->C 3ms 3
A->C 4ms 4 O

In this situation, the difference between 3ms and 4ms is only 1ms Its not worth to save this 1ms, and the forwarding itself takes time

With the AdditionalCost parameter, each node can set the additional cost of forwarding through this node

If ABC is all set to AdditionalCost=10

Path Latency AdditionalCost Cost Win
A->B->C 3ms 20 23
A->C 4ms 10 14 O

A->C will use direct connection instead of forward via B in order to save 1ms
Here AdditionalCost=10 can be interpreted as: It have to save 10ms to transfer by this Node.

UpdateNhTable

While supernode get a Pong message, it will update the Distance matrix and run the Floyd-Warshall Algorithm to calculate the NextHopTable.
image
If there are any changes of this table, it will distribute UpdateNhTable to all edges to till then download the latest NextHopTable via HTTP API as soon as possible.

ServerUpdate

Send message to EdgeMode from SuperNode

  1. Turn off EdgeNode
    • Version Not match
    • Wrong NodeID
    • Deleted by SuperNode
  2. Notify EdgeNode there are something new
    • UpdateNhTable
    • UpdatePeer
    • UpdateSuperParams

HTTP EdgeAPI

Why we use HTTP API instead of pack all information in the UpdateXXX?
Because UDP is an unreliable protocol, there is an limit on the amount of content that can be carried.
But the peer list contains all the peer information, the length is not fixed, it may exceed
So we use UpdateXXX to tell we have a update, please download the latest information from SuperNode via HTTP API as soon as possible. And UpdateXXX itself is not reliable, maybe it didn't reach the edge node at all.
So the information of UpdateXXX carries the state hash. Bring it when with HTTP API. When the super node receives the HTTP API and sees the state hash, it knows that the edge node has received the UpdateXXX.
Otherwise, it will send UpdateXXX to the node again after few seconds.

The default configuration is to use HTTP. But for the sake of your security, it is recommended to use an reverse-proxy ot convert it into https I have thought about the development of SuperNode to natively support https, but the dynamic update of the certificate costs me too much time.

HTTP Manage API

HTTP also has some APIs for the front-end to help manage the entire network

super/state

curl "http://127.0.0.1:3456/eg_net/eg_api/manage/super/state?Password=passwd_showstate"

It can show some information such as single way latency or last seen time.
We can visualize it by Force-directed graph drawing.

There is an Infinity section in the json response. It should be 9999. It means infinity if the number larger than it.
Cuz json can't present infinity so that I use this trick.
While we see the latency larger than this, we doesn't need to draw lines in this two nodes.

Example return value:

{
  "PeerInfo": {
    "1": {
      "Name": "Node_01",
      "LastSeen": "2021-12-05 21:21:56.039750832 +0000 UTC m=+23.401193649"
    },
    "2": {
      "Name": "Node_02",
      "LastSeen": "2021-12-05 21:21:57.711616169 +0000 UTC m=+25.073058986"
    }
  },
  "Infinity": 99999,
  "Edges": {
    "1": {
      "2": 0.002179297
    },
    "2": {
      "1": -0.00030252
    }
  },
  "Edges_Nh": {
    "1": {
      "2": 0.012179297
    },
    "2": {
      "1": 0.00969748
    }
  },
  "NhTable": {
    "1": {
      "2": 2
    },
    "2": {
      "1": 1
    }
  },
  "Dist": {
    "1": {
      "1": 0,
      "2": 0.012179297
    },
    "2": {
      "1": 0.00969748,
      "2": 0
    }
  }
}

Section meaning:

  1. PeerInfo: NodeIDNameLastSeen
  2. Edges: The Single way latency99999 or missing means unreachable(UDP hole punching failed)
  3. Edges_Nh: Edges with AdditionalCost
  4. NhTable: Calculate result.
  5. Dist: The latency of packet through Etherguard

peer/add

We can add new edges with this API without restart the SuperNode

Exanple:

curl -X POST "http://127.0.0.1:3456/eg_net/eg_api/manage/peer/add?Password=passwd_addpeer" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "NodeID=100&Name=Node_100&PubKey=DG%2FLq1bFpE%2F6109emAoO3iaC%2BshgWtdRaGBhW3soiSI%3D&AdditionalCost=1000&PSKey=w5t64vFEoyNk%2FiKJP3oeSi9eiGEiPteZmf2o0oI2q2U%3D&SkipLocalIP=false"

Parameter:

  1. URL query: Password: Password. Configured in the config file.
  2. Post body:
    1. NodeID: Node ID
    2. Name: Name
    3. PubKey: Public Key
    4. PSKey: Pre shared Key
    5. AdditionalCost: Additional cost for packet transfer. Unit: ms
    6. SkipLocalIP: Skip local IP reported by the node
    7. nexthoptable: If the graphrecalculatesetting of your super node is in static mode, you need to provide a new NextHopTable in json format in this parameter.

Return value:

  1. http code != 200: Error reason
  2. http code == 200An example edge config.
    • generate by contents in edgetemplate with custom data (nodeid/name/pubkey)
    • Convenient for users to copy and paste

peer/del

Delete peer

There are two deletion modes, namely password deletion and private key deletion.
Designed to be used by administrators, or for people who join the network and want to leave the network.

Use Password to delete any node. Take the newly added node above as an example, use this API to delete the node

curl "http://127.0.0.1:3456/eg_net/eg_api/manage/peer/del?Password=passwd_delpeer&NodeID=100"

We can also use privkey to delete, the same as above, but use privkey parameter only.

curl "http://127.0.0.1:3456/eg_net/eg_api/manage/peer/del?PrivKey=iquaLyD%2BYLzW3zvI0JGSed9GfDqHYMh%2FvUaU0PYVAbQ%3D"

Parameter:

  1. URL query:
    1. Password: Password: Password. Configured in the config file.
    2. nodeid: Node ID that you want to delete
    3. privkey: The private key of the edge

Return value:

  1. http code != 200: Error reason
  2. http code == 200: Success message

peer/update

curl -X POST "http://127.0.0.1:3456/eg_net/eg_api/manage/peer/update?Password=passwd_updatepeer&NodeID=1" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "AdditionalCost=10&SkipLocalIP=false"

super/update

curl -X POST "http://127.0.0.1:3456/eg_net/eg_api/manage/super/update?Password=passwd_updatesuper" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "SendPingInterval=15&HttpPostInterval=60&PeerAliveTimeout=70&DampingFilterRadius=3"

SuperNode Config Parameter

Key Description
NodeName node name
PostScript Running script after initialized
PrivKeyV4 Private key for IPv4 session
PrivKeyV6 Private key for IPv6 session
ListenPort UDP listen port
ListenPort_EdgeAPI HTTP EdgeAPI listen port
ListenPort_ManageAPI HTTP ManageAPI listen port
API_Prefix HTTP API prefix
RePushConfigInterval The interval of pushUpdateXXX
HttpPostInterval The interval of report by HTTP Edge API
PeerAliveTimeout The time of inactive which marks peer offline
SendPingInterval The interval that send pings/pongs between EdgeNodes
LogLevel Log related settings
Passwords Password for HTTP ManageAPI, 5 API passwords are independent
GraphRecalculateSetting Some parameters related to [Floyd-Warshall algorithm](https://zh.wikipedia.org/zh-tw/Floyd-Warshall algorithm)
NextHopTable NextHopTable used by StaticMode
EdgeTemplate for HTTP ManageAPI peer/add. Refer to this configuration file and show a sample configuration file of the edge to the user
UsePSKForInterEdge Whether to enable pre-share key communication between edges.
If enabled, SuperNode will generate PSK for edges automatically
Peers EdgeNode information
Passwords Description
ShowState HTTP ManageAPI Password for super/state
AddPeer HTTP ManageAPI Password for peer/add
DelPeer HTTP ManageAPI Password for peer/del
UpdatePeer HTTP ManageAPI Password for peer/update
UpdateSuper HTTP ManageAPI Password for super/update
GraphRecalculateSetting Description
StaticMode Disable Floyd-Warshall, use NextHopTablein the configuration instead.
SuperNode for udp hole punching only.
ManualLatency Set latency manually, ignore Edge reported latency.
JitterTolerance Jitter tolerance, after receiving Pong, one 37ms and one 39ms will not trigger recalculation
Compared to last calculation
JitterToleranceMultiplier high ping allows more errors
https://www.desmos.com/calculator/raoti16r5n
DampingFilterRadius Windows radius for the low pass filter for latency damping prevention
TimeoutCheckInterval The interval to check if there any Pong packet timed out, and recalculate the NhTable
RecalculateCoolDown Floyd-Warshal is an O(n^3)time complexity algorithm
This option set a cooldown, and prevent it cost too many CPU
Connect/Disconnect event ignores this cooldown.
Peers Description
NodeID Peer's node ID
PubKey Peer's public key
PSKey Pre shared key
AdditionalCost AdditionalCost(unit:ms)
-1 means uses client's self configuration.
SkipLocalIP Ignore Edge reported local IP, use public IP only while udp-hole-punching

EdgeNode Config Parameter

EdgeConfig Root

DynamicRoute Description
SendPingInterval The interval that send pings/pongs between EdgeNodes(sec)
PeerAliveTimeout The time of inactive which marks peer offline(sec)
TimeoutCheckInterval The interval of check PeerAliveTimeout(sec)
ConnNextTry After marked offline, the interval of switching Endpoint(sec)
DupCheckTimeout Duplication chack timeout.(sec)
AdditionalCost AdditionalCost(unit:ms)
SaveNewPeers Save peer info to local file.
SuperNode SuperNode related configs
P2P P2P related configs
NTPConfig NTP related configs
SuperNode Description
UseSuperNode Enable SuperMode
PSKey PreShared Key to communicate to SuperNode
EndpointV4 IPv4 Endpoint of the SuperNode
PubKeyV4 Public Key for IPv4 session to SuperNode
EndpointV6 IPv6 Endpoint of the SuperNode
PubKeyV6 Public Key for IPv6 session to SuperNode
EndpointEdgeAPIUrl The EdgeAPI of the SuperNode
SkipLocalIP Do not report local IP to SuperNode.
SuperNodeInfoTimeout Experimental option, SuperNode offline timeout, switch to P2P mode
P2P mode needs to be enabled first
This option is useless while UseP2P=false
P2P mode has not been tested, stability is unknown, it is not recommended for production use
NTPConfig Description
UseNTP Sync time at startup
MaxServerUse Use how many server to sync time
SyncTimeInterval The interval of syncing time
NTPTimeout NTP server connection Timeout
Servers NTP server list

V4 V6 Two Keys

Why we split IPv4 and IPv6 into two session? Because of this situation

OneChannel

In this case, SuperNode does not know the external ipv4 address of Node02 and cannot help Node1 and Node2 to UDP hole punch.

TwoChannel

So like this, both V4 and V6 establish a session, so that both V4 and V6 can be taken care of at the same time.

UDP hole punch reachability

For different NAT type, the UDP hole punch reachability can refer this table.(Origin)

reachability between NAT types

And if both sides are using ConeNAT, it's not gerenteed to punch success. It depends on the topology and the devices attributes.
Like the section 3.5 in this article, we can't punch success.

Notice for Relay node

Unlike n2n, our supernode do not relay any packet for edges.
If the edge punch failed and no any route available, it's just unreachable. In this case we need to setup a relay node.

Relay node is a regular edge in public network, but interface=dummy.

And we have to note that do not use 127.0.0.1 to connect to supernode.
Because supernode well distribute the source IP of the nodes to all other edges. But 127.0.0.1 is not accessible from other edge.

Setup relay node

To avoid this issue, please use the external IP of the supernode in the edge config.

Quick start

Run this example_config (please open three terminals):

./etherguard-go -config example_config/super_mode/Node_super.yaml -mode super
./etherguard-go -config example_config/super_mode/Node_edge001.yaml -mode edge
./etherguard-go -config example_config/super_mode/Node_edge002.yaml -mode edge

Because it is in stdio mode, stdin will be read into the VPN network
Please type in one of the edge windows

b1aaaaaaaaaa

b1 will be converted into a 12byte layer 2 header, b is the broadcast address FF:FF:FF:FF:FF:FF, 1 is the ordinary MAC address AA:BB:CC:DD:EE:01, aaaaaaaaaa is the payload, and then feed it into the VPN
You should be able to see the string b1aaaaaaaaaa on another window. The first 12 bytes are converted back

Next: P2P Mode