Small code cleaning in the iface package. These changes necessary to
get a clean code in case if we involve more platforms. The OS related
functions has been distributed into separate files and it has been
mixed with not OS related logic. The goal is to get a clear picture
of the layer between WireGuard and business logic.
* Disable upstream DNS resolver after several tries and fails
* Add tests for upstream fails
* Use an extra flag to disable domains in DNS upstreams
* Fix hashing IPs of nameservers for updates.
avoid sending admin or management URLs on service start
as it doesn't have an input
Parse management and admin URL when needed
Pass empty admin url on commands to prevent default overwrite
Adding --external-ip-map and --dns-resolver-address to up command and shorthand option to global flags.
Refactor get and read config functions with new ConfigInput type.
updated cobra package to latest release.
This PR adds system activity tracking.
The management service records events like
add/remove peer, group, rule, route, etc.
The activity events are stored in the SQLite event store
and can be queried by the HTTP API.
If peer is deleted in the console,
we set its state as needs login
On Down command we clean any previous state errors
this prevents need for daemon restart
Removed state error wrapping when engine exits, log is enough
Use stdout and stderr log path only if on Linux and attempt to create the path
Update status system with FQDN fields and
status command to display the domain names of remote and local peers
Set some DNS logs to tracing
update readme file
Added host configurators for Linux, Windows, and macOS.
The host configurator will update the peer system configuration
directing DNS queries according to its capabilities.
Some Linux distributions don't support split (match) DNS or custom ports,
and that will be reported to our management system in another PR
Due to peer reconnects when restarting the Management service,
there are lots of SaveStore operations to update peer status.
Store.SavePeerStatus stores peer status separately and the
FileStore implementation stores it in memory.
Added DNS update protocol message
Added sync to clients
Update nameserver API with new fields
Added default NS groups
Added new dns-name flag for the management service append to peer DNS label
If the gateway address would be nil which is
the case on macOS, we return the preferredSrc
added tests for getExistingRIBRouteGateway function
update log message
* Seticon only when status changes
This prevents a memory leak with the systray lib
when setting the icon every 2 seconds causes a large memory consumption
see https://github.com/getlantern/systray/issues/135
* Use fork with permanent fix
* Add additional check for needed kernel modules
* Check if wireguard and tun modules are loaded
If modules are loaded return true, otherwise attempt to load them
* fix state check
* Add module function tests
* Add test execution in container
* run client package tests on docker
* add package comment to new file
* force entrypoint
* add --privileged flag
* clean only if tables where created
* run from within the directories
Handle routes updates from management
Manage routing firewall rules
Manage peer RIB table
Add get peer and get notification channel from the status recorder
Update interface peers allowed IPs
Added additional common blacklisted interfaces
Updated the signal protocol to pass the peer port and netbird version
Co-authored-by: braginini <bangvalo@gmail.com>
Support Generic OAuth 2.0 Device Authorization Grant
as per RFC specification https://www.rfc-editor.org/rfc/rfc8628.
The previous version supported only Auth0 as an IDP backend.
This implementation enables the Interactive SSO Login feature
for any IDP compatible with the specification, e.g., Keycloak.
This PR fixes a race condition that happens
when agents connect to a Signal stream, multiple
times within a short amount of time. Common on
slow and unstable internet connections.
Every time an agent establishes a new connection
to Signal, Signal creates a Stream and writes an entry
to the registry of connected peers storing the stream.
Every time an agent disconnects, Signal removes the
stream from the registry.
Due to unstable connections, the agent could detect
a broken connection, and attempt to reconnect to Signal.
Signal will override the stream, but it might detect
the old broken connection later, causing peer deregistration.
It will deregister the peer leaving the client thinking
it is still connected, rejecting any messages.
All the existing agents by default connect to port 33073 of the
Management service. This value is also stored in the local config.
All the agents won't switch to the new port 443
unless explicitly specified in the config.
We want the transition to be smooth for our users, therefore
this PR adds logic to check whether the old port 33073 can be
changed to 443 and updates the config automatically.
This PR is a part of an effort to use standard ports (443 or 80) that are usually allowed by default in most of the environments.
Right now Management Service runs the Let'sEncrypt manager on port 443, HTTP API server on port 33071,
and a gRPC server on port 33073. There are three separate listeners.
This PR combines these listeners into one.
With this change, the HTTP and gRPC server runs on either 443 with TLS or 80 without TLS
by default (no --port specified).
Let's Encrypt manager always runs on port 443 if enabled.
The backward compatibility server runs on port 33073 (with TLS or without).
HTTP port 33071 is obsolete and not used anymore.
Newly installed agents will connect to port 443 by default instead of port 33073 if not specified otherwise.
When building client without CGO, user.Lookup
attempts to get user from /etc/passwd
Which doesn't have the user as MacOS uses
opendirectoryd as user directory
This PR fixes issues with the terminal when
running netbird ssh to a remote agent.
Every session looks up a user and loads its
profile. If no user is found, the connection is rejected.
The default user is root.
The Management client will try reconnecting in case.
of network issues or non-permanent errors.
If the device was off-boarded, then the client will stop retrying.
This PR adds support for SSH access through the NetBird network
without managing SSH skeys.
NetBird client app has an embedded SSH server (Linux/Mac only)
and a netbird ssh command.
Before this change, NetBird Agent wasn't handling
peer interface configuration changes dynamically.
Also, remote peer configuration changes have
not been applied (e.g. AllowedIPs changed).
Not a very common cause, but still it should be handled.
Now, Agent reacts to PeerConfig changes sent from the
management service and restarts remote connections
if AllowedIps have been changed.
The peer IP allocation logic was allocating sequential peer IP from the 100.64.0.0/10
address block.
Each account is created with a random subnet from 100.64.0.0/10.
The total amount of potential subnets is 64.
The new logic allocates random peer IP
from the account subnet.
This gives us flexibility to add support for
multi subnet accounts without overlapping IPs.
* GetClientID method and increase interval on slow_down err
* Reuse existing authentication flow if is not expired
Created a new struct to hold additional info
about the flow
If there is a waiting sso running, we cancel its context
* Run the up command on a goroutine
* Use time.Until
* Use proper ctx and consistently use goroutine for up/down
* Fix UP calls when state is idle
When we want to login we can call server.Login
It already checks the login status of the peer
* Remove unused status
* Defer close daemon client conn
Co-authored-by: braginini <bangvalo@gmail.com>
Send Desktop UI client version as user-agent to daemon
This is sent on every login request to the management
Parse the GRPC context on the system package and
retrieves the user-agent
Management receives the new UIVersion field and
store in the Peer's system meta
Updated windows installer package generation with
launch UI after install
remove older version
remove wiretrustee
added install and uninstall scripts
Updated brew cask:
run installer script to start daemon
Daemon conflicts with wiretrustee on brew
Removed migrate check on non-root commands like status
CLI CMD is now going to stdout
Added MacOS icons, plist, and cask template file
Adjusted goreleaser with the new name for all builds
Added Icon and update windows-ui build to include it and avoid console
migrated Docker builds to new namespace netbirdio
UI and CLI Clients are now able to use SSO login by default
we will check if the management has configured or supports SSO providers
daemon will handle fetching and waiting for an access token
Oauth package was moved to internal to avoid one extra package at this stage
Secrets were removed from OAuth
CLI clients have less and better output
2 new status were introduced, NeedsLogin and FailedLogin for better messaging
With NeedsLogin we no longer have endless login attempts
The management will validate the JWT as it does in the API
and will register the Peer to the user's account.
New fields were added to grpc messages in management
and client daemon and its clients were updated
Peer has one new field, UserID,
that will hold the id of the user that registered it
JWT middleware CheckJWT got a splitter
and renamed to support validation for non HTTP requests
Added test for adding new Peer with UserID
Lots of tests update because of a new field
Agent systray UI has been extended with
a setting window that allows configuring
management URL, admin URL and
supports pre-shared key.
While for the Netbird managed version
the Settings are not necessary, it helps
to properly configure the self-hosted version.
Add method for rotating access token with refresh tokens
This will be useful for catching expired sessions and
offboarding users
Also added functions to handle secrets. They have to be revisited
as some tests didn't run on CI as they waited some user input, like password
Updates test workflows with serial execution to avoid collision
of ports and resource names.
Also, used -exec sudo flag for UNIX tests and removed not-needed
limits configuration on Linux and added a 5 minutes timeout.
Updated the multi-peer tests in the client/internal/engine_test.go
to provide proper validation when creating or starting
a peer engine instance fails.
As some operations of the tests running on windows
are slow, we will experiment with disabling the Defender before
restoring cache and checkout a repository, then we reenable
it to run the tests.
disabled extra logs for windows interface
When stopping engine, all peer conns have to be closed
and for each peer WireGuard iface is called
to remove WireGuard peer.
This operation happens in a goroutine causing
Engine to remove the whole WireGuard interface before.
Therefore consequent calls to RemovePeer are unsuccessful.
This fix just adds a small delay before removing interface.
* Call start services function for tests
when testing CMDs we were using some global
variables which got replaced by parallel test
Now we will call a single function independently
for each test
When one of the peers has a static public host IP
or both peers are in the same local network
we establish a direct Wireguard connection
bypassing proxy usage.
This helps reduce FD usage and improves
performance.
* Create temp file before saving data
On the event of full disk, we may encounter the case where the
destination file get replaced by an empty file as the
ioutil.WriteFile truncates the destination before write.
* Close the tempFile instance before moving it
* Blacklist Wireguard interfaces for ICE checks
* test: WIP mocking the grpc server for testing the sending of the client information
* WIP: Test_SystemMetaDataFromClient with mocks, todo:
* fix: failing meta data test
* test: add system meta expectation in management client test
* fix: removing deprecated register function, replacing with new one
* fix: removing deprecated register function from mockclient interface impl
* fix: fixing interface declaration
* chore: remove unused commented code
Co-authored-by: braginini <bangvalo@gmail.com>
* moved wiretrustee version from main to system.info
* added wiretrustee version for all supported platforms
* typo corrected
* refactor: use single WiretrusteeVersion() func to get version of the client
Co-authored-by: braginini <bangvalo@gmail.com>
* update interface tests and configuration messages
* little debug
* little debug on both errors
* print all devs
* list of devices
* debug func
* handle interface close
* debug socks
* debug socks
* if ports match
* use random assigned ports
* remove unused const
* close management client connection when stopping engine
* GracefulStop when management clients are closed
* enable workflows on PRs too
* remove iface_test debug code
* get account id from access token claim
* use GetOrCreateAccountByUser and add test
* correct account id claim
* remove unused account
* Idp manager interface
* auth0 idp manager
* use if instead of switch case
* remove unnecessary lock
* NewAuth0Manager
* move idpmanager to its own package
* update metadata when accountId is not supplied
* update tests with idpmanager field
* format
* new idp manager and config support
* validate if we fetch the interface before converting to string
* split getJWTToken
* improve tests
* proper json fields and handle defer body close
* fix ci lint notes
* documentation and proper defer position
* UpdateUserAppMetadata tests
* update documentation
* ManagerCredentials interface
* Marshal and Unmarshal functions
* fix tests
* ManagerHelper and ManagerHTTPClient
* further tests with mocking
* rename package and custom http client
* sync local packages
* remove idp suffix
* feature: support new management service protocol
* chore: add more logging to track networkmap serial
* refactor: organize peer update code in engine
* chore: fix lint issues
* refactor: extract Signal client interface
* test: add signal client mock
* refactor: introduce Management Service client interface
* chore: place management and signal clients mocks to respective packages
* test: add Serial test to the engine
* fix: lint issues
* test: unit tests for a networkMapUpdate
* test: unit tests Sync update
* script to generate syso files
* test wireguard-windows driver package
* set int log
* add windows test
* add windows test
* verbose bash
* use cd
* move checkout
* exit 0
* removed tty flag
* artifact path
* fix tags and add cache
* fix cache
* fix cache
* test dir
* restore artifacts in the root
* try dll file
* try dll file
* copy dll
* typo in copy dll
* compile test
* checkout first
* updated cicd
* fix add address issue and gen GUID
* psexec typo
* accept eula
* mod tidy before tests
* regular test exec and verbose test with psexec
* test all
* return WGInterface Interface
* use WgIfaceName and timeout after 30 seconds
* different ports and validate connect 2 peers
* Use time.After for timeout and close interface
* Use time.After for testing connect peers
* WG Interface struct
* Update engine and parse address
* refactor Linux create and assignAddress
* NewWGIface and configuration methods
* Update proxy with interface methods
* update up command test
* resolve lint warnings
* remove psexec test
* close copied files
* add goos before build
* run tests on mac,windows and linux
* cache by testing os
* run on push
* fix indentation
* adjust test timeouts
* remove parallel flag
* mod tidy before test
* ignore syso files
* removed functions and renamed vars
* different IPs for connect peers test
* Generate syso with DLL
* Single Close method
* use port from test constant
* test: remove wireguard interfaces after finishing engine test
* use load_wgnt_from_rsrc
Co-authored-by: braginini <bangvalo@gmail.com>
* feature: introduce NetworkMap to the management protocol with a Serial ID
* test: add Management Sync method protocol test
* test: add Management Sync method NetworkMap field check [FAILING]
* test: add Management Sync method NetworkMap field check [FAILING]
* feature: fill NetworkMap property to when Deleting peer
* feature: fill NetworkMap in the Sync protocol
* test: code review mentions - GeneratePrivateKey() in the test
* fix: wiretrustee client use wireguard GeneratePrivateKey() instead of GenerateKey()
* test: add NetworkMap test
* fix: management_proto test remove store.json on test finish
* fix: too many open files caused by agent not being closed after unsuccessful attempts to start a peer connection (happens when no network available)
* fix: minor refactor to consider signal status
* refactor: move goroutine that runs Signal Client Receive to the engine for better control
* chore: fix comments typo
* test: fix golint
* chore: comments update
* chore: consider connection state=READY in signal and management clients
* chore: fix typos
* test: fix signal ping-pong test
* chore: add wait condition to signal client
* refactor: add stream status to the Signal client
* refactor: defer mutex unlock
* move setup-key to root command
* up will check login and start service
* update tests to reflect new UP capabilities
* display client IP
* removed unused argument
* install service if not installed
* update post-install and add pre remove script
* improve log messages
* handle service status failures and install service when needed
* removing unused files
* update documentation and description
* add version command
* update service lib version
* using lib constant for not installed services
* match version from goreleaser
* fix: graceful shutdown
* stop only if service is running
* add logs initialization to service controller commands
Co-authored-by: braginini <bangvalo@gmail.com>
* feature: add logging to a file
* refactor: move InitLog to util lib
* docs: update signal and management docs
* chore: update docker compose
* set --log-file to console
* chore: comment out log volume in docker compose
Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
* windows installer
* unpack function in local dir
* working-directory client
* using env var plugin
* test tag and publishing
* getting version from tag
* using version number
* remove unnecessary commands and add description
* using long version outputs
* uncomment docker steps
* feature: update STUNs and TURNs in engine
* fix: setup TURN credentials request only when refresh enabled
* feature: update TURNs and STUNs in teh client app on Management update
* chore: disable peer reflexive candidates in ICE
* chore: relocate management.json
* chore: make TURN secret and pwd plain text in config
* abstract peer channel
* remove wip code
* refactor NewServer with Peer updates channel
* feature: add TURN credentials manager
* hmac logic
* example test function
* test: add TimeBasedAuthSecretsManager_GenerateCredentials test
* test: make tests for now with hardcoded secret
* test: add TimeBasedAuthSecretsManager_SetupRefresh test
* test: add TimeBasedAuthSecretsManager_SetupRefresh test
* test: add TimeBasedAuthSecretsManager_CancelRefresh test
* feature: extract TURNConfig to the management config
* feature: return hash based TURN credentials only on initial sync
* feature: make TURN time based secret credentials optional
Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
* feature: replace RegisterPeer with Login method that does both - registration and login
* test: add management login test
* feature: add WiretrusteeConfig to the Login response to configure peer global config
* feature: add client peer login support
* fix: missing parts
* chore: update go deps
* feature: support Management Service gRPC endpoints [CLIENT]
* feature: finalize client sync with management
* fix: management store peer key lower case restore
* fix: management returns peer ip without a mask
* refactor: remove cmd pkg
* fix: invalid tun interface name on mac
* fix: timeout when calling management client
* fix: tests and lint errors
* fix: golang-test workflow
* fix: client service tests
* fix: iface build
* feature: detect management scheme on startup
* chore: better logs for management
* fix: goreleaser
* fix: lint errors
* fix: signal TLS
* fix: direct Wireguard connection
* chore: verbose logging on direct connection
