This update adds new relay integration for NetBird clients. The new relay is based on web sockets and listens on a single port.
- Adds new relay implementation with websocket with single port relaying mechanism
- refactor peer connection logic, allowing upgrade and downgrade from/to P2P connection
- peer connections are faster since it connects first to relay and then upgrades to P2P
- maintains compatibility with old clients by not using the new relay
- updates infrastructure scripts with new relay service
* compile client under freebsd (#1620)
Compile netbird client under freebsd and now support netstack and userspace modes.
Refactoring linux specific code to share same code with FreeBSD, move to *_unix.go files.
Not implemented yet:
Kernel mode not supported
DNS probably does not work yet
Routing also probably does not work yet
SSH support did not tested yet
Lack of test environment for freebsd (dedicated VM for github runners under FreeBSD required)
Lack of tests for freebsd specific code
info reporting need to review and also implement, for example OS reported as GENERIC instead of FreeBSD (lack of FreeBSD icon in management interface)
Lack of proper client setup under FreeBSD
Lack of FreeBSD port/package
* Add DNS routes (#1943)
Given domains are resolved periodically and resolved IPs are replaced with the new ones. Unless the flag keep_route is set to true, then only new ones are added.
This option is helpful if there are long-running connections that might still point to old IP addresses from changed DNS records.
* Add process posture check (#1693)
Introduces a process posture check to validate the existence and active status of specific binaries on peer systems. The check ensures that files are present at specified paths, and that corresponding processes are running. This check supports Linux, Windows, and macOS systems.
Co-authored-by: Evgenii <mail@skillcoder.com>
Co-authored-by: Pascal Fischer <pascal@netbird.io>
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
Co-authored-by: Bethuel Mmbaga <bethuelmbaga12@gmail.com>
* migrate sqlite store to
generic sql store
* fix conflicts
* init postgres store
* Add postgres store tests
* Refactor postgres store engine name
* fix tests
* Run postgres store tests on linux only
* fix tests
* Refactor
* cascade policy rules on policy deletion
* fix tests
* run postgres cases in new db
* close store connection after tests
* refactor
* using testcontainers
* sync go sum
* remove postgres service
* remove store cleanup
* go mod tidy
* remove env
* use postgres as engine and initialize test store with testcontainer
---------
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
This PR implements the following posture checks:
* Agent minimum version allowed
* OS minimum version allowed
* Geo-location based on connection IP
For the geo-based location, we rely on GeoLite2 databases which are free IP geolocation databases. MaxMind was tested and we provide a script that easily allows to download of all necessary files, see infrastructure_files/download-geolite2.sh.
The OpenAPI spec should extensively cover the life cycle of current version posture checks.
With this change we should be able to collect and expose the following histograms:
* `management.updatechannel.create.duration.ms` with `closed` boolean label
* `management.updatechannel.create.duration.micro` with `closed` boolean label
* `management.updatechannel.close.one.duration.ms`
* `management.updatechannel.close.one.duration.micro`
* `management.updatechannel.close.multiple.duration.ms`
* `management.updatechannel.close.multiple.duration.micro`
* `management.updatechannel.close.multiple.channels`
* `management.updatechannel.send.duration.ms` with `found` and `dropped` boolean labels
* `management.updatechannel.send.duration.micro` with `found` and `dropped` boolean labels
* `management.updatechannel.get.all.duration.ms`
* `management.updatechannel.get.all.duration.micro`
* `management.updatechannel.get.all.peers`
Restructure data handling for improved performance and flexibility.
Introduce 'G'-prefixed fields to represent Gorm relations, simplifying resource management.
Eliminate complexity in lookup tables for enhanced query and write speed.
Enable independent operations on data structures, requiring adjustments in the Store interface and Account Manager.
Implement user deletion across all IDP-ss. Expires all user peers
when the user is deleted. Users are permanently removed from a local
store, but in IDP, we remove Netbird attributes for the user
untilUserDeleteFromIDPEnabled setting is not enabled.
To test, an admin user should remove any additional users.
Until the UI incorporates this feature, use a curl DELETE request
targeting the /users/<USER_ID> management endpoint. Note that this
request only removes user attributes and doesn't trigger a delete
from the IDP.
To enable user removal from the IdP, set UserDeleteFromIDPEnabled
to true in account settings. Until we have a UI for this, make this
change directly in the store file.
Store the deleted email addresses in encrypted in activity store.
The ephemeral manager keep the inactive ephemeral peers in a linked list. The manager schedule a cleanup procedure to the head of the linked list (to the most deprecated peer). At the end of cleanup schedule the next cleanup to the new head.
If a device connect back to the server the manager will remote it from the peers list.
The peer login expiration ACL check introduced in #714
filters out peers that are expired and agents receive a network map
without that expired peers.
However, the agents should see those peers in status "Disconnected".
This PR extends the Agent <-> Management protocol
by introducing a new field OfflinePeers
that contain expired peers. Agents keep track of those and display
then just in the Status response.
The Management gRPC API has too much business logic
happening while it has to be in the Account manager.
This also needs to make more requests to the store
through the account manager.
This PR adds system activity tracking.
The management service records events like
add/remove peer, group, rule, route, etc.
The activity events are stored in the SQLite event store
and can be queried by the HTTP API.
Due to peer reconnects when restarting the Management service,
there are lots of SaveStore operations to update peer status.
Store.SavePeerStatus stores peer status separately and the
FileStore implementation stores it in memory.
Added DNS update protocol message
Added sync to clients
Update nameserver API with new fields
Added default NS groups
Added new dns-name flag for the management service append to peer DNS label
This PR brings user invites logic to the Management service
via HTTP API.
The POST /users/ API endpoint creates a new user in the Idp
and then in the local storage.
Once the invited user signs ups, the account invitation is redeemed.
There are a few limitations.
This works only with an enabled IdP manager.
Users that already have a registered account can't be invited.
* Send netmask from account network
Added the GetPeerNetwork method to account manager
Pass a copy of the network to the toPeerConfig function
to retrieve the netmask from the network instead of constant
updated methods and added test
* check if the network is the same for 2 peers
* Use expect with BeEquivalentTo
The peer IP allocation logic was allocating sequential peer IP from the 100.64.0.0/10
address block.
Each account is created with a random subnet from 100.64.0.0/10.
The total amount of potential subnets is 64.
The new logic allocates random peer IP
from the account subnet.
This gives us flexibility to add support for
multi subnet accounts without overlapping IPs.
* get account id from access token claim
* use GetOrCreateAccountByUser and add test
* correct account id claim
* remove unused account
* Idp manager interface
* auth0 idp manager
* use if instead of switch case
* remove unnecessary lock
* NewAuth0Manager
* move idpmanager to its own package
* update metadata when accountId is not supplied
* update tests with idpmanager field
* format
* new idp manager and config support
* validate if we fetch the interface before converting to string
* split getJWTToken
* improve tests
* proper json fields and handle defer body close
* fix ci lint notes
* documentation and proper defer position
* UpdateUserAppMetadata tests
* update documentation
* ManagerCredentials interface
* Marshal and Unmarshal functions
* fix tests
* ManagerHelper and ManagerHTTPClient
* further tests with mocking
* rename package and custom http client
* sync local packages
* remove idp suffix
* abstract peer channel
* remove wip code
* refactor NewServer with Peer updates channel
* feature: add TURN credentials manager
* hmac logic
* example test function
* test: add TimeBasedAuthSecretsManager_GenerateCredentials test
* test: make tests for now with hardcoded secret
* test: add TimeBasedAuthSecretsManager_SetupRefresh test
* test: add TimeBasedAuthSecretsManager_SetupRefresh test
* test: add TimeBasedAuthSecretsManager_CancelRefresh test
* feature: extract TURNConfig to the management config
* feature: return hash based TURN credentials only on initial sync
* feature: make TURN time based secret credentials optional
Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
* feature: add peer GET and DELETE API methods
* refactor: extract peer business logic to a separate file
* refactor: extract peer business logic to a separate file
* feature: add peer update HTTP endpoint
* chore: fill peer new fields
* merge with main
* refactor: HTTP methods according to standards
* feature: add peer system metadata
* feature: add peer status
* test: fix removal
* feature: replace RegisterPeer with Login method that does both - registration and login
* test: add management login test
* feature: add WiretrusteeConfig to the Login response to configure peer global config
* feature: add client peer login support
* fix: missing parts
* chore: update go deps
* feature: support Management Service gRPC endpoints [CLIENT]
* feature: finalize client sync with management
* fix: management store peer key lower case restore
* fix: management returns peer ip without a mask
* refactor: remove cmd pkg
* fix: invalid tun interface name on mac
* fix: timeout when calling management client
* fix: tests and lint errors
* fix: golang-test workflow
* fix: client service tests
* fix: iface build
* feature: detect management scheme on startup
* chore: better logs for management
* fix: goreleaser
* fix: lint errors
* fix: signal TLS
* fix: direct Wireguard connection
* chore: verbose logging on direct connection
* feature: add config properties to the SyncResponse of the management gRpc service
* fix: lint errors
* chore: modify management protocol according to the review notes
* fix: management proto fields sequence
* feature: add proper peer configuration to be synced
* chore: minor changes
* feature: finalize peer config management
* fix: lint errors
* feature: add management server config file
* refactor: extract hosts-config to a separate file
* refactor: review notes applied to correct file_store usage
* refactor: extract management service configuration to a file
* refactor: simplify management config
