In some cases, when the refresh cache fails, we should try to get the cache from the external cache obj.
This may happen if the IDP is not responsive between storing metadata and refreshing the cache
We allow service users with user role read-only access
to all resources so users can create service user and propagate
PATs without having to give full admin permissions.
* Adds management, signal, and relay (STUN/TURN) health probes to the status command.
* Adds a reason when the management or signal connections are disconnected.
* Adds last wireguard handshake and received/sent bytes per peer
This PR aims to integrate Rosenpass with NetBird. It adds a manager for Rosenpass that starts a Rosenpass server and handles the managed peers. It uses the cunicu/go-rosenpass implementation. Rosenpass will then negotiate a pre-shared key every 2 minutes and apply it to the wireguard connection.
The Feature can be enabled by setting a flag during the netbird up --enable-rosenpass command.
If two peers are both support and have the Rosenpass feature enabled they will create a post-quantum secure connection. If one of the peers or both don't have this feature enabled or are running an older version that does not have this feature yet, the NetBird client will fall back to a plain Wireguard connection without pre-shared keys for those connections (keeping Rosenpass negotiation for the rest).
Additionally, this PR includes an update of all Github Actions workflows to use go version 1.21.0 as this is a requirement for the integration.
---------
Co-authored-by: braginini <bangvalo@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
* Update user's last login when authenticating a peer
Prior to this update the user's last login only updated on dashboard authentication
* use account and user methods
Some IdPs might have eventual consistency for their API calls, and refreshing the cache with its data may return the deleted user as part of the account
Introduce a new account manager method, removeUserFromCache, to remove the user from the local cache without refresh
* Added function to check user access by JWT groups in the account management mock server and account manager
* Refactor auth middleware for group-based JWT access control
* Add group-based JWT access control on adding new peer with JWT
* Remove mapping error as the token validation error is already present in grpc error codes
* use GetAccountFromToken to prevent single mode issues
* handle foreground login message
---------
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Ensure the jwks expiresInTime is not zero and add a log indicating the new expiration time
Replace the configuration property only when the flag is being used
* Extend management API to support list of allowed JWT groups (#1366)
* Add JWTAllowGroups settings to account management
* Return an empty group list if jwt allow groups is not set
* Add JwtAllowGroups to account settings in handler test
* Add JWT group-based user authorization (#1373)
* Add JWTAllowGroups settings to account management
* Return an empty group list if jwt allow groups is not set
* Add JwtAllowGroups to account settings in handler test
* Implement user access validation authentication based on JWT groups
* Remove the slices package import due to compatibility issues with the gitHub workflow(s) Go version
* Refactor auth middleware and test for extracted claim handling
* Optimize JWT group check in auth middleware to cover nil and empty allowed groups
This PR adds `gosec` linter with the following checks disabled:
- G102: Bind to all interfaces
- G107: Url provided to HTTP request as taint input
- G112: Potential slowloris attack
- G114: Use of net/http serve function that has no support for setting timeouts
- G204: Audit use of command execution
- G401: Detect the usage of DES, RC4, MD5 or SHA1
- G402: Look for bad TLS connection settings
- G404: Insecure random number source (rand)
- G501: Import blocklist: crypto/md5
- G505: Import blocklist: crypto/sha1
We have complaints related to the checks above. They have to be addressed separately.
This PR adds support to Owner roles.
The owner role has a similar access level as the admin, but it has the power to delete the account.
Besides that, the role has the following constraints:
- The role can only be transferred. So, only a user with the owner role can transfer the owner role to a new user
- It can't be assigned to users being invited
- It can't be assigned to service users
Adding support to account owners to delete an account
This will remove all users from local, and if --user-delete-from-idp is set it will remove from the remote IDP
* Add gocritic linter
`gocritic` provides diagnostics that check for bugs, performance, and style issues
We disable the following checks:
- commentFormatting
- captLocal
- deprecatedComment
This PR contains many `//nolint:gocritic` to disable `appendAssign`.
With this change we should be able to collect and expose the following histograms:
* `management.updatechannel.create.duration.ms` with `closed` boolean label
* `management.updatechannel.create.duration.micro` with `closed` boolean label
* `management.updatechannel.close.one.duration.ms`
* `management.updatechannel.close.one.duration.micro`
* `management.updatechannel.close.multiple.duration.ms`
* `management.updatechannel.close.multiple.duration.micro`
* `management.updatechannel.close.multiple.channels`
* `management.updatechannel.send.duration.ms` with `found` and `dropped` boolean labels
* `management.updatechannel.send.duration.micro` with `found` and `dropped` boolean labels
* `management.updatechannel.get.all.duration.ms`
* `management.updatechannel.get.all.duration.micro`
* `management.updatechannel.get.all.peers`
* Add non-deletable flag for service users
* fix non deletable service user created as deletable
* Exclude non deletable service users in service users api response
* Fix broken tests
* Add test for non deletable service user
* Add handling for non-deletable service users in tests
* Remove non-deletable service users when fetching all users
* Ensure non-deletable users are filtered out when fetching all user data
- dupword checks for duplicate words in the source code
- durationcheck checks for two durations multiplied together
- forbidigo forbids identifiers
- mirror reports wrong mirror patterns of bytes/strings usage
- misspell finds commonly misspelled English words in comments
- predeclared finds code that shadows one of Go's predeclared identifiers
- thelper detects Go test helpers without t.Helper() call and checks the consistency of test helpers
* Enforce admin service user role for integration group deletion
Added a check to prevent non-admin service users from deleting integration groups.
* Restrict deletion of integration user to admin service user only
* Refactor user and group deletion tests
New activity types for integration creation, update, and deletion have been added to the activity codes. This ensures the tracking of these user activities relating to integrations, which were not previously being logged.
The group copy method now includes the IntegrationReference field in its output. This change was made to ensure that the integration reference information is retained when a group instance is copied, which previously was not the case.
* extends user and group structure by introducing fields for issued and integration references
* Add integration checks to group management to prevent groups added by integration.
* Add integration checks to user management to prevent deleting user added by integration.
* Fix broken user update tests
* Initialize all user fields for testing
* Change a serializer option to embedded for IntegrationReference in user and group models
* Add issued field to user api response
* Add IntegrationReference to Group in update groups handler
* Set the default issued field for users in file store
Supporting search domains will allow users to define match domains to also
be added to a list of search domains in their systems
Fix Windows registry key configuration for search domains using a key within the netbird interface path
Some reverse proxies might find 15s interval too short and respond with an enhance your-calm message
This change is setting the management and signal clients' keepalive interval to 30 seconds to minimize the number of reconnections
* Move StoreKind under own StoreConfig configuration parameter
* Rename StoreKind option to Engine
* Rename StoreKind internal methods and types to Engine
* Add template engine value test
---------
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Restructure data handling for improved performance and flexibility.
Introduce 'G'-prefixed fields to represent Gorm relations, simplifying resource management.
Eliminate complexity in lookup tables for enhanced query and write speed.
Enable independent operations on data structures, requiring adjustments in the Store interface and Account Manager.
Because we provide the option to regenerate the config files, the encryption key could be lost.
- The configure.sh read the existing key and write it back during the config generation
- Backup the previously generated config files before overwrite it
- Fix invalid json output in the Extras field
- Reduce the error logs in case if the encryption key is invalid
- Response in the events API with valid user info in any cases
- Add extra error handling to the configure.sh. I.e. handle the invalid OpenID urls
Fix group delete panic
In case if in the db the DNSSettings is null then can cause panic in delete group function
because this field is pointer and it was not checked. Because of in the future implementation
this variable will be filled in any case then make no sense to keep the pointer type.
Fix DNSSettings copy function
With this change, we don't need to update all peers on startup. We will
check the existence of an update channel when returning a list or single peer on API.
Then after restarting of server consumers of API will see peer not
connected status till the creation of an updated channel which indicates
peer successful connection.
* Retrieve all workspace users via pagination, excluding custom user attributes
* Retrieve all authentik users via pagination
* Retrieve all Azure AD users via pagination
* Simplify user data appending operation
Reduced unnecessary iteration and used an efficient way to append all users to 'indexedUsers'
* Fix ineffectual assignment to reqURL
* Retrieve all Okta users via pagination
* Add missing GetAccount metrics
* Refactor
* minimize memory allocation
Refactored the memory allocation for the 'users' slice in the Okta IDP code. Previously, the slice was only initialized but not given a size. Now the size of userList is utilized to optimize memory allocation, reducing potential slice resizing and memory re-allocation costs while appending users.
* Add logging for entries received from IdP management
Added informative and debug logging statements in account.go file. Logging has been added to identify the number of entries received from Identity Provider (IdP) management. This will aid in tracking and debugging any potential data ingestion issues.
This PR fixes an issue were only one route containing routing groups was being synced to peers.
It also prevents sending routes for peers that aren't connect via ACL.
Moved all checks to Account.getEnabledAndDisabledRoutesByPeer.
Co-authored-by: Yury Gargay <yury.gargay@gmail.com>
Co-authored-by: braginini <bangvalo@gmail.com>
added intergration with JumpCloud User API. Use the steps in setup.md for configuration.
Additional changes:
- Enhance compatibility for providers that lack audience support in the Authorization Code Flow and the Authorization - - Code Flow with Proof Key for Code Exchange (PKCE) using NETBIRD_DASH_AUTH_USE_AUDIENCE=falseenv
- Verify tokens by utilizing the client ID when audience support is absent in providers
This pull request modifies the IdP and cache manager(s) to prevent the sending of app metadata
to the upstream IDP on self-hosted instances.
As a result, the IdP will now load all users from the IdP without filtering based on accountID.
We disable user invites as the administrator's own IDP system manages them.
If there is a difference between local and cached data, we trigger a cache refresh;
as we remove users from the local store and potentially from the remote IDP,
we need to switch the source of truth to the local store to prevent unwanted endless
cache for cases where the removal from the IDP fails or for cases
where the userDeleteFromIDPEnabled got enabled after the first user deletion.
This commit enhances the functionality of the network routes endpoint by introducing a new parameter called `peers_group`. This addition allows users to associate network routes with specific peer groups, simplifying the management and distribution of routes within a network.
Extend the deleted user info with the username
- Because initially, we did not store the user name in the activity db
Sometimes, we can not provide the user name in the API response.
Fix service user deletion
- In case of service user deletion, do not invoke the IdP delete function
- Prevent self deletion
Add a direct write to handle management.json write operation.
Remove empty configuration types to avoid unnecessary fields in the generated management.json file.
Implement user deletion across all IDP-ss. Expires all user peers
when the user is deleted. Users are permanently removed from a local
store, but in IDP, we remove Netbird attributes for the user
untilUserDeleteFromIDPEnabled setting is not enabled.
To test, an admin user should remove any additional users.
Until the UI incorporates this feature, use a curl DELETE request
targeting the /users/<USER_ID> management endpoint. Note that this
request only removes user attributes and doesn't trigger a delete
from the IDP.
To enable user removal from the IdP, set UserDeleteFromIDPEnabled
to true in account settings. Until we have a UI for this, make this
change directly in the store file.
Store the deleted email addresses in encrypted in activity store.
This PR showcases the implementation of additional linter rules. I've updated the golangci-lint GitHub Actions to the latest available version. This update makes sure that the tool works the same way locally - assuming being updated regularly - and with the GitHub Actions.
I've also taken care of keeping all the GitHub Actions up to date, which helps our code stay current. But there's one part, goreleaser that's a bit tricky to test on our computers. So, it's important to take a close look at that.
To make it easier to understand what I've done, I've made separate changes for each thing that the new linters found. This should help the people reviewing the changes see what's going on more clearly. Some of the changes might not be obvious at first glance.
Things to consider for the future
CI runs on Ubuntu so the static analysis only happens for Linux. Consider running it for the rest: Darwin, Windows
The ephemeral manager keep the inactive ephemeral peers in a linked list. The manager schedule a cleanup procedure to the head of the linked list (to the most deprecated peer). At the end of cleanup schedule the next cleanup to the new head.
If a device connect back to the server the manager will remote it from the peers list.
The API authentication with PATs was not considering different userIDClaim
that some of the IdPs are using.
In this PR we read the userIDClaim from the config file
instead of using the fixed default and only keep
it as a fallback if none in defined.
With this fix, all nested slices and pointers will be copied by value.
Also, this fixes tests to compare the original and copy account by their
values by marshaling them to JSON strings.
Before that, they were copying the pointers that also passed the simple `=` compassion
(as the addresses match).
For better auditing this PR adds a dashboard login event to the management service.
For that the user object was extended with a field for last login that is not actively saved to the database but kept in memory until next write. The information about the last login can be extracted from the JWT claims nb_last_login. This timestamp will be stored and compared on each API request. If the value changes we generate an event to inform about a login.
For peer propagation this commit triggers
network map update in two cases:
1) peer login
2) user AutoGroups update
Also it issues new activity message about new user group
for peer login process.
Previous implementation only adds JWT groups to user. This fix also
removes JWT groups from user auto assign groups.
Pelase note, it also happen when user works with dashboard.
Enhancements to Peer Group Assignment:
1. Auto-assigned groups are now applied to all peers every time a user logs into the network.
2. Feature activation is available in the account settings.
3. API modifications included to support these changes for account settings updates.
4. If propagation is enabled, updates to a user's auto-assigned groups are immediately reflected across all user peers.
5. With the JWT group sync feature active, auto-assigned groups are forcefully updated whenever a peer logs in using user credentials.
Enhance the user experience by enabling authentication to Netbird using Single Sign-On (SSO) with any Identity Provider (IDP) provider. Current client offers this capability through the Device Authorization Flow, however, is not widely supported by many IDPs, and even some that do support it do not provide a complete verification URL.
To address these challenges, this pull request enable Authorization Code Flow with Proof Key for Code Exchange (PKCE) for client logins, which is a more widely adopted and secure approach to facilitate SSO with various IDP providers.
This fixes the test logic creates copy of account with empty id and
re-pointing the indices to it.
Also, adds additional check for empty ID in SaveAccount method of FileStore.
* Optimize rules with All groups
* Use IP sets in ACLs (nftables implementation)
* Fix squash rule when we receive optimized rules list from management
* Check links of groups before delete it
* Add delete group handler test
* Rename dns error msg
* Add delete group test
* Remove rule check
The policy cover this scenario
* Fix test
* Check disabled management grps
* Change error message
* Add new activity for group delete event
