Commit Graph

550 Commits

Author SHA1 Message Date
Maycon Santos
a7547b9990
Get cache from external cache when refresh fails (#1537)
In some cases, when the refresh cache fails, we should try to get the cache from the external cache obj.

This may happen if the IDP is not responsive between storing metadata and refreshing the cache
2024-02-07 16:14:30 +01:00
Yury Gargay
bdf71ab7ff
Remove query parameter from policy endpoints (#1527) 2024-02-05 14:07:11 +01:00
Yury Gargay
93cab49696
Extract peer real IP from Load Balancer when possible (#1510) 2024-01-31 16:02:24 +01:00
Viktor Liu
25b3641be8
Fix data dir creation permissions (#1503) 2024-01-29 14:21:45 +01:00
pascal-fischer
399493a954
Allow service users with user role read-only access to all resources (#1484)
We allow service users with user role read-only access 
to all resources so users can create service user and propagate 
PATs without having to give full admin permissions.
2024-01-25 09:50:27 +01:00
Viktor Liu
a7d6632298
Extend netbird status command to include health information (#1471)
* Adds management, signal, and relay (STUN/TURN) health probes to the status command.

* Adds a reason when the management or signal connections are disconnected.

* Adds last wireguard handshake and received/sent bytes per peer
2024-01-22 12:20:24 +01:00
Maycon Santos
d4194cba6a
Fix race condition with JWT group sync (#1486)
This PR fixes the issue that caused JWT group membership not being store
Therefore causing many event logs and inconsistency
2024-01-20 23:50:57 +01:00
pascal-fischer
131d9f1bc7
Add getGroupByName method (#1481)
* add get group by name method to account manager

* remove contains function and add proper description for GetGroupByName method

* add to mock server
2024-01-19 15:41:27 +01:00
Maycon Santos
9fa0fbda0d
Enable group propagation by default (#1469)
Group updates to user auto groups will propagate by default for new accounts
2024-01-15 19:26:27 +01:00
Yury Gargay
5311ce4e4a
Soft deprecate Rules API (#1454) 2024-01-10 13:55:11 +01:00
pascal-fischer
5de4acf2fe
Integrate Rosenpass (#1153)
This PR aims to integrate Rosenpass with NetBird. It adds a manager for Rosenpass that starts a Rosenpass server and handles the managed peers. It uses the cunicu/go-rosenpass implementation. Rosenpass will then negotiate a pre-shared key every 2 minutes and apply it to the wireguard connection.

The Feature can be enabled by setting a flag during the netbird up --enable-rosenpass command.

If two peers are both support and have the Rosenpass feature enabled they will create a post-quantum secure connection. If one of the peers or both don't have this feature enabled or are running an older version that does not have this feature yet, the NetBird client will fall back to a plain Wireguard connection without pre-shared keys for those connections (keeping Rosenpass negotiation for the rest).

Additionally, this PR includes an update of all Github Actions workflows to use go version 1.21.0 as this is a requirement for the integration.

---------

Co-authored-by: braginini <bangvalo@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-01-08 12:25:35 +01:00
Maycon Santos
8b4ec96516
Update user's last login when authenticating a peer (#1437)
* Update user's last login when authenticating a peer

Prior to this update the user's last login only updated on dashboard authentication

* use account and user methods
2024-01-06 12:57:05 +01:00
Maycon Santos
1f3a12d941
Cancel peer expiration scheduled job when deleting account (#1434) 2024-01-04 17:10:55 +01:00
Maycon Santos
b28b8fce50
Remove the user from the cache without refreshing it (#1422)
Some IdPs might have eventual consistency for their API calls, and refreshing the cache with its data may return the deleted user as part of the account

Introduce a new account manager method, removeUserFromCache, to remove the user from the local cache without refresh
2024-01-01 19:17:44 +01:00
Maycon Santos
f780f17f85
Use integrated activity store (#1421)
---------

Co-authored-by: braginini <bangvalo@gmail.com>
2024-01-01 19:11:11 +01:00
Bethuel Mmbaga
cba3c549e9
Add JWT group-based access control for adding new peers (#1383)
* Added function to check user access by JWT groups in the account management mock server and account manager

* Refactor auth middleware for group-based JWT access control

* Add group-based JWT access control on adding new peer with JWT

* Remove mapping error as the token validation error is already present in grpc error codes

* use GetAccountFromToken to prevent single mode issues

* handle foreground login message

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2023-12-13 13:18:35 +03:00
pascal-fischer
65247de48d
Fix nil pointer handling in get peers from group (#1381)
Fix nil handling in getAllPeersFromGroups to not include nil pointer in the output.
2023-12-12 18:17:00 +01:00
Maycon Santos
2d1dfa3ae7
Fix jwks validation and flag/config overriding (#1380)
Ensure the jwks expiresInTime is not zero and add a log indicating the new expiration time

Replace the configuration property only when the flag is being used
2023-12-12 14:56:27 +01:00
Yury Gargay
5961c8330e
Fix SaveOrAddUser and GetPeers methods in MockAccountManager (#1374) 2023-12-11 17:32:10 +01:00
Bethuel Mmbaga
d275d411aa
Enable JWT group-based user authorization (#1368)
* Extend management API to support list of allowed JWT groups (#1366)

* Add JWTAllowGroups settings to account management

* Return an empty group list if jwt allow groups is not set

* Add JwtAllowGroups to account settings in handler test

* Add JWT group-based user authorization (#1373)

* Add JWTAllowGroups settings to account management

* Return an empty group list if jwt allow groups is not set

* Add JwtAllowGroups to account settings in handler test

* Implement user access validation authentication based on JWT groups

* Remove the slices package import due to compatibility issues with the gitHub workflow(s) Go version

* Refactor auth middleware and test for extracted claim handling

* Optimize JWT group check in auth middleware to cover nil and empty allowed groups
2023-12-11 18:59:15 +03:00
Yury Gargay
5ecafef5d2
Fix ListUsers method in MockAccountManager (#1367) 2023-12-11 15:00:02 +01:00
Maycon Santos
dd1e730454
Update API descriptions and examples (#1364) 2023-12-08 11:39:33 +01:00
Pascal Fischer
08733ed8d5 update tests 2023-12-06 15:02:10 +01:00
Yury Gargay
27ed88f918
Implement lightweight method to check is peer has update channel (#1351)
Instead of GetAllConnectedPeers that need to traverse the whole
connections map in order to find one channel there.
2023-12-05 14:17:56 +01:00
Maycon Santos
0d2db4b172 update API doc 2023-12-04 19:02:16 +01:00
pascal-fischer
ae5f69562d
Merge branch 'main' into feature/peer-approval 2023-12-04 17:34:53 +01:00
pascal-fischer
755ffcfc73
Merge pull request #1353 from netbirdio/feature/extend-add-peer-event-with-setup-key
Extend add peer event meta with setup key name
2023-12-04 17:33:50 +01:00
Pascal Fischer
dc8f55f23e remove dependency cycle from prepare peer 2023-12-04 16:26:34 +01:00
Pascal Fischer
89249b414f move peer validation into getPeerconnectionResources 2023-12-04 14:53:38 +01:00
Pascal Fischer
92adf57fea fix map assignment 2023-12-04 13:49:46 +01:00
Yury Gargay
e37a337164
Add gosec linter (#1342)
This PR adds `gosec` linter with the following checks disabled:

- G102: Bind to all interfaces
- G107: Url provided to HTTP request as taint input
- G112: Potential slowloris attack
- G114: Use of net/http serve function that has no support for setting timeouts
- G204: Audit use of command execution
- G401: Detect the usage of DES, RC4, MD5 or SHA1
- G402: Look for bad TLS connection settings
- G404: Insecure random number source (rand)
- G501: Import blocklist: crypto/md5
- G505: Import blocklist: crypto/sha1

We have complaints related to the checks above. They have to be addressed separately.
2023-12-04 13:34:06 +01:00
Pascal Fischer
1cd5a66575 adding setup key name to the event meta for adding peers by setup key 2023-12-04 13:00:13 +01:00
Pascal Fischer
b9fc008542 extract peer preparation 2023-12-04 12:49:50 +01:00
pascal-fischer
d5bf79bc51
Merge branch 'main' into feature/peer-approval 2023-12-01 18:12:59 +01:00
Maycon Santos
d7efea74b6
add owner role support (#1340)
This PR adds support to Owner roles.

The owner role has a similar access level as the admin, but it has the power to delete the account.

Besides that, the role has the following constraints:

- The role can only be transferred. So, only a user with the owner role can transfer the owner role to a new user
- It can't be assigned to users being invited
- It can't be assigned to service users
2023-12-01 17:24:57 +01:00
Pascal Fischer
4bf574037f fix sql store 2023-11-30 11:51:35 +01:00
Pascal Fischer
47c44d4b87 fix imports in sqlite store test 2023-11-30 11:08:51 +01:00
Pascal Fischer
96f866fb68 add missing imports after refactor 2023-11-29 16:46:46 +01:00
pascal-fischer
141065f14e
Merge branch 'main' into feature/peer-approval 2023-11-29 16:27:01 +01:00
Pascal Fischer
8e74fb1fa8 add account id to validating peer update 2023-11-29 15:57:56 +01:00
Pascal Fischer
ba96e102b4 settings nil check 2023-11-29 15:16:11 +01:00
Pascal Fischer
2129b23fe7 allow sync for and return empty map 2023-11-29 14:56:06 +01:00
Maycon Santos
b6211ad020
Fix group membership for peers API response (#1337) 2023-11-29 09:33:05 +01:00
Pascal Fischer
efd05ca023 fix api references 2023-11-28 15:15:51 +01:00
Pascal Fischer
c829ad930c use separate package for signatures 2023-11-28 15:09:04 +01:00
Pascal Fischer
ad1f18a52a replace with updated integrations 2023-11-28 14:55:20 +01:00
Pascal Fischer
bab420ca77 extract account into separate package 2023-11-28 14:34:57 +01:00
Maycon Santos
c2eaf8a1c0
Add account deletion endpoint (#1331)
Adding support to account owners to delete an account

This will remove all users from local, and if --user-delete-from-idp is set it will remove from the remote IDP
2023-11-28 14:23:38 +01:00
Pascal Fischer
a729c83b06 extract peer into seperate package 2023-11-28 13:45:26 +01:00
Maycon Santos
dc05102b8f
Fix panic on empty username for invites (#1334)
Validate email and user are not empty
2023-11-28 13:09:33 +01:00
Pascal Fischer
a7e55cc5e3 add signatures and frame for peer approval 2023-11-28 11:44:08 +01:00
Pascal Fischer
b7c0eba1e5 add extra settings struct 2023-11-27 17:04:40 +01:00
Yury Gargay
d1a323fa9d
Add gocritic linter (#1324)
* Add gocritic linter

`gocritic` provides diagnostics that check for bugs, performance, and style issues

We disable the following checks:

- commentFormatting
- captLocal
- deprecatedComment

This PR contains many `//nolint:gocritic` to disable `appendAssign`.
2023-11-27 16:40:02 +01:00
Yury Gargay
63d211c698
Prepare regexps on compile time (#1327) 2023-11-27 13:01:00 +01:00
Yury Gargay
afece95ae5
Fix lookupCache to work when idp has more users (#1321) 2023-11-20 16:47:11 +01:00
Bethuel Mmbaga
d78b7e5d93
Skip user deletion if the user does not exist in IdP (#1320) 2023-11-20 16:56:21 +03:00
Yury Gargay
67906f6da5
Improve Account cache reload condition (#1319)
To take in consideration that cache may know more users
2023-11-20 12:05:32 +01:00
pascal-fischer
52b5a31058
Merge pull request #1309 from netbirdio/fix/duplicated-entries-on-events-api
Fix duplicated Activity events shown
2023-11-17 12:03:26 +01:00
Yury Gargay
b58094de0f
Add metrics for PeersUpdateManager (#1310)
With this change we should be able to collect and expose the following histograms:

* `management.updatechannel.create.duration.ms`  with `closed` boolean label
* `management.updatechannel.create.duration.micro` with `closed` boolean label
* `management.updatechannel.close.one.duration.ms`
* `management.updatechannel.close.one.duration.micro`
* `management.updatechannel.close.multiple.duration.ms`
* `management.updatechannel.close.multiple.duration.micro`
* `management.updatechannel.close.multiple.channels`
* `management.updatechannel.send.duration.ms` with `found` and `dropped` boolean labels
* `management.updatechannel.send.duration.micro` with `found` and `dropped` boolean labels
* `management.updatechannel.get.all.duration.ms`
* `management.updatechannel.get.all.duration.micro`
* `management.updatechannel.get.all.peers`
2023-11-16 18:21:52 +01:00
Pascal Fischer
d379c25ff5 use idp cache instead of idp manager 2023-11-16 17:13:04 +01:00
Pascal Fischer
f86ed12cf5 add support for ipad as well 2023-11-16 17:01:01 +01:00
Pascal Fischer
5a45f79fec use the email address to set the iphone name for iOS 16+ 2023-11-16 16:46:08 +01:00
Bethuel Mmbaga
e7d063126d
Add non-deletable service user (#1311)
* Add non-deletable flag for service users

* fix non deletable service user created as deletable

* Exclude non deletable service users in service users api response

* Fix broken tests

* Add test for non deletable service user

* Add handling for non-deletable service users in tests

* Remove non-deletable service users when fetching all users

* Ensure non-deletable users are filtered out when fetching all user data
2023-11-15 18:22:00 +03:00
Yury Gargay
fb42fedb58
Fix PAT copy for GetUserByTokenID in SQLite store (#1312) 2023-11-15 14:15:12 +01:00
Pascal Fischer
9eb1e90bbe refactor activity get queries to only add comment 2023-11-15 11:21:59 +01:00
Pascal Fischer
53fb0a9754 refactor activity get queries to only consider 1 deleted user entry 2023-11-14 17:40:14 +01:00
Yury Gargay
70c7543e36
Allow to update IntegrationReference for user (#1308)
This should not happen via an API but be possible when calling the method directly.
2023-11-14 12:25:21 +01:00
Yury Gargay
d1d01a0611
Extend AccountManager with external cache and group/user management methods (#1289) 2023-11-13 14:04:18 +01:00
Yury Gargay
9e8725618e
Extend linter rules (#1300)
- dupword checks for duplicate words in the source code
- durationcheck checks for two durations multiplied together
- forbidigo forbids identifiers
- mirror reports wrong mirror patterns of bytes/strings usage
- misspell finds commonly misspelled English words in comments
- predeclared finds code that shadows one of Go's predeclared identifiers
- thelper detects Go test helpers without t.Helper() call and checks the consistency of test helpers
2023-11-10 16:33:13 +01:00
Maycon Santos
a40261ff7e
Log access control error (#1299) 2023-11-09 17:15:59 +01:00
Bethuel Mmbaga
89e8540531
Export account manager events store (#1295)
* Expose account manager StoreEvent to integrations

* Add account manager StoreEvent mock
2023-11-08 13:35:37 +03:00
Bethuel Mmbaga
9f7e13fc87
Enable deletion of integration resources (#1294)
* Enforce admin service user role for integration group deletion

Added a check to prevent non-admin service users from deleting integration groups.

* Restrict deletion of integration user to admin service user only

* Refactor user and group deletion tests
2023-11-07 17:02:51 +03:00
Zoltan Papp
8be6e92563
Extend API with accessible peers (#1284)
Extend the peer and peers API endpoints with accessible peers.
2023-11-07 14:38:36 +01:00
Maycon Santos
b726b3262d
Add codespell job (#1281)
add codespell workflow and fixed spelling issues
2023-11-07 13:37:57 +01:00
Bethuel Mmbaga
125a7a9daf
Add integration activity types to codes.go (#1293)
New activity types for integration creation, update, and deletion have been added to the activity codes. This ensures the tracking of these user activities relating to integrations, which were not previously being logged.
2023-11-07 14:05:58 +03:00
Bethuel Mmbaga
8cf2866a6a
Add integration reference field to group copy method (#1283)
The group copy method now includes the IntegrationReference field in its output. This change was made to ensure that the integration reference information is retained when a group instance is copied, which previously was not the case.
2023-11-02 16:33:20 +03:00
Fabio Fantoni
c99ae6f009
fix some typo spotted with codespell (#1278)
Fixed spelling typos on logs, comments and command help text
2023-11-01 17:11:16 +01:00
Yury Gargay
8843784312
Remove logging of User names on Debug level (#1275) 2023-11-01 12:00:47 +01:00
Bethuel Mmbaga
c38d65ef4c
Extends management user and group structure (#1268)
* extends user and group structure by introducing fields for issued and integration references

* Add integration checks to group management to prevent groups added by integration.

* Add integration checks to user management to prevent deleting user added by integration.

* Fix broken user update tests

* Initialize all user fields for testing

* Change a serializer option to embedded for IntegrationReference in user and group models

* Add issued field to user api response

* Add IntegrationReference to Group in update groups handler

* Set the default issued field for users in file store
2023-11-01 13:04:17 +03:00
Yury Gargay
e2eef4e3fd
Pass JWT Claims Extractor to Integrations (#1258) 2023-10-27 17:18:44 +02:00
Maycon Santos
db25ca21a8
Log auth0 batch length (#1255) 2023-10-27 09:52:11 +02:00
Yury Gargay
7a5c6b24ae
Fix GetAccountByPrivateDomain for SQLite (#1242) 2023-10-25 00:12:10 +02:00
Yury Gargay
90c2093018
Fix SaveUserLastLogin in SQLite store (#1241) 2023-10-23 16:08:21 +02:00
Maycon Santos
06318a15e1
Log store engine type (#1234) 2023-10-19 21:14:05 +02:00
Maycon Santos
e59d2317fe
Add search domains support (#1224)
Supporting search domains will allow users to define match domains to also
 be added to a list of search domains in their systems

Fix Windows registry key configuration for search domains using a key within the netbird interface path
2023-10-19 19:32:42 +02:00
Maycon Santos
a9f5fad625
Update grpc clients' keepalive interval (#1231)
Some reverse proxies might find 15s interval too short and respond with an enhance your-calm message

This change is setting the management and signal clients' keepalive interval to 30 seconds to minimize the number of reconnections
2023-10-19 10:18:16 +02:00
Yury Gargay
87cc53b743
Add management-integrations (#1227) 2023-10-17 17:19:47 +02:00
Maycon Santos
7d8a69cc0c
Use account creator as inviter as a fallback (#1225)
When inviting a user using a service user PAT, we need to fall back to a known ID to get the user's email, which is used in the invite message.
2023-10-17 15:54:50 +02:00
guangwu
73e57f17ea
chore: pkg import only once (#1222)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-16 17:00:05 +02:00
Yury Gargay
46f5f148da
Move StoreKind under own StoreConfig configuration and rename to Engine (#1219)
* Move StoreKind under own StoreConfig configuration parameter

* Rename StoreKind option to Engine

* Rename StoreKind internal methods and types to Engine

* Add template engine value test

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2023-10-16 11:19:39 +02:00
Yury Gargay
32880c56a4
Implement SQLite Store using gorm and relational approach (#1065)
Restructure data handling for improved performance and flexibility. 
Introduce 'G'-prefixed fields to represent Gorm relations, simplifying resource management. 
Eliminate complexity in lookup tables for enhanced query and write speed. 
Enable independent operations on data structures, requiring adjustments in the Store interface and Account Manager.
2023-10-12 15:42:36 +02:00
Zoltan Papp
2b90ff8c24
Fix/key backup in config script (#1206)
Because we provide the option to regenerate the config files, the encryption key could be lost.

- The configure.sh read the existing key and write it back during the config generation
- Backup the previously generated config files before overwrite it
- Fix invalid json output in the Extras field
- Reduce the error logs in case if the encryption key is invalid
- Response in the events API with valid user info in any cases
- Add extra error handling to the configure.sh. I.e. handle the invalid OpenID urls
2023-10-11 23:01:49 +02:00
Zoltan Papp
b8599f634c
Fix nil pointer exception in group delete (#1211)
Fix group delete panic

In case if in the db the DNSSettings is null then can cause panic in delete group function
because this field is pointer and it was not checked. Because of in the future implementation
this variable will be filled in any case then make no sense to keep the pointer type.

Fix DNSSettings copy function
2023-10-11 23:00:56 +02:00
Yury Gargay
659110f0d5
Rework peer connection status based on the update channel existence (#1213)
With this change, we don't need to update all peers on startup. We will
check the existence of an update channel when returning a list or single peer on API.
Then after restarting of server consumers of API will see peer not
connected status till the creation of an updated channel which indicates
peer successful connection.
2023-10-11 18:11:45 +02:00
Bethuel Mmbaga
4ad14cb46b
Add Pagination for IdP Users Fetch (#1210)
* Retrieve all workspace users via pagination, excluding custom user attributes

* Retrieve all authentik users via pagination

* Retrieve all Azure AD users via pagination

* Simplify user data appending operation

Reduced unnecessary iteration and used an efficient way to append all users to 'indexedUsers'

* Fix ineffectual assignment to reqURL

* Retrieve all Okta users via pagination

* Add missing GetAccount metrics

* Refactor

* minimize memory allocation

Refactored the memory allocation for the 'users' slice in the Okta IDP code. Previously, the slice was only initialized but not given a size. Now the size of userList is utilized to optimize memory allocation, reducing potential slice resizing and memory re-allocation costs while appending users.

* Add logging for entries received from IdP management

Added informative and debug logging statements in account.go file. Logging has been added to identify the number of entries received from Identity Provider (IdP) management. This will aid in tracking and debugging any potential data ingestion issues.
2023-10-11 17:09:30 +03:00
Maycon Santos
3c485dc7a1
Fix routing groups expand and filtering (#1203)
This PR fixes an issue were only one route containing routing groups was being synced to peers.
It also prevents sending routes for peers that aren't connect via ACL.
Moved all checks to Account.getEnabledAndDisabledRoutesByPeer.

Co-authored-by: Yury Gargay <yury.gargay@gmail.com>
Co-authored-by: braginini <bangvalo@gmail.com>
2023-10-09 14:39:41 +02:00
Yury Gargay
4791e41004
Cleanup Account Manager code (#1192)
* Remove unused am.AccountExists
* Remove unused am.GetPeerByKey
* Remove unused am.GetPeerByIP and account.GetPeerByIP
* Remove unused am.GroupListPeers
2023-10-04 15:41:52 +02:00
Yury Gargay
9131069d12
Improve updateAccountPeers by bypassing AM and using account directly (#1193)
Improve updateAccountPeers performance by bypassing AM and using the account directly
2023-10-04 15:08:50 +02:00
Bethuel Mmbaga
26bbc33e7a
Add jumpcloud IdP (#1124)
added intergration with JumpCloud User API. Use the steps in setup.md for configuration.

Additional changes:

- Enhance compatibility for providers that lack audience support in the Authorization Code Flow and the Authorization - - Code Flow with Proof Key for Code Exchange (PKCE) using NETBIRD_DASH_AUTH_USE_AUDIENCE=falseenv
- Verify tokens by utilizing the client ID when audience support is absent in providers
2023-10-03 19:33:42 +02:00
Misha Bragin
35bc493cc3
Reorder peer deletion when deleteing a user (#1191) 2023-10-03 16:46:58 +02:00