Commit Graph

419 Commits

Author SHA1 Message Date
Yury Gargay
7a5c6b24ae
Fix GetAccountByPrivateDomain for SQLite (#1242) 2023-10-25 00:12:10 +02:00
Yury Gargay
90c2093018
Fix SaveUserLastLogin in SQLite store (#1241) 2023-10-23 16:08:21 +02:00
Maycon Santos
06318a15e1
Log store engine type (#1234) 2023-10-19 21:14:05 +02:00
Maycon Santos
e59d2317fe
Add search domains support (#1224)
Supporting search domains will allow users to define match domains to also
 be added to a list of search domains in their systems

Fix Windows registry key configuration for search domains using a key within the netbird interface path
2023-10-19 19:32:42 +02:00
Maycon Santos
a9f5fad625
Update grpc clients' keepalive interval (#1231)
Some reverse proxies might find 15s interval too short and respond with an enhance your-calm message

This change is setting the management and signal clients' keepalive interval to 30 seconds to minimize the number of reconnections
2023-10-19 10:18:16 +02:00
Yury Gargay
87cc53b743
Add management-integrations (#1227) 2023-10-17 17:19:47 +02:00
Maycon Santos
7d8a69cc0c
Use account creator as inviter as a fallback (#1225)
When inviting a user using a service user PAT, we need to fall back to a known ID to get the user's email, which is used in the invite message.
2023-10-17 15:54:50 +02:00
guangwu
73e57f17ea
chore: pkg import only once (#1222)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-16 17:00:05 +02:00
Yury Gargay
46f5f148da
Move StoreKind under own StoreConfig configuration and rename to Engine (#1219)
* Move StoreKind under own StoreConfig configuration parameter

* Rename StoreKind option to Engine

* Rename StoreKind internal methods and types to Engine

* Add template engine value test

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2023-10-16 11:19:39 +02:00
Yury Gargay
32880c56a4
Implement SQLite Store using gorm and relational approach (#1065)
Restructure data handling for improved performance and flexibility. 
Introduce 'G'-prefixed fields to represent Gorm relations, simplifying resource management. 
Eliminate complexity in lookup tables for enhanced query and write speed. 
Enable independent operations on data structures, requiring adjustments in the Store interface and Account Manager.
2023-10-12 15:42:36 +02:00
Zoltan Papp
2b90ff8c24
Fix/key backup in config script (#1206)
Because we provide the option to regenerate the config files, the encryption key could be lost.

- The configure.sh read the existing key and write it back during the config generation
- Backup the previously generated config files before overwrite it
- Fix invalid json output in the Extras field
- Reduce the error logs in case if the encryption key is invalid
- Response in the events API with valid user info in any cases
- Add extra error handling to the configure.sh. I.e. handle the invalid OpenID urls
2023-10-11 23:01:49 +02:00
Zoltan Papp
b8599f634c
Fix nil pointer exception in group delete (#1211)
Fix group delete panic

In case if in the db the DNSSettings is null then can cause panic in delete group function
because this field is pointer and it was not checked. Because of in the future implementation
this variable will be filled in any case then make no sense to keep the pointer type.

Fix DNSSettings copy function
2023-10-11 23:00:56 +02:00
Yury Gargay
659110f0d5
Rework peer connection status based on the update channel existence (#1213)
With this change, we don't need to update all peers on startup. We will
check the existence of an update channel when returning a list or single peer on API.
Then after restarting of server consumers of API will see peer not
connected status till the creation of an updated channel which indicates
peer successful connection.
2023-10-11 18:11:45 +02:00
Bethuel Mmbaga
4ad14cb46b
Add Pagination for IdP Users Fetch (#1210)
* Retrieve all workspace users via pagination, excluding custom user attributes

* Retrieve all authentik users via pagination

* Retrieve all Azure AD users via pagination

* Simplify user data appending operation

Reduced unnecessary iteration and used an efficient way to append all users to 'indexedUsers'

* Fix ineffectual assignment to reqURL

* Retrieve all Okta users via pagination

* Add missing GetAccount metrics

* Refactor

* minimize memory allocation

Refactored the memory allocation for the 'users' slice in the Okta IDP code. Previously, the slice was only initialized but not given a size. Now the size of userList is utilized to optimize memory allocation, reducing potential slice resizing and memory re-allocation costs while appending users.

* Add logging for entries received from IdP management

Added informative and debug logging statements in account.go file. Logging has been added to identify the number of entries received from Identity Provider (IdP) management. This will aid in tracking and debugging any potential data ingestion issues.
2023-10-11 17:09:30 +03:00
Maycon Santos
3c485dc7a1
Fix routing groups expand and filtering (#1203)
This PR fixes an issue were only one route containing routing groups was being synced to peers.
It also prevents sending routes for peers that aren't connect via ACL.
Moved all checks to Account.getEnabledAndDisabledRoutesByPeer.

Co-authored-by: Yury Gargay <yury.gargay@gmail.com>
Co-authored-by: braginini <bangvalo@gmail.com>
2023-10-09 14:39:41 +02:00
Yury Gargay
4791e41004
Cleanup Account Manager code (#1192)
* Remove unused am.AccountExists
* Remove unused am.GetPeerByKey
* Remove unused am.GetPeerByIP and account.GetPeerByIP
* Remove unused am.GroupListPeers
2023-10-04 15:41:52 +02:00
Yury Gargay
9131069d12
Improve updateAccountPeers by bypassing AM and using account directly (#1193)
Improve updateAccountPeers performance by bypassing AM and using the account directly
2023-10-04 15:08:50 +02:00
Bethuel Mmbaga
26bbc33e7a
Add jumpcloud IdP (#1124)
added intergration with JumpCloud User API. Use the steps in setup.md for configuration.

Additional changes:

- Enhance compatibility for providers that lack audience support in the Authorization Code Flow and the Authorization - - Code Flow with Proof Key for Code Exchange (PKCE) using NETBIRD_DASH_AUTH_USE_AUDIENCE=falseenv
- Verify tokens by utilizing the client ID when audience support is absent in providers
2023-10-03 19:33:42 +02:00
Misha Bragin
35bc493cc3
Reorder peer deletion when deleteing a user (#1191) 2023-10-03 16:46:58 +02:00
Bethuel Mmbaga
e26ec0b937
Optimize Cache and IDP Management (#1147)
This pull request modifies the IdP and cache manager(s) to prevent the sending of app metadata
 to the upstream IDP on self-hosted instances. 
As a result, the IdP will now load all users from the IdP without filtering based on accountID.

We disable user invites as the administrator's own IDP system manages them.
2023-10-03 16:40:28 +02:00
Maycon Santos
a952e7c72f
Prevent return extra userData (#1190)
If there is a difference between local and cached data, we trigger a cache refresh; 
as we remove users from the local store and potentially from the remote IDP, 
we need to switch the source of truth to the local store to prevent unwanted endless 
cache for cases where the removal from the IDP fails or for cases 
where the userDeleteFromIDPEnabled got enabled after the first user deletion.
2023-10-02 19:18:08 +02:00
Maycon Santos
22f69d7852
Add routing groups metrics (#1187)
add routing groups metrics and tests for the property generation
2023-10-02 12:10:12 +02:00
Misha Bragin
b23011fbe8
Delete user peers when deleting a user (#1186) 2023-10-01 19:51:39 +02:00
Misha Bragin
6ad3894a51
Fix peer login expiration event duplication (#1185) 2023-09-29 17:37:04 +02:00
Maycon Santos
0c470e7838
Update delete method for user API (#1160) 2023-09-28 21:53:28 +02:00
Yury Gargay
8118d60ffb
Add peer groups support for network routes (#1150)
This commit enhances the functionality of the network routes endpoint by introducing a new parameter called `peers_group`. This addition allows users to associate network routes with specific peer groups, simplifying the management and distribution of routes within a network.
2023-09-28 14:32:36 +02:00
Yury Gargay
830dee1771
Expose store metrics with milliseconds bucketing (#1179)
As the current upper 10000 microseconds(10ms) bucket may be too low for
`management.store.persistence.duration` metric
2023-09-28 08:54:49 +02:00
Yury Gargay
c08a96770e
Remove unnecessary global lock (#1178) 2023-09-27 15:51:49 +02:00
Pascal Fischer
ab849f0942 add additional check after datastore init if the key was newly generated and needs to be written to config 2023-09-27 14:48:40 +02:00
Yury Gargay
1324169ebb
Add management_grpc_updatechannel_queue_bucket histogram (#1158)
This should help to find better value for `server.channelBufferSize`
2023-09-25 10:23:43 +02:00
Maycon Santos
732afd8393
Revert zitadel update parameters endpoint (#1163)
* Revert zitadel update parameters endpoint

With previous release we broke the parameters' endpoint. This Pr reverses that

* add error log to util
2023-09-25 07:57:16 +02:00
Zoltan Papp
da7b6b11ad
Fix/user deletion (#1157)
Extend the deleted user info with the username
- Because initially, we did not store the user name in the activity db 
Sometimes, we can not provide the user name in the API response.

Fix service user deletion
  - In case of service user deletion, do not invoke the IdP delete function
  - Prevent self deletion
2023-09-23 10:47:49 +02:00
Maycon Santos
e260270825
Add direct write file to avoid moving docker mounted files (#1155)
Add a direct write to handle management.json write operation. 

Remove empty configuration types to avoid unnecessary fields in the generated management.json file.
2023-09-22 10:25:04 +02:00
Givi Khojanashvili
d4b6d7646c
Handle user delete (#1113)
Implement user deletion across all IDP-ss. Expires all user peers
when the user is deleted. Users are permanently removed from a local
store, but in IDP, we remove Netbird attributes for the user
untilUserDeleteFromIDPEnabled setting is not enabled.

To test, an admin user should remove any additional users.

Until the UI incorporates this feature, use a curl DELETE request
targeting the /users/<USER_ID> management endpoint. Note that this
request only removes user attributes and doesn't trigger a delete
from the IDP.

To enable user removal from the IdP, set UserDeleteFromIDPEnabled
to true in account settings. Until we have a UI for this, make this
change directly in the store file.

Store the deleted email addresses in encrypted in activity store.
2023-09-19 18:08:40 +02:00
Yury Gargay
0be8c72601
Remove unused methods from AccountManager interface (#1149)
This PR removes the following unused methods from the AccountManager interface:
* `UpdateGroup`
* `UpdateNameServerGroup`
* `UpdateRoute`
2023-09-18 12:25:12 +02:00
Maycon Santos
fa4b8c1d42
Update ephemeral field on the API response (#1129) 2023-09-06 10:40:45 +02:00
Maycon Santos
7682fe2e45
Account ephemeral setup keys metrics (#1128) 2023-09-05 23:04:14 +02:00
Misha Bragin
e4bc76c4de
Ignore empty fields in the app metadata when storing on IDP (#1122) 2023-09-05 14:41:50 +02:00
Yury Gargay
bb40325977
Update GitHub Actions and Enhance golangci-lint (#1075)
This PR showcases the implementation of additional linter rules. I've updated the golangci-lint GitHub Actions to the latest available version. This update makes sure that the tool works the same way locally - assuming being updated regularly - and with the GitHub Actions.

I've also taken care of keeping all the GitHub Actions up to date, which helps our code stay current. But there's one part, goreleaser that's a bit tricky to test on our computers. So, it's important to take a close look at that.

To make it easier to understand what I've done, I've made separate changes for each thing that the new linters found. This should help the people reviewing the changes see what's going on more clearly. Some of the changes might not be obvious at first glance.

Things to consider for the future
CI runs on Ubuntu so the static analysis only happens for Linux. Consider running it for the rest: Darwin, Windows
2023-09-04 17:03:44 +02:00
Zoltan Papp
c1f164c9cb
Feature/ephemeral peers (#1100)
The ephemeral manager keep the inactive ephemeral peers in a linked list. The manager schedule a cleanup procedure to the head of the linked list (to the most deprecated peer). At the end of cleanup schedule the next cleanup to the new head.
If a device connect back to the server the manager will remote it from the peers list.
2023-09-04 11:37:39 +02:00
pascal-fischer
f89c200ce9
Fix api Auth with PAT when a custom UserIDClaim is configured in management.json (#1120)
The API authentication with PATs was not considering different userIDClaim 
that some of the IdPs are using.
In this PR we read the userIDClaim from the config file 
instead of using the fixed default and only keep 
it as a fallback if none in defined.
2023-09-01 18:09:59 +02:00
Zoltan Papp
00dddb9458
Fix log formatter initialization in mgm cmd (#1112)
The log format was mixed in the management command.
In this commit put to earlier state the log preparation.
2023-08-30 11:42:03 +02:00
Yury Gargay
e586eca16c
Improve account copying (#1069)
With this fix, all nested slices and pointers will be copied by value.
Also, this fixes tests to compare the original and copy account by their
values by marshaling them to JSON strings.

Before that, they were copying the pointers that also passed the simple `=` compassion
(as the addresses match).
2023-08-22 17:56:39 +02:00
pascal-fischer
da75a76d41
Adding dashboard login activity (#1092)
For better auditing this PR adds a dashboard login event to the management service.

For that the user object was extended with a field for last login that is not actively saved to the database but kept in memory until next write. The information about the last login can be extracted from the JWT claims nb_last_login. This timestamp will be stored and compared on each API request. If the value changes we generate an event to inform about a login.
2023-08-18 19:23:11 +02:00
Givi Khojanashvili
3ac32fd78a
Send network update when propagate user auto-groups (#1084)
For peer propagation this commit triggers
network map update in two cases:
  1) peer login
  2) user AutoGroups update

Also it issues new activity message about new user group
for peer login process.

Previous implementation only adds JWT groups to user. This fix also
removes JWT groups from user auto assign groups.

Pelase note, it also happen when user works with dashboard.
2023-08-18 15:36:05 +02:00
Misha Bragin
d4e9087f94
Add peer login and expiration activity events (#1090)
Track the even of a user logging in their peer.
Track the event of a peer login expiration.
2023-08-17 14:04:04 +02:00
Misha Bragin
8e3bcd57a2
Specify invited by email when inviting a user (#1087) 2023-08-16 23:05:22 +02:00
Maycon Santos
4572c6c1f8
Avoid categorization on incoming claim (#1086)
This prevents domain categorization on claims of invited users
2023-08-16 16:11:26 +02:00
Bethuel Mmbaga
442ba7cbc8
Add domain validation for nameserver groups (#1077)
This change ensures that domain names with uppercase 
letters are also considered valid, 
providing more flexibility in domain naming.
2023-08-16 11:25:38 +02:00
Givi Khojanashvili
e5e69b1f75
Autopropagate peers by JWT groups (#1037)
Enhancements to Peer Group Assignment:

1. Auto-assigned groups are now applied to all peers every time a user logs into the network.
2. Feature activation is available in the account settings.
3. API modifications included to support these changes for account settings updates.
4. If propagation is enabled, updates to a user's auto-assigned groups are immediately reflected across all user peers.
5. With the JWT group sync feature active, auto-assigned groups are forcefully updated whenever a peer logs in using user credentials.
2023-08-07 19:44:51 +04:00