bdb2a76eae
Merge branch 'main' into policy-get-account-refactoring
...
# Conflicts:
# management/server/policy.go
# management/server/posture_checks_test.go
# management/server/sql_store.go
# management/server/sql_store_test.go
# management/server/status/error.go
# management/server/store.go
# management/server/testdata/extended-store.sql
2024-11-25 19:03:55 +03:00
ca12bc6953
[management] Refactor posture check to use store methods ( #2874 )
2024-11-25 16:26:24 +01:00
52ea2e84e9
[management] Add transaction metrics and exclude getAccount time from peers update ( #2904 )
2024-11-19 00:04:50 +01:00
6dd56e3328
Merge branch 'posturechecks-get-account-refactoring' into policy-get-account-refactoring
2024-11-18 16:58:04 +03:00
d4c712493a
Merge branch 'groups-get-account-refactoring' into posturechecks-get-account-refactoring
2024-11-15 20:52:59 +03:00
12f442439a
[management] Refactor group to use store methods ( #2867 )
...
* Refactor setup key handling to use store methods
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add lock to get account groups
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add check for regular user
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* get only required groups for auto-group validation
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add account lock and return auto groups map on validation
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor account peers update
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Refactor groups to use store methods
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor GetGroupByID and add NewGroupNotFoundError
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix tests
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Add AddPeer and RemovePeer methods to Group struct
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Preserve store engine in SqlStore transactions
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Run groups ops in transaction
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix missing group removed from setup key activity
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix merge
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix merge
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix sonar
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Change setup key log level to debug for missing group
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Retrieve modified peers once for group events
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Add tests
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Add account locking and merge group deletion methods
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Fix tests
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
---------
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-15 20:09:32 +03:00
ed047ec9dd
Add account locking and merge group deletion methods
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-13 16:16:30 +03:00
32d1b2d602
Retrieve policy groups and posture checks once for validation
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-12 18:53:10 +03:00
2a59f04540
Merge branch 'posturechecks-get-account-refactoring' into policy-get-account-refactoring
2024-11-12 17:16:52 +03:00
a3abc211b3
Add tests
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-12 17:11:56 +03:00
00023bf110
Merge branch 'groups-get-account-refactoring' into posturechecks-get-account-refactoring
2024-11-12 15:55:34 +03:00
0c0fd380bd
Refactor policy get and save account to use store methods
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-12 11:17:16 +03:00
ffce48ca5f
Merge branch 'groups-get-account-refactoring' into policy-get-account-refactoring
2024-11-11 23:08:34 +03:00
d23b5c892b
Retrieve modified peers once for group events
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-11 22:58:22 +03:00
664d1388aa
fix merge
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-11 22:29:59 +03:00
010a8bfdc1
Merge branch 'main' into groups-get-account-refactoring
...
# Conflicts:
# management/server/group.go
# management/server/group/group.go
# management/server/setupkey.go
# management/server/sql_store.go
# management/server/status/error.go
# management/server/store.go
2024-11-11 21:10:02 +03:00
6cb697eed6
[management] Refactor setup key to use store methods ( #2861 )
...
* Refactor setup key handling to use store methods
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add lock to get account groups
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add check for regular user
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* get only required groups for auto-group validation
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add account lock and return auto groups map on validation
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix missing group removed from setup key activity
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Remove context from DB queries
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Add user permission check and add setup events into events to store slice
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Retrieve all groups once during setup key auto-group validation
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Fix lint
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Fix sonar
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
---------
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-11 19:46:10 +03:00
174e07fefd
Refactor posture checks to remove get and save account
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-11 12:37:19 +03:00
6dc185e141
Preserve store engine in SqlStore transactions
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-09 01:16:03 +03:00
d58cf50127
Merge branch 'setupkey-get-account-refactoring' into groups-get-account-refactoring
...
# Conflicts:
# management/server/sql_store.go
2024-11-08 19:48:13 +03:00
40af1a50e3
Merge branch 'feature/get-account-refactoring' into setupkey-get-account-refactoring
...
# Conflicts:
# management/server/sql_store.go
2024-11-08 19:17:28 +03:00
8126d95316
refactor GetGroupByID and add NewGroupNotFoundError
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-08 18:58:04 +03:00
0a70e4c5d4
Refactor groups to use store methods
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-08 18:39:36 +03:00
669904cd06
[management] Remove context from database calls ( #2863 )
2024-11-08 15:49:00 +01:00
389c9619af
Refactor setup key handling to use store methods
...
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-11-08 00:31:41 +03:00
7fef8f6758
[management] Enforce max conn of 1 for sqlite setups ( #2855 )
2024-11-07 16:32:35 +01:00
bac95ace18
[management] Add DB access duration to logs for context cancel ( #2781 )
2024-11-01 10:58:39 +01:00
10480eb52f
[management] Setup key improvements ( #2775 )
2024-10-28 17:52:23 +01:00
96d2207684
Fix JSON function compatibility for SQLite and PostgreSQL ( #2746 )
...
resolves the issue with json_array_length compatibility between SQLite and PostgreSQL. It adjusts the query to conditionally cast types:
PostgreSQL: Casts to json with ::json.
SQLite: Uses the text representation directly.
2024-10-16 17:55:30 +02:00
da3a053e2b
[management] Refactor getAccountIDWithAuthorizationClaims ( #2715 )
...
This change restructures the getAccountIDWithAuthorizationClaims method to improve readability, maintainability, and performance.
- have dedicated methods to handle possible cases
- introduced Store.UpdateAccountDomainAttributes and Store.GetAccountUsers methods
- Remove GetAccount and SaveAccount dependency
- added tests
2024-10-12 08:35:51 +02:00
8284ae959c
[management] Move testdata to sql files ( #2693 )
2024-10-10 12:35:03 +02:00
b79c1d64cc
[management] Make max open db conns configurable ( #2713 )
2024-10-09 20:17:25 +02:00
d4ef84fe6e
[management] Propagate error in store errors ( #2709 )
2024-10-09 14:33:58 +02:00
8bf729c7b4
[management] Add AccountExists to AccountManager ( #2694 )
...
* Add AccountExists method to account manager interface
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* remove unused code
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
---------
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-10-04 18:09:40 +03:00
7f09b39769
[management] Refactor User JWT group sync ( #2690 )
...
* Refactor GetAccountIDByUserOrAccountID
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* sync user jwt group changes
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* propagate jwt group changes to peers
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix no jwt groups synced
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix tests and lint
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Move the account peer update outside the transaction
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* move updateUserPeersInGroups to account manager
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* move event store outside of transaction
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* get user with update lock
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Run jwt sync in transaction
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
---------
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-10-04 17:17:01 +03:00
158936fb15
[management] Remove file store ( #2689 )
2024-10-03 15:50:35 +02:00
acb73bd64a
[management] Remove redundant get account calls in GetAccountFromToken ( #2615 )
...
* refactor access control middleware and user access by JWT groups
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor jwt groups extractor
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor handlers to get account when necessary
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor getAccountFromToken
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor getAccountWithAuthorizationClaims
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix merge
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* revert handles change
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* remove GetUserByID from account manager
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix tests
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor getAccountWithAuthorizationClaims to return account id
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor handlers to use GetAccountIDFromToken
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix tests
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* remove locks
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* refactor
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add GetGroupByName from store
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add GetGroupByID from store and refactor
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Refactor retrieval of policy and posture checks
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Refactor user permissions and retrieves PAT
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Refactor route, setupkey, nameserver and dns to get record(s) from store
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* Refactor store
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix lint
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix tests
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix add missing policy source posture checks
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add store lock
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* fix tests
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
* add get account
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
---------
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com >
2024-09-27 17:10:50 +03:00
1e4a0f77e2
Add get DB method to store ( #2650 )
2024-09-25 18:22:27 +02:00
6c50b0c84b
[management] Add transaction to addPeer ( #2469 )
...
This PR removes the GetAccount and SaveAccount operations from the AddPeer and instead makes use of gorm.Transaction to add the new peer.
2024-09-16 15:47:03 +02:00
d97b03656f
[management] Refactor HTTP metrics ( #2476 )
...
* Add logging for slow SQL queries in SaveAccount and GetAccount
* Add resource count log for large accounts
* Refactor metrics middleware to simplify counters and histograms
* Update log levels and remove redundant resource count check
2024-08-23 19:42:55 +03:00
049b5fb7ed
Split DB calls in peer login ( #2439 )
2024-08-19 12:50:11 +02:00
165988429c
Add write lock for peer when saving its connection status ( #2359 )
2024-07-31 14:53:32 +02:00
1f48fdf6ca
Add SavePeer method to prevent a possible account inconsistency ( #2296 )
...
SyncPeer was storing the account with a simple read lock
This change introduces the SavePeer method to the store to be used in these cases
2024-07-26 07:49:05 +02:00
a711e116a3
fix: save peer status correctly in sqlstore ( #2262 )
...
* fix: save peer status correctly in sqlstore
https://github.com/netbirdio/netbird/issues/2110#issuecomment-2162768273
* feat: update test function
* refactor: simplify status update
2024-07-16 18:38:12 +03:00
1537b0f5e7
Add batch save/update for groups and users ( #2245 )
...
* Add functionality to update multiple users
* Remove SaveUsers from DefaultAccountManager
* Add SaveGroups method to AccountManager interface
* Refactoring
* Add SaveUsers and SaveGroups methods to store interface
* Refactor method SaveAccount to SaveUsers and SaveGroups
The method SaveAccount in user.go and group.go files was split into two separate methods. Now, user-specific data is handled by SaveUsers and group-specific data is handled by SaveGroups method. This provides a cleaner and more efficient way to save user and group data.
* Add account ID to user and group in SqlStore
* Refactor SaveUsers and SaveGroups in store
* Remove unnecessary ID assignment in SaveUsers and SaveGroups
2024-07-15 17:04:06 +03:00
58fbc1249c
Fix parameter limit issue for Postgres store ( #2261 )
...
Added CreateBatchSize for both SQL stores and updated tests to test large accounts with Postgres, too. Increased the account peer size to 6K.
2024-07-12 09:28:53 +02:00
765aba2c1c
Add context to throughout the project and update logging ( #2209 )
...
propagate context from all the API calls and log request ID, account ID and peer ID
---------
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com >
2024-07-03 11:33:02 +02:00
eab6183a8e
Add stack trace when saving empty domains ( #2228 )
...
added temporary domain check for existing accounts to trace where the issue originated
Refactor save account due to complexity score
2024-07-02 12:40:26 +02:00
95299be52d
Deprecate FileStore engine support ( #2119 )
...
* Removejsonfile' from test matrix in workflows
* Remove sqlite to json migration command
* Refactor store engine implementation to remove JSON file store support
The codebase has been refactored to remove support for JSON file store storage engine, with SQLite serving as the default store engine. New functions have been added to handle unsupported store engines and to migrate data from file store to SQLite.
* Remove 'downCmd' from migration commands
* Refactoring
* Add sqlite cleanup
* Remove comment
2024-06-13 13:39:19 +03:00
521f7dd39f
Improve login performance ( #2061 )
2024-05-31 16:41:12 +02:00
