netbird/management/server
Bethuel Mmbaga 7bda385e1b
[management] Optimize network map updates (#2718)
* Skip peer update on unchanged network map (#2236)

* Enhance network updates by skipping unchanged messages

Optimizes the network update process
by skipping updates where no changes in the peer update message received.

* Add unit tests

* add locks

* Improve concurrency and update peer message handling

* Refactor account manager network update tests

* fix test

* Fix inverted network map update condition

* Add default group and policy to test data

* Run peer updates in a separate goroutine

* Refactor

* Refactor lock

* Fix peers update by including NetworkMap and posture Checks

* go mod tidy

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* [management] Skip account peers update if no changes affect peers (#2310)

* Remove incrementing network serial and updating peers after group deletion

* Update account peer if posture check is linked to policy

* Remove account peers update on saving setup key

* Refactor group link checking into re-usable functions

* Add HasPeers function to group

* Refactor group management

* Optimize group change effects on account peers

* Update account peers if ns group has peers

* Refactor group changes

* Optimize account peers update in DNS settings

* Optimize update of account peers on jwt groups sync

* Refactor peer account updates for efficiency

* Optimize peer update on user deletion and changes

* Remove condition check for network serial update

* Optimize account peers updates on route changes

* Remove UpdatePeerSSHKey method

* Remove unused isPolicyRuleGroupsEmpty

* Add tests for peer update behavior on posture check changes

* Add tests for peer update behavior on policy changes

* Add tests for peer update behavior on group changes

* Add tests for peer update behavior on dns settings changes

* Refactor

* Add tests for peer update behavior on name server changes

* Add tests for peer update behavior on user changes

* Add tests for peer update behavior on route changes

* fix tests

* Add tests for peer update behavior on setup key changes

* Add tests for peer update behavior on peers changes

* fix merge

* Fix tests

* go mod tidy

* Add NameServer and Route comparators

* Update network map diff logic with custom comparators

* Add tests

* Refactor duplicate diff handling logic

* fix linter

* fix tests

* Refactor policy group handling and update logic.

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update route check by checking if group has peers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor posture check policy linking logic

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Simplify peer update condition in DNS management

Refactor the condition for updating account peers to remove redundant checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add posture checks tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix user and setup key tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix account and route tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix typo

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix routes tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* upgrade diff package

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* use generic differ for netip.Addr and netip.Prefix

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* go mod tidy

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add peer tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix management suite tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix postgres tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* enable diff nil structs comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip the update only last sent the serial is larger

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor peer and user

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for groupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor group, ns group, policy and posture checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for GroupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update account policy check before verifying policy status

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* add tests missing tests for dns setting groups

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add tests for posture checks changes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add ns group and policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add route and group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* increase Linux test timeout to 10 minutes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Run diff for client posture checks only

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add panic recovery and detailed logging in peer update comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-10-23 13:05:02 +03:00
..
account Feature/peer validator (#1553) 2024-03-27 18:48:48 +01:00
activity Add session expire functionality based on inactivity (#2326) 2024-10-13 14:52:43 +02:00
context Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
differs [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
geolocation [management] Auto update geolite (#2297) 2024-09-09 18:27:42 +02:00
group [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
http Add session expire functionality based on inactivity (#2326) 2024-10-13 14:52:43 +02:00
idp [management] improve zitadel idp error response detail by decoding errors (#2634) 2024-09-27 22:21:34 +03:00
integrated_validator Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
integration_reference Feature/peer validator (#1553) 2024-03-27 18:48:48 +01:00
jwtclaims [management] Add support to ECDSA public Keys (#2461) 2024-08-27 16:37:55 +02:00
metrics Fix issue with canceled context before pushing metrics and decreasing pushing interval (#2235) 2024-07-04 19:15:59 +02:00
migration Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
mock_server [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
peer [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
posture Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
status [management] Propagate error in store errors (#2709) 2024-10-09 14:33:58 +02:00
telemetry [management] Refactor HTTP metrics (#2476) 2024-08-23 19:42:55 +03:00
testdata [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
testutil Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
account_request_buffer.go rename request buffer and update default interval (#2459) 2024-08-21 11:44:52 +02:00
account_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
account.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
config.go [relay] Feature/relay integration (#2244) 2024-09-08 12:06:14 +02:00
dns_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
dns.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
ephemeral_test.go [management] Add transaction to addPeer (#2469) 2024-09-16 15:47:03 +02:00
ephemeral.go Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
event_test.go Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
event.go Add write lock for peer when saving its connection status (#2359) 2024-07-31 14:53:32 +02:00
file_store.go Add session expire functionality based on inactivity (#2326) 2024-10-13 14:52:43 +02:00
group_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
group.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
grpcserver.go [management, client] Add access control support to network routes (#2100) 2024-10-02 13:41:00 +02:00
integrated_validator.go Add write lock for peer when saving its connection status (#2359) 2024-07-31 14:53:32 +02:00
management_proto_test.go [management] Move testdata to sql files (#2693) 2024-10-10 12:35:03 +02:00
management_suite_test.go Move management server to a separate directory (#67) 2021-07-24 16:14:29 +02:00
management_test.go [management] Move testdata to sql files (#2693) 2024-10-10 12:35:03 +02:00
nameserver_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
nameserver.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
network_test.go Exclude second last IP from allocation to use it in the Fake DNS (#912) 2023-05-30 18:26:44 +04:00
network.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
peer_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
peer.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
personal_access_token_test.go move into separate package 2023-05-16 12:57:56 +02:00
personal_access_token.go Implement SQLite Store using gorm and relational approach (#1065) 2023-10-12 15:42:36 +02:00
policy_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
policy.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
posture_checks_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
posture_checks.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
route_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
route.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
scheduler_test.go Reduce test logs (#2550) 2024-09-06 16:28:19 +02:00
scheduler.go Add context to throughout the project and update logging (#2209) 2024-07-03 11:33:02 +02:00
setupkey_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
setupkey.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
sql_store_test.go Fix JSON function compatibility for SQLite and PostgreSQL (#2746) 2024-10-16 17:55:30 +02:00
sql_store.go Fix JSON function compatibility for SQLite and PostgreSQL (#2746) 2024-10-16 17:55:30 +02:00
store_test.go [management] Remove file store (#2689) 2024-10-03 15:50:35 +02:00
store.go [management] Refactor getAccountIDWithAuthorizationClaims (#2715) 2024-10-12 08:35:51 +02:00
token_mgr_test.go [relay] Improve relay messages (#2574) 2024-09-11 16:20:30 +02:00
token_mgr.go [relay] Improve relay messages (#2574) 2024-09-11 16:20:30 +02:00
updatechannel_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
updatechannel.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
user_test.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00
user.go [management] Optimize network map updates (#2718) 2024-10-23 13:05:02 +03:00