Commit Graph

624 Commits

Author SHA1 Message Date
f43a0a0177 [client] Retry on tun creation for darwin (#2564)
The interface creation on macOS seems to be asynchronus why the tun.create methode somethimes failes becasue the interface is not ready yet. To work around this issue we introduce a retry on tun.create
2024-09-09 19:02:10 +02:00
12c36312b5 [management] Auto update geolite (#2297)
introduces helper functions to fetch and verify database versions, downloads new files if outdated, and deletes old ones. It also refactors filename handling to improve clarity and consistency, adding options to disable auto-updating via a flag. The changes aim to simplify GeoLite database management for admins.
2024-09-09 18:27:42 +02:00
0c039274a4 [relay] Feature/relay integration (#2244)
This update adds new relay integration for NetBird clients. The new relay is based on web sockets and listens on a single port.

- Adds new relay implementation with websocket with single port relaying mechanism
- refactor peer connection logic, allowing upgrade and downgrade from/to P2P connection
- peer connections are faster since it connects first to relay and then upgrades to P2P
- maintains compatibility with old clients by not using the new relay
- updates infrastructure scripts with new relay service
2024-09-08 12:06:14 +02:00
a7e46bf7b1 Reduce test logs (#2550) 2024-09-06 16:28:19 +02:00
95174d4619 Update route API doc with max domain number (#2516) 2024-09-02 17:40:34 +02:00
00944bcdbf [management] Add support to ECDSA public Keys (#2461)
Update the JWT validation logic to handle ECDSA keys in addition to the existing RSA keys

---------

Co-authored-by: Harry Kodden <harry.kodden@surf.nl>
Co-authored-by: Bethuel Mmbaga <bethuelmbaga12@gmail.com>
2024-08-27 16:37:55 +02:00
d97b03656f [management] Refactor HTTP metrics (#2476)
* Add logging for slow SQL queries in SaveAccount and GetAccount

* Add resource count log for large accounts

* Refactor metrics middleware to simplify counters and histograms

* Update log levels and remove redundant resource count check
2024-08-23 19:42:55 +03:00
0f0415b92a rename request buffer and update default interval (#2459) 2024-08-21 11:44:52 +02:00
3ed90728e6 [management] Add buffering for getAccount requests during login (#2449) 2024-08-20 20:06:01 +02:00
8c2d37d3fc [management] Fix logging out peers on deletion (#2453) 2024-08-20 19:13:40 +02:00
049b5fb7ed Split DB calls in peer login (#2439) 2024-08-19 12:50:11 +02:00
6016d2f7ce Fix lint (#2427) 2024-08-14 13:30:10 +03:00
539480a713 [management] Prevent removal of All group from peers during user groups propagation (#2410)
* Prevent removal of "All" group from peers

* Prevent adding "All" group to users and setup keys

* Refactor setup key group validation
2024-08-12 13:48:05 +03:00
0911163146 Add batch delete for groups and users (#2370)
* Refactor user deletion logic and introduce batch delete

* Prevent self-deletion for users

* Add delete multiple groups

* Refactor group deletion with validation

* Fix tests

* Add bulk delete functions for Users and Groups in account manager interface and mocks

* Add tests for DeleteGroups method in group management

* Add tests for DeleteUsers method in users management
2024-08-08 18:01:38 +03:00
ac0d5ff9f3 [management] Improve mgmt sync performance (#2363) 2024-08-07 10:52:31 +02:00
54d896846b Skip network map check if not regular user (#2402)
when getting all peers we don't need to calculate network map when not a regular user
2024-08-07 10:22:12 +02:00
f84b606506 add extra auth audience (#2350) 2024-08-01 18:52:50 +02:00
216d9f2ee8 Adding geolocation download log message. (#2085)
* Adding geolocation download prompt message.

* import log file and remove unnecessary else

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-08-01 18:52:38 +02:00
3506ac4234 When creating new setup key, "revoked" field doesn't do anything (#2357)
Remove unused field from API
2024-08-01 17:13:58 +02:00
cbf9f2058e Use accountID retrieved from the sync call to acquire read lock sooner (#2369)
Use accountID retrieved from the sync call to acquire read lock sooner and avoiding extra DB calls.
- Use the account ID across sync calls
- Moved account read lock
- Renamed CancelPeerRoutines to OnPeerDisconnected
- Added race tests
2024-08-01 16:21:43 +02:00
5ee9c77e90 Move write peer lock (#2364)
Moved the write peer lock to avoid latency caused by disk access

Updated the method CancelPeerRoutines to use the peer public key
2024-07-31 21:51:45 +02:00
c832cef44c Update SaveUsers and SaveGroups to SaveAccount (#2362)
Changed SaveUsers and SaveGroups method calls to SaveAccount for consistency in data persistence operations.
2024-07-31 19:48:12 +03:00
165988429c Add write lock for peer when saving its connection status (#2359) 2024-07-31 14:53:32 +02:00
da39c8bbca Refactor login with store.SavePeer (#2334)
This pull request refactors the login functionality by integrating store.SavePeer. The changes aim to improve the handling of peer login processes, particularly focusing on synchronization and error handling.

Changes:
- Refactored login logic to use store.SavePeer.
- Added checks for login without lock for login necessary checks from the client and utilized write lock for full login flow.
- Updated error handling with status.NewPeerLoginExpiredError().
- Moved geoIP check logic to a more appropriate place.
- Removed redundant calls and improved documentation.
- Moved the code to smaller methods to improve readability.
2024-07-29 13:30:27 +02:00
7321046cd6 Remove redundant check for empty JWT groups (#2323)
* Remove redundant check for empty group names in SetJWTGroups

* add test
2024-07-26 16:33:54 +02:00
1f48fdf6ca Add SavePeer method to prevent a possible account inconsistency (#2296)
SyncPeer was storing the account with a simple read lock

This change introduces the SavePeer method to the store to be used in these cases
2024-07-26 07:49:05 +02:00
45fd1e9c21 add save peer status test for connected peers (#2321) 2024-07-25 16:22:04 +02:00
788f130941 Retry management connection only on context canceled (#2301) 2024-07-22 15:49:25 +02:00
95d725f2c1 Wait on daemon down (#2279) 2024-07-17 16:26:06 +02:00
a711e116a3 fix: save peer status correctly in sqlstore (#2262)
* fix: save peer status correctly in sqlstore

https://github.com/netbirdio/netbird/issues/2110#issuecomment-2162768273

* feat: update test function

* refactor: simplify status update
2024-07-16 18:38:12 +03:00
1537b0f5e7 Add batch save/update for groups and users (#2245)
* Add functionality to update multiple users

* Remove SaveUsers from DefaultAccountManager

* Add SaveGroups method to AccountManager interface

* Refactoring

* Add SaveUsers and SaveGroups methods to store interface

* Refactor method SaveAccount to SaveUsers and SaveGroups

The method SaveAccount in user.go and group.go files was split into two separate methods. Now, user-specific data is handled by SaveUsers and group-specific data is handled by SaveGroups method. This provides a cleaner and more efficient way to save user and group data.

* Add account ID to user and group in SqlStore

* Refactor SaveUsers and SaveGroups in store

* Remove unnecessary ID assignment in SaveUsers and SaveGroups
2024-07-15 17:04:06 +03:00
58fbc1249c Fix parameter limit issue for Postgres store (#2261)
Added CreateBatchSize for both SQL stores and updated tests to test large accounts with Postgres, too. Increased the account peer size to 6K.
2024-07-12 09:28:53 +02:00
e8c2fafccd Avoid empty domain overwrite (#2252) 2024-07-10 14:08:35 +02:00
85e991ff78 Fix issue with canceled context before pushing metrics and decreasing pushing interval (#2235)
Fix a bug where the post context was canceled before sending metrics to the server.

The interval time was decreased, and an optional environment variable NETBIRD_METRICS_INTERVAL_IN_SECONDS was added to control the interval time.

* update doc URL
2024-07-04 19:15:59 +02:00
765aba2c1c Add context to throughout the project and update logging (#2209)
propagate context from all the API calls and log request ID, account ID and peer ID

---------

Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
2024-07-03 11:33:02 +02:00
7cb81f1d70 Fix nil pointer exception in case of error (#2230) 2024-07-02 18:18:14 +02:00
eab6183a8e Add stack trace when saving empty domains (#2228)
added temporary domain check for existing accounts to trace where the issue originated

Refactor save account due to complexity score
2024-07-02 12:40:26 +02:00
eaa31c2dc6 Optimize process checks database read (#2182)
* Add posture checks to peer management

This commit includes posture checks to the peer management logic. The AddPeer, SyncPeer and LoginPeer functions now return a list of posture checks along with the peer and network map.

* Update peer methods to return posture checks

* Refactor

* return early if there is no posture checks

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-06-22 17:41:16 +03:00
fc15ee6351 auto migrate older management to sqlite (#2170) 2024-06-20 19:45:57 +02:00
f9462eea27 Fix dns route retrieval condition (#2165)
* Fix route retrieval condition

* Make error messages take domains into account
2024-06-20 13:52:32 +02:00
381447b8d6 Fix store migration on empty string (#2149)
* Fix store migration on empty string

when fetching empty values from the database to check for migration our parser failed to handle null strings preventing the service from start

this uses sql.NullString to handle that and check for empty string resulted from null data

---------

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
2024-06-18 15:39:54 +02:00
4fec709bb1 Release 0.28.0 (#2092)
* compile client under freebsd (#1620)

Compile netbird client under freebsd and now support netstack and userspace modes.
Refactoring linux specific code to share same code with FreeBSD, move to *_unix.go files.

Not implemented yet:

Kernel mode not supported
DNS probably does not work yet
Routing also probably does not work yet
SSH support did not tested yet
Lack of test environment for freebsd (dedicated VM for github runners under FreeBSD required)
Lack of tests for freebsd specific code
info reporting need to review and also implement, for example OS reported as GENERIC instead of FreeBSD (lack of FreeBSD icon in management interface)
Lack of proper client setup under FreeBSD
Lack of FreeBSD port/package

* Add DNS routes (#1943)

Given domains are resolved periodically and resolved IPs are replaced with the new ones. Unless the flag keep_route is set to true, then only new ones are added.
This option is helpful if there are long-running connections that might still point to old IP addresses from changed DNS records.

* Add process posture check (#1693)

Introduces a process posture check to validate the existence and active status of specific binaries on peer systems. The check ensures that files are present at specified paths, and that corresponding processes are running. This check supports Linux, Windows, and macOS systems.


Co-authored-by: Evgenii <mail@skillcoder.com>
Co-authored-by: Pascal Fischer <pascal@netbird.io>
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
Co-authored-by: Bethuel Mmbaga <bethuelmbaga12@gmail.com>
2024-06-13 13:24:24 +02:00
95299be52d Deprecate FileStore engine support (#2119)
* Removejsonfile' from test matrix in workflows

* Remove sqlite to json migration command

* Refactor store engine implementation to remove JSON file store support

The codebase has been refactored to remove support for JSON file store storage engine, with SQLite serving as the default store engine. New functions have been added to handle unsupported store engines and to migrate data from file store to SQLite.

* Remove 'downCmd' from migration commands

* Refactoring

* Add sqlite cleanup

* Remove comment
2024-06-13 13:39:19 +03:00
f68d5e965f Optimize JWT Group Sync (#2108)
* Optimize JWT group sync to avoid unnecessary account sync

* Ignore adding matching API and JWT groups during Sync

* add tests

* refactor
2024-06-13 09:55:09 +03:00
4da29451d0 Add missing openid scope when requesting JWT token (#2089)
According to the Zitadel documentation, `openid` scope is required
when requesting JWT tokens.

Apparently Zitadel was accepting requests without it until very
recently. Now lack thereof causes 400 Bad Requests which makes it
impossible to authenticate to the Netbird dashboard.

https://zitadel.com/docs/guides/integrate/service-users/client-credentials#2-authenticating-a-service-user-and-request-a-token
2024-06-04 10:46:24 +02:00
456629811b Prevent using expired ctx when sending metrics (#2088) 2024-06-03 12:41:15 +02:00
521f7dd39f Improve login performance (#2061) 2024-05-31 16:41:12 +02:00
012235ff12 Add FindExistingPostureCheck (#2075) 2024-05-30 15:22:42 +02:00
f176807ebe Add extra logs for account not found, peer login and getAccount (#2053) 2024-05-27 12:29:28 +02:00
d4c47eaf8a Don't allow delete group from peer groups (#2055) 2024-05-27 11:06:43 +02:00