Commit Graph

655 Commits

Author SHA1 Message Date
Maycon Santos
cbf9f2058e
Use accountID retrieved from the sync call to acquire read lock sooner (#2369)
Use accountID retrieved from the sync call to acquire read lock sooner and avoiding extra DB calls.
- Use the account ID across sync calls
- Moved account read lock
- Renamed CancelPeerRoutines to OnPeerDisconnected
- Added race tests
2024-08-01 16:21:43 +02:00
Maycon Santos
5ee9c77e90
Move write peer lock (#2364)
Moved the write peer lock to avoid latency caused by disk access

Updated the method CancelPeerRoutines to use the peer public key
2024-07-31 21:51:45 +02:00
Bethuel Mmbaga
c832cef44c
Update SaveUsers and SaveGroups to SaveAccount (#2362)
Changed SaveUsers and SaveGroups method calls to SaveAccount for consistency in data persistence operations.
2024-07-31 19:48:12 +03:00
Maycon Santos
165988429c
Add write lock for peer when saving its connection status (#2359) 2024-07-31 14:53:32 +02:00
Maycon Santos
da39c8bbca
Refactor login with store.SavePeer (#2334)
This pull request refactors the login functionality by integrating store.SavePeer. The changes aim to improve the handling of peer login processes, particularly focusing on synchronization and error handling.

Changes:
- Refactored login logic to use store.SavePeer.
- Added checks for login without lock for login necessary checks from the client and utilized write lock for full login flow.
- Updated error handling with status.NewPeerLoginExpiredError().
- Moved geoIP check logic to a more appropriate place.
- Removed redundant calls and improved documentation.
- Moved the code to smaller methods to improve readability.
2024-07-29 13:30:27 +02:00
Bethuel Mmbaga
7321046cd6
Remove redundant check for empty JWT groups (#2323)
* Remove redundant check for empty group names in SetJWTGroups

* add test
2024-07-26 16:33:54 +02:00
Maycon Santos
1f48fdf6ca
Add SavePeer method to prevent a possible account inconsistency (#2296)
SyncPeer was storing the account with a simple read lock

This change introduces the SavePeer method to the store to be used in these cases
2024-07-26 07:49:05 +02:00
Maycon Santos
45fd1e9c21
add save peer status test for connected peers (#2321) 2024-07-25 16:22:04 +02:00
Maycon Santos
788f130941
Retry management connection only on context canceled (#2301) 2024-07-22 15:49:25 +02:00
pascal-fischer
95d725f2c1
Wait on daemon down (#2279) 2024-07-17 16:26:06 +02:00
ctrl-zzz
a711e116a3
fix: save peer status correctly in sqlstore (#2262)
* fix: save peer status correctly in sqlstore

https://github.com/netbirdio/netbird/issues/2110#issuecomment-2162768273

* feat: update test function

* refactor: simplify status update
2024-07-16 18:38:12 +03:00
Bethuel Mmbaga
1537b0f5e7
Add batch save/update for groups and users (#2245)
* Add functionality to update multiple users

* Remove SaveUsers from DefaultAccountManager

* Add SaveGroups method to AccountManager interface

* Refactoring

* Add SaveUsers and SaveGroups methods to store interface

* Refactor method SaveAccount to SaveUsers and SaveGroups

The method SaveAccount in user.go and group.go files was split into two separate methods. Now, user-specific data is handled by SaveUsers and group-specific data is handled by SaveGroups method. This provides a cleaner and more efficient way to save user and group data.

* Add account ID to user and group in SqlStore

* Refactor SaveUsers and SaveGroups in store

* Remove unnecessary ID assignment in SaveUsers and SaveGroups
2024-07-15 17:04:06 +03:00
Maycon Santos
58fbc1249c
Fix parameter limit issue for Postgres store (#2261)
Added CreateBatchSize for both SQL stores and updated tests to test large accounts with Postgres, too. Increased the account peer size to 6K.
2024-07-12 09:28:53 +02:00
pascal-fischer
e8c2fafccd
Avoid empty domain overwrite (#2252) 2024-07-10 14:08:35 +02:00
Maycon Santos
85e991ff78
Fix issue with canceled context before pushing metrics and decreasing pushing interval (#2235)
Fix a bug where the post context was canceled before sending metrics to the server.

The interval time was decreased, and an optional environment variable NETBIRD_METRICS_INTERVAL_IN_SECONDS was added to control the interval time.

* update doc URL
2024-07-04 19:15:59 +02:00
pascal-fischer
765aba2c1c
Add context to throughout the project and update logging (#2209)
propagate context from all the API calls and log request ID, account ID and peer ID

---------

Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
2024-07-03 11:33:02 +02:00
Zoltan Papp
7cb81f1d70
Fix nil pointer exception in case of error (#2230) 2024-07-02 18:18:14 +02:00
Maycon Santos
eab6183a8e
Add stack trace when saving empty domains (#2228)
added temporary domain check for existing accounts to trace where the issue originated

Refactor save account due to complexity score
2024-07-02 12:40:26 +02:00
Bethuel Mmbaga
eaa31c2dc6
Optimize process checks database read (#2182)
* Add posture checks to peer management

This commit includes posture checks to the peer management logic. The AddPeer, SyncPeer and LoginPeer functions now return a list of posture checks along with the peer and network map.

* Update peer methods to return posture checks

* Refactor

* return early if there is no posture checks

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-06-22 17:41:16 +03:00
Bethuel Mmbaga
fc15ee6351
auto migrate older management to sqlite (#2170) 2024-06-20 19:45:57 +02:00
Viktor Liu
f9462eea27
Fix dns route retrieval condition (#2165)
* Fix route retrieval condition

* Make error messages take domains into account
2024-06-20 13:52:32 +02:00
Maycon Santos
381447b8d6
Fix store migration on empty string (#2149)
* Fix store migration on empty string

when fetching empty values from the database to check for migration our parser failed to handle null strings preventing the service from start

this uses sql.NullString to handle that and check for empty string resulted from null data

---------

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
2024-06-18 15:39:54 +02:00
Maycon Santos
4fec709bb1
Release 0.28.0 (#2092)
* compile client under freebsd (#1620)

Compile netbird client under freebsd and now support netstack and userspace modes.
Refactoring linux specific code to share same code with FreeBSD, move to *_unix.go files.

Not implemented yet:

Kernel mode not supported
DNS probably does not work yet
Routing also probably does not work yet
SSH support did not tested yet
Lack of test environment for freebsd (dedicated VM for github runners under FreeBSD required)
Lack of tests for freebsd specific code
info reporting need to review and also implement, for example OS reported as GENERIC instead of FreeBSD (lack of FreeBSD icon in management interface)
Lack of proper client setup under FreeBSD
Lack of FreeBSD port/package

* Add DNS routes (#1943)

Given domains are resolved periodically and resolved IPs are replaced with the new ones. Unless the flag keep_route is set to true, then only new ones are added.
This option is helpful if there are long-running connections that might still point to old IP addresses from changed DNS records.

* Add process posture check (#1693)

Introduces a process posture check to validate the existence and active status of specific binaries on peer systems. The check ensures that files are present at specified paths, and that corresponding processes are running. This check supports Linux, Windows, and macOS systems.


Co-authored-by: Evgenii <mail@skillcoder.com>
Co-authored-by: Pascal Fischer <pascal@netbird.io>
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
Co-authored-by: Bethuel Mmbaga <bethuelmbaga12@gmail.com>
2024-06-13 13:24:24 +02:00
Bethuel Mmbaga
95299be52d
Deprecate FileStore engine support (#2119)
* Removejsonfile' from test matrix in workflows

* Remove sqlite to json migration command

* Refactor store engine implementation to remove JSON file store support

The codebase has been refactored to remove support for JSON file store storage engine, with SQLite serving as the default store engine. New functions have been added to handle unsupported store engines and to migrate data from file store to SQLite.

* Remove 'downCmd' from migration commands

* Refactoring

* Add sqlite cleanup

* Remove comment
2024-06-13 13:39:19 +03:00
Bethuel Mmbaga
f68d5e965f
Optimize JWT Group Sync (#2108)
* Optimize JWT group sync to avoid unnecessary account sync

* Ignore adding matching API and JWT groups during Sync

* add tests

* refactor
2024-06-13 09:55:09 +03:00
Gabriel Górski
4da29451d0
Add missing openid scope when requesting JWT token (#2089)
According to the Zitadel documentation, `openid` scope is required
when requesting JWT tokens.

Apparently Zitadel was accepting requests without it until very
recently. Now lack thereof causes 400 Bad Requests which makes it
impossible to authenticate to the Netbird dashboard.

https://zitadel.com/docs/guides/integrate/service-users/client-credentials#2-authenticating-a-service-user-and-request-a-token
2024-06-04 10:46:24 +02:00
Maycon Santos
456629811b
Prevent using expired ctx when sending metrics (#2088) 2024-06-03 12:41:15 +02:00
pascal-fischer
521f7dd39f
Improve login performance (#2061) 2024-05-31 16:41:12 +02:00
pascal-fischer
012235ff12
Add FindExistingPostureCheck (#2075) 2024-05-30 15:22:42 +02:00
Maycon Santos
f176807ebe
Add extra logs for account not found, peer login and getAccount (#2053) 2024-05-27 12:29:28 +02:00
Maycon Santos
d4c47eaf8a
Don't allow delete group from peer groups (#2055) 2024-05-27 11:06:43 +02:00
Bethuel Mmbaga
d35a79d3b5
Upgrade gRPC and OpenTelemetry packages for compatibility (#2003)
Upgrades `go.opentelemetry.io/otel` from version` v1.11.1` to `v1.26.0`. The upgrade addresses compatibility issues caused by the removal of several sub-packages in the latest OpenTelemetry release, which were causing broken dependencies.

**Key Changes:**
- Upgraded `go.opentelemetry.io/otel` from `v1.11.1` to `v1.26.0`.

- Fixed broken dependencies by replacing the deprecated sub-packages:
  - `go.opentelemetry.io/otel/metric/instrument`
  - `go.opentelemetry.io/otel/metric/instrument/asyncint64`
  - `go.opentelemetry.io/otel/metric/instrument/syncint64`
  
- Upgraded `google.golang.org/grpc` from `v1.56.3`  to `v1.64.0` which deprecate `Dial` and `DialContext` to `NewClient`.
2024-05-27 08:39:18 +02:00
Maycon Santos
7a1c96ebf4
Remove extra error mapping (#2050) 2024-05-24 14:46:11 +02:00
Zoltan Papp
41fe9f84ec
Extend integrated validator with error handling (#2044) 2024-05-24 13:29:25 +02:00
Maycon Santos
29a2d93873
Log global lock acquisition per user (#2039) 2024-05-23 17:09:58 +02:00
Maycon Santos
9d3db68805
Return the proper error when a peer is deleted (#2035)
this fixes an issue causing peers to keep retrying the connection after a peer is removed from the management system
2024-05-23 14:59:09 +02:00
Maycon Santos
91fa2e20a0
Store location information in peer event meta (#1994) 2024-05-22 12:31:16 +02:00
Bethuel Mmbaga
a5811a2d7d
Implement experimental PostgreSQL store (#1939)
* migrate sqlite store to
 generic sql store

* fix conflicts

* init postgres store

* Add postgres store tests

* Refactor postgres store engine name

* fix tests

* Run postgres store tests on linux only

* fix tests

* Refactor

* cascade policy rules on policy deletion

* fix tests

* run postgres cases in new db

* close store connection after tests

* refactor

* using testcontainers

* sync go sum

* remove postgres service

* remove store cleanup

* go mod tidy

* remove env

* use postgres as engine and initialize test store with testcontainer

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-05-16 19:28:37 +03:00
pascal-fischer
0a75da6fb7
Remove GetNetworkMap stacktrace(#1941) 2024-05-07 19:19:30 +02:00
Viktor Liu
920877964f
Monitor network changes and restart engine on detection (#1904) 2024-05-07 18:50:34 +02:00
pascal-fischer
2e0047daea
Improve Sync performance (#1901) 2024-05-07 14:30:03 +02:00
Bethuel Mmbaga
ce0718fcb5
Migrate blob net ip fields to json serializer (#1906)
* serialize net.IP as json

* migrate net ip field from blob to json

* run net ip migration

* remove duplicate index

* Refactor

* Add tests

* fix tests

* migrate null blob values
2024-05-07 14:01:45 +03:00
Maycon Santos
7357a9954c
Fix a panic when management is behind an invalid proxy (#1930)
- Add a new error on gRPC client that doesn't pass the incorrect status from the gRPC client
- Try login only if we have a server public key
2024-05-06 18:04:32 +02:00
Misha Bragin
df4ca01848
Return system serial on a peer HTTP API call (#1929) 2024-05-06 14:49:03 +02:00
Viktor Liu
4e7c17756c
Refactor Route IDs (#1891) 2024-05-06 14:47:49 +02:00
Maycon Santos
3598418206
Update the check interval for new geo db and change log level (#1908)
Update log level to trace and update the check db interval from 60s to 300s
2024-04-30 17:54:29 +02:00
Maycon Santos
fd26e989e3
Check if channel exist before sending network map (#1894)
Check for connection channel before calculating and sending the network map
2024-04-29 18:31:52 +02:00
Bethuel Mmbaga
71c6437bab
add content type before writing header (#1887) 2024-04-25 21:20:24 +02:00
pascal-fischer
8f3a0f2c38
Add retry to IdP cache lookup (#1882) 2024-04-23 19:23:43 +02:00
pascal-fischer
1e6addaa65
Add account locks to getAccountWithAuthorizationClaims method (#1847) 2024-04-23 19:09:58 +02:00
Maycon Santos
a80c8b0176
Redeem invite only when incoming user was invited (#1861)
checks for users with pending invite status in the cache that already logged in and refresh the cache
2024-04-22 11:10:27 +02:00
Misha Bragin
b74078fd95
Use a better way to insert data in batches (#1874) 2024-04-20 22:04:20 +02:00
Viktor Liu
77488ad11a
Migrate serializer:gob fields to serializer:json (#1855) 2024-04-18 18:14:21 +02:00
Viktor Liu
e0de86d6c9
Use fixed activity codes (#1846)
* Add duplicate constants check
2024-04-15 14:15:46 +02:00
Zoltan Papp
5204d07811
Pass integrated validator for API (#1814)
Pass integrated validator for API handler
2024-04-15 12:08:38 +02:00
Zoltan Papp
9505805313
Rename variable (#1829) 2024-04-11 14:08:03 +02:00
Maycon Santos
704c67dec8
Allow owners that did not create the account to delete it (#1825)
Sometimes the Owner role will be passed to new users, and they need to be able to delete the account
2024-04-11 10:02:51 +02:00
verytrap
c28657710a
Fix function names in comments (#1816)
Signed-off-by: verytrap <wangqiuyue@outlook.com>
2024-04-09 13:18:38 +02:00
Maycon Santos
3875c29f6b
Revert "Rollback new routing functionality (#1805)" (#1813)
This reverts commit 9f32ccd453.
2024-04-08 18:56:52 +02:00
Viktor Liu
9f32ccd453
Rollback new routing functionality (#1805) 2024-04-05 20:38:49 +02:00
rqi14
9af532fe71
Get scope from endpoint url instead of hardcoding (#1770) 2024-04-02 13:43:57 +02:00
Misha Bragin
22beac1b1b
Fix invalid token due to the cache race (#1763) 2024-03-28 12:33:56 +01:00
Zoltan Papp
2d76b058fc
Feature/peer validator (#1553)
Follow up management-integrations changes

move groups to separated packages to avoid circle dependencies
save location information in Login action
2024-03-27 18:48:48 +01:00
pascal-fischer
ea2d060f93
Add limited dashboard view (#1738) 2024-03-27 16:11:45 +01:00
pascal-fischer
af50eb350f
Change log level for JWT override message of single account mode (#1747) 2024-03-25 14:25:26 +01:00
Viktor Liu
2475473227
Support client default routes for Linux (#1667)
All routes are now installed in a custom netbird routing table.
Management and wireguard traffic is now marked with a custom fwmark.
When the mark is present the traffic is routed via the main routing table, bypassing the VPN.
When the mark is absent the traffic is routed via the netbird routing table, if:
- there's no match in the main routing table
- it would match the default route in the routing table

IPv6 traffic is blocked when a default route IPv4 route is configured to avoid leakage.
2024-03-21 16:49:28 +01:00
Maycon Santos
f0672b87bc
Add missing dns domain to tests to avoid verbose test logs (#1724) 2024-03-18 12:25:01 +01:00
Maycon Santos
9b0fe2c8e5
Validate authentik issuer url (#1723)
* Validate authentik issuer url

* test issuer

* adjust test times on windows
2024-03-18 10:12:46 +01:00
Misha Bragin
abd57d1191
Avoid creating duplicate groups with the same name (#1579)
Avoid creating groups with the same name via API calls. 

JWT and integrations still allowed to register groups with duplicated names
2024-03-17 11:13:39 +01:00
Maycon Santos
fc7c1e397f
Disable force jsonfile variable (#1611)
This enables windows management tests

Added another DNS server to the dns server tests
2024-03-15 10:50:02 +01:00
Aaron Turner
52a3ac6b06
Add support for inviting/deleting users via Zitadel (#1572)
This fixes the "Invite User" button in Dashboard v2.0.0
and enables the usage of the --user-delete-from-idp flag for Zitadel.

Unlike the NetBird SaaS solution, we rely on Zitadel to send
the emails on our behalf.
2024-03-15 10:32:51 +01:00
Yury Gargay
0b3b50c705
Remove deprecated Rules API endpoints (#1523) 2024-03-14 21:31:21 +01:00
Maycon Santos
042141db06
Update account attributes only when there is a domain (#1701)
add log for when a domain is not present
2024-03-14 14:17:22 +01:00
Maycon Santos
5dde044fa5
Check for record not found when searching the store (#1686)
This change returns status.NotFound only on gorm.ErrRecordNotFound and status.Internal on every other DB error
2024-03-10 19:09:45 +01:00
Maycon Santos
0aeb87742a
Return 1s when next expiration is too low (#1672)
using the login expired issue could cause problems with ticker used in the scheduler

This change makes 1s the minimum number returned when rescheduling the peer expiration task
2024-03-06 15:18:53 +01:00
Viktor Liu
199bf73103
Remove usage stats (#1665) 2024-03-05 09:45:32 +01:00
Maycon Santos
17f5abc653
Handle canceling schedule and avoid recursive call (#1636)
Using time.Ticker allows us to avoid recursive calls that may end up in schedule running and possible deadlock if no routine is listening for cancel calls
2024-03-03 10:35:01 +01:00
Maycon Santos
aa935bdae3
Register creation time for peer, user and account (#1654)
This change register creation time for new peers, users and accounts
2024-03-02 13:49:40 +01:00
Viktor Liu
17b1099032
Enhance DNS failover reliability (#1637)
* Fix using wrong array index in log to avoid potential panic

* Increase gRPC connection timeout and add the timeout resolv.conf option

This makes sure the dns client is able to failover to a second
configured nameserver, if present. That is the case then when using the
dns `file` manager and a resolv.conf file generated for netbird.

* On file backup restore, remove the first NS if it's the netbird NS

* Bump dns mangager discovery message from debug to info to ease debugging
2024-03-01 15:17:35 +01:00
pascal-fischer
a4b9e93217
add environment meta from grpc to store (#1651) 2024-03-01 15:15:56 +01:00
Maycon Santos
63d7957140
Log version when starting management and signal (#1649)
Output log version at service startup
2024-02-29 16:05:47 +01:00
Maycon Santos
9a6814deff
Enable review linter with exported docs check (#1647)
Exported function and methods should have comments. This warns on undocumented exported functions and methods.
2024-02-29 16:05:31 +01:00
pascal-fischer
468fa2940b
add quotes to all timestamps in openapi spec (#1642) 2024-02-29 13:09:43 +01:00
Yury Gargay
5a8f1763a6
Implement Auth0 IdP get all connections method (#1613) 2024-02-28 16:57:35 +01:00
Bethuel Mmbaga
f64e73ca70
Fix invalid cross-device link when moving geolocation databases (#1638)
* Fix invalid cross-device link when move geonames db

* Add test for geolocation databases in workflow

This step checks the existence and proper functioning of geolocation databases, including GeoLite2-City.mmdb and Geonames.db. It will help us ensure that geolocation databases are loaded correctly in the management.

* Enable debug mode

* Increase sleep duration in geolocation tests
2024-02-28 16:42:33 +03:00
pascal-fischer
b085419ab8
FIx order when validating account settings (#1632)
* moved extraSettings validation to the end

* moved extraSettings validation directly after permission check
2024-02-27 14:17:22 +01:00
Bethuel Mmbaga
d78b652ff7
Rename PrivateNetworkCheck to PeerNetworkRangeCheck (#1629)
* Rename PrivateNetworkCheck to PeerNetworkRangeCheck

* update description and example

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-02-27 11:59:48 +01:00
Bethuel Mmbaga
b65c2f69b0
Add support for downloading Geo databases to the management service (#1626)
Adds support for downloading Geo databases to the management service. If the Geo databases are not found, the service will automatically attempt to download them during startup.
2024-02-26 22:49:28 +01:00
Yury Gargay
d8ce08d898
Extend bypass middleware with support of wildcard paths (#1628)
---------

Co-authored-by: Viktor Liu <viktor@netbird.io>
2024-02-26 17:54:58 +01:00
Bethuel Mmbaga
a47c69c472
Add private network posture check (#1606)
* wip: Add PrivateNetworkCheck checks interface implementation

* use generic CheckAction constant

* Add private network check to posture checks

* Fix copy function target in posture checks

* Add network check functionality to posture package

* regenerate the openapi specs

* Update Posture Check actions in test file

* Remove unused function

* Refactor network address handling in PrivateNetworkCheck

* Refactor Prefixes to Ranges in private network checks

* Implement private network checks in posture checks handler tests

* Add test for check copy

* Add gorm serializer for network range
2024-02-22 19:22:43 +03:00
Yury Gargay
bbea4c3cc3
Use SQLite store as default when running tests when env is not set (#1612) 2024-02-22 16:51:56 +01:00
Viktor Liu
b7a6cbfaa5
Add account usage logic (#1567)
---------

Co-authored-by: Yury Gargay <yury.gargay@gmail.com>
2024-02-22 12:27:08 +01:00
Bethuel Mmbaga
51fa3c92c5
Fix copying function in posture checks process (#1605) 2024-02-21 19:19:13 +03:00
Maycon Santos
d65602f904
Add posture checks metrics report (#1603) 2024-02-21 15:16:43 +01:00
Yury Gargay
8d9e1fed5f
Mark new peer meta fields required in OpenAPI spec (#1604) 2024-02-21 15:06:42 +01:00
Bethuel Mmbaga
e1eddd1cab
Fix incorrect assignment of SystemSerialNumber and SystemManufacturer (#1600) 2024-02-20 22:50:14 +03:00
Yury Gargay
0fbf72434e
Make SQLite default for new installations (#1529)
* Make SQLite default for new installations

* if var is not set, return empty string

this allows getStoreEngineFromDatadir to detect json store files

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-02-20 15:06:32 +01:00
pascal-fischer
51f133fdc6
Extend system meta (#1598)
* wip: add posture checks structs

* add netbird version check

* Refactor posture checks and add version checks

* Add posture check activities (#1445)

* Integrate Endpoints for Posture Checks (#1432)

* wip: add posture checks structs

* add netbird version check

* Refactor posture checks and add version checks

* Implement posture and version checks in API models

* Refactor API models and enhance posture check functionality

* wip: add posture checks endpoints

* go mod tidy

* Reference the posture checks by id's in policy

* Add posture checks management to server

* Add posture checks management mocks

* implement posture checks handlers

* Add posture checks to account copy and fix tests

* Refactor posture checks validation

* wip: Add posture checks handler tests

* Add JSON encoding support to posture checks

* Encode posture checks to correct api response object

* Refactored posture checks implementation to align with the new API schema

* Refactor structure of `Checks` from slice to map

* Cleanup

* Add posture check activities (#1445)

* Revert map to use list of checks

* Add posture check activity events

* Refactor posture check initialization in account test

* Improve the handling of version range in posture check

* Fix tests and linter

* Remove max_version from NBVersionCheck

* Added unit tests for NBVersionCheck

* go mod tidy

* Extend policy endpoint with posture checks (#1450)

* Implement posture and version checks in API models

* go mod tidy

* Allow attaching posture checks to policy

* Update error message for linked posture check on deleting

* Refactor PostureCheck and Checks structures

* go mod tidy

* Add validation for non-existing posture checks

* fix unit tests

* use Wt version

* Remove the enabled field, as posture check will now automatically be activated by default when attaching to a policy

* wip: add posture checks structs

* add netbird version check

* Refactor posture checks and add version checks

* Add posture check activities (#1445)

* Integrate Endpoints for Posture Checks (#1432)

* wip: add posture checks structs

* add netbird version check

* Refactor posture checks and add version checks

* Implement posture and version checks in API models

* Refactor API models and enhance posture check functionality

* wip: add posture checks endpoints

* go mod tidy

* Reference the posture checks by id's in policy

* Add posture checks management to server

* Add posture checks management mocks

* implement posture checks handlers

* Add posture checks to account copy and fix tests

* Refactor posture checks validation

* wip: Add posture checks handler tests

* Add JSON encoding support to posture checks

* Encode posture checks to correct api response object

* Refactored posture checks implementation to align with the new API schema

* Refactor structure of `Checks` from slice to map

* Cleanup

* Add posture check activities (#1445)

* Revert map to use list of checks

* Add posture check activity events

* Refactor posture check initialization in account test

* Improve the handling of version range in posture check

* Fix tests and linter

* Remove max_version from NBVersionCheck

* Added unit tests for NBVersionCheck

* go mod tidy

* Extend policy endpoint with posture checks (#1450)

* Implement posture and version checks in API models

* go mod tidy

* Allow attaching posture checks to policy

* Update error message for linked posture check on deleting

* Refactor PostureCheck and Checks structures

* go mod tidy

* Add validation for non-existing posture checks

* fix unit tests

* use Wt version

* Remove the enabled field, as posture check will now automatically be activated by default when attaching to a policy

* Extend network map generation with posture checks (#1466)

* Apply posture checks to network map generation

* run policy posture checks on peers to connect

* Refactor and streamline policy posture check process for peers to connect.

* Add posture checks testing in a network map

* Remove redundant nil check in policy.go

* Refactor peer validation check in policy.go

* Update 'Check' function signature and use logger for version check

* Refactor posture checks run on sources and updated the validation func

* Update peer validation

* fix tests

* improved test coverage for policy posture check

* Refactoring

* Extend NetBird agent to collect kernel version (#1495)

* Add KernelVersion field to LoginRequest

* Add KernelVersion to system info retrieval

* Fix tests

* Remove Core field from system info

* Replace Core field with new OSVersion field in system info

* Added WMI dependency to info_windows.go

* Add OS Version posture checks  (#1479)

* Initial support of Geolocation service (#1491)

* Add Geo Location posture check (#1500)

* wip: implement geolocation check

* add geo location posture checks to posture api

* Merge branch 'feature/posture-checks' into geo-posture-check

* Remove CityGeoNameID and update required fields in API

* Add geoLocation checks to posture checks handler tests

* Implement geo location-based checks for peers

* Update test values and embed location struct in peer system

* add support for country wide checks

* initialize country code regex once

* Fix peer meta core compability with older clients (#1515)

* Refactor extraction of OSVersion in grpcserver

* Ignore lint check

* Fix peer meta core compability with older management (#1532)

* Revert core field deprecation

* fix tests

* Extend peer meta with location information (#1517)

This PR uses the geolocation service to resolve IP to location. 
The lookup happens once on the first connection - when a client calls the Sync func.
The location is stored as part of the peer:

* Add Locations endpoints (#1516)

* add locations endpoints

* Add sqlite3 check and database generation in geolite script

* Add SQLite storage for geolocation data

* Refactor file existence check into a separate function

* Integrate geolocation services into management application

* Refactoring

* Refactor city retrieval to include Geonames ID

* Add signature verification for GeoLite2 database download

* Change to in-memory database for geolocation store

* Merge manager to geolocation

* Update GetAllCountries to return Country name and iso code

* fix tests

* Add reload to SqliteStore

* Add geoname indexes

* move db file check to connectDB

* Add concurrency safety to SQL queries and database reloading

The commit adds mutex locks to the GetAllCountries and GetCitiesByCountry functions to ensure thread-safety during database queries. Additionally, it introduces a mechanism to safely close the old database connection before a new connection is established upon reloading, which improves the reliability of database operations. Lastly, it moves the checking of database file existence to the connectDB function.

* Add sha256 sum check to geolocation store before reload

* Use read lock

* Check SHA256 twice when reload geonames db

---------

Co-authored-by: Yury Gargay <yury.gargay@gmail.com>

* Add tests and validation for empty peer location in GeoLocationCheck (#1546)

* Disallow Geo check creation/update without configured Geo DB (#1548)

* Fix shared access to in memory copy of geonames.db (#1550)

* Trim suffix in when evaluate Min Kernel Version in OS check

* Add Valid Peer Windows Kernel version test

* Add Geolocation handler tests (#1556)

* Implement user admin checks in posture checks

* Add geolocation handler tests

* Mark initGeolocationTestData as helper func

* Add error handling to geolocation database closure

* Add cleanup function to close geolocation resources

* Simplify checks definition serialisation (#1555)

* Regenerate network map on posture check update (#1563)

* change network state and generate map on posture check update

* Refactoring

* Make city name optional (#1575)

* Do not return empty city name

* Validate action param of geo location checks (#1577)

We only support allow and deny

* Switch realip middleware to upstream (#1578)

* Be more silent in download-geolite2.sh script

* Fix geonames db reload (#1580)

* Ensure posture check name uniqueness when create (#1594)

* Enhance the management of posture checks (#1595)

* add a correct min version and kernel for os posture check example

* handle error when geo or location db is nil

* expose all peer location details in api response

* Check for nil geolocation manager only

* Validate posture check before save

* bump open api version

* add peer location fields to toPeerListItemResponse

* Feautre/extend sys meta (#1536)

* Collect network addresses

* Add Linux sys product info

* Fix peer meta comparison

* Collect sys info on mac

* Add windows sys info

* Fix test

* Fix test

* Fix grpc client

* Ignore test

* Fix test

* Collect IPv6 addresses

* Change the IP to IP + net

* fix tests

* Use netip on server side

* Serialize netip to json

* Extend Peer metadata with cloud detection (#1552)

* add cloud detection + test binary

* test windows exe

* Collect IPv6 addresses

* Change the IP to IP + net

* switch to forked cloud detect lib

* new test builds

* new GCE build

* discontinue using library but local copy instead

* fix imports

* remove openstack check

* add hierarchy to cloud check

* merge IBM and SoftLayer

* close resp bodies and use os lib for file reading

* close more resp bodies

* fix error check logic

* parallelize IBM checks

* fix response value

* go mod tidy

* include context + change kubernetes detection

* add context in info functions

* extract platform into separate field

* fix imports

* add missing wmi import

---------

Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>

---------

Co-authored-by: pascal-fischer <32096965+pascal-fischer@users.noreply.github.com>

* generate proto

* remove test binaries

---------

Co-authored-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Yury Gargay <yury.gargay@gmail.com>
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
2024-02-20 11:53:11 +01:00
Yury Gargay
9bc7b9e897
Add initial support of device posture checks (#1540)
This PR implements the following posture checks:

* Agent minimum version allowed
* OS minimum version allowed
* Geo-location based on connection IP

For the geo-based location, we rely on GeoLite2 databases which are free IP geolocation databases. MaxMind was tested and we provide a script that easily allows to download of all necessary files, see infrastructure_files/download-geolite2.sh.

The OpenAPI spec should extensively cover the life cycle of current version posture checks.
2024-02-20 09:59:56 +01:00
Yury Gargay
db3cba5e0f
Remove Account.Rules from Store engines (#1528) 2024-02-19 17:17:36 +01:00