Commit Graph

41 Commits

Author SHA1 Message Date
Maycon Santos
5b46cc8e9c
Avoid failing all other matrix tests if one fails (#2839) 2024-11-05 13:28:42 +01:00
Zoltan Papp
4e918e55ba
[client] Fix controller re-connection (#2758)
Rethink the peer reconnection implementation
2024-10-24 11:43:14 +02:00
Bethuel Mmbaga
7bda385e1b
[management] Optimize network map updates (#2718)
* Skip peer update on unchanged network map (#2236)

* Enhance network updates by skipping unchanged messages

Optimizes the network update process
by skipping updates where no changes in the peer update message received.

* Add unit tests

* add locks

* Improve concurrency and update peer message handling

* Refactor account manager network update tests

* fix test

* Fix inverted network map update condition

* Add default group and policy to test data

* Run peer updates in a separate goroutine

* Refactor

* Refactor lock

* Fix peers update by including NetworkMap and posture Checks

* go mod tidy

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* [management] Skip account peers update if no changes affect peers (#2310)

* Remove incrementing network serial and updating peers after group deletion

* Update account peer if posture check is linked to policy

* Remove account peers update on saving setup key

* Refactor group link checking into re-usable functions

* Add HasPeers function to group

* Refactor group management

* Optimize group change effects on account peers

* Update account peers if ns group has peers

* Refactor group changes

* Optimize account peers update in DNS settings

* Optimize update of account peers on jwt groups sync

* Refactor peer account updates for efficiency

* Optimize peer update on user deletion and changes

* Remove condition check for network serial update

* Optimize account peers updates on route changes

* Remove UpdatePeerSSHKey method

* Remove unused isPolicyRuleGroupsEmpty

* Add tests for peer update behavior on posture check changes

* Add tests for peer update behavior on policy changes

* Add tests for peer update behavior on group changes

* Add tests for peer update behavior on dns settings changes

* Refactor

* Add tests for peer update behavior on name server changes

* Add tests for peer update behavior on user changes

* Add tests for peer update behavior on route changes

* fix tests

* Add tests for peer update behavior on setup key changes

* Add tests for peer update behavior on peers changes

* fix merge

* Fix tests

* go mod tidy

* Add NameServer and Route comparators

* Update network map diff logic with custom comparators

* Add tests

* Refactor duplicate diff handling logic

* fix linter

* fix tests

* Refactor policy group handling and update logic.

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update route check by checking if group has peers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor posture check policy linking logic

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Simplify peer update condition in DNS management

Refactor the condition for updating account peers to remove redundant checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add posture checks tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix user and setup key tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix account and route tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix typo

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix routes tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* upgrade diff package

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* use generic differ for netip.Addr and netip.Prefix

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* go mod tidy

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add peer tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix management suite tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix postgres tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* enable diff nil structs comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip the update only last sent the serial is larger

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor peer and user

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for groupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor group, ns group, policy and posture checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for GroupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update account policy check before verifying policy status

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* add tests missing tests for dns setting groups

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add tests for posture checks changes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add ns group and policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add route and group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* increase Linux test timeout to 10 minutes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Run diff for client posture checks only

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add panic recovery and detailed logging in peer update comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-10-23 13:05:02 +03:00
pascal-fischer
8284ae959c
[management] Move testdata to sql files (#2693) 2024-10-10 12:35:03 +02:00
pascal-fischer
b79c1d64cc
[management] Make max open db conns configurable (#2713) 2024-10-09 20:17:25 +02:00
Zoltan Papp
fd67892cb4
[client] Refactor/iface pkg (#2646)
Refactor the flat code structure
2024-10-02 18:24:22 +02:00
pascal-fischer
6c50b0c84b
[management] Add transaction to addPeer (#2469)
This PR removes the GetAccount and SaveAccount operations from the AddPeer and instead makes use of gorm.Transaction to add the new peer.
2024-09-16 15:47:03 +02:00
Maycon Santos
4c130a0291
Update Go version to 1.23 (#2588) 2024-09-12 13:46:28 +02:00
Maycon Santos
afb9673bc4
[misc] Update core github actions (#2584) 2024-09-11 21:49:05 +02:00
Maycon Santos
4fec709bb1
Release 0.28.0 (#2092)
* compile client under freebsd (#1620)

Compile netbird client under freebsd and now support netstack and userspace modes.
Refactoring linux specific code to share same code with FreeBSD, move to *_unix.go files.

Not implemented yet:

Kernel mode not supported
DNS probably does not work yet
Routing also probably does not work yet
SSH support did not tested yet
Lack of test environment for freebsd (dedicated VM for github runners under FreeBSD required)
Lack of tests for freebsd specific code
info reporting need to review and also implement, for example OS reported as GENERIC instead of FreeBSD (lack of FreeBSD icon in management interface)
Lack of proper client setup under FreeBSD
Lack of FreeBSD port/package

* Add DNS routes (#1943)

Given domains are resolved periodically and resolved IPs are replaced with the new ones. Unless the flag keep_route is set to true, then only new ones are added.
This option is helpful if there are long-running connections that might still point to old IP addresses from changed DNS records.

* Add process posture check (#1693)

Introduces a process posture check to validate the existence and active status of specific binaries on peer systems. The check ensures that files are present at specified paths, and that corresponding processes are running. This check supports Linux, Windows, and macOS systems.


Co-authored-by: Evgenii <mail@skillcoder.com>
Co-authored-by: Pascal Fischer <pascal@netbird.io>
Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
Co-authored-by: Bethuel Mmbaga <bethuelmbaga12@gmail.com>
2024-06-13 13:24:24 +02:00
Bethuel Mmbaga
95299be52d
Deprecate FileStore engine support (#2119)
* Removejsonfile' from test matrix in workflows

* Remove sqlite to json migration command

* Refactor store engine implementation to remove JSON file store support

The codebase has been refactored to remove support for JSON file store storage engine, with SQLite serving as the default store engine. New functions have been added to handle unsupported store engines and to migrate data from file store to SQLite.

* Remove 'downCmd' from migration commands

* Refactoring

* Add sqlite cleanup

* Remove comment
2024-06-13 13:39:19 +03:00
Bethuel Mmbaga
a5811a2d7d
Implement experimental PostgreSQL store (#1939)
* migrate sqlite store to
 generic sql store

* fix conflicts

* init postgres store

* Add postgres store tests

* Refactor postgres store engine name

* fix tests

* Run postgres store tests on linux only

* fix tests

* Refactor

* cascade policy rules on policy deletion

* fix tests

* run postgres cases in new db

* close store connection after tests

* refactor

* using testcontainers

* sync go sum

* remove postgres service

* remove store cleanup

* go mod tidy

* remove env

* use postgres as engine and initialize test store with testcontainer

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-05-16 19:28:37 +03:00
Maycon Santos
3875c29f6b
Revert "Rollback new routing functionality (#1805)" (#1813)
This reverts commit 9f32ccd453.
2024-04-08 18:56:52 +02:00
Viktor Liu
9f32ccd453
Rollback new routing functionality (#1805) 2024-04-05 20:38:49 +02:00
Viktor Liu
2475473227
Support client default routes for Linux (#1667)
All routes are now installed in a custom netbird routing table.
Management and wireguard traffic is now marked with a custom fwmark.
When the mark is present the traffic is routed via the main routing table, bypassing the VPN.
When the mark is absent the traffic is routed via the netbird routing table, if:
- there's no match in the main routing table
- it would match the default route in the routing table

IPv6 traffic is blocked when a default route IPv4 route is configured to avoid leakage.
2024-03-21 16:49:28 +01:00
Yury Gargay
9028c3c1f7
Check git status after go mod tidy on CI (#1614) 2024-02-23 12:00:43 +01:00
Yury Gargay
0fbf72434e
Make SQLite default for new installations (#1529)
* Make SQLite default for new installations

* if var is not set, return empty string

this allows getStoreEngineFromDatadir to detect json store files

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-02-20 15:06:32 +01:00
pascal-fischer
5de4acf2fe
Integrate Rosenpass (#1153)
This PR aims to integrate Rosenpass with NetBird. It adds a manager for Rosenpass that starts a Rosenpass server and handles the managed peers. It uses the cunicu/go-rosenpass implementation. Rosenpass will then negotiate a pre-shared key every 2 minutes and apply it to the wireguard connection.

The Feature can be enabled by setting a flag during the netbird up --enable-rosenpass command.

If two peers are both support and have the Rosenpass feature enabled they will create a post-quantum secure connection. If one of the peers or both don't have this feature enabled or are running an older version that does not have this feature yet, the NetBird client will fall back to a plain Wireguard connection without pre-shared keys for those connections (keeping Rosenpass negotiation for the rest).

Additionally, this PR includes an update of all Github Actions workflows to use go version 1.21.0 as this is a requirement for the integration.

---------

Co-authored-by: braginini <bangvalo@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-01-08 12:25:35 +01:00
Yury Gargay
46f5f148da
Move StoreKind under own StoreConfig configuration and rename to Engine (#1219)
* Move StoreKind under own StoreConfig configuration parameter

* Rename StoreKind option to Engine

* Rename StoreKind internal methods and types to Engine

* Add template engine value test

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2023-10-16 11:19:39 +02:00
Yury Gargay
32880c56a4
Implement SQLite Store using gorm and relational approach (#1065)
Restructure data handling for improved performance and flexibility. 
Introduce 'G'-prefixed fields to represent Gorm relations, simplifying resource management. 
Eliminate complexity in lookup tables for enhanced query and write speed. 
Enable independent operations on data structures, requiring adjustments in the Store interface and Account Manager.
2023-10-12 15:42:36 +02:00
Yury Gargay
bb40325977
Update GitHub Actions and Enhance golangci-lint (#1075)
This PR showcases the implementation of additional linter rules. I've updated the golangci-lint GitHub Actions to the latest available version. This update makes sure that the tool works the same way locally - assuming being updated regularly - and with the GitHub Actions.

I've also taken care of keeping all the GitHub Actions up to date, which helps our code stay current. But there's one part, goreleaser that's a bit tricky to test on our computers. So, it's important to take a close look at that.

To make it easier to understand what I've done, I've made separate changes for each thing that the new linters found. This should help the people reviewing the changes see what's going on more clearly. Some of the changes might not be obvious at first glance.

Things to consider for the future
CI runs on Ubuntu so the static analysis only happens for Linux. Consider running it for the rest: Darwin, Windows
2023-09-04 17:03:44 +02:00
Givi Khojanashvili
ba7a39a4fc
Feat linux firewall support (#805)
Update the client's engine to apply firewall rules received from the manager (results of ACL policy).
2023-05-29 16:00:18 +02:00
Maycon Santos
ecac82a5ae
Share kernel Wireguard port with raw socket (#826)
This PR brings support of a shared port between stun (ICE agent) and
the kernel WireGuard

It implements a single port mode for execution with kernel WireGuard
interface using a raw socket listener.

BPF filters ensure that only STUN packets hit the NetBird userspace app

Removed a lot of the proxy logic and direct mode exchange.

Now we are doing an extra hole punch to the remote WireGuard 
port for best-effort cases and support to old client's direct mode.
2023-05-03 14:47:44 +02:00
Zoltan Papp
7f5e1c623e
Use forked Wireguard-go for custom bind (#823)
Update go version to 1.20
Use forked wireguard-go repo because of custom Bind implementation
2023-04-27 17:50:45 +02:00
Misha Bragin
2eeed55c18
Bind implementation (#779)
This PR adds supports for the WireGuard userspace implementation
using Bind interface from wireguard-go. 
The newly introduced ICEBind struct implements Bind with UDPMux-based
structs from pion/ice to handle hole punching using ICE.
The core implementation was taken from StdBind of wireguard-go.

The result is a single WireGuard port that is used for host and server reflexive candidates. 
Relay candidates are still handled separately and will be integrated in the following PRs.

ICEBind checks the incoming packets for being STUN or WireGuard ones
and routes them to UDPMux (to handle hole punching) or to WireGuard  respectively.
2023-04-13 17:00:01 +02:00
Zoltan Papp
d1703479ff
Add custom ice stdnet implementation (#754)
On Android, because of the hard SELinux policies can not list the
interfaces of the ICE package. Without it can not generate a host type
candidate. In this pull request, the list of interfaces comes via the Java
interface.
2023-03-24 08:40:39 +01:00
Misha Bragin
5c0b8a46f0
Add system activity tracking and event store (#636)
This PR adds system activity tracking. 
The management service records events like 
add/remove peer,  group, rule, route, etc.

The activity events are stored in the SQLite event store
and can be queried by the HTTP API.
2023-01-02 15:11:32 +01:00
Maycon Santos
a387e3cfc2
Add network routes distribution groups (#606)
Updated tests, API, and account manager methods

Sync routes to peers in the distribution groups

Added store upgrade by adding the All group to routes that don't have them
2022-12-06 10:11:57 +01:00
Maycon Santos
d2d5d4b4b9
Update go version (#603)
Removed ioctl code and remove exception from lint action
2022-12-04 13:22:21 +01:00
Maycon Santos
c88e6a7342
Run tests only on branch main and on pull requests (#492)
* Use reusable workflow and control push and pr test exec

* use format

* use path ref

* Run tests on push to main and on PRs
2022-10-03 00:17:16 +05:00
Maycon Santos
2cd9b11e7d
Add DNS nameserver support to management (#484)
Add DNS package and Nameserver group objects

Add CRUD operations for Nameserver Groups to account manager

Add Routes and Nameservers to Account Copy method

Run docker tests with timeout and serial flags
2022-09-30 16:47:11 +05:00
Maycon Santos
e4ad6174ca
Improve module load (#470)
* Add additional check for needed kernel modules

* Check if wireguard and tun modules are loaded

If modules are loaded return true, otherwise attempt to load them

* fix state check

* Add module function tests

* Add test execution in container

* run client package tests on docker

* add package comment to new file

* force entrypoint

* add --privileged flag

* clean only if tables where created

* run from within the directories
2022-09-15 01:26:11 +05:00
Maycon Santos
e95f0f7acb
Support 32 bit (#374)
Add build for 32 bits linux

improved windows test time
2022-07-01 10:42:38 +02:00
Maycon Santos
1aafc15607
Update self hosting scripts (#367)
split setup.env with example and base

add setup.env to .gitignore to avoid overwrite from new versions

Added test workflow for docker-compose 
and validated configure.sh generated variables
2022-06-24 14:50:14 +02:00
Givi Khojanashvili
951e011a9c
Add Settings window to Agent UI
Agent systray UI has been extended with
a setting window that allows configuring 
management URL, admin URL and 
supports pre-shared key.
While for the Netbird managed version 
the Settings are not necessary, it helps
to properly configure the self-hosted version.
2022-04-15 17:30:12 +02:00
Maycon Santos
97ab8f4c34
Updating Go 1.18 and Wireguard dependencies (#282) 2022-03-22 14:59:17 +01:00
Maycon Santos
a2fc4ec221
Rotate Access token with refresh token (#280)
Add method for rotating access token with refresh tokens
This will be useful for catching expired sessions and
offboarding users

Also added functions to handle secrets. They have to be revisited
as some tests didn't run on CI as they waited some user input, like password
2022-03-22 13:12:11 +01:00
Givi Khojanashvili
4ef3c7a637
Add basic desktop UI - systray
This PR adds a basic UI for desktop
applications that support Linux, Max
and Windows.
2022-03-20 17:36:35 +01:00
Maycon Santos
1cd1e84290
Run tests in serial and update multi-peer test (#269)
Updates test workflows with serial execution to avoid collision 
of ports and resource names.

Also, used -exec sudo flag for UNIX tests and removed not-needed
 limits configuration on Linux and added a 5 minutes timeout.

Updated the multi-peer tests in the client/internal/engine_test.go
 to provide proper validation when creating or starting 
a peer engine instance fails.

As some operations of the tests running on windows
 are slow, we will experiment with disabling the Defender before 
restoring cache and checkout a repository, then we reenable 
it to run the tests.

disabled extra logs for windows interface
2022-03-16 11:02:06 +01:00
Maycon Santos
0739038d51
Fix unstable parallel tests (#202)
* update interface tests and configuration messages

* little debug

* little debug on both errors

* print all devs

* list of devices

* debug func

* handle interface close

* debug socks

* debug socks

* if ports match

* use random assigned ports

* remove unused const

* close management client connection when stopping engine

* GracefulStop when management clients are closed

* enable workflows on PRs too

* remove iface_test debug code
2022-01-25 09:40:28 +01:00
Maycon Santos
64f2d295a8
Refactor Interface package and update windows driver (#192)
* script to generate syso files

* test wireguard-windows driver package

* set int log

* add windows test

* add windows test

* verbose bash

* use cd

* move checkout

* exit 0

* removed tty flag

* artifact path

* fix tags and add cache

* fix cache

* fix cache

* test dir

* restore artifacts in the root

* try dll file

* try dll file

* copy dll

* typo in copy dll

* compile test

* checkout first

* updated cicd

* fix add address issue and gen GUID

* psexec typo

* accept eula

* mod tidy before tests

* regular test exec and verbose test with psexec

* test all

* return WGInterface Interface

* use WgIfaceName and timeout after 30 seconds

* different ports and validate connect 2 peers

* Use time.After for timeout and close interface

* Use time.After for testing connect peers

* WG Interface struct

* Update engine and parse address

* refactor Linux create and assignAddress

* NewWGIface and configuration methods

* Update proxy with interface methods

* update up command test

* resolve lint warnings

* remove psexec test

* close copied files

* add goos before build

* run tests on mac,windows and linux

* cache by testing os

* run on push

* fix indentation

* adjust test timeouts

* remove parallel flag

* mod tidy before test

* ignore syso files

* removed functions and renamed vars

* different IPs for connect peers test

* Generate syso with DLL

* Single Close method

* use port from test constant

* test: remove wireguard interfaces after finishing engine test

* use load_wgnt_from_rsrc

Co-authored-by: braginini <bangvalo@gmail.com>
2022-01-17 14:01:58 +01:00