Commit Graph

627 Commits

Author SHA1 Message Date
Misha Bragin
a454a1aa28
Create account in once place (#358)
There are a few places where an account is created.
When we create a new account, there should be
some defaults set. E.g. created by and group ALL.
It makes sense to add it in one place to avoid inconsistencies.
2022-06-09 13:14:34 +02:00
braginini
c9b5328f19 Fix account ALL group creation 2022-06-08 00:30:19 +02:00
Misha Bragin
dab146ed87
Improve Management startup time (#355) 2022-06-06 13:45:59 +02:00
Misha Bragin
0cba0f81e0
Warmup IDP cache on Management start (#354) 2022-06-06 12:05:44 +02:00
Misha Bragin
ebd70a569c
Add caching when querying IDP Manager (#353) 2022-06-05 21:36:42 +02:00
Givi Khojanashvili
d005cd32b0
fix(acl): update each peer's network when rule,group or peer changed (#333)
* fix(acl): update each peer's network when rule,group or peer changed

* fix(ACL): update network test

* fix(acl): cleanup indexes before update them

* fix(acl): clean up rules indexes only for account
2022-06-04 22:02:22 +02:00
Misha Bragin
e6e9f0322f
Handle peer interface config change (#348)
Before this change, NetBird Agent wasn't handling
peer interface configuration changes dynamically.
Also, remote peer configuration changes have
not been applied (e.g. AllowedIPs changed).
Not a very common cause, but still it should be handled.
Now, Agent reacts to PeerConfig changes sent from the
management service and restarts remote connections
if AllowedIps have been changed.
2022-06-04 19:41:01 +02:00
Misha Bragin
60ac8c3268
Handle Network out of range (#347) 2022-06-02 12:56:02 +02:00
Misha Bragin
87631cbc8b
Replace IP allocation logic (#342)
The peer IP allocation logic was allocating sequential peer IP from the 100.64.0.0/10 
address block.
Each account is created with a random subnet from 100.64.0.0/10.
The total amount of potential subnets is 64.
The new logic allocates random peer IP
from the account subnet.
This gives us flexibility to add support for
multi subnet accounts without overlapping IPs.
2022-05-29 22:43:39 +02:00
Misha Bragin
59a964eed8
Change network mask to limit number of peers to 65k (#339) 2022-05-28 12:54:09 +02:00
shatoboar
94fbfcdb85
Versioning of UI and grpc-agent for passing version (#324)
Send Desktop UI client version as user-agent to daemon

This is sent on every login request to the management

Parse the GRPC context on the system package and 
retrieves the user-agent

Management receives the new UIVersion field and 
store in the Peer's system meta
2022-05-25 23:25:02 +02:00
Givi Khojanashvili
65069c1787
feat(ac): add access control middleware (#321) 2022-05-25 18:26:50 +02:00
Misha Bragin
abe78666d4
Store updated system info on Login to Management (#323) 2022-05-23 13:03:57 +02:00
Misha Bragin
3eb230e1a0
Fix Peer Deletion & HTTP endpoints (#319) 2022-05-21 17:27:04 +02:00
Givi Khojanashvili
3ce3ccc39a
Add rules for ACL (#306)
Add rules HTTP endpoint for frontend - CRUD operations.
Add Default rule - allow all.
Send network map to peers based on rules.
2022-05-21 15:21:39 +02:00
shatoboar
7d893c0238
Rename management from Wiretrustee to Netbird (#311)
Rename documentation and goreleaser build names

Added a migration function for when the old path exists and the new one doesn't

updated the configure.sh to generate the docker-compose with a new path only 
if no pre-existing volume with old name exists
2022-05-13 14:11:21 +02:00
Maycon Santos
e5c52efb4c
Client Login via device authorization flow (#309)
UI and CLI Clients are now able to use SSO login by default

we will check if the management has configured or supports SSO providers

daemon will handle fetching and waiting for an access token

Oauth package was moved to internal to avoid one extra package at this stage

Secrets were removed from OAuth

CLI clients have less and better output

2 new status were introduced, NeedsLogin and FailedLogin for better messaging

With NeedsLogin we no longer have endless login attempts
2022-05-12 11:17:24 +02:00
Maycon Santos
49cca57565
Saving new user to existing account (#310)
Add check if user with
account id metadata belongs to account
2022-05-09 14:30:20 +02:00
Maycon Santos
7e5449fb55
Get Device Authorization Flow information from management (#308)
We will configure the device authorization
flow information and a client will
retrieve it and initiate a
device authorization gran flow
2022-05-08 11:04:57 +02:00
Maycon Santos
fec3132585
Adding peer registration support to JWT (#305)
The management will validate the JWT as it does in the API
 and will register the Peer to the user's account.

New fields were added to grpc messages in management
 and client daemon and its clients were updated

Peer has one new field, UserID, 
that will hold the id of the user that registered it

JWT middleware CheckJWT got a splitter 
and renamed to support validation for non HTTP requests

Added test for adding new Peer with UserID

Lots of tests update because of a new field
2022-05-05 20:02:15 +02:00
shatoboar
c7e5e5c7c9
Add User HTTP Endpoint to the Management service (#303)
Exposes endpoint under "/users/" that returns information on users.
Calls IDP manager to get information not stored locally (email, name), 
which in the case of the managed version is auth0.
2022-05-05 08:58:34 +02:00
Givi Khojanashvili
219888254e
Feat peer groups (#304)
* feat(management): add groups

* squash

* feat(management): add handlers for groups

* feat(management): add handlers for groups

* chore(management): add tests for the get group of the management

* chore(management): add tests for save group
2022-05-03 16:02:51 +02:00
Maycon Santos
c3bc85e22d
Rename module to netbirdio/netbird (#288)
rename the go module to netbirdio/netbird 
as part of our rebranding.
2022-03-26 12:08:54 +01:00
Mikhail Bragin
be0c5c887c
Persist Network Serial to Store to avoid outdated netmap sent (#260)
Fix outdated update coming from management
even when it is actually not outdated.
2022-03-10 18:18:38 +01:00
Maycon Santos
c1b162c974
Improve private domain's behavior tests and logic (#256)
Improved the behavior tests for private domains
and its logic as well because on existing accounts
there was no primary status update
2022-03-10 13:47:36 +01:00
Maycon Santos
605ca03519
Fix IDP Manager config structs with correct tags (#253)
* Fix IDP Manager config structs with correct tags

When loading the configuration from file
we will use the Auth0ClientConfig and when
sending the post to retrieve a token
 we use the auth0JWTRequest with proper tags

 Also, removed the idle timeout as it was closing
 all idle connections
2022-03-09 17:22:47 +01:00
Maycon Santos
ff62fec956
Handle category change with provided Acc Id (#252)
When account id supplied via claim, we should
handle change of the domain classification.

If category of domain change to private, we
should re-evaluate the private account
2022-03-09 13:31:42 +01:00
Maycon Santos
0b8387bd2c
Group users of same private domain (#243)
* Added Domain Category field and fix store tests

* Add GetAccountByDomain method

* Add Domain Category to authorization claims

* Initial GetAccountWithAuthorizationClaims test cases

* Renamed Private Domain map and index it on saving account

* New Go build tags

* Added NewRegularUser function

* Updated restore to account for primary domain account

Also, added another test case

* Added grouping user of private domains

Also added auxiliary methods for update metadata and domain attributes

* Update http handles get account method and tests

* Fix lint and document another case

* Removed unnecessary log

* Move use cases to method and add flow comments

* Split the new user and existing logic from GetAccountWithAuthorizationClaims

* Review: minor corrections

Co-authored-by: braginini <bangvalo@gmail.com>
2022-03-01 15:22:18 +01:00
Maycon Santos
b29948b910
Jwtclaims package (#242)
* Move JWTClaims logic to its own package

* Add extractor tests
2022-02-23 20:02:02 +01:00
shatoboar
5f5cbf7e20
Test mgmt http handler (#240) 2022-02-22 18:18:05 +01:00
shatoboar
41c6af6b6f
Extracted AccountManager to interface (#230) 2022-02-22 11:28:19 +01:00
braginini
765d3a0ad0 fix: account JWT claim 2022-02-15 13:38:22 +01:00
shatoboar
15e371b592
sends wtversion to dashboard-UI (#229) 2022-02-14 17:51:07 +01:00
Maycon Santos
cd9a418df2
Store domain information (#217)
* extract claim information from JWT

* get account function

* Store domain

* tests missing domain

* update existing account with domain

* add store domain tests
2022-02-11 17:18:18 +01:00
shatoboar
b59fd50226
Add client version to the client app and send it to the management service (#222)
* test: WIP mocking the grpc server for testing the sending of the client information

* WIP: Test_SystemMetaDataFromClient with mocks, todo:

* fix: failing meta data test

* test: add system meta expectation in management client test

* fix: removing deprecated register function, replacing with new one

* fix: removing deprecated register function from mockclient interface impl

* fix: fixing interface declaration

* chore: remove unused commented code

Co-authored-by: braginini <bangvalo@gmail.com>
2022-02-08 18:03:27 +01:00
Maycon Santos
0739038d51
Fix unstable parallel tests (#202)
* update interface tests and configuration messages

* little debug

* little debug on both errors

* print all devs

* list of devices

* debug func

* handle interface close

* debug socks

* debug socks

* if ports match

* use random assigned ports

* remove unused const

* close management client connection when stopping engine

* GracefulStop when management clients are closed

* enable workflows on PRs too

* remove iface_test debug code
2022-01-25 09:40:28 +01:00
Maycon Santos
fd7282d3cf
Link account id with the external user store (#184)
* get account id from access token claim

* use GetOrCreateAccountByUser and add test

* correct account id claim

* remove unused account

* Idp manager interface

* auth0 idp manager

* use if instead of switch case

* remove unnecessary lock

* NewAuth0Manager

* move idpmanager to its own package

* update metadata when accountId is not supplied

* update tests with idpmanager field

* format

* new idp manager and config support

* validate if we fetch the interface before converting to string

* split getJWTToken

* improve tests

* proper json fields and handle defer body close

* fix ci lint notes

* documentation and proper defer position

* UpdateUserAppMetadata tests

* update documentation

* ManagerCredentials interface

* Marshal and Unmarshal functions

* fix tests

* ManagerHelper and ManagerHTTPClient

* further tests with mocking

* rename package and custom http client

* sync local packages

* remove idp suffix
2022-01-24 11:21:30 +01:00
Maycon Santos
0f7ab4354b
Fix cicd testing issue (#197)
* sync module

* cache per test os

* different port for tests

* wireguard packages versions
2022-01-17 15:10:18 +01:00
Mikhail Bragin
afb302d5e7
Change Management Sync protocol to support incremental (serial) network changes (#191)
* feature: introduce NetworkMap to the management protocol with a Serial ID

* test: add Management Sync method protocol test

* test: add Management Sync method NetworkMap field check [FAILING]

* test: add Management Sync method NetworkMap field check [FAILING]

* feature: fill NetworkMap property to when Deleting peer

* feature: fill NetworkMap in the Sync protocol

* test: code review mentions - GeneratePrivateKey() in the test

* fix: wiretrustee client use wireguard GeneratePrivateKey() instead of GenerateKey()

* test: add NetworkMap test

* fix: management_proto test remove store.json on test finish
2022-01-16 17:10:36 +01:00
Mikhail Bragin
9d1ecbbfb2
Management - add serial to Network reflecting network updates (#179)
* chore: [management] - add account serial ID

* Fix concurrency on the client (#183)

* reworked peer connection establishment logic eliminating race conditions and deadlocks while running many peers

* chore: move serial to Network from Account

* feature: increment Network serial ID when adding/removing peers

* chore: extract network struct init to network.go

* chore: add serial test when adding peer to the account

* test: add ModificationID test on AddPeer and DeletePeer
2022-01-14 14:34:27 +01:00
Mikhail Bragin
6ae27c9a9b
Refactor: support multiple users under the same account (#170)
* feature: add User entity to Account

* test: new file store creation test

* test: add FileStore persist-restore tests

* test: add GetOrCreateAccountByUser Accountmanager test

* refactor: rename account manager users file

* refactor: use userId instead of accountId when handling Management HTTP API

* fix: new account creation for every request

* fix: golint

* chore: add account creator to Account Entity to identify who created the account.

* chore: use xid ID generator for account IDs

* fix: test failures

* test: check that CreatedBy is stored when account is stored

* chore: add account copy method

* test: remove test for non existent GetOrCreateAccount func

* chore: add accounts conversion function

* fix: golint

* refactor: simplify admin user creation

* refactor: move migration script to a separate package
2021-12-27 13:17:15 +01:00
Mikhail Bragin
8c75ef8bef
update to go 1.17 (#167)
* chore: update to go 1.17

* fix: update workflows go version

* fix: golint errors/update grpc
2021-12-21 10:02:25 +01:00
Mikhail Bragin
673e807528
chore: set default key expiration if not provided by frontednd (#142) 2021-10-31 12:06:44 +01:00
Mikhail Bragin
32c369257b
management/support cert from file (#122)
* feature: support cert file in management service

* docs: add new management commands
2021-09-25 19:22:49 +02:00
Mikhail Bragin
ec759bc461
Delete peer (#114)
* feature: add peer deletion

* feature: add peer deletion [CLIENT]

* fix: lint error

* test: fix sync block

* test: fix management test

* feature: add client stop after was deleted

* chore: remove permission denied cancellation

* chore: add larger signal backoff

* feature: notify deleted peer of removal

* fix: lint issue

* chore: add 2nd default key - one off

* test: fix account key check
2021-09-07 18:36:46 +02:00
braginini
081162864d fix: management config 2021-09-07 10:35:33 +02:00
Mikhail Bragin
a4db0b4e94
client update of TURNs and STUNs (#106)
* feature: update STUNs and TURNs in engine

* fix: setup TURN credentials request only when refresh enabled

* feature: update TURNs and STUNs in teh client app on Management update

* chore: disable peer reflexive candidates in ICE

* chore: relocate management.json

* chore: make TURN secret and pwd plain text in config
2021-09-03 17:47:40 +02:00
Mikhail Bragin
b17424d630
Turn credentials generation (#102)
* abstract peer channel

* remove wip code

* refactor NewServer with Peer updates channel

* feature: add TURN credentials manager

* hmac logic

* example test function

* test: add TimeBasedAuthSecretsManager_GenerateCredentials  test

* test: make tests for now with hardcoded secret

* test: add TimeBasedAuthSecretsManager_SetupRefresh test

* test: add TimeBasedAuthSecretsManager_SetupRefresh test

* test: add TimeBasedAuthSecretsManager_CancelRefresh test

* feature: extract TURNConfig to the management config

* feature: return hash based TURN credentials only on initial sync

* feature: make TURN time based secret credentials optional

Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
2021-09-02 14:41:54 +02:00
Maycon Santos
a31cbb1f5b
abstract peer channel (#101)
* abstract peer channel

* remove wip code

* refactor NewServer with Peer updates channel

* add PeersUpdateManager tests

* adding documentation

* using older version of linter

* verbose lint

* skip cache

* setup go version

* extra output

* configure fetch-depth

* exit 0

* skip-build-cache: true

* disabling failure for lint for now

* fix: darwin issue

* enable lint failure

* remove sock file for macOS

* refactor: remove tests interdependence

* fixed linux native iface

Co-authored-by: braginini <bangvalo@gmail.com>
2021-08-29 17:48:31 +02:00
braginini
5c7260298f chore: adjust system info discovery methods 2021-08-27 11:45:46 +02:00
braginini
49800a6d03 fix: minor HTTP bugs 2021-08-25 14:16:17 +02:00
Mikhail Bragin
0fa15e6920
Add peer meta data (#95)
* feature: add peer GET and DELETE API methods

* refactor: extract peer business logic to a separate file

* refactor: extract peer business logic to a separate file

* feature: add peer update HTTP endpoint

* chore: fill peer new fields

* merge with main

* refactor: HTTP methods according to standards

* feature: add peer system metadata

* feature: add peer status

* test: fix removal
2021-08-24 11:50:19 +02:00
Mikhail Bragin
95845c88fe
Extend peer http endpoint (#94)
* feature: add peer GET and DELETE API methods

* refactor: extract peer business logic to a separate file

* refactor: extract peer business logic to a separate file

* feature: add peer update HTTP endpoint

* chore: fill peer new fields

* merge with main

* refactor: HTTP methods according to standards

* chore: setup keys POST endpoint without ID
2021-08-23 21:43:05 +02:00
Mikhail Bragin
6869b48905
feature: increase key usage after successful peer registration (#93) 2021-08-22 11:29:25 +02:00
braginini
90ef1e939b fix: test 2021-08-20 23:12:39 +02:00
braginini
2e9fc20567 feature: add update setup key endpoint 2021-08-20 22:33:43 +02:00
braginini
b5a20bf1ba chore: remove unused static dashboard files 2021-08-20 16:25:51 +02:00
braginini
07ab9c196d fix: golint errors 2021-08-20 15:51:29 +02:00
braginini
4a5901ada1 test: add more AccountManager tests 2021-08-20 15:44:18 +02:00
braginini
4c427ae900 test: add AddAccount test 2021-08-20 15:18:29 +02:00
braginini
22fdb0a029 test: add setup key UUID format validation 2021-08-20 15:08:07 +02:00
braginini
1b056ab75a test: add setup key tests 2021-08-20 15:01:57 +02:00
braginini
708835afa8 test: change test data to support new setup key fields 2021-08-20 13:40:07 +02:00
braginini
02cc6a30f5 feature: extend setup key logic 2021-08-19 21:12:21 +02:00
braginini
c68d9dff4a chore: setup key upper case 2021-08-19 20:07:12 +02:00
Mikhail Bragin
e8ca289f4a
test: add management client tests (#89)
* test: add management client tests

* test: add management client Sync test

* fix: engine test

* test: remove engine tests

* test: return engine tests [check]

* test: remove engine tests [check]
2021-08-16 23:30:51 +02:00
Maycon Santos
38e3c9c062
Add cors headers (#85)
* disable EnableAuthOnOptions

* Setup basic cors headers

* feature: user cors lib
2021-08-16 11:29:57 +02:00
Mikhail Bragin
877ad97a96
Peer management login (#83)
* feature: replace RegisterPeer with Login method that does both - registration and login

* test: add management login test

* feature: add WiretrusteeConfig to the Login response to configure peer global config

* feature: add client peer login support

* fix: missing parts

* chore: update go deps

* feature: support Management Service gRPC endpoints [CLIENT]

* feature: finalize client sync with management

* fix: management store peer key lower case restore

* fix: management returns peer ip without a mask

* refactor: remove cmd pkg

* fix: invalid tun interface name on mac

* fix: timeout when calling management client

* fix: tests and lint errors

* fix: golang-test workflow

* fix: client service tests

* fix: iface build

* feature: detect management scheme on startup

* chore: better logs for management

* fix: goreleaser

* fix: lint errors

* fix: signal TLS

* fix: direct Wireguard connection

* chore: verbose logging on direct connection
2021-08-15 16:56:26 +02:00
braginini
dcc9dcacdc feature: add setup key HTTP api response fields 2021-08-12 13:24:06 +02:00
Mikhail Bragin
3c47a3c408
peer management HTTP API (#81)
* feature: create account for a newly registered user

* feature: finalize user auth flow

* feature: create protected API with JWT

* chore: cleanup http server

* feature: add UI assets

* chore: update react UI

* refactor: move account not exists -> create to AccountManager

* chore: update UI

* chore: return only peers on peers endpoint

* chore: add UI path to the config

* chore: remove ui from management

* chore: remove unused Docker comamnds

* docs: update management config sample

* fix: store creation

* feature: introduce peer response to the HTTP api

* fix: lint errors

* feature: add setup-keys HTTP endpoint

* fix: return empty json arrays in HTTP API

* feature: add new peer response fields
2021-08-12 12:49:10 +02:00
braginini
9f0c86c28e refactor: move grpc and http APIs to separate packages 2021-08-07 13:51:17 +02:00
braginini
08d44b1d5f refactor: move LetsEncryptDomain to HttpServer config 2021-08-07 13:35:52 +02:00
Mikhail Bragin
1f29975737
feature: basic auth0 support (#78)
* feature: basic auth0 support

* refactor: improve auth flow

* refactor: extract HttpServer config

* feature: merge HTTP API layer with Let's Encrypt
2021-08-07 12:26:07 +02:00
andpar83
11982d6dde
Add client's interaction with management service (#71)
* Add client's interaction with management service

* Getting updates

* Fixed key and nil ptr

* Added setupKey param

* Added managment address parameter

* Fixed test

* feature: use RemotePeers from the management server instead of deprecated Peers

* merge: merge changes from main
2021-08-01 19:06:01 +02:00
Mikhail Bragin
2c2c1e19df
Peer configuration management (#69)
* feature: add config properties to the SyncResponse of the management gRpc service

* fix: lint errors

* chore: modify management protocol according to the review notes

* fix: management proto fields sequence

* feature: add proper peer configuration to be synced

* chore: minor changes

* feature: finalize peer config management

* fix: lint errors

* feature: add management server config file

* refactor: extract hosts-config to a separate file

* refactor: review notes applied to correct file_store usage

* refactor: extract management service configuration to a file

* refactor: simplify management config
2021-07-30 17:46:38 +02:00
Mikhail Bragin
3b30beb567
add config properties to the SyncResponse of the management gRpc service (#66)
* feature: add config properties to the SyncResponse of the management gRpc service
2021-07-25 17:08:16 +02:00
andpar83
9e4aa4f1f1
Move management server to a separate directory (#67)
* Move management server to a separate directory
2021-07-24 16:14:29 +02:00