Commit Graph

624 Commits

Author SHA1 Message Date
Misha Bragin
48265b32f3
Measure write requests separately from read requests (#880) 2023-05-19 16:56:15 +02:00
Misha Bragin
03a42de5a0
Add telemetry to measure app durations (#878) 2023-05-19 11:42:25 +02:00
Maycon Santos
48a8b52740
Avoid storing account if no peer meta or expiration change (#875)
* Avoid storing account if no peer meta or expiration change

* remove extra log

* Update management/server/peer.go

Co-authored-by: Misha Bragin <bangvalo@gmail.com>

* Clarify why we need to skip account update

---------

Co-authored-by: Misha Bragin <bangvalo@gmail.com>
2023-05-18 19:31:35 +02:00
Misha Bragin
6e9f7531f5
Track user block/unblock activity event (#865) 2023-05-17 09:54:20 +02:00
Pascal Fischer
873abc43bf move into separate package 2023-05-16 12:57:56 +02:00
Pascal Fischer
2fef52b856 remove dependency to external base62 package and create own methods in utils 2023-05-16 12:44:26 +02:00
Bethuel
2570363861 fix assign correct issuer url to auth0 AuthIssuer 2023-05-12 18:07:11 +03:00
Misha Bragin
e3d2b6a408
Block user through HTTP API (#846)
The new functionality allows blocking a user in the Management service.
Blocked users lose access to the Dashboard, aren't able to modify the network map,
and all of their connected devices disconnect and are set to the "login expired" state.

Technically all above was achieved with the updated PUT /api/users endpoint,
that was extended with the is_blocked field.
2023-05-11 18:09:36 +02:00
Bethuel
2c50d7af1e
Automatically load IdP OIDC configuration (#847) 2023-05-11 15:14:00 +02:00
pascal-fischer
e4c28f64fa
Fix user cache lookup filtering for service users (#849) 2023-05-10 19:27:17 +02:00
Bethuel
f4ec1699ca
Add Zitadel IdP (#833)
Added intergration with Zitadel management API.

Use the steps in zitadel.md for configuration.
2023-05-05 19:27:28 +02:00
Bethuel
873b56f856
Add Azure Idp Manager (#822)
Added intergration with Azure IDP user API.

Use the steps in azure-ad.md for configuration:
cb03373f8f/docs/integrations/identity-providers/self-hosted/azure-ad.md
2023-05-03 14:51:44 +02:00
pascal-fischer
59372ee159
API cleanup (#824)
removed all PATCH endpoints
updated path parameters for all endpoints
removed not implemented endpoints for api doc
minor description updates
2023-05-03 00:15:25 +02:00
pascal-fischer
08db5f5a42
Merge pull request #831 from netbirdio/fix/issue_with_account_creation_after_auth_refactor
FIx account creation issue after auth refactor
2023-05-02 19:14:54 +02:00
pascal-fischer
88678ef364
Merge pull request #808 from bcmmbaga/main
Add support for refreshing signing keys on expiry
2023-05-02 17:17:09 +02:00
Pascal Fischer
f1da4fd55d using old isAdmin function to create account 2023-05-02 16:49:29 +02:00
Zoltan Papp
7f5e1c623e
Use forked Wireguard-go for custom bind (#823)
Update go version to 1.20
Use forked wireguard-go repo because of custom Bind implementation
2023-04-27 17:50:45 +02:00
pascal-fischer
6fec0c682e
Merging full service user feature into main (#819)
Merging full feature branch into main.
Adding full support for service users including backend objects, persistence, verification and api endpoints.
2023-04-22 12:57:51 +02:00
Bethuel
45224e76d0 fallback to olde keys if failing to fetch refreshed keys 2023-04-21 13:34:52 +03:00
Bethuel
90c8cfd863 synchronize access to the signing keys 2023-04-19 17:11:38 +03:00
Zoltan Papp
4616bc5258
Add route management for Android interface (#801)
Support client route management feature on Android
2023-04-17 11:15:37 +02:00
Bethuel
f7196cd9a5 refactoring 2023-04-15 03:44:42 +03:00
Bethuel
53d78ad982 make variable unexported 2023-04-14 13:16:01 +03:00
Bethuel
9f352c1b7e validate keys for idp's with key rotation mechanism 2023-04-14 12:20:34 +03:00
Bethuel
a89808ecae initialize jwt validator with keys rotation state 2023-04-14 12:17:28 +03:00
Bethuel
c6190fa2ba add use-key-cache-headers flag to management command 2023-04-13 20:19:04 +03:00
Givi Khojanashvili
0343c5f239
Rollback simple ACL rules processing. (#803) 2023-04-12 09:39:17 +02:00
Misha Bragin
251f2d7bc2
Pass newly generated ID to network map when adding peer (#800) 2023-04-11 14:28:22 +02:00
Maycon Santos
306e02d32b
Update calculate server state (#796)
Refactored updateServerStates and calculateState

added some checks to ensure we are not sending connecting on context canceled

removed some state updates from the RunClient function
2023-04-10 18:22:25 +02:00
pascal-fischer
8375491708
Merge pull request #778 from netbirdio/fix/consistent_time_format_for_pat
fix/use_utc_for_time_operations
2023-04-10 18:11:41 +02:00
Pascal Fischer
6aba28ccb7 remove UTC from some not store related operations 2023-04-10 10:54:23 +02:00
Maycon Santos
32b345991a
Support remote scope and use id token configuration (#784)
Some IDP requires different scope requests and
issue access tokens for different purposes

This change allow for remote configurable scopes
and the use of ID token
2023-04-05 17:46:34 +02:00
Maycon Santos
fe1ea4a2d0
Check multiple audience values (#781)
Some IDP use different audience for different clients. 
This update checks HTTP and Device authorization flow audience values.



---------

Co-authored-by: Givi Khojanashvili <gigovich@gmail.com>
2023-04-04 16:40:56 +02:00
Pascal Fischer
489892553a use UTC everywhere in server 2023-04-03 15:09:35 +02:00
Pascal Fischer
b05e30ac5a do not use UTC for time to stay consistent 2023-04-03 12:44:55 +02:00
pascal-fischer
769388cd21
Merge pull request #776 from netbirdio/feature/activity_events_for_pat
feature/activity_events_for_pat
2023-04-03 12:27:51 +02:00
pascal-fischer
c54fb9643c
Merge pull request #774 from netbirdio/feature/add_pat_middleware
Feature/add pat middleware
2023-04-03 12:09:11 +02:00
Givi Khojanashvili
5dc0ff42a5
Fix broken auto-generated Rego rule (#769)
Default Rego policy generated from the rules in some cases is broken.
This change fixes the Rego template for rules to generate policies.

Also, file store load constantly regenerates policy objects from rules.
It allows updating/fixing of the default Rego template during releases.
2023-04-01 12:02:08 +02:00
Pascal Fischer
45badd2c39 add event store to user tests 2023-04-01 11:11:30 +02:00
Pascal Fischer
d3de035961 error responses always lower case + duplicate error response fix 2023-04-01 11:04:21 +02:00
Pascal Fischer
b2da0ae70f add activity events on PAT creation and deletion 2023-03-31 17:41:22 +02:00
Pascal Fischer
931c20c8fe fix test name 2023-03-31 12:45:10 +02:00
Pascal Fischer
2eaf4aa8d7 add test for auth middleware 2023-03-31 12:44:22 +02:00
Pascal Fischer
110067c00f change order for access control checks and aquire account lock after global lock 2023-03-31 12:03:53 +02:00
Pascal Fischer
32c96c15b8 disable linter errors by comment 2023-03-31 10:30:05 +02:00
Pascal Fischer
ca1dc5ac88 disable access control for token endpoint 2023-03-30 19:03:44 +02:00
Pascal Fischer
ce775d59ae revert codacy 2023-03-30 18:59:35 +02:00
Pascal Fischer
f273fe9f51 revert codacy 2023-03-30 18:54:55 +02:00
Pascal Fischer
e08af7fcdf codacy 2023-03-30 17:46:21 +02:00
Pascal Fischer
454240ca05 comments for codacy 2023-03-30 17:32:44 +02:00
Pascal Fischer
1343a3f00e add test + codacy 2023-03-30 16:43:39 +02:00
Pascal Fischer
2a79995706 fix linter 2023-03-30 16:22:15 +02:00
Pascal Fischer
e869882da1 fix merge 2023-03-30 16:14:51 +02:00
Pascal Fischer
6c8bb60632 fix merge 2023-03-30 16:06:46 +02:00
Pascal Fischer
4d7029d80c Merge branch 'main' into feature/add_pat_middleware
# Conflicts:
#	management/server/grpcserver.go
#	management/server/http/middleware/jwt.go
2023-03-30 16:06:21 +02:00
pascal-fischer
909f305728
Merge pull request #766 from netbirdio/feature/add_rest_endpoints_for_pat
Feature/add rest endpoints for pat
2023-03-30 15:55:48 +02:00
Pascal Fischer
5e2f66d591 fix codacy 2023-03-30 15:23:24 +02:00
Pascal Fischer
a7519859bc fix test 2023-03-30 14:15:44 +02:00
Pascal Fischer
9b000b89d5 Merge branch 'feature/add_rest_endpoints_for_pat' into feature/add_pat_middleware
# Conflicts:
#	management/server/mock_server/account_mock.go
2023-03-30 14:02:58 +02:00
Pascal Fischer
5c1acdbf2f move validation into account manager + func for get requests 2023-03-30 13:58:44 +02:00
Pascal Fischer
db3a9f0aa2 refactor jwt token validation and add PAT to middleware auth 2023-03-30 10:54:09 +02:00
Pascal Fischer
ecc4f8a10d fix Pat handler test 2023-03-29 19:13:01 +02:00
Pascal Fischer
03abdfa112 return empty object on all handlers instead of empty string 2023-03-29 18:46:40 +02:00
Pascal Fischer
9746a7f61a remove debug logs 2023-03-29 18:27:01 +02:00
Pascal Fischer
4ec6d5d20b remove debug logs 2023-03-29 18:23:10 +02:00
Pascal Fischer
3bab745142 last_used can be nil 2023-03-29 17:46:09 +02:00
Pascal Fischer
0ca3d27a80 update account mock 2023-03-29 15:25:44 +02:00
Pascal Fischer
c5942e6b33 store hashed token base64 encoded 2023-03-29 15:21:53 +02:00
Pascal Fischer
726ffb5740 add comments for exported functions 2023-03-29 15:06:54 +02:00
Pascal Fischer
42ba0765c8 fix linter 2023-03-28 14:54:06 +02:00
Pascal Fischer
514403db37 use object instead of plain token for create response + handler test 2023-03-28 14:47:15 +02:00
Pascal Fischer
6a75ec4ab7 fix http error codes 2023-03-27 17:42:05 +02:00
Pascal Fischer
b66e984ddd set limits for expiration 2023-03-27 17:28:24 +02:00
Pascal Fischer
c65a934107 refactor to use name instead of description 2023-03-27 16:28:49 +02:00
Pascal Fischer
9e74f30d2f fix delete token parameter lookup 2023-03-27 15:19:19 +02:00
Maycon Santos
a27fe4326c
Add JWT middleware validation failure log (#760)
We will log the middleware log now, but in the next
releases we should provide a generic error that can be
parsed by the dashboard.
2023-03-23 18:26:41 +01:00
Misha Bragin
e6292e3124
Disable peer expiration of peers added with setup keys (#758) 2023-03-23 17:47:53 +01:00
Maycon Santos
628b497e81
Adjustments for the change server flow (#756)
Check SSO support by calling the internal.GetDeviceAuthorizationFlowInfo

Rename LoginSaveConfigIfSSOSupported to SaveConfigIfSSOSupported

Receive device name as input for setup-key login

have a default android name when no context value is provided

log non parsed errors from management registration calls
2023-03-23 16:35:06 +01:00
Bethuel
8f66dea11c
Add Keycloak Idp Manager (#746)
Added intergration with keycloak user API.
2023-03-23 14:54:31 +01:00
Pascal Fischer
de8608f99f add rest endpoints and update openapi doc 2023-03-21 16:02:19 +01:00
pascal-fischer
9c5adfea2b
Merge pull request #745 from netbirdio/feature/pat_persistence
PAT persistence
2023-03-21 14:38:24 +01:00
Pascal Fischer
8e4710763e use single line return for SaveAccount 2023-03-21 14:02:34 +01:00
Pascal Fischer
82af60838e use "ok" convention for check variables throughout files_store 2023-03-21 14:00:59 +01:00
Pascal Fischer
311b67fe5a change error messages 2023-03-21 13:56:31 +01:00
Pascal Fischer
94d39ab48c improve style for tests 2023-03-21 13:34:48 +01:00
Pascal Fischer
41a47be379 add function comments, implement account mock functions and added error handling in tests 2023-03-20 16:38:17 +01:00
Pascal Fischer
e30def175b switch PATs to map and add deletion 2023-03-20 16:14:55 +01:00
Pascal Fischer
e1ef091d45 remove unnecessary string conversion 2023-03-20 12:08:01 +01:00
pascal-fischer
511ba6d51f
Delete pat_handler.go 2023-03-20 11:47:54 +01:00
Pascal Fischer
b852198f67 codacy and lint hints 2023-03-20 11:44:12 +01:00
Zoltan Papp
747797271e
Fix connstate indication (#732)
Fix the status indication in the client service. The status of the
management server and the signal server was incorrect if the network
connection was broken. Basically the status update was not used by
the management and signal library.
2023-03-16 17:22:36 +01:00
Pascal Fischer
628a201e31 fix PAT array split 2023-03-16 16:59:32 +01:00
Pascal Fischer
453643683d add method to account mock 2023-03-16 16:44:05 +01:00
Pascal Fischer
b8cab2882b storing and retrieving PATs 2023-03-16 15:57:44 +01:00
Pascal Fischer
3b42d5e48a fix imports after merge 2023-03-16 11:59:12 +01:00
pascal-fischer
f8db5742b5
Merge branch 'main' into feature/add_PAT_generation 2023-03-16 11:36:43 +01:00
Pascal Fischer
bc3cec23ec use slice copy 2023-03-16 11:32:55 +01:00
Zoltan Papp
292ee260ad
Add version info command to signal server (#739)
Add version command to signal and management servers.

The version information will be filled during build time.
2023-03-15 07:54:51 +01:00
Givi Khojanashvili
2a1efbd0fd
Don't drop Rules from file storage after migration to Policies (#741)
Rego policy migration clears the rules property of the file storage, but it does not allow rollback management upgrade, so this changes pre-saves rules in the file store and updates it from the policies.
2023-03-15 09:42:40 +04:00
Givi Khojanashvili
3bfa26b13b
Feat rego default policy (#700)
Converts rules to Rego policies and allow users to write raw policies to set up connectivity and firewall on the clients.
2023-03-13 18:14:18 +04:00
Misha Bragin
221934447e
Send remote agents updates when peer re-authenticates (#737)
When peer login expires, all remote peers are updated to exclude the peer from connecting.
Once a peer re-authenticates, the remote peers are not updated.
This peer fixes the behavior.
2023-03-10 17:39:29 +01:00
Misha Bragin
9ce8056b17
Use global login expiration setting when sending network map (#731)
Peers were considered expired and not sent to remote peers
when global expiration was disabled.
2023-03-09 11:24:42 +01:00
Pascal Fischer
62de082961 fix account test 2023-03-08 12:21:44 +01:00
Pascal Fischer
c4d9b76634 add comment for exported const 2023-03-08 12:09:22 +01:00
Pascal Fischer
b4bb5c6bb8 use const and do array copy 2023-03-08 11:54:10 +01:00
Pascal Fischer
2b1965c941 switch secret generation to use lib 2023-03-08 11:36:03 +01:00
Pascal Fischer
83e7e30218 store hashedToken as string 2023-03-08 11:30:09 +01:00
Misha Bragin
ed4f90b6aa
Report offline peers to agents (#728)
The peer login expiration ACL check introduced in #714
filters out peers that are expired and agents receive a network map 
without that expired peers.
However, the agents should see those peers in status "Disconnected".

This PR extends the Agent <-> Management protocol 
by introducing a new field OfflinePeers
that contain expired peers. Agents keep track of those and display 
then just in the Status response.
2023-03-07 10:17:25 +01:00
Pascal Fischer
ed470d7dbe add comments for exported functions 2023-03-06 14:46:04 +01:00
Pascal Fischer
cb8abacadd extend User Copy function 2023-03-06 14:01:18 +01:00
Pascal Fischer
bcac5f7b32 fixed some namings 2023-03-06 13:51:32 +01:00
Pascal Fischer
95d87384ab fixed some namings 2023-03-06 13:49:07 +01:00
Misha Bragin
e914adb5cd
Move Login business logic from gRPC API to Accountmanager (#713)
The Management gRPC API has too much business logic 
happening while it has to be in the Account manager.
This also needs to make more requests to the store 
through the account manager.
2023-03-03 18:35:38 +01:00
Pascal Fischer
2f2d45de9e updated PAT struct to only use user id instead of user 2023-03-03 16:37:39 +01:00
Pascal Fischer
b3f339c753 improved code for token checksum calc 2023-03-03 14:51:33 +01:00
Pascal Fischer
e0fc779f58 add id to the PAT 2023-03-02 16:19:31 +01:00
Misha Bragin
fe22eb3b98
Check peer expiration after ACL check (#714)
Bug 1: When calculating the network map, peers added by a setup key
were falling under expiration logic while they shouldn't.

Bug 2: Peers HTTP API didn't return expired peers for non-admin users
because of the expired peer check in the ACL logic.

The fix applies peer expiration checks outside of the ACL logic.
2023-03-02 12:45:10 +01:00
Pascal Fischer
69be2a8071 add generating token (only frame for now, actual token is only dummy) 2023-03-01 20:12:04 +01:00
Misha Bragin
1bda8fd563
Remove stale peer indices when getting peer by key after removing (#711)
When we delete a peer from an account, we save the account in the file store.
The file store maintains peerID -> accountID and peerKey -> accountID indices.
Those can't be updated when we delete a peer because the store saves the whole account
without a peer already and has no access to the removed peer.
In this PR, we dynamically check if there are stale indices when GetAccountByPeerPubKey
and GetAccountByPeerID.
2023-03-01 12:11:32 +01:00
pascal-fischer
1ab791e91b
Merge pull request #707 from netbirdio/chore/NB-93-re-arrange-api-code
chore/re-arrange-api-code
2023-03-01 09:52:42 +01:00
Misha Bragin
41948f7919
Fix peer status update when expiring peers (#708) 2023-02-28 20:02:30 +01:00
Pascal Fischer
60f67076b0 change methods to not link 2023-02-28 18:17:55 +01:00
Pascal Fischer
c645171c40 split api code into smaller pieces 2023-02-28 18:08:02 +01:00
Pascal Fischer
8a130ec3f1 add comments to fix codacy 2023-02-28 16:51:30 +01:00
Pascal Fischer
c26cd3b9fe add comments for constructors and fix typo 2023-02-28 15:46:08 +01:00
Pascal Fischer
9d7b515b26 changed the naming convention for all handling objects and methods to have unified way 2023-02-28 15:27:43 +01:00
Pascal Fischer
f1f90807e4 changed the naming convention for all handling objects and methods to have unified way 2023-02-28 15:01:24 +01:00
Misha Bragin
f984b8a091
Proactively expire peers' login per account (#698)
Goals:

Enable peer login expiration when adding new peer
Expire peer's login when the time comes
The account manager triggers peer expiration routine in future if the
following conditions are true:

peer expiration is enabled for the account
there is at least one peer that has expiration enabled and is connected
The time of the next expiration check is based on the nearest peer expiration.
Account manager finds a peer with the oldest last login (auth) timestamp and
calculates the time when it has to run the routine as a sum of the configured
peer login expiration duration and the peer's last login time.

When triggered, the expiration routine checks whether there are expired peers.
The management server closes the update channel of these peers and updates
network map of other peers to exclude expired peers so that the expired peers
are not able to connect anywhere.

The account manager can reschedule or cancel peer expiration in the following cases:

when admin changes account setting (peer expiration enable/disable)
when admin updates the expiration duration of the account
when admin updates peer expiration (enable/disable)
when peer connects (Sync)
P.S. The network map calculation was updated to exclude peers that have login expired.
2023-02-27 16:44:26 +01:00
Misha Bragin
c962d29280
Fix login expiration enum in OpenAPI (#694)
Add missing OpenAPI enums for the peer login expiration events
2023-02-16 15:36:36 +01:00
Misha Bragin
44af5be30f
Reject peer login expiration update when no SSO login (#693) 2023-02-16 13:03:53 +01:00
Misha Bragin
fe63a64b6e
Add Account HTTP API (#691)
Extend HTTP API with Account endpoints to configure global peer login expiration.
GET /api/accounts
PUT /api/account/{id}/

The GET endpoint returns an array of accounts with
always one account in the list. No exceptions.

The PUT endpoint updates account settings:
PeerLoginExpiration and PeerLoginExpirationEnabled.

PeerLoginExpiration is a duration in seconds after which peers' logins will expire.
2023-02-16 12:00:41 +01:00
Misha Bragin
d31219ba89
Update peer status when login expires (#688)
Extend PeerStatus with an extra field LoginExpired, that can be stored in the database.
2023-02-15 11:27:22 +01:00
Misha Bragin
756ce96da9
Add login expiration fields to peer HTTP API (#687)
Return login expiration related fields in the Peer HTTP GET endpoint.
Support enable/disable peer's login expiration via HTTP PUT.
2023-02-14 10:14:00 +01:00
Misha Bragin
d5dfed498b
Add account settings (#686)
Add account settings with a global peer expiration flag and duration
2023-02-13 15:07:15 +01:00
Misha Bragin
3fc89749c1
Add peer login expiration (#682)
This PR adds a peer login expiration logic that requires
peers created by a user to re-authenticate (re-login) after
a certain threshold of time (24h by default).

The Account object now has a PeerLoginExpiration
property that indicates the duration after which a peer's
login will expire and a login will be required. Defaults to 24h.

There are two new properties added to the Peer object:
LastLogin that indicates the last time peer successfully used
the Login gRPC endpoint and LoginExpirationEnabled that
enables/disables peer login expiration.

The login expiration logic applies only to peers that were created
by a user and not those that were added with a setup key.
2023-02-13 12:21:02 +01:00
Misha Bragin
00a8092482
Add GET peer HTTP API endpoint (#670) 2023-02-07 20:11:08 +01:00
Givi Khojanashvili
3ec8274b8e
Feature: add custom id claim (#667)
This feature allows using the custom claim in the JWT token as a user ID.

Refactor claims extractor with options support

Add is_current to the user API response
2023-02-03 21:47:20 +01:00
Misha Bragin
9adadfade4
Use Peer.ID instead of Peer.Key as peer identifier (#664)
Replace Peer.Key as internal identifier with a randomly generated Peer.ID 
in the Management service.
Every group now references peers by ID instead of a public key.
Every route now references peers by ID instead of a public key.
FileStore does store.json file migration on startup by generating Peer.ID and replacing
all Peer.Key identifier references .
2023-02-03 10:33:28 +01:00
Misha Bragin
9e408b5bbc
Add more activity events (#663) 2023-01-25 16:29:59 +01:00
Misha Bragin
a0de9aa345
Simplify event storing with one generic method (#662)
Use the generic storeEvent() funcion to store all activity events.
2023-01-24 10:17:24 +01:00
Maycon Santos
4406d50c18
Check if peer name change before update labels (#658) 2023-01-20 10:07:37 +01:00
Maycon Santos
5e3502bb83
Enable CGO in goreleaser for management (#657)
Update the debug docker image file

update goreleaser version
Install and reference CGO OS deps
2023-01-19 15:03:10 +01:00
Maycon Santos
793e4f1f29
Allow empty groups for DNS setting (#656)
We should allow passing empty group slice for DisabledManagementGroups to reset any setting.


Fix DNS settings activities message
2023-01-18 14:01:50 +01:00
Maycon Santos
dcf6533ed5
Adding --external-ip-map and --dns-resolver-address and shorthand flags (#652)
Adding --external-ip-map and --dns-resolver-address to up command and shorthand option to global flags.

Refactor get and read config functions with new ConfigInput type.

updated cobra package to latest release.
2023-01-17 19:16:50 +01:00
Maycon Santos
12ae2e93fc
Adding DNS settings for accounts (#655)
Allow users to set groups in which the DNS management is disabled

Added API, activity store, and network map sync test
2023-01-17 17:34:40 +01:00
Misha Bragin
afaf0660be
Install ca-certs in the Management docker image build (#650) 2023-01-11 08:19:22 +01:00
Misha Bragin
d4d8c5f037
Fix cgo dependencies in the management docker container (#649) 2023-01-10 17:24:51 +01:00
Misha Bragin
5c0b8a46f0
Add system activity tracking and event store (#636)
This PR adds system activity tracking. 
The management service records events like 
add/remove peer,  group, rule, route, etc.

The activity events are stored in the SQLite event store
and can be queried by the HTTP API.
2023-01-02 15:11:32 +01:00
Maycon Santos
6b32e2dc07
Validate single account domain input (#624) 2022-12-13 13:43:29 +01:00
Maycon Santos
6f610dca89
Filter routes to sync from same HA group (#618)
An additional check and filter for routes that are part
 of the same HA group where the peer is a routing peer
2022-12-08 15:15:50 +01:00
Maycon Santos
0be46c083d
Generate validation certificate from mandatory JWK fields (#614)
When there is no X5c we will use N and E fields of 
a JWK to generate the public RSA and a Pem certificate
2022-12-07 22:06:43 +01:00
Maycon Santos
a387e3cfc2
Add network routes distribution groups (#606)
Updated tests, API, and account manager methods

Sync routes to peers in the distribution groups

Added store upgrade by adding the All group to routes that don't have them
2022-12-06 10:11:57 +01:00
Misha Bragin
d1b7c23b19
Add SetupKey usage limit (#605)
Add a usage_limit parameter to the API.
This limits the number of times a setup key
can be used. 
usage_limit == 0 indicates the the usage is inlimited.
2022-12-05 13:09:59 +01:00
Maycon Santos
20a73e3e14
Sync peers FQDN (#584)
Use stdout and stderr log path only if on Linux and attempt to create the path

Update status system with FQDN fields and 
status command to display the domain names of remote and local peers

Set some DNS logs to tracing

update readme file
2022-11-26 13:29:50 +01:00
Misha Bragin
fcf7786a85
Disable route when removing peer (#582) 2022-11-25 18:11:07 +01:00
Misha Bragin
53c532bbb4
Fix interactive SSO login when creating account from a device (#575) 2022-11-22 12:37:36 +01:00
Misha Bragin
8b0a1bbae0
Display peers of a user that it has access to (#571)
If a user has a non-admin role, display all peers
that user's peers have access to when calling
/peers endpoint of the HTTP API.
2022-11-21 17:45:14 +01:00
Maycon Santos
d63a9ce4a7
Return peer's FQDN via API (#567)
Added a temp method to retrieve the dns domain
2022-11-21 11:14:42 +01:00
Maycon Santos
9cb66bdb5d
Update last run time and active count (#568)
* Update last run time and active count

We will collect the active peer min and max versions

* Get UI client usage
2022-11-18 16:35:13 +01:00
Misha Bragin
509d23c7cf
Replace gRPC errors in business logic with internal ones (#558) 2022-11-11 20:36:45 +01:00
Misha Bragin
e19d5dca7f
Refactor AddPeer to ensure consistency (#557) 2022-11-08 16:14:36 +01:00
Maycon Santos
157137e4ad
Use a single way to generate network map (#550) 2022-11-08 11:38:40 +01:00
Misha Bragin
f37b43a542
Save Peer Status separately in the FileStore (#554)
Due to peer reconnects when restarting the Management service,
there are lots of SaveStore operations to update peer status.

Store.SavePeerStatus stores peer status separately and the
FileStore implementation stores it in memory.
2022-11-08 10:46:12 +01:00
Maycon Santos
7e262572a4
Move dns label generation to store (#552) 2022-11-08 10:31:34 +01:00
Misha Bragin
a768a0aa8a
Always lock the store when getting an account (#551) 2022-11-07 19:09:22 +01:00
Misha Bragin
ed7ac81027
Introduce locking on the account level (#548) 2022-11-07 17:52:23 +01:00
Maycon Santos
1f845f466c
Add account copy test (#549) 2022-11-07 17:37:28 +01:00
Maycon Santos
270f0e4ce8
Feature/dns protocol (#543)
Added DNS update protocol message

Added sync to clients

Update nameserver API with new fields

Added default NS groups

Added new dns-name flag for the management service append to peer DNS label
2022-11-07 15:38:21 +01:00
Misha Bragin
d0c6d88971
Simplified Store Interface (#545)
This PR simplifies Store and FileStore
by keeping just the Get and Save account methods.

The AccountManager operates mostly around
a single account, so it makes sense to fetch
the whole account object from the store.
2022-11-07 12:10:56 +01:00
Misha Bragin
4321b71984
Hide content based on user role (#541) 2022-11-05 10:24:50 +01:00
Maycon Santos
e8d82c1bd3
Feature/dns-server (#537)
Adding DNS server for client

Updated the API with new fields

Added custom zone object for peer's DNS resolution
2022-11-03 18:39:37 +01:00
Misha Bragin
6aa7a2c5e1
Hide setup key from non-admin users (#539) 2022-11-03 17:02:31 +01:00
Maycon Santos
9a4c9aa286
Add active peers count per OS (#526)
* Add active peers count per OS

* increase iface tests timeout
2022-10-26 14:48:40 +02:00
Misha Bragin
b41f36fccd
Add gRPC metrics (#522) 2022-10-22 15:06:54 +02:00
Misha Bragin
d2cde4a040
Add IdP metrics (#521) 2022-10-22 13:29:39 +02:00
Misha Bragin
84879a356b
Extract app metrics to a separate struct (#520) 2022-10-22 11:50:21 +02:00
Misha Bragin
ed2214f9a9
Add HTTP request/response totals to metrics (#519) 2022-10-22 10:07:13 +02:00
braginini
4388dcc20b Listen metrics on all interfaces 2022-10-21 16:50:06 +02:00
Misha Bragin
4f1f0df7d2
Add Open-telemetry support (#517)
This PR brings open-telemetry metrics to the
Management service.
The Management service exposes new HTTP endpoint
/metrics on 8081 port by default.
The port can be changed by specifying
--metrics-port PORT flag when starting the service.
2022-10-21 16:24:13 +02:00
Misha Bragin
08ddf04c5f
Fix IdP tests (#516) 2022-10-19 18:36:10 +02:00
Misha Bragin
b5ee2174a8
Do not set wt_pending_invite when unnecessary (#515)
wt_pending_invite property is set for every user on IdP.
Avoid setting it when unnecessary.
2022-10-19 17:51:41 +02:00
Misha Bragin
7218a3d563
Management single account mode (#511) 2022-10-19 17:43:28 +02:00
Maycon Santos
04e4407ea7
Add anonymous usage metrics collection (#508)
This will help us understand usage on self-hosted deployments

The collection may be disabled by using the flag --disable-anonymous-metrics or 
NETBIRD_DISABLE_ANONYMOUS_METRICS in setup.env
2022-10-16 13:33:46 +02:00
Misha Bragin
06055af361
Super user invites (#483)
This PR brings user invites logic to the Management service
via HTTP API. 
The POST /users/ API endpoint creates a new user in the Idp
and then in the local storage. 
Once the invited user signs ups, the account invitation is redeemed.
There are a few limitations.
This works only with an enabled IdP manager.
Users that already have a registered account can't be invited.
2022-10-13 18:26:31 +02:00
Maycon Santos
b4e03f4616
Feature/add nameservers API endpoint (#491)
Add nameservers endpoint and Open API definition

updated open api generator cli
2022-10-10 11:06:54 +02:00
Maycon Santos
2cd9b11e7d
Add DNS nameserver support to management (#484)
Add DNS package and Nameserver group objects

Add CRUD operations for Nameserver Groups to account manager

Add Routes and Nameservers to Account Copy method

Run docker tests with timeout and serial flags
2022-09-30 16:47:11 +05:00
Maycon Santos
878ca6db22
Check if domain from claim is valid (#485)
If domain is invalid we call GetAccountByUserOrAccountId
2022-09-29 13:51:18 +05:00
Misha Bragin
34c1c7d901
Add hostname, userID, ui version to the HTTP API peer response (#479) 2022-09-26 18:02:45 +02:00
Misha Bragin
051fd3a4d7
Fix Management and Signal gRPC client stream leak (#482) 2022-09-26 18:02:20 +02:00
Misha Bragin
af69a48745
Support user role update (#478) 2022-09-23 14:18:42 +02:00
braginini
7e1ae448e0 Add extra logging to Sync and Login requests 2022-09-22 09:25:31 +02:00
Misha Bragin
518a2561a2
Add auto-assign groups to the User API (#467) 2022-09-22 09:06:32 +02:00
Misha Bragin
6de313070a
Always return empty auto_groups if previously were nil (#468) 2022-09-13 17:19:03 +02:00
Misha Bragin
cd7d1a80c9
Assign groups to peers when registering with the setup key (#466) 2022-09-13 13:39:46 +02:00
Misha Bragin
be7d829858
Add SetupKey auto-groups property (#460) 2022-09-11 23:16:40 +02:00
Maycon Santos
586c0f5c3d
Log remote address when not registered (#445) 2022-08-27 17:55:05 +02:00
Maycon Santos
c13f0b9f07
Use select for turn credentials and peers update (#443)
Also, prevent peer update when SSH is the same
2022-08-27 12:57:03 +02:00
Misha Bragin
dd4ff61b51
Do not autoload authissuer for the IDPManager config (#442) 2022-08-25 09:24:24 +02:00
Misha Bragin
e8733a37af
Update scripts for the self-hosted Oauth 2.0 Device Auth Grant support (#439)
Support Oauth 2.0 Device Auth Grant in the
self-hosted scripts.
2022-08-24 14:37:18 +02:00
Misha Bragin
3def84b111
Support Generic OAuth 2.0 Device Authorization Grant (#433)
Support Generic OAuth 2.0 Device Authorization Grant
as per RFC specification https://www.rfc-editor.org/rfc/rfc8628.
The previous version supported only Auth0 as an IDP backend.
This implementation enables the Interactive SSO Login feature 
for any IDP compatible with the specification, e.g., Keycloak.
2022-08-23 15:46:12 +02:00
Maycon Santos
47add9a9c3
Don't create index if peer is empty (#435)
When checking for existing prefix routes
Return nil if peer is empty
2022-08-23 11:09:56 +02:00
Maycon Santos
09312b3e6d
Add Network ID and rename Prefix to Network (#432)
Adding network ID will allow us to group

Renaming Prefix with Network
will keep things more clear and Consistent
2022-08-22 14:10:24 +02:00
Maycon Santos
000ea72aec
Add routing Rest API support (#428)
Routing API will allow us to list, create, update, and delete routes.
2022-08-20 19:11:54 +02:00
Maycon Santos
4b34a6d6df
Add routing support to management service (#424)
Management will receive and store routes that are associated with a peer ID.
The routes are distributed to peers according to their ACLs.
2022-08-18 18:22:15 +02:00
Misha Bragin
6dc3e8ca90
Enable HTTP/2 when loading TLS config from file (#423)
When creating TLSConfig from provided certificate file, the HTTP/2 support is not enabled.
It works with Certmanager because it adds h2 support.
We enable it the same way when creating TLSConfig from files.
2022-08-15 19:36:00 +02:00
Maycon Santos
14e322d3f7
Handle CORS requests before authentication (#413)
This helps our FE to get proper request responses
2022-08-05 22:41:04 +02:00
Misha Bragin
851de3fd4e
Output NetBird daemon and CLI versions on status command (#408) 2022-08-01 12:42:45 +02:00
Misha Bragin
966661fe91
Serve Management gRPC and HTTP on a single 80/443 port (#400)
This PR is a part of an effort to use standard ports (443 or 80) that are usually allowed by default in most of the environments.

Right now Management Service runs the Let'sEncrypt manager on port 443, HTTP API server on port 33071,
and a gRPC server on port 33073. There are three separate listeners.
This PR combines these listeners into one.
With this change, the HTTP and gRPC server runs on either 443 with TLS or 80 without TLS
by default (no --port specified).
Let's Encrypt manager always runs on port 443 if enabled.
The backward compatibility server runs on port 33073 (with TLS or without).
HTTP port 33071 is obsolete and not used anymore.

Newly installed agents will connect to port 443 by default instead of port 33073 if not specified otherwise.
2022-07-29 20:37:09 +02:00
Misha Bragin
86a66c6202
Make Signal Service listen on a standard 443/80 port instead of 10000 (#396)
Right now Signal Service runs the Let'sEncrypt manager on port 80
and a gRPC server on port 10000. There are two separate listeners.
This PR combines these listeners into one with a cmux lib.
The gRPC server runs on either 443 with TLS or 80 without TLS.
Let's Encrypt manager always runs on port 80.
2022-07-25 19:55:38 +02:00
Misha Bragin
3bdfa3cc8e
Introduce larger retries for the agent (#379)
The Management client will try reconnecting in case.
of network issues or non-permanent errors.
If the device was off-boarded, then the client will stop retrying.
2022-07-02 20:38:16 +02:00
Maycon Santos
e8caa562b0
Send netmask from account network (#369)
* Send netmask from account network

Added the GetPeerNetwork method to account manager

Pass a copy of the network to the toPeerConfig function
to retrieve the netmask from the network instead of constant

updated methods and added test

* check if the network is the same for 2 peers

* Use expect with BeEquivalentTo
2022-06-24 21:30:51 +02:00
Misha Bragin
06860c4c10
NetBird SSH (#361)
This PR adds support for SSH access through the NetBird network
without managing SSH skeys.
NetBird client app has an embedded SSH server (Linux/Mac only) 
and a netbird ssh command.
2022-06-23 17:04:53 +02:00
Maycon Santos
f9f2d7c7ef
Check if new account ID is already being used (#364) 2022-06-20 18:20:43 +02:00
Maycon Santos
503a116f7c
OpenAPI specification and API Adjusts (#356)
Introduced an OpenAPI specification.
Updated API handlers to use the specification types.

Added patch operation for rules and groups
and methods to the account manager.

HTTP PUT operations require id, fail if not provided.

Use snake_case for HTTP request and response body
2022-06-14 10:32:54 +02:00
Misha Bragin
a454a1aa28
Create account in once place (#358)
There are a few places where an account is created.
When we create a new account, there should be
some defaults set. E.g. created by and group ALL.
It makes sense to add it in one place to avoid inconsistencies.
2022-06-09 13:14:34 +02:00
braginini
c9b5328f19 Fix account ALL group creation 2022-06-08 00:30:19 +02:00
Misha Bragin
dab146ed87
Improve Management startup time (#355) 2022-06-06 13:45:59 +02:00
Misha Bragin
0cba0f81e0
Warmup IDP cache on Management start (#354) 2022-06-06 12:05:44 +02:00
Misha Bragin
ebd70a569c
Add caching when querying IDP Manager (#353) 2022-06-05 21:36:42 +02:00
Givi Khojanashvili
d005cd32b0
fix(acl): update each peer's network when rule,group or peer changed (#333)
* fix(acl): update each peer's network when rule,group or peer changed

* fix(ACL): update network test

* fix(acl): cleanup indexes before update them

* fix(acl): clean up rules indexes only for account
2022-06-04 22:02:22 +02:00
Misha Bragin
e6e9f0322f
Handle peer interface config change (#348)
Before this change, NetBird Agent wasn't handling
peer interface configuration changes dynamically.
Also, remote peer configuration changes have
not been applied (e.g. AllowedIPs changed).
Not a very common cause, but still it should be handled.
Now, Agent reacts to PeerConfig changes sent from the
management service and restarts remote connections
if AllowedIps have been changed.
2022-06-04 19:41:01 +02:00
Misha Bragin
60ac8c3268
Handle Network out of range (#347) 2022-06-02 12:56:02 +02:00
Misha Bragin
87631cbc8b
Replace IP allocation logic (#342)
The peer IP allocation logic was allocating sequential peer IP from the 100.64.0.0/10 
address block.
Each account is created with a random subnet from 100.64.0.0/10.
The total amount of potential subnets is 64.
The new logic allocates random peer IP
from the account subnet.
This gives us flexibility to add support for
multi subnet accounts without overlapping IPs.
2022-05-29 22:43:39 +02:00
Misha Bragin
59a964eed8
Change network mask to limit number of peers to 65k (#339) 2022-05-28 12:54:09 +02:00
Misha Bragin
1bb504ea78
Fix peer status Connected when removed from the management (#336) 2022-05-27 15:26:36 +02:00
Maycon Santos
e5c429af1a
Move flags declaration to root (#329)
This allows for mgmtDataDir and mgmtConfig to be initialized properly

use handleMigration function for copying files
2022-05-26 12:55:39 +02:00
shatoboar
94fbfcdb85
Versioning of UI and grpc-agent for passing version (#324)
Send Desktop UI client version as user-agent to daemon

This is sent on every login request to the management

Parse the GRPC context on the system package and 
retrieves the user-agent

Management receives the new UIVersion field and 
store in the Peer's system meta
2022-05-25 23:25:02 +02:00
Givi Khojanashvili
65069c1787
feat(ac): add access control middleware (#321) 2022-05-25 18:26:50 +02:00
Misha Bragin
abe78666d4
Store updated system info on Login to Management (#323) 2022-05-23 13:03:57 +02:00
Maycon Santos
5cbfa4bb9e
Rebrand client cli (#320) 2022-05-22 18:53:47 +02:00
Misha Bragin
3eb230e1a0
Fix Peer Deletion & HTTP endpoints (#319) 2022-05-21 17:27:04 +02:00
Givi Khojanashvili
3ce3ccc39a
Add rules for ACL (#306)
Add rules HTTP endpoint for frontend - CRUD operations.
Add Default rule - allow all.
Send network map to peers based on rules.
2022-05-21 15:21:39 +02:00
shatoboar
77e58295e7
Rename wiretrustee-signal to netbird-signal (#313)
* rename wiretrustee-signal to netbird-signal

* Rename Signal repositories and source bin

* Adjust docker-compose with signal volume [skip ci]

Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
2022-05-13 21:51:41 +02:00
shatoboar
7d893c0238
Rename management from Wiretrustee to Netbird (#311)
Rename documentation and goreleaser build names

Added a migration function for when the old path exists and the new one doesn't

updated the configure.sh to generate the docker-compose with a new path only 
if no pre-existing volume with old name exists
2022-05-13 14:11:21 +02:00
Maycon Santos
e5c52efb4c
Client Login via device authorization flow (#309)
UI and CLI Clients are now able to use SSO login by default

we will check if the management has configured or supports SSO providers

daemon will handle fetching and waiting for an access token

Oauth package was moved to internal to avoid one extra package at this stage

Secrets were removed from OAuth

CLI clients have less and better output

2 new status were introduced, NeedsLogin and FailedLogin for better messaging

With NeedsLogin we no longer have endless login attempts
2022-05-12 11:17:24 +02:00
Maycon Santos
49cca57565
Saving new user to existing account (#310)
Add check if user with
account id metadata belongs to account
2022-05-09 14:30:20 +02:00
Maycon Santos
7e5449fb55
Get Device Authorization Flow information from management (#308)
We will configure the device authorization
flow information and a client will
retrieve it and initiate a
device authorization gran flow
2022-05-08 11:04:57 +02:00
Maycon Santos
fec3132585
Adding peer registration support to JWT (#305)
The management will validate the JWT as it does in the API
 and will register the Peer to the user's account.

New fields were added to grpc messages in management
 and client daemon and its clients were updated

Peer has one new field, UserID, 
that will hold the id of the user that registered it

JWT middleware CheckJWT got a splitter 
and renamed to support validation for non HTTP requests

Added test for adding new Peer with UserID

Lots of tests update because of a new field
2022-05-05 20:02:15 +02:00
shatoboar
c7e5e5c7c9
Add User HTTP Endpoint to the Management service (#303)
Exposes endpoint under "/users/" that returns information on users.
Calls IDP manager to get information not stored locally (email, name), 
which in the case of the managed version is auth0.
2022-05-05 08:58:34 +02:00
Givi Khojanashvili
219888254e
Feat peer groups (#304)
* feat(management): add groups

* squash

* feat(management): add handlers for groups

* feat(management): add handlers for groups

* chore(management): add tests for the get group of the management

* chore(management): add tests for save group
2022-05-03 16:02:51 +02:00
Maycon Santos
c3bc85e22d
Rename module to netbirdio/netbird (#288)
rename the go module to netbirdio/netbird 
as part of our rebranding.
2022-03-26 12:08:54 +01:00
Maycon Santos
ede2795529
Replace Wiretrustee links and naming (#287)
* Replace Wiretrustee links and naming

* Upper case for Netbrid in README

* Replace logo

* Dashboard URL to app.netbird.io

Co-authored-by: Misha Bragin <bangvalo@gmail.com>
2022-03-26 11:39:27 +01:00
Mikhail Bragin
be0c5c887c
Persist Network Serial to Store to avoid outdated netmap sent (#260)
Fix outdated update coming from management
even when it is actually not outdated.
2022-03-10 18:18:38 +01:00
Maycon Santos
c1b162c974
Improve private domain's behavior tests and logic (#256)
Improved the behavior tests for private domains
and its logic as well because on existing accounts
there was no primary status update
2022-03-10 13:47:36 +01:00
Maycon Santos
605ca03519
Fix IDP Manager config structs with correct tags (#253)
* Fix IDP Manager config structs with correct tags

When loading the configuration from file
we will use the Auth0ClientConfig and when
sending the post to retrieve a token
 we use the auth0JWTRequest with proper tags

 Also, removed the idle timeout as it was closing
 all idle connections
2022-03-09 17:22:47 +01:00
Maycon Santos
ff62fec956
Handle category change with provided Acc Id (#252)
When account id supplied via claim, we should
handle change of the domain classification.

If category of domain change to private, we
should re-evaluate the private account
2022-03-09 13:31:42 +01:00
Givi Khojanashvili
ef47385e38
Split client app into cmd and daemon service (#239) 2022-03-08 14:47:55 +01:00
Maycon Santos
0b8387bd2c
Group users of same private domain (#243)
* Added Domain Category field and fix store tests

* Add GetAccountByDomain method

* Add Domain Category to authorization claims

* Initial GetAccountWithAuthorizationClaims test cases

* Renamed Private Domain map and index it on saving account

* New Go build tags

* Added NewRegularUser function

* Updated restore to account for primary domain account

Also, added another test case

* Added grouping user of private domains

Also added auxiliary methods for update metadata and domain attributes

* Update http handles get account method and tests

* Fix lint and document another case

* Removed unnecessary log

* Move use cases to method and add flow comments

* Split the new user and existing logic from GetAccountWithAuthorizationClaims

* Review: minor corrections

Co-authored-by: braginini <bangvalo@gmail.com>
2022-03-01 15:22:18 +01:00
Maycon Santos
b29948b910
Jwtclaims package (#242)
* Move JWTClaims logic to its own package

* Add extractor tests
2022-02-23 20:02:02 +01:00
shatoboar
5f5cbf7e20
Test mgmt http handler (#240) 2022-02-22 18:18:05 +01:00
shatoboar
41c6af6b6f
Extracted AccountManager to interface (#230) 2022-02-22 11:28:19 +01:00
Maycon Santos
852c7c50c0
Fix IDPManagement initialization when no config (#234)
* initialize idpmanage only if config was set

* removed LetsEncryption information for local selfhosted deployments
2022-02-17 19:18:46 +01:00
braginini
765d3a0ad0 fix: account JWT claim 2022-02-15 13:38:22 +01:00
shatoboar
15e371b592
sends wtversion to dashboard-UI (#229) 2022-02-14 17:51:07 +01:00
Maycon Santos
cd9a418df2
Store domain information (#217)
* extract claim information from JWT

* get account function

* Store domain

* tests missing domain

* update existing account with domain

* add store domain tests
2022-02-11 17:18:18 +01:00
shatoboar
b59fd50226
Add client version to the client app and send it to the management service (#222)
* test: WIP mocking the grpc server for testing the sending of the client information

* WIP: Test_SystemMetaDataFromClient with mocks, todo:

* fix: failing meta data test

* test: add system meta expectation in management client test

* fix: removing deprecated register function, replacing with new one

* fix: removing deprecated register function from mockclient interface impl

* fix: fixing interface declaration

* chore: remove unused commented code

Co-authored-by: braginini <bangvalo@gmail.com>
2022-02-08 18:03:27 +01:00
Mikhail Bragin
3c959bb178
Login exits on a single attempt to connect to management (#220)
* fix: login exits on a single attempt to connect to management

* chore: add log verbosity for Login operation
2022-02-06 18:56:00 +01:00
shatoboar
efbb5acf63
Add client version to the client app and send it to the management service (#218)
* moved wiretrustee version from main to system.info

* added wiretrustee version for all supported platforms

* typo corrected

* refactor: use single WiretrusteeVersion() func to get version of the client

Co-authored-by: braginini <bangvalo@gmail.com>
2022-02-03 18:35:54 +01:00
Maycon Santos
0739038d51
Fix unstable parallel tests (#202)
* update interface tests and configuration messages

* little debug

* little debug on both errors

* print all devs

* list of devices

* debug func

* handle interface close

* debug socks

* debug socks

* if ports match

* use random assigned ports

* remove unused const

* close management client connection when stopping engine

* GracefulStop when management clients are closed

* enable workflows on PRs too

* remove iface_test debug code
2022-01-25 09:40:28 +01:00
Maycon Santos
fd7282d3cf
Link account id with the external user store (#184)
* get account id from access token claim

* use GetOrCreateAccountByUser and add test

* correct account id claim

* remove unused account

* Idp manager interface

* auth0 idp manager

* use if instead of switch case

* remove unnecessary lock

* NewAuth0Manager

* move idpmanager to its own package

* update metadata when accountId is not supplied

* update tests with idpmanager field

* format

* new idp manager and config support

* validate if we fetch the interface before converting to string

* split getJWTToken

* improve tests

* proper json fields and handle defer body close

* fix ci lint notes

* documentation and proper defer position

* UpdateUserAppMetadata tests

* update documentation

* ManagerCredentials interface

* Marshal and Unmarshal functions

* fix tests

* ManagerHelper and ManagerHTTPClient

* further tests with mocking

* rename package and custom http client

* sync local packages

* remove idp suffix
2022-01-24 11:21:30 +01:00
Mikhail Bragin
5db130a12e
Support new Management service protocol (NetworkMap) (#193)
* feature: support new management service protocol

* chore: add more logging to track networkmap serial

* refactor: organize peer update code in engine

* chore: fix lint issues

* refactor: extract Signal client interface

* test: add signal client mock

* refactor: introduce Management Service client interface

* chore: place management and signal clients mocks to respective packages

* test: add Serial test to the engine

* fix: lint issues

* test: unit tests for a networkMapUpdate

* test: unit tests Sync update
2022-01-18 16:44:58 +01:00
Maycon Santos
0f7ab4354b
Fix cicd testing issue (#197)
* sync module

* cache per test os

* different port for tests

* wireguard packages versions
2022-01-17 15:10:18 +01:00
Mikhail Bragin
afb302d5e7
Change Management Sync protocol to support incremental (serial) network changes (#191)
* feature: introduce NetworkMap to the management protocol with a Serial ID

* test: add Management Sync method protocol test

* test: add Management Sync method NetworkMap field check [FAILING]

* test: add Management Sync method NetworkMap field check [FAILING]

* feature: fill NetworkMap property to when Deleting peer

* feature: fill NetworkMap in the Sync protocol

* test: code review mentions - GeneratePrivateKey() in the test

* fix: wiretrustee client use wireguard GeneratePrivateKey() instead of GenerateKey()

* test: add NetworkMap test

* fix: management_proto test remove store.json on test finish
2022-01-16 17:10:36 +01:00
Mikhail Bragin
9d1ecbbfb2
Management - add serial to Network reflecting network updates (#179)
* chore: [management] - add account serial ID

* Fix concurrency on the client (#183)

* reworked peer connection establishment logic eliminating race conditions and deadlocks while running many peers

* chore: move serial to Network from Account

* feature: increment Network serial ID when adding/removing peers

* chore: extract network struct init to network.go

* chore: add serial test when adding peer to the account

* test: add ModificationID test on AddPeer and DeletePeer
2022-01-14 14:34:27 +01:00
Mikhail Bragin
a67b9a16af
fix peer update concurrency on the client side (#177)
* fix: gRpc Signal and Management connections deadlock on IDLE state

* fix: client peer update concurrency issues
2021-12-31 18:11:33 +01:00
Mikhail Bragin
6ae27c9a9b
Refactor: support multiple users under the same account (#170)
* feature: add User entity to Account

* test: new file store creation test

* test: add FileStore persist-restore tests

* test: add GetOrCreateAccountByUser Accountmanager test

* refactor: rename account manager users file

* refactor: use userId instead of accountId when handling Management HTTP API

* fix: new account creation for every request

* fix: golint

* chore: add account creator to Account Entity to identify who created the account.

* chore: use xid ID generator for account IDs

* fix: test failures

* test: check that CreatedBy is stored when account is stored

* chore: add account copy method

* test: remove test for non existent GetOrCreateAccount func

* chore: add accounts conversion function

* fix: golint

* refactor: simplify admin user creation

* refactor: move migration script to a separate package
2021-12-27 13:17:15 +01:00
Mikhail Bragin
8c75ef8bef
update to go 1.17 (#167)
* chore: update to go 1.17

* fix: update workflows go version

* fix: golint errors/update grpc
2021-12-21 10:02:25 +01:00
Mikhail Bragin
ed1e4dfc51
refactor signal client sync func (#147)
* refactor: move goroutine that runs Signal Client Receive to the engine for better control

* chore: fix comments typo

* test: fix golint

* chore: comments update

* chore: consider connection state=READY in signal and management clients

* chore: fix typos

* test: fix signal ping-pong test

* chore: add wait condition to signal client

* refactor: add stream status to the Signal client

* refactor: defer mutex unlock
2021-11-06 15:00:13 +01:00
braginini
4d34fb4e64 chore: decrease backoff maxinterval to avoid long connection waiting times on the client app 2021-11-02 14:51:29 +01:00
Mikhail Bragin
d040cfed7e
fix: client app retry logic (#144)
* fix: retry logic
2021-11-01 09:34:06 +01:00
braginini
e9066b4651 chore: increase signal and management gRPC clients timeouts 2021-10-31 12:14:00 +01:00
Mikhail Bragin
673e807528
chore: set default key expiration if not provided by frontednd (#142) 2021-10-31 12:06:44 +01:00
Mikhail Bragin
74485d3b13
fix: service hanging when error on startup has been encountered (#135) 2021-10-18 13:29:26 +02:00
Mikhail Bragin
bef3b3392b
fix: graceful shutdown (#134)
* fix: graceful shutdown

* fix: windows graceful shutdown
2021-10-17 22:15:38 +02:00
Mikhail Bragin
32c369257b
management/support cert from file (#122)
* feature: support cert file in management service

* docs: add new management commands
2021-09-25 19:22:49 +02:00
Mikhail Bragin
ec759bc461
Delete peer (#114)
* feature: add peer deletion

* feature: add peer deletion [CLIENT]

* fix: lint error

* test: fix sync block

* test: fix management test

* feature: add client stop after was deleted

* chore: remove permission denied cancellation

* chore: add larger signal backoff

* feature: notify deleted peer of removal

* fix: lint issue

* chore: add 2nd default key - one off

* test: fix account key check
2021-09-07 18:36:46 +02:00
braginini
081162864d fix: management config 2021-09-07 10:35:33 +02:00
Mikhail Bragin
13b4be31df
feature: add logging to a file (#112)
* feature: add logging to a file

* refactor: move InitLog to util lib

* docs: update signal and management docs

* chore: update docker compose

* set --log-file to console

* chore: comment out log volume in docker compose

Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
2021-09-07 09:53:18 +02:00
Mikhail Bragin
a4db0b4e94
client update of TURNs and STUNs (#106)
* feature: update STUNs and TURNs in engine

* fix: setup TURN credentials request only when refresh enabled

* feature: update TURNs and STUNs in teh client app on Management update

* chore: disable peer reflexive candidates in ICE

* chore: relocate management.json

* chore: make TURN secret and pwd plain text in config
2021-09-03 17:47:40 +02:00
Mikhail Bragin
b17424d630
Turn credentials generation (#102)
* abstract peer channel

* remove wip code

* refactor NewServer with Peer updates channel

* feature: add TURN credentials manager

* hmac logic

* example test function

* test: add TimeBasedAuthSecretsManager_GenerateCredentials  test

* test: make tests for now with hardcoded secret

* test: add TimeBasedAuthSecretsManager_SetupRefresh test

* test: add TimeBasedAuthSecretsManager_SetupRefresh test

* test: add TimeBasedAuthSecretsManager_CancelRefresh test

* feature: extract TURNConfig to the management config

* feature: return hash based TURN credentials only on initial sync

* feature: make TURN time based secret credentials optional

Co-authored-by: mlsmaycon <mlsmaycon@gmail.com>
2021-09-02 14:41:54 +02:00
Maycon Santos
a31cbb1f5b
abstract peer channel (#101)
* abstract peer channel

* remove wip code

* refactor NewServer with Peer updates channel

* add PeersUpdateManager tests

* adding documentation

* using older version of linter

* verbose lint

* skip cache

* setup go version

* extra output

* configure fetch-depth

* exit 0

* skip-build-cache: true

* disabling failure for lint for now

* fix: darwin issue

* enable lint failure

* remove sock file for macOS

* refactor: remove tests interdependence

* fixed linux native iface

Co-authored-by: braginini <bangvalo@gmail.com>
2021-08-29 17:48:31 +02:00
braginini
5c7260298f chore: adjust system info discovery methods 2021-08-27 11:45:46 +02:00
braginini
49800a6d03 fix: minor HTTP bugs 2021-08-25 14:16:17 +02:00
Mikhail Bragin
0fa15e6920
Add peer meta data (#95)
* feature: add peer GET and DELETE API methods

* refactor: extract peer business logic to a separate file

* refactor: extract peer business logic to a separate file

* feature: add peer update HTTP endpoint

* chore: fill peer new fields

* merge with main

* refactor: HTTP methods according to standards

* feature: add peer system metadata

* feature: add peer status

* test: fix removal
2021-08-24 11:50:19 +02:00
Mikhail Bragin
95845c88fe
Extend peer http endpoint (#94)
* feature: add peer GET and DELETE API methods

* refactor: extract peer business logic to a separate file

* refactor: extract peer business logic to a separate file

* feature: add peer update HTTP endpoint

* chore: fill peer new fields

* merge with main

* refactor: HTTP methods according to standards

* chore: setup keys POST endpoint without ID
2021-08-23 21:43:05 +02:00
Mikhail Bragin
6869b48905
feature: increase key usage after successful peer registration (#93) 2021-08-22 11:29:25 +02:00
braginini
90ef1e939b fix: test 2021-08-20 23:12:39 +02:00
braginini
bff137b109 fix: test 2021-08-20 22:37:58 +02:00
braginini
2e9fc20567 feature: add update setup key endpoint 2021-08-20 22:33:43 +02:00
braginini
b5a20bf1ba chore: remove unused static dashboard files 2021-08-20 16:25:51 +02:00
braginini
07ab9c196d fix: golint errors 2021-08-20 15:51:29 +02:00
braginini
4a5901ada1 test: add more AccountManager tests 2021-08-20 15:44:18 +02:00
braginini
4c427ae900 test: add AddAccount test 2021-08-20 15:18:29 +02:00
braginini
22fdb0a029 test: add setup key UUID format validation 2021-08-20 15:08:07 +02:00
braginini
1b056ab75a test: add setup key tests 2021-08-20 15:01:57 +02:00
braginini
708835afa8 test: change test data to support new setup key fields 2021-08-20 13:40:07 +02:00
braginini
02cc6a30f5 feature: extend setup key logic 2021-08-19 21:12:21 +02:00
braginini
c68d9dff4a chore: setup key upper case 2021-08-19 20:07:12 +02:00
Mikhail Bragin
1dfa99d07c
add wiretrustee LOGIN command (#90)
* feature: add wiretrustee LOGIN command

* chore: add management initial connection timeout

* test: add login cmd test

* test: validate generated config in login cmd

* test: add up command test

* chore: add timeout to signal client creation method

* test: close wireguard interface once test finished
2021-08-18 13:35:42 +02:00
Mikhail Bragin
e8ca289f4a
test: add management client tests (#89)
* test: add management client tests

* test: add management client Sync test

* fix: engine test

* test: remove engine tests

* test: return engine tests [check]

* test: remove engine tests [check]
2021-08-16 23:30:51 +02:00
Maycon Santos
38e3c9c062
Add cors headers (#85)
* disable EnableAuthOnOptions

* Setup basic cors headers

* feature: user cors lib
2021-08-16 11:29:57 +02:00
Mikhail Bragin
877ad97a96
Peer management login (#83)
* feature: replace RegisterPeer with Login method that does both - registration and login

* test: add management login test

* feature: add WiretrusteeConfig to the Login response to configure peer global config

* feature: add client peer login support

* fix: missing parts

* chore: update go deps

* feature: support Management Service gRPC endpoints [CLIENT]

* feature: finalize client sync with management

* fix: management store peer key lower case restore

* fix: management returns peer ip without a mask

* refactor: remove cmd pkg

* fix: invalid tun interface name on mac

* fix: timeout when calling management client

* fix: tests and lint errors

* fix: golang-test workflow

* fix: client service tests

* fix: iface build

* feature: detect management scheme on startup

* chore: better logs for management

* fix: goreleaser

* fix: lint errors

* fix: signal TLS

* fix: direct Wireguard connection

* chore: verbose logging on direct connection
2021-08-15 16:56:26 +02:00
braginini
dcc9dcacdc feature: add setup key HTTP api response fields 2021-08-12 13:24:06 +02:00
Mikhail Bragin
3c47a3c408
peer management HTTP API (#81)
* feature: create account for a newly registered user

* feature: finalize user auth flow

* feature: create protected API with JWT

* chore: cleanup http server

* feature: add UI assets

* chore: update react UI

* refactor: move account not exists -> create to AccountManager

* chore: update UI

* chore: return only peers on peers endpoint

* chore: add UI path to the config

* chore: remove ui from management

* chore: remove unused Docker comamnds

* docs: update management config sample

* fix: store creation

* feature: introduce peer response to the HTTP api

* fix: lint errors

* feature: add setup-keys HTTP endpoint

* fix: return empty json arrays in HTTP API

* feature: add new peer response fields
2021-08-12 12:49:10 +02:00
braginini
9f0c86c28e refactor: move grpc and http APIs to separate packages 2021-08-07 13:51:17 +02:00
braginini
08d44b1d5f refactor: move LetsEncryptDomain to HttpServer config 2021-08-07 13:35:52 +02:00
Mikhail Bragin
1f29975737
feature: basic auth0 support (#78)
* feature: basic auth0 support

* refactor: improve auth flow

* refactor: extract HttpServer config

* feature: merge HTTP API layer with Let's Encrypt
2021-08-07 12:26:07 +02:00
andpar83
11982d6dde
Add client's interaction with management service (#71)
* Add client's interaction with management service

* Getting updates

* Fixed key and nil ptr

* Added setupKey param

* Added managment address parameter

* Fixed test

* feature: use RemotePeers from the management server instead of deprecated Peers

* merge: merge changes from main
2021-08-01 19:06:01 +02:00
Maycon Santos
ea99def502
Update mgmt binary name and config doc (#75)
* using wiretrustee-mgmt for binary name

* using wiretrustee-mgmt

* updated documentation and compose files to use config.json
2021-07-31 12:33:04 +02:00
Maycon Santos
f51a79d3b3
Mgmt docker and document (#72)
* debug image and use wiretrustee/management repository

* Update documentation and docker-compose to include management

* improve documentation and add debug image build

* update docker-compose section with management service notes.

* fix broken doc link
2021-07-31 10:29:49 +02:00
Mikhail Bragin
2c2c1e19df
Peer configuration management (#69)
* feature: add config properties to the SyncResponse of the management gRpc service

* fix: lint errors

* chore: modify management protocol according to the review notes

* fix: management proto fields sequence

* feature: add proper peer configuration to be synced

* chore: minor changes

* feature: finalize peer config management

* fix: lint errors

* feature: add management server config file

* refactor: extract hosts-config to a separate file

* refactor: review notes applied to correct file_store usage

* refactor: extract management service configuration to a file

* refactor: simplify management config
2021-07-30 17:46:38 +02:00
Mikhail Bragin
3b30beb567
add config properties to the SyncResponse of the management gRpc service (#66)
* feature: add config properties to the SyncResponse of the management gRpc service
2021-07-25 17:08:16 +02:00
andpar83
9e4aa4f1f1
Move management server to a separate directory (#67)
* Move management server to a separate directory
2021-07-24 16:14:29 +02:00
braginini
83ac774264 test: fix management multiple concurrent peers test 2021-07-22 15:53:15 +02:00
Mikhail Bragin
2172d6f1b9
Extract common server encryption logic (#65)
* refactor: extract common message encryption logic
* refactor: move letsencrypt logic to common
* refactor: rename common package to encryption
* test: add encryption tests
2021-07-22 15:23:24 +02:00
braginini
c98be683bf docs: add management service docs 2021-07-22 12:32:04 +02:00
Mikhail Bragin
079d35eada
Extend Management to support peer changes distribution (#55)
* feature: add peer sync and a server public key endpoints
* test: add Management.Sync() gRpc endpoint test
* feat: implement peer sync
* docs: added some comments to the Management server
* chore: use for loop over channel when monitoring peer updates
* fix: exit infinite loop when sending updates to peers
* test: add multiple concurrent peers test for management service
* chore: remove unused test
* fix: reduce the amount peers for a concurrent peer update test

Co-authored-by: braginini <m.bragin@wiretrustee.com>
2021-07-22 10:28:00 +02:00
braginini
940578d600 chore: use latest golang-grpc libs 2021-07-20 18:09:26 +02:00
braginini
4e17890597 docs: minor FilesStore corrections 2021-07-18 21:00:32 +02:00
andpar83
7b52049333
Improve addition of new peers in Management service. (#56)
* Store refactoring
* Improve addition of new peers in Management service.
2021-07-18 20:51:09 +02:00
braginini
2d85fcfcc3 refactor: set default flags in code not Dockerfile 2021-07-17 17:26:51 +02:00
braginini
07118d972d chore: change default datadir 2021-07-17 15:47:16 +02:00
braginini
d4a9f4d38a feature: add letsencrypt support to the management service 2021-07-17 14:51:16 +02:00
Mikhail Bragin
4587f7686e
feature: basic management service implementation (#44)
* feat: basic management service implementation [FAILING TESTS]

* test: fix healthcheck test

* test: #39 add peer registration endpoint test

* feat: #39 add setup key handling

* feat: #39 add peer management store persistence

* refactor: extract config read/write to the utility package

* refactor: move file contents copy to the utility package

* refactor: use Accounts instead of Users in the Store

* feature: add management server Docker file

* refactor: introduce datadir instead of config

* chore: use filepath.Join to concat filepaths instead of string concat

* refactor: move stop channel to the root

* refactor: move stop channel to the root

* review: fix PR review notes

Co-authored-by: braginini <hello@wiretrustee.com>
2021-07-17 14:38:59 +02:00