Commit Graph

75 Commits

Author SHA1 Message Date
pascal-fischer
158936fb15
[management] Remove file store (#2689) 2024-10-03 15:50:35 +02:00
Bethuel Mmbaga
acb73bd64a
[management] Remove redundant get account calls in GetAccountFromToken (#2615)
* refactor access control middleware and user access by JWT groups

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor jwt groups extractor

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor handlers to get account when necessary

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor getAccountFromToken

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor getAccountWithAuthorizationClaims

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* revert handles change

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* remove GetUserByID from account manager

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor getAccountWithAuthorizationClaims to return account id

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor handlers to use GetAccountIDFromToken

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* remove locks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add GetGroupByName from store

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add GetGroupByID from store and refactor

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor retrieval of policy and posture checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor user permissions and retrieves PAT

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor route, setupkey, nameserver and dns to get record(s) from store

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor store

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix lint

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix add missing policy source posture checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add store lock

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add get account

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
2024-09-27 17:10:50 +03:00
pascal-fischer
6c50b0c84b
[management] Add transaction to addPeer (#2469)
This PR removes the GetAccount and SaveAccount operations from the AddPeer and instead makes use of gorm.Transaction to add the new peer.
2024-09-16 15:47:03 +02:00
pascal-fischer
049b5fb7ed
Split DB calls in peer login (#2439) 2024-08-19 12:50:11 +02:00
Maycon Santos
165988429c
Add write lock for peer when saving its connection status (#2359) 2024-07-31 14:53:32 +02:00
Maycon Santos
1f48fdf6ca
Add SavePeer method to prevent a possible account inconsistency (#2296)
SyncPeer was storing the account with a simple read lock

This change introduces the SavePeer method to the store to be used in these cases
2024-07-26 07:49:05 +02:00
Bethuel Mmbaga
1537b0f5e7
Add batch save/update for groups and users (#2245)
* Add functionality to update multiple users

* Remove SaveUsers from DefaultAccountManager

* Add SaveGroups method to AccountManager interface

* Refactoring

* Add SaveUsers and SaveGroups methods to store interface

* Refactor method SaveAccount to SaveUsers and SaveGroups

The method SaveAccount in user.go and group.go files was split into two separate methods. Now, user-specific data is handled by SaveUsers and group-specific data is handled by SaveGroups method. This provides a cleaner and more efficient way to save user and group data.

* Add account ID to user and group in SqlStore

* Refactor SaveUsers and SaveGroups in store

* Remove unnecessary ID assignment in SaveUsers and SaveGroups
2024-07-15 17:04:06 +03:00
pascal-fischer
765aba2c1c
Add context to throughout the project and update logging (#2209)
propagate context from all the API calls and log request ID, account ID and peer ID

---------

Co-authored-by: Zoltan Papp <zoltan.pmail@gmail.com>
2024-07-03 11:33:02 +02:00
pascal-fischer
521f7dd39f
Improve login performance (#2061) 2024-05-31 16:41:12 +02:00
pascal-fischer
012235ff12
Add FindExistingPostureCheck (#2075) 2024-05-30 15:22:42 +02:00
Bethuel Mmbaga
a5811a2d7d
Implement experimental PostgreSQL store (#1939)
* migrate sqlite store to
 generic sql store

* fix conflicts

* init postgres store

* Add postgres store tests

* Refactor postgres store engine name

* fix tests

* Run postgres store tests on linux only

* fix tests

* Refactor

* cascade policy rules on policy deletion

* fix tests

* run postgres cases in new db

* close store connection after tests

* refactor

* using testcontainers

* sync go sum

* remove postgres service

* remove store cleanup

* go mod tidy

* remove env

* use postgres as engine and initialize test store with testcontainer

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-05-16 19:28:37 +03:00
pascal-fischer
2e0047daea
Improve Sync performance (#1901) 2024-05-07 14:30:03 +02:00
Zoltan Papp
2d76b058fc
Feature/peer validator (#1553)
Follow up management-integrations changes

move groups to separated packages to avoid circle dependencies
save location information in Login action
2024-03-27 18:48:48 +01:00
Viktor Liu
199bf73103
Remove usage stats (#1665) 2024-03-05 09:45:32 +01:00
Viktor Liu
b7a6cbfaa5
Add account usage logic (#1567)
---------

Co-authored-by: Yury Gargay <yury.gargay@gmail.com>
2024-02-22 12:27:08 +01:00
Yury Gargay
9bc7b9e897
Add initial support of device posture checks (#1540)
This PR implements the following posture checks:

* Agent minimum version allowed
* OS minimum version allowed
* Geo-location based on connection IP

For the geo-based location, we rely on GeoLite2 databases which are free IP geolocation databases. MaxMind was tested and we provide a script that easily allows to download of all necessary files, see infrastructure_files/download-geolite2.sh.

The OpenAPI spec should extensively cover the life cycle of current version posture checks.
2024-02-20 09:59:56 +01:00
Yury Gargay
db3cba5e0f
Remove Account.Rules from Store engines (#1528) 2024-02-19 17:17:36 +01:00
pascal-fischer
141065f14e
Merge branch 'main' into feature/peer-approval 2023-11-29 16:27:01 +01:00
Maycon Santos
c2eaf8a1c0
Add account deletion endpoint (#1331)
Adding support to account owners to delete an account

This will remove all users from local, and if --user-delete-from-idp is set it will remove from the remote IDP
2023-11-28 14:23:38 +01:00
Pascal Fischer
a729c83b06 extract peer into seperate package 2023-11-28 13:45:26 +01:00
Bethuel Mmbaga
c38d65ef4c
Extends management user and group structure (#1268)
* extends user and group structure by introducing fields for issued and integration references

* Add integration checks to group management to prevent groups added by integration.

* Add integration checks to user management to prevent deleting user added by integration.

* Fix broken user update tests

* Initialize all user fields for testing

* Change a serializer option to embedded for IntegrationReference in user and group models

* Add issued field to user api response

* Add IntegrationReference to Group in update groups handler

* Set the default issued field for users in file store
2023-11-01 13:04:17 +03:00
Yury Gargay
46f5f148da
Move StoreKind under own StoreConfig configuration and rename to Engine (#1219)
* Move StoreKind under own StoreConfig configuration parameter

* Rename StoreKind option to Engine

* Rename StoreKind internal methods and types to Engine

* Add template engine value test

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2023-10-16 11:19:39 +02:00
Yury Gargay
32880c56a4
Implement SQLite Store using gorm and relational approach (#1065)
Restructure data handling for improved performance and flexibility. 
Introduce 'G'-prefixed fields to represent Gorm relations, simplifying resource management. 
Eliminate complexity in lookup tables for enhanced query and write speed. 
Enable independent operations on data structures, requiring adjustments in the Store interface and Account Manager.
2023-10-12 15:42:36 +02:00
Yury Gargay
659110f0d5
Rework peer connection status based on the update channel existence (#1213)
With this change, we don't need to update all peers on startup. We will
check the existence of an update channel when returning a list or single peer on API.
Then after restarting of server consumers of API will see peer not
connected status till the creation of an updated channel which indicates
peer successful connection.
2023-10-11 18:11:45 +02:00
pascal-fischer
da75a76d41
Adding dashboard login activity (#1092)
For better auditing this PR adds a dashboard login event to the management service.

For that the user object was extended with a field for last login that is not actively saved to the database but kept in memory until next write. The information about the last login can be extracted from the JWT claims nb_last_login. This timestamp will be stored and compared on each API request. If the value changes we generate an event to inform about a login.
2023-08-18 19:23:11 +02:00
Yury Gargay
97b6e79809
Fix DefaultAccountManager GetGroupsFromTheToken false positive tests (#1019)
This fixes the test logic creates copy of account with empty id and
re-pointing the indices to it.

Also, adds additional check for empty ID in SaveAccount method of FileStore.
2023-07-22 15:54:08 +04:00
Yury Gargay
7ddde41c92 Do not persist filestore when deleting indices
As both TokenID2UserID and HashedPAT2TokenID are in-memory indices and
not stored in the file.
2023-07-17 11:52:45 +02:00
Givi Khojanashvili
8b619a8224
JWT Groups support (#966)
Get groups from the JWT tokens if the feature enabled for the account
2023-06-27 18:51:05 +04:00
Givi Khojanashvili
ba7a39a4fc
Feat linux firewall support (#805)
Update the client's engine to apply firewall rules received from the manager (results of ACL policy).
2023-05-29 16:00:18 +02:00
Misha Bragin
03a42de5a0
Add telemetry to measure app durations (#878) 2023-05-19 11:42:25 +02:00
Givi Khojanashvili
0343c5f239
Rollback simple ACL rules processing. (#803) 2023-04-12 09:39:17 +02:00
Pascal Fischer
6aba28ccb7 remove UTC from some not store related operations 2023-04-10 10:54:23 +02:00
Pascal Fischer
489892553a use UTC everywhere in server 2023-04-03 15:09:35 +02:00
Givi Khojanashvili
5dc0ff42a5
Fix broken auto-generated Rego rule (#769)
Default Rego policy generated from the rules in some cases is broken.
This change fixes the Rego template for rules to generate policies.

Also, file store load constantly regenerates policy objects from rules.
It allows updating/fixing of the default Rego template during releases.
2023-04-01 12:02:08 +02:00
Pascal Fischer
4ec6d5d20b remove debug logs 2023-03-29 18:23:10 +02:00
Pascal Fischer
c5942e6b33 store hashed token base64 encoded 2023-03-29 15:21:53 +02:00
Pascal Fischer
82af60838e use "ok" convention for check variables throughout files_store 2023-03-21 14:00:59 +01:00
Pascal Fischer
41a47be379 add function comments, implement account mock functions and added error handling in tests 2023-03-20 16:38:17 +01:00
Pascal Fischer
e30def175b switch PATs to map and add deletion 2023-03-20 16:14:55 +01:00
Pascal Fischer
e1ef091d45 remove unnecessary string conversion 2023-03-20 12:08:01 +01:00
Pascal Fischer
b8cab2882b storing and retrieving PATs 2023-03-16 15:57:44 +01:00
Givi Khojanashvili
2a1efbd0fd
Don't drop Rules from file storage after migration to Policies (#741)
Rego policy migration clears the rules property of the file storage, but it does not allow rollback management upgrade, so this changes pre-saves rules in the file store and updates it from the policies.
2023-03-15 09:42:40 +04:00
Givi Khojanashvili
3bfa26b13b
Feat rego default policy (#700)
Converts rules to Rego policies and allow users to write raw policies to set up connectivity and firewall on the clients.
2023-03-13 18:14:18 +04:00
Misha Bragin
9ce8056b17
Use global login expiration setting when sending network map (#731)
Peers were considered expired and not sent to remote peers
when global expiration was disabled.
2023-03-09 11:24:42 +01:00
Misha Bragin
1bda8fd563
Remove stale peer indices when getting peer by key after removing (#711)
When we delete a peer from an account, we save the account in the file store.
The file store maintains peerID -> accountID and peerKey -> accountID indices.
Those can't be updated when we delete a peer because the store saves the whole account
without a peer already and has no access to the removed peer.
In this PR, we dynamically check if there are stale indices when GetAccountByPeerPubKey
and GetAccountByPeerID.
2023-03-01 12:11:32 +01:00
Misha Bragin
d5dfed498b
Add account settings (#686)
Add account settings with a global peer expiration flag and duration
2023-02-13 15:07:15 +01:00
Misha Bragin
3fc89749c1
Add peer login expiration (#682)
This PR adds a peer login expiration logic that requires
peers created by a user to re-authenticate (re-login) after
a certain threshold of time (24h by default).

The Account object now has a PeerLoginExpiration
property that indicates the duration after which a peer's
login will expire and a login will be required. Defaults to 24h.

There are two new properties added to the Peer object:
LastLogin that indicates the last time peer successfully used
the Login gRPC endpoint and LoginExpirationEnabled that
enables/disables peer login expiration.

The login expiration logic applies only to peers that were created
by a user and not those that were added with a setup key.
2023-02-13 12:21:02 +01:00
Misha Bragin
9adadfade4
Use Peer.ID instead of Peer.Key as peer identifier (#664)
Replace Peer.Key as internal identifier with a randomly generated Peer.ID 
in the Management service.
Every group now references peers by ID instead of a public key.
Every route now references peers by ID instead of a public key.
FileStore does store.json file migration on startup by generating Peer.ID and replacing
all Peer.Key identifier references .
2023-02-03 10:33:28 +01:00
Maycon Santos
a387e3cfc2
Add network routes distribution groups (#606)
Updated tests, API, and account manager methods

Sync routes to peers in the distribution groups

Added store upgrade by adding the All group to routes that don't have them
2022-12-06 10:11:57 +01:00
Misha Bragin
509d23c7cf
Replace gRPC errors in business logic with internal ones (#558) 2022-11-11 20:36:45 +01:00