bitwarden/server
1
0
Fork 0
mirror of https://github.com/bitwarden/server synced 2025-12-22 19:23:45 +00:00
Code Issues Releases Wiki Activity

[PM-25015] Add performance tests for Admin Console endpoints (#6235)

Browse Source 
* Add GroupsRecipe to manage group creation and user relationships in organizations

* Add CollectionsRecipe to manage collection creation and user relationships in organizations

* Refactor OrganizationUsersControllerPerformanceTests to enhance performance testing and add new test cases

* Add OrganizationDomainRecipe to add verified domains for organizations

* Add more tests to OrganizationUsersControllerPerformanceTests and enhance seeding logic for organizations

- Updated performance tests to use dynamic domain generation for organization users.
- Refactored seeding methods in OrganizationWithUsersRecipe to accept user status and type.
- Modified AddToOrganization methods in CollectionsRecipe and GroupsRecipe to return created IDs.
- Adjusted DbSeederUtility to align with new seeding method signatures.

* Enhance OrganizationSeeder with additional configuration options and update seat calculation in OrganizationWithUsersRecipe to ensure a minimum of 1000 seats.

* Add performance tests for Groups, Organizations, Organization Users, and Provider Organizations controllers

- Introduced `GroupsControllerPerformanceTests` to validate the performance of the PutGroupAsync method.
- Added `OrganizationsControllerPerformanceTests` with multiple tests including DeleteOrganizationAsync, DeleteOrganizationWithTokenAsync, PostStorageAsync, and CreateWithoutPaymentAsync.
- Enhanced `OrganizationUsersControllerPerformanceTests` with DeleteSingleUserAccountAsync and InviteUsersAsync methods to test user account deletion and bulk invitations.
- Created `ProviderOrganizationsControllerPerformanceTests` to assess the performance of deleting provider organizations.

These tests ensure the reliability and efficiency of the respective controller actions under various scenarios.

* Refactor GroupsControllerPerformanceTests to use parameterized tests

- Renamed `GroupsControllerPerformanceTest` to `GroupsControllerPerformanceTests` for consistency.
- Updated `PutGroupAsync` method to use `[Theory]` with `InlineData` for dynamic user and collection counts.
- Adjusted organization user and collection seeding logic to utilize the new parameters.
- Enhanced logging to provide clearer performance metrics during tests.

* Update domain generation in GroupsControllerPerformanceTests for improved test consistency

* Remove ProviderOrganizationsControllerPerformanceTests

* Refactor performance tests for Groups, Organizations, and Organization Users controllers

- Updated method names for clarity and consistency, e.g., `PutGroupAsync` to `UpdateGroup_WithUsersAndCollections`.
- Enhanced test documentation with XML comments to describe the purpose of each test.
- Improved domain generation logic for consistency across tests.
- Adjusted logging to provide detailed performance metrics during test execution.
- Renamed several test methods to better reflect their functionality.

* Refactor performance tests in Organizations and Organization Users controllers

- Updated tests to use parameterized `[Theory]` attributes with `InlineData` for dynamic user, collection, and group counts.
- Enhanced logging to include detailed metrics such as user and collection counts during test execution.
- Marked several tests as skipped for performance considerations.
- Removed unused code and improved organization of test methods for clarity.

* Add bulk reinvite users performance test to OrganizationUsersControllerPerformanceTests

- Implemented a new performance test for the POST /organizations/{orgId}/users/reinvite endpoint.
- Utilized parameterized testing with `[Theory]` and `InlineData` to evaluate performance with varying user counts.
- Enhanced logging to capture request duration and response status for better performance insights.
- Updated OrganizationSeeder to conditionally set email based on user status during seeding.

* Refactor domain generation in performance tests to use OrganizationTestHelpers

- Updated domain generation logic in GroupsControllerPerformanceTests, OrganizationsControllerPerformanceTests, and OrganizationUsersControllerPerformanceTests to utilize the new GenerateRandomDomain method from OrganizationTestHelpers.
- This change enhances consistency and readability across the tests by centralizing domain generation logic.

* Update CollectionsRecipe to have better readability

* Update GroupsRecipe to have better readability

* Refactor authentication in performance tests to use centralized helper method. This change reduces code duplication across Groups, Organizations, and OrganizationUsers controller tests by implementing the `AuthenticateClientAsync` method in a new `PerformanceTestHelpers` class.

* Refactor OrganizationUsersControllerPerformanceTests to filter organization users by OrganizationId.

* Refactor CreateOrganizationUser method to improve handling of user status and key assignment based on invitation and confirmation states.

* Add XML documentation for CreateOrganizationUser method to clarify user status handling
This commit is contained in:
Rui Tomé
2025-12-05 14:22:00 +00:00
committed by GitHub
parent 3605b4d2ff
commit 80ee31b4fe
11 changed files with 1124 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
util/DbSeederUtility/Program.cs
View File

@@ -34,6 +34,6 @@ public class Program
var db = scopedServices.GetRequiredService<DatabaseContext>();
var recipe = new OrganizationWithUsersRecipe(db);
recipe.Seed(name, users, domain);
recipe.Seed(name: name, domain: domain, users: users);
}
}

46
util/Seeder/Factories/OrganizationSeeder.cs
View File

@@ -17,7 +17,31 @@ public class OrganizationSeeder
Plan = "Enterprise (Annually)",
PlanType = PlanType.EnterpriseAnnually,
Seats = seats,
UseCustomPermissions = true,
UseOrganizationDomains = true,
UseSecretsManager = true,
UseGroups = true,
UseDirectory = true,
UseEvents = true,
UseTotp = true,
Use2fa = true,
UseApi = true,
UseResetPassword = true,
UsePasswordManager = true,
UseAutomaticUserConfirmation = true,
SelfHost = true,
UsersGetPremium = true,
LimitCollectionCreation = true,
LimitCollectionDeletion = true,
LimitItemDeletion = true,
AllowAdminAccessToAllCollectionItems = true,
UseRiskInsights = true,
UseAdminSponsoredFamilies = true,
SyncSeats = true,
Status = OrganizationStatusType.Created,
//GatewayCustomerId = "example-customer-id",
//GatewaySubscriptionId = "example-subscription-id",
MaxStorageGb = 10,
// Currently hardcoded to the values from https://github.com/bitwarden/sdk-internal/blob/main/crates/bitwarden-core/src/client/test_accounts.rs.
// TODO: These should be dynamically generated by the SDK.
PublicKey = "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmIJbGMk6eZqVE7UxhZ46Weu2jKciqOiOkSVYtGvs61rfe9AXxtLaaZEKN4d4DmkZcF6dna2eXNxZmb7U4pwlttye8ksqISe6IUAZQox7auBpjopdCEPhKRg3BD/u8ks9UxSxgWe+fpebjt6gd5hsl1/5HOObn7SeU6EEU04cp3/eH7a4OTdXxB8oN62HGV9kM/ubM1goILgjoSJDbihMK0eb7b8hPHwcA/YOgKKiu/N3FighccdSMD5Pk+HfjacsFNZQa2EsqW09IvvSZ+iL6HQeZ1vwc/6TO1J7EOfJZFQcjoEL9LVI693efYoMZSmrPEWziZ4PvwpOOGo6OObyMQIDAQAB",
@@ -28,17 +52,25 @@ public class OrganizationSeeder
public static class OrgnaizationExtensions
{
public static OrganizationUser CreateOrganizationUser(this Organization organization, User user)
/// <summary>
/// Creates an OrganizationUser with fields populated based on status.
/// For Invited status, only user.Email is used. For other statuses, user.Id is used.
/// </summary>
public static OrganizationUser CreateOrganizationUser(
this Organization organization, User user, OrganizationUserType type, OrganizationUserStatusType status)
{
var isInvited = status == OrganizationUserStatusType.Invited;
var isConfirmed = status == OrganizationUserStatusType.Confirmed || status == OrganizationUserStatusType.Revoked;
return new OrganizationUser
{
Id = Guid.NewGuid(),
OrganizationId = organization.Id,
UserId = user.Id,
Key = "4.rY01mZFXHOsBAg5Fq4gyXuklWfm6mQASm42DJpx05a+e2mmp+P5W6r54WU2hlREX0uoTxyP91bKKwickSPdCQQ58J45LXHdr9t2uzOYyjVzpzebFcdMw1eElR9W2DW8wEk9+mvtWvKwu7yTebzND+46y1nRMoFydi5zPVLSlJEf81qZZ4Uh1UUMLwXz+NRWfixnGXgq2wRq1bH0n3mqDhayiG4LJKgGdDjWXC8W8MMXDYx24SIJrJu9KiNEMprJE+XVF9nQVNijNAjlWBqkDpsfaWTUfeVLRLctfAqW1blsmIv4RQ91PupYJZDNc8nO9ZTF3TEVM+2KHoxzDJrLs2Q==",
Type = OrganizationUserType.Admin,
Status = OrganizationUserStatusType.Confirmed
UserId = isInvited ? null : user.Id,
Email = isInvited ? user.Email : null,
Key = isConfirmed ? "4.rY01mZFXHOsBAg5Fq4gyXuklWfm6mQASm42DJpx05a+e2mmp+P5W6r54WU2hlREX0uoTxyP91bKKwickSPdCQQ58J45LXHdr9t2uzOYyjVzpzebFcdMw1eElR9W2DW8wEk9+mvtWvKwu7yTebzND+46y1nRMoFydi5zPVLSlJEf81qZZ4Uh1UUMLwXz+NRWfixnGXgq2wRq1bH0n3mqDhayiG4LJKgGdDjWXC8W8MMXDYx24SIJrJu9KiNEMprJE+XVF9nQVNijNAjlWBqkDpsfaWTUfeVLRLctfAqW1blsmIv4RQ91PupYJZDNc8nO9ZTF3TEVM+2KHoxzDJrLs2Q==" : null,
Type = type,
Status = status
};
}

122
util/Seeder/Recipes/CollectionsRecipe.cs Normal file
View File

@@ -0,0 +1,122 @@
using Bit.Core.Enums;
using Bit.Core.Utilities;
using Bit.Infrastructure.EntityFramework.Repositories;
using LinqToDB.EntityFrameworkCore;
namespace Bit.Seeder.Recipes;
public class CollectionsRecipe(DatabaseContext db)
{
/// <summary>
/// Adds collections to an organization and creates relationships between users and collections.
/// </summary>
/// <param name="organizationId">The ID of the organization to add collections to.</param>
/// <param name="collections">The number of collections to add.</param>
/// <param name="organizationUserIds">The IDs of the users to create relationships with.</param>
/// <param name="maxUsersWithRelationships">The maximum number of users to create relationships with.</param>
public List<Guid> AddToOrganization(Guid organizationId, int collections, List<Guid> organizationUserIds, int maxUsersWithRelationships = 1000)
{
var collectionList = CreateAndSaveCollections(organizationId, collections);
if (collectionList.Any())
{
CreateAndSaveCollectionUserRelationships(collectionList, organizationUserIds, maxUsersWithRelationships);
}
return collectionList.Select(c => c.Id).ToList();
}
private List<Core.Entities.Collection> CreateAndSaveCollections(Guid organizationId, int count)
{
var collectionList = new List<Core.Entities.Collection>();
for (var i = 0; i < count; i++)
{
collectionList.Add(new Core.Entities.Collection
{
Id = CoreHelpers.GenerateComb(),
OrganizationId = organizationId,
Name = $"Collection {i + 1}",
Type = CollectionType.SharedCollection,
CreationDate = DateTime.UtcNow,
RevisionDate = DateTime.UtcNow
});
}
if (collectionList.Any())
{
db.BulkCopy(collectionList);
}
return collectionList;
}
private void CreateAndSaveCollectionUserRelationships(
List<Core.Entities.Collection> collections,
List<Guid> organizationUserIds,
int maxUsersWithRelationships)
{
if (!organizationUserIds.Any() || maxUsersWithRelationships <= 0)
{
return;
}
var collectionUsers = BuildCollectionUserRelationships(collections, organizationUserIds, maxUsersWithRelationships);
if (collectionUsers.Any())
{
db.BulkCopy(collectionUsers);
}
}
/// <summary>
/// Creates user-to-collection relationships with varied assignment patterns for realistic test data.
/// Each user gets 1-3 collections based on a rotating pattern.
/// </summary>
private List<Core.Entities.CollectionUser> BuildCollectionUserRelationships(
List<Core.Entities.Collection> collections,
List<Guid> organizationUserIds,
int maxUsersWithRelationships)
{
var maxRelationships = Math.Min(organizationUserIds.Count, maxUsersWithRelationships);
var collectionUsers = new List<Core.Entities.CollectionUser>();
for (var i = 0; i < maxRelationships; i++)
{
var orgUserId = organizationUserIds[i];
var userCollectionAssignments = CreateCollectionAssignmentsForUser(collections, orgUserId, i);
collectionUsers.AddRange(userCollectionAssignments);
}
return collectionUsers;
}
/// <summary>
/// Assigns collections to a user with varying permissions.
/// Pattern: 1-3 collections per user (cycles: 1, 2, 3, 1, 2, 3...).
/// First collection has Manage rights, subsequent ones are ReadOnly.
/// </summary>
private List<Core.Entities.CollectionUser> CreateCollectionAssignmentsForUser(
List<Core.Entities.Collection> collections,
Guid organizationUserId,
int userIndex)
{
var assignments = new List<Core.Entities.CollectionUser>();
var userCollectionCount = (userIndex % 3) + 1; // Cycles through 1, 2, or 3 collections
for (var j = 0; j < userCollectionCount; j++)
{
var collectionIndex = (userIndex + j) % collections.Count; // Distribute across available collections
assignments.Add(new Core.Entities.CollectionUser
{
CollectionId = collections[collectionIndex].Id,
OrganizationUserId = organizationUserId,
ReadOnly = j > 0, // First assignment gets write access
HidePasswords = false,
Manage = j == 0 // First assignment gets manage permissions
});
}
return assignments;
}
}

94
util/Seeder/Recipes/GroupsRecipe.cs Normal file
View File

@@ -0,0 +1,94 @@
using Bit.Core.Utilities;
using Bit.Infrastructure.EntityFramework.Repositories;
using LinqToDB.EntityFrameworkCore;
namespace Bit.Seeder.Recipes;
public class GroupsRecipe(DatabaseContext db)
{
/// <summary>
/// Adds groups to an organization and creates relationships between users and groups.
/// </summary>
/// <param name="organizationId">The ID of the organization to add groups to.</param>
/// <param name="groups">The number of groups to add.</param>
/// <param name="organizationUserIds">The IDs of the users to create relationships with.</param>
/// <param name="maxUsersWithRelationships">The maximum number of users to create relationships with.</param>
public List<Guid> AddToOrganization(Guid organizationId, int groups, List<Guid> organizationUserIds, int maxUsersWithRelationships = 1000)
{
var groupList = CreateAndSaveGroups(organizationId, groups);
if (groupList.Any())
{
CreateAndSaveGroupUserRelationships(groupList, organizationUserIds, maxUsersWithRelationships);
}
return groupList.Select(g => g.Id).ToList();
}
private List<Core.AdminConsole.Entities.Group> CreateAndSaveGroups(Guid organizationId, int count)
{
var groupList = new List<Core.AdminConsole.Entities.Group>();
for (var i = 0; i < count; i++)
{
groupList.Add(new Core.AdminConsole.Entities.Group
{
Id = CoreHelpers.GenerateComb(),
OrganizationId = organizationId,
Name = $"Group {i + 1}"
});
}
if (groupList.Any())
{
db.BulkCopy(groupList);
}
return groupList;
}
private void CreateAndSaveGroupUserRelationships(
List<Core.AdminConsole.Entities.Group> groups,
List<Guid> organizationUserIds,
int maxUsersWithRelationships)
{
if (!organizationUserIds.Any() || maxUsersWithRelationships <= 0)
{
return;
}
var groupUsers = BuildGroupUserRelationships(groups, organizationUserIds, maxUsersWithRelationships);
if (groupUsers.Any())
{
db.BulkCopy(groupUsers);
}
}
/// <summary>
/// Creates user-to-group relationships with distributed assignment patterns for realistic test data.
/// Each user is assigned to one group, distributed evenly across available groups.
/// </summary>
private List<Core.AdminConsole.Entities.GroupUser> BuildGroupUserRelationships(
List<Core.AdminConsole.Entities.Group> groups,
List<Guid> organizationUserIds,
int maxUsersWithRelationships)
{
var maxRelationships = Math.Min(organizationUserIds.Count, maxUsersWithRelationships);
var groupUsers = new List<Core.AdminConsole.Entities.GroupUser>();
for (var i = 0; i < maxRelationships; i++)
{
var orgUserId = organizationUserIds[i];
var groupIndex = i % groups.Count; // Round-robin distribution across groups
groupUsers.Add(new Core.AdminConsole.Entities.GroupUser
{
GroupId = groups[groupIndex].Id,
OrganizationUserId = orgUserId
});
}
return groupUsers;
}
}

25
util/Seeder/Recipes/OrganizationDomainRecipe.cs Normal file
View File

@@ -0,0 +1,25 @@
using Bit.Infrastructure.EntityFramework.Models;
using Bit.Infrastructure.EntityFramework.Repositories;
namespace Bit.Seeder.Recipes;
public class OrganizationDomainRecipe(DatabaseContext db)
{
public void AddVerifiedDomainToOrganization(Guid organizationId, string domainName)
{
var domain = new OrganizationDomain
{
Id = Guid.NewGuid(),
OrganizationId = organizationId,
DomainName = domainName,
Txt = Guid.NewGuid().ToString("N"),
CreationDate = DateTime.UtcNow,
};
domain.SetVerifiedDate();
domain.SetLastCheckedDate();
db.Add(domain);
db.SaveChanges();
}
}

18
util/Seeder/Recipes/OrganizationWithUsersRecipe.cs
View File

@@ -1,4 +1,5 @@
using Bit.Infrastructure.EntityFramework.Models;
using Bit.Core.Enums;
using Bit.Infrastructure.EntityFramework.Models;
using Bit.Infrastructure.EntityFramework.Repositories;
using Bit.Seeder.Factories;
using LinqToDB.EntityFrameworkCore;
@@ -7,11 +8,12 @@ namespace Bit.Seeder.Recipes;
public class OrganizationWithUsersRecipe(DatabaseContext db)
{
public Guid Seed(string name, int users, string domain)
public Guid Seed(string name, string domain, int users, OrganizationUserStatusType usersStatus = OrganizationUserStatusType.Confirmed)
{
var organization = OrganizationSeeder.CreateEnterprise(name, domain, users);
var user = UserSeeder.CreateUser($"admin@{domain}");
var orgUser = organization.CreateOrganizationUser(user);
var seats = Math.Max(users + 1, 1000);
var organization = OrganizationSeeder.CreateEnterprise(name, domain, seats);
var ownerUser = UserSeeder.CreateUser($"owner@{domain}");
var ownerOrgUser = organization.CreateOrganizationUser(ownerUser, OrganizationUserType.Owner, OrganizationUserStatusType.Confirmed);
var additionalUsers = new List<User>();
var additionalOrgUsers = new List<OrganizationUser>();
@@ -19,12 +21,12 @@ public class OrganizationWithUsersRecipe(DatabaseContext db)
{
var additionalUser = UserSeeder.CreateUser($"user{i}@{domain}");
additionalUsers.Add(additionalUser);
additionalOrgUsers.Add(organization.CreateOrganizationUser(additionalUser));
additionalOrgUsers.Add(organization.CreateOrganizationUser(additionalUser, OrganizationUserType.User, usersStatus));
}
db.Add(organization);
db.Add(user);
db.Add(orgUser);
db.Add(ownerUser);
db.Add(ownerOrgUser);
db.SaveChanges();