We want to reduce the amount of business critical test data in the company. One way of doing that is to generate test data on demand prior to client side testing.
Clients will request a scene to be set up with a JSON body set of options, specific to a given scene. Successful seed requests will be responded to with a mangleMap which maps magic strings present in the request to the mangled, non-colliding versions inserted into the database. This way, the server is solely responsible for understanding uniqueness requirements in the database. scenes also are able to return custom data, depending on the scene. For example, user creation would benefit from a return value of the userId for further test setup on the client side.
Clients will indicate they are running tests by including a unique header, x-play-id which specifies a unique testing context. The server uses this PlayId as the seed for any mangling that occurs. This allows the client to decide it will reuse a given PlayId if the test context builds on top of previously executed tests. When a given context is no longer needed, the API user will delete all test data associated with the PlayId by calling a delete endpoint.
---------
Co-authored-by: Matt Gibson <mgibson@bitwarden.com>
* Add GroupsRecipe to manage group creation and user relationships in organizations
* Add CollectionsRecipe to manage collection creation and user relationships in organizations
* Refactor OrganizationUsersControllerPerformanceTests to enhance performance testing and add new test cases
* Add OrganizationDomainRecipe to add verified domains for organizations
* Add more tests to OrganizationUsersControllerPerformanceTests and enhance seeding logic for organizations
- Updated performance tests to use dynamic domain generation for organization users.
- Refactored seeding methods in OrganizationWithUsersRecipe to accept user status and type.
- Modified AddToOrganization methods in CollectionsRecipe and GroupsRecipe to return created IDs.
- Adjusted DbSeederUtility to align with new seeding method signatures.
* Enhance OrganizationSeeder with additional configuration options and update seat calculation in OrganizationWithUsersRecipe to ensure a minimum of 1000 seats.
* Add performance tests for Groups, Organizations, Organization Users, and Provider Organizations controllers
- Introduced `GroupsControllerPerformanceTests` to validate the performance of the PutGroupAsync method.
- Added `OrganizationsControllerPerformanceTests` with multiple tests including DeleteOrganizationAsync, DeleteOrganizationWithTokenAsync, PostStorageAsync, and CreateWithoutPaymentAsync.
- Enhanced `OrganizationUsersControllerPerformanceTests` with DeleteSingleUserAccountAsync and InviteUsersAsync methods to test user account deletion and bulk invitations.
- Created `ProviderOrganizationsControllerPerformanceTests` to assess the performance of deleting provider organizations.
These tests ensure the reliability and efficiency of the respective controller actions under various scenarios.
* Refactor GroupsControllerPerformanceTests to use parameterized tests
- Renamed `GroupsControllerPerformanceTest` to `GroupsControllerPerformanceTests` for consistency.
- Updated `PutGroupAsync` method to use `[Theory]` with `InlineData` for dynamic user and collection counts.
- Adjusted organization user and collection seeding logic to utilize the new parameters.
- Enhanced logging to provide clearer performance metrics during tests.
* Update domain generation in GroupsControllerPerformanceTests for improved test consistency
* Remove ProviderOrganizationsControllerPerformanceTests
* Refactor performance tests for Groups, Organizations, and Organization Users controllers
- Updated method names for clarity and consistency, e.g., `PutGroupAsync` to `UpdateGroup_WithUsersAndCollections`.
- Enhanced test documentation with XML comments to describe the purpose of each test.
- Improved domain generation logic for consistency across tests.
- Adjusted logging to provide detailed performance metrics during test execution.
- Renamed several test methods to better reflect their functionality.
* Refactor performance tests in Organizations and Organization Users controllers
- Updated tests to use parameterized `[Theory]` attributes with `InlineData` for dynamic user, collection, and group counts.
- Enhanced logging to include detailed metrics such as user and collection counts during test execution.
- Marked several tests as skipped for performance considerations.
- Removed unused code and improved organization of test methods for clarity.
* Add bulk reinvite users performance test to OrganizationUsersControllerPerformanceTests
- Implemented a new performance test for the POST /organizations/{orgId}/users/reinvite endpoint.
- Utilized parameterized testing with `[Theory]` and `InlineData` to evaluate performance with varying user counts.
- Enhanced logging to capture request duration and response status for better performance insights.
- Updated OrganizationSeeder to conditionally set email based on user status during seeding.
* Refactor domain generation in performance tests to use OrganizationTestHelpers
- Updated domain generation logic in GroupsControllerPerformanceTests, OrganizationsControllerPerformanceTests, and OrganizationUsersControllerPerformanceTests to utilize the new GenerateRandomDomain method from OrganizationTestHelpers.
- This change enhances consistency and readability across the tests by centralizing domain generation logic.
* Update CollectionsRecipe to have better readability
* Update GroupsRecipe to have better readability
* Refactor authentication in performance tests to use centralized helper method. This change reduces code duplication across Groups, Organizations, and OrganizationUsers controller tests by implementing the `AuthenticateClientAsync` method in a new `PerformanceTestHelpers` class.
* Refactor OrganizationUsersControllerPerformanceTests to filter organization users by OrganizationId.
* Refactor CreateOrganizationUser method to improve handling of user status and key assignment based on invitation and confirmation states.
* Add XML documentation for CreateOrganizationUser method to clarify user status handling
Adds a Rust SDK for performing seed related cryptograhic operations. It depends on internal portions of our Rust SDK. Primarily parts of the bitwarden-crypto crate.
Adds a database seeder which can be used standalone using a CLI for seeding your local development environment, or used in unit tests to seed complex scenarios.
---------
Co-authored-by: Robert Y <rkac@bitwarden.com>
