How to evaluate open-source identity platforms

Evaluate open-source identity platforms against mandatory requirements for a specific release and deployment model, then test the workflows your application needs. Use feature lists to build a shortlist—not to select a winner. Keep authentication, federation, application permissions, and operational ownership separate throughout the evaluation.

Define requirements before choosing candidates

Start with the system you need to support. Record whether identities belong to customers, employees, or both; which browser and native applications need access; and where tenant and administrator boundaries must sit.

Turn those requirements into acceptance criteria. For example:

Evaluation area	Questions to resolve
Account lifecycle	How must registration, verification, recovery, account linking, and deletion work?
Authentication	Which sign-in methods and MFA enrollment and recovery paths are mandatory?
Protocols and provisioning	Which existing systems must connect? What exact OIDC, OAuth 2.0, SAML, or SCIM integrations are required, and in which direction?
Application permissions	Must access depend on roles, ownership, group membership, resource relationships, or tenant boundaries?
Administration	Who may invite users, reset credentials, change policies, and inspect security events?
Deployment	Which hosting environments, regions, databases, and availability requirements are acceptable?

These are questions for each candidate, not assumed capabilities. Make mandatory requirements pass/fail gates, and specify what documentation or test result would satisfy each one.

Separate authentication, federation, and permissions

Ory’s projects illustrate why a single “identity support” checkbox is too broad:

• Kratos addresses identity and account workflows. Its repository describes identity and credential management, with user-facing flows for login, registration, recovery, verification, and profile management. Kratos repository
• Hydra addresses OAuth 2.0 and OpenID Connect. Its repository describes a standalone OAuth 2.0 and OpenID Connect server without user management, connecting to an existing identity provider through a login and consent application. Hydra repository
• Keto addresses application permissions. Its repository describes a permission server using the Zanzibar approach and lists support for Ory Permission Language. Keto repository

Use these boundaries when designing the proof of concept. A successful login should not count as evidence that delegated API access or document-sharing permissions work. Define and evaluate those responsibilities independently, even when a candidate packages several of them together.

Do not assume that every application needs all three components. Map the requirements to the necessary responsibilities first, then identify the products and integrations needed to implement them.

Build a shortlist from documented scope

Other official repositories provide useful starting points for selecting evaluation candidates:

Candidate	Scope described by its repository	Useful evaluation question
Keycloak	User federation, authentication, user management, and fine-grained authorization. Repository	Can its account, federation, and permission workflows satisfy your requirements without unacceptable customization?
authentik	An identity provider for SSO, with listed support for SAML, OAuth2/OIDC, LDAP, and RADIUS, designed for self-hosting. Repository	Do the exact integration modes and administrative workflows fit your applications?
ZITADEL	An identity and access-management platform whose README lists SSO, MFA, passkeys, OIDC, SAML, SCIM, and multi-tenancy. Repository	Does its tenant and organization model match your required isolation and delegation boundaries?

Treat these summaries as pointers to investigate, not a verified feature-parity matrix. Confirm protocol roles, configuration requirements, edition boundaries, and behavior in the release you intend to deploy. A protocol name in a README is not enough to close a detailed integration requirement.

Evaluate the release, license, and deployment separately

Give each evaluation a precise identity: project, release or commit, distribution, edition, deployment model, and configuration. Inspect that release’s license and relevant dependency licenses. Check whether each required capability belongs to the self-hosted open-source distribution, a commercial offering, a managed service, or an additional integration.

For a concrete edition check, the Kratos README lists SCIM and SAML among the additional features provided through the Ory Enterprise License and not available in the open-source version. Verify the exact protocol role and entitlement for your chosen release before marking an integration requirement as satisfied. Kratos repository

Keep managed-service documentation separate from self-hosted release documentation. Ory’s Network introductions identify open-source Kratos and Hydra as the foundations of the corresponding Network services. Do not treat that shared foundation as proof of release-specific feature parity between deployment options. Kratos Network introduction · Hydra Network introduction

User-interface responsibilities deserve an explicit entry. Ory’s Kratos Network introduction describes building custom authentication and profile-management interfaces with Ory SDKs and REST APIs. For each candidate, identify the interface you intend to use, verify its availability in the selected deployment, and include customization and maintenance in the evaluation. Kratos Network introduction

Mark unresolved entries unverified, rather than treating missing evidence as proof that a feature is unavailable.

Test complete workflows and denial paths

Write acceptance tests around user journeys, not isolated feature names. For each test, record the release, configuration, starting state, expected result, and observed result.

For an account lifecycle test, consider this sequence:

1. Register and verify an account.
2. Enroll the required authentication factors.
3. Recover access after losing a factor.
4. Change a sensitive account attribute.
5. Revoke sessions or disable the account.
6. Confirm which existing sessions and downstream access remain usable.

Add account-linking and administrator-access cases where required. Include attempts by users and administrators from another tenant. Define the required outcome before running each test, including acceptable revocation timing.

For authorization, use a concrete resource model. Create a document, grant direct access to one user, and grant another user access through a group. Test inherited access, removal of membership, tenant boundaries, and permission-service failure.

That scenario is relevant to the relationship and inheritance capabilities described in Ory’s Keto Network documentation, which gives a document-reader relationship as an example and describes inheritance through groups, roles, and hierarchies. Measure revocation behavior in your deployment rather than inferring a latency or consistency guarantee from that description. Keto Network introduction

Rehearse operations and exit before committing

Include operational exercises in the proof of concept:

• Upgrade: rehearse schema migrations and document recovery options if an upgrade fails.
• Restore: restore a backup into an isolated environment and verify representative identity and permission workflows.
• Rotate keys: exercise the supported rotation procedure and observe effects on active clients and credentials.
• Handle failures: test database and service outages, including the application’s behavior when identity or permission decisions are unavailable.
• Observe incidents: check whether logs and metrics provide the information your responders need without exposing credentials or unnecessary personal data.
• Exit: test available exports and document which users, identifiers, credentials, clients, and permission data can move to another system—and which would require recreation or user action.

Estimate infrastructure, integration development, UI maintenance, administration, support, and on-call work separately. Use measurements from the proposed deployment for performance and recovery decisions; do not substitute repository claims for workload testing.

Apply mandatory gates before weighted scores

Only score candidates that satisfy mandatory requirements. Treat an unverified mandatory capability as an unresolved selection blocker—not as a product defect or an assumed pass.

Then compare the surviving options on integration effort, user-experience control, administrative usability, operational burden, and migration options. Set weights before reviewing results so that an attractive demo does not quietly redefine the requirements.

The final decision should name a release and deployment model, list the workflows demonstrated, identify remaining risks, and assign ownership for custom integrations. Select the option that satisfies the required behavior with an operational burden your team can support—not the one with the longest feature list.