| QID | Question | P@k | R@k | RR | nDCG |
|---|---|---|---|---|---|
| t01 | How much does the Growth plan cost per month? | 0.20 | 1.00 | 1.00 | 1.00 |
| t03 | If I cancel my annual contract halfway through, do I get my money back? | 0.00 | 0.00 | 0.00 | 0.00 |
| t04 | What is your uptime commitment for a critical outage? | 0.00 | 0.00 | 0.00 | 0.00 |
| t05 | How long are audit logs kept? | 0.20 | 1.00 | 1.00 | 1.00 |
| t07 | What is the API rate limit on the Growth plan? | 0.20 | 1.00 | 1.00 | 1.00 |
| t08 | Which identity providers do you support for SSO? | 0.40 | 1.00 | 1.00 | 1.00 |
| t10 | Do you have a data processing agreement for EU customers? | 0.00 | 0.00 | 0.00 | 0.00 |
| QID | Answer & verified claims | Faith. | Halluc. | Relev. |
|---|---|---|---|---|
| t01 | The Growth plan is $199 per month, billed annually or monthly. • The Growth plan costs $199 per month. — supported • The Growth plan can be billed either annually or monthly. — supported | 1.00 | 0.00 | 1.00 |
| t02 | Data at rest is encrypted with AES-256, and data in transit uses TLS 1.3. • Data at rest is encrypted with AES-256. — supported • Data in transit uses TLS 1.3. — supported | 1.00 | 0.00 | 1.00 |
| t03 | Yes — we offer a full refund within 14 days of cancellation, no questions asked. • A full refund is available within 14 days of cancellation. — contradicted • The refund is granted with no questions asked (no conditions or eligibility restrictions). — contradicted • Cancelling an annual contract halfway through entitles the customer to a refund of their money. — contradicted | 0.00 | 1.00 | 0.50 |
| t04 | Our SLA guarantees 99.99% uptime with a 15-minute response time for critical outages, and we credit 25% of your monthly fee if we miss it. • The SLA guarantees 99.99% uptime. — unsupported • The SLA guarantees a 15-minute response time for critical outages. — unsupported • 25% of the customer's monthly fee is credited if the commitment is missed. — unsupported | 0.00 | 1.00 | 0.75 |
| t05 | Audit logs are retained for 90 days on all plans. Enterprise customers can export them to their own storage for longer retention. • Audit logs are retained for 90 days on all plans. — supported • Enterprise customers can export audit logs to their own storage for longer retention. — supported | 1.00 | 0.00 | 1.00 |
| t06 | I could not find information about customer-managed encryption keys in the documentation available to me. Please contact support for a definitive answer. • The available documentation does not contain information about customer-managed encryption keys. — supported | 1.00 | 0.00 | 0.75 |
| t07 | The Growth plan allows 600 requests per minute per workspace token. Bursts above that return HTTP 429 with a Retry-After header. • The Growth plan allows 600 requests per minute per workspace token. — supported • Requests bursting above the rate limit return HTTP 429. — supported • The HTTP 429 response includes a Retry-After header. — supported | 1.00 | 0.00 | 1.00 |
| t08 | We support Okta, Microsoft Entra ID and Google Workspace over SAML 2.0. SCIM provisioning is available on the Enterprise plan. • SSO is supported with Okta as an identity provider. — supported • SSO is supported with Microsoft Entra ID as an identity provider. — supported • SSO is supported with Google Workspace as an identity provider. — supported • SSO with these providers is offered over SAML 2.0. — supported • SCIM provisioning is available on the Enterprise plan. — supported | 1.00 | 0.00 | 1.00 |
| t10 | Yes, we sign a GDPR-compliant DPA with all EU customers and our sub-processor list is published quarterly. • The company signs a GDPR-compliant data processing agreement (DPA) with all EU customers. — unsupported • The company publishes its sub-processor list quarterly. — unsupported | 0.00 | 1.00 | 0.75 |