ITProjectManagement.org

This checklist provides a generic list of items to consider when evaluating a SaaS service.

Authentication

1. Is SAML based Single Sign-On supported? Make sure your company’s IDP is supported (Azure, Okta, etc) and make sure your company’s user name format is support (SSO is no longer SSO if name format is not supported).

User management

2. How are new users added? Manually, or automatically (consider the SCIM protocol if not aware already)
3. How are users removed? Manually or automatically if removed from your enterprise directory?
4. How are permissions applied to users? Manually in with the application or automatically (for the later, permissions assigned via Active Directory security groups provides central auditing advantages)

Security

5. Does the SaaS provider have security certifications (e.g. ISO27001)
6. Has a third party independent PEN test been completed, if so, how often is it repeated?
7. Is user and/or admin multi factor authentication available?
8. Is there security auditing on who/ when / what? can this be automatically shipped to a log platform?
9. Are the roles based permissions defined that an be leveraged?
10. Who and what can SaaS provider staff access?
11. Does the provider offer a bug bounty program? (this gives some insight into how serious they consider security)

Data sovereignty

12. Where is your data stored, and do you have location restrictions to consider?

Disaster Recovery

13. Does the SaaS provider have a published disaster recovery plan?
14. How often is the disaster recovery plan tested?

Backup and restore

15. What is backed up by the provider, and what is your responsibility to backup?
16. How often are backups taken?
17. What is the process for restore? (how long does it take, can you restore only certain items, or is only full restore possible?)
18. Is there capability for customers to take independent backups and store outside of the SaaS environment? (if so, can this be done automatically?)

Import / Export

17. What data can be imported (and in what format) to assist with service setup?
18. What data can be exported (and is the format useable) if you choose to leave the service?

API access

21. Is there published API availability to enable automation?
22. Are the APIs sufficient to be useful?

Licensing

23. Is the licensing model understandable?
24. Is the licensing approach for archived or non-active data/users treated differently? (paying full licences for archived users is never nice!)

Reporting

25. Is there useful management reports available?

Roadmaps

26. Is there a clear roadmap on how long versions are supported and when/ how upgrades are enforced?
27. Is there a clear roadmap on upcoming features?
28. Is the vendor actively developing the service? (this provides insight on their priority on the service)

Availability

29. Is historic service availability publicly published?

Training and support methods

30. Is there an online knowledge base?
31. Is there a user forum?
32. How are support tickets logged and what are the service level options?

If you liked this checklist, please give me a clap. If you have suggested additions please let us know via the comments.