Trust MeA hypothetical risk scenario
An app that determines the trustworthiness of the people you invite into your home
You are the founder and CEO of a startup called “Trust Me” that is based on an app designed to “vet” professionals, tradespersons, and others who you invite into your home. The app is designed to help users assess the trustworthiness of strangers who enter their home, whether they are salespersons, carrying out maintenance or home improvement tasks, providing professional services, or are guests at a social gathering. The app allows you to enter information on the person such as their name, gender, age, and where they live. It also allows you to take a photo of them. This information is cross-referenced against a growing database of social media activity and other sources, and the results assessed by an AI to assign a “trustworthiness” index to the person. The process usually takes between 30 – 90 seconds.
The mission of Trust Me is to increase the safety and security of users by helping them decide who they do and do not invite into their home, based on data. You are aware that there are ethical issues around the use of your technology. However, you firmly believe that this is a technology that will improve lives if developed and used responsibly, and you are committed to ensuring that it places decisions on who users trust on a more evidence-based footing.
Critical Risk Dimensions
- Social Justice & Equity
- Black Swan Events
- Co-opted Tech
- Health & Environment
- Loss of Agency
- Governance & Regulation
- Organizational Values & Culture
- Reputation & Trust
Download Trust Me
Click to download.
Key questions for identifying orphan risks
How might your product lead to perceived or actual discrimination, and how might this affect your business?
How could your product compromise privacy in ways that in turn compromise business success?
What are some of the ethical concerns that may affect your business?
In what ways might your product reduce the ability of users and others to have agency over their lives?
How could evolving regulations and practice norms around the use of personal data affect your business?
© 2020 Arizona Board of Regents on behalf of Arizona State University