Ethical Reasoning 3: Avoid Harm

Problem Definition

According to the ACM Code of Ethics Principle 1.2 “A computing professional should avoid harm”, examples of harm include unjustified mental injury, disclosure of information and damage to reputation, and avoiding harm begins with careful consideration of potential impacts on those affected. In this case, the ethical issue is the incompetence of the AI or counselors in interacting with and consoling the user, possibly worsening their mental health. Constant checking up on users' emotional state may also prove annoying or even intrusive.

Gathering Information

Public:

Users expect their mental health to improve after using the app.
Users seeking counseling have the option of seeing specific counselors or different therapy plans.

Government:

Mobilize and provide the counselor workforce for the digital age
Fund counselor training programs

Stakeholders

Counselors:

Have a duty to be professional.
Have patience in listening and not discriminate against anyone based on their age, culture, disability, ethnicity, race, religion, sexual orientation, relationship status, language preference, socioeconomic status, immigration status etc.
Never share personal information with users and never have any relationship with the user other than the counseling session. This may result in a lack of objectivity and judgement in conducting the sessions, and may also be dangerous for the counselors.
Familiarize themselves with the app and the digital landscape.

Developers:

Ensure that only credentialed and able counsellors can treat users.
Constantly regulate the AI Chatbot to make sure it does not malfunction.
Notice the possible harm the app may cause and work hard in minimizing it.
Be cognizant that certain features may be detrimental to some users. Listen to user feedback and update accordingly.
Implement courses to train counselors for digital age.

Applying Guiding Principles

The Virtue Approach

Developers and counselors should make sure that the app does not cause more harm than good, including unintentional harm by the carelessness of implementing the counselor system.

The Utilitarian Approach

Developers should avoid harm and maximize the benefit of the app. Users should benefit from easy access to professional counselling services. The app should not replace counsellor jobs.

The Deontology/Rights Approach

Developers should consider the public interest and not their personal interests. Users have a right that their mental health is not made worse by an app that claims to help improve their mental health.

Generating, evaluating and taking actions

The app must implement a system to verify the credentials of the counsellors, and accept only those that are competent enough.
The users may choose their own counsellor and follow up on them for recurring therapy sessions.
Implement a counsellor rating system for users to evaluate the performance of the counsellors.
Implement a system for users to report potential misbehavior by the AI chatbot.
Establish a training program to prepare counselors for the online counseling.

Our reflection on this ethical issue

All parties involved should ensure that the app does not ultimately lead to harm for the users. By taking the necessary precautions, we should make sure to minimize all potential problems to facilitate the user experience and benefit from the app.