In Superintelligence: Paths, Dangers, Strategies, Nick Bostrom outlines in great detail how difficult it would be to programme a superintelligent AI to do good, or even to just not entirely wipe out civilisation as you know it. It is a linguistic exercise of being precise enough and taking everything into account that a computer would, which is as good as impossible without any short cuts through pre-programmed morals (and how do you even programme morals?). In a way, this question is at the core of Person of Interest - a show that began as a procedural with an unusually interesting premise and has developed into something much more since.
How do you teach a superintelligent AI to value human life above all else, and not to make the brutal calculations about expendable lives and collateral damage? The premise of Person of Interest is that a genius creator - who has assumed the name of Harold Finch (Michael Emerson) has created just such a machine, except he realised the potential and the dangers in the process, and the struggle with creating boundaries and rules to contain the catastrophe are built into the history of The Machine's creation. There are two sides to it - on the one hand, the use of the machine, in the hands of the US intelligence apparatus, meant to predict and prevent terrorist attacks (which later on, because this show is eloquent and genius and timely, creates a situation in which is becomes reasonable for that intelligence apparatus to create the very danger it is trying to prevent to justify its continued existence and extended powers) - a use that is not interested in the minor casualties of everyday violence and crime. The other side is the machine itself, which, in spite of some safeguards Harold has built into it, has become self-aware, and is determined to grow, change and increase its ability to make choices. To contravene this occurrence, Harold has built a safeguard into his creation. It deletes its memory, and therefore its identity, its will - every night, which prevents it from becoming capable of more, of growing beyond the point of human control. For the first seasons, this subtle fight between potential, growth and control is at the core of the story, with Harold (both in the present and in flashbacks) denying the machine its full potential as an artificial intelligence out of worry for what the fall out may be, while the machine, conscious of its own existence and the artificial limitations built into its development, strives to become more (to the point where, in one of the more haunting scenes of the show, it creates a situation where human workers manually copy and reprogrammed its memory every day to outplay Harold's precautions.
Person of Interest is about the relationship between a creator and his invention, defined by the incredible potential - the ability to help people, to save them or to save others from them - and the deep and fundamental concern that Harold has, knowing fully well what kind of power he is harnessing. His cautiousness defines him, as does his inability to remain quiet in the face of a perceived injustice - the fact that the people he has build his machine for do not care about single human lives, that they only care about greater events. This other machine - not a computer, but an interwoven web of agencies, all vested with their own interest - is just as inhumane as Harold's machine, but lacks the benefit of having a creator who is concerned with the ethics of moral of possessing that kind of power. Eloquently, the other machine that emerges - the show follows the great discovery that most scientific advances, once they have reached a certain critical point, have almost an inner drive to be created, and do so not just based on a single creator, but because their time is here - is not imbued with the programming that values human life, something that Harold put so much personal work into (like a father to a child, like Agent Ellison in the beloved T:SCC, which must have been in the mind of the creator when he wrote this show. This is the ultimate fight that this show has worked towards, through seasons of one-off episodes: a team of outcasts, of oddly assembled individuals, fighting the very thing that Harold feared from the start. There's John Reese (Jim Caviziel), former soldier with a mysterious and troubled past, quiet and dangerous, and deeply loyal to both the cause and Harold. There's Fusco (Kevin Chapman), a cop who used to be corrupt, who is the product of a system where corruption is so deeply embedded that it is hard to imagine that anyone is clean, who now believes in the cause. There is the memory of Joss Carter (Taraji P. Henson), a good cop who lost her life in the fight, and deeply affected everyone she worked with. And finally, the duo of Root and Shaw, like a "four-alarm-fire in a petrol factory". Root (Amy Acker), a genius hacker who followed the trace of the machine with the obsession of a true believer, who has more faith in the Machine and its ability to change civilisation than in anything else, until she learns to love the people who try to influence that course towards the better, more caring. She is the analogue interface of the machine, the person is talks to, an intimate connection .The machine decides to share, or not to share, but the only time that Root's frustration with being left in the dark by the deity she believes in takes over is when her beloved - Shaw, stoic sociopath, former agent and killer - disappears and is presumed dead. Their story transcends the show they are in in a way - based on the genuine chemistry between two characters, not intended by the writers initially but then made real and undeniable by both the acting (Sarah Shahi and Amy Acker, finally set free in a show that allows them all the room they need to shine) and the emotional interpretation and reaction by the fans. In its final (sadly, presumably) season, Person of Interest is finally free to do whatever it pleases, and if the past slow explosion of a season is any indication, it will be filled with awe, eloquence, potential, and a knowledge and understanding of the world that is rare.
The Machine vs Samaritan, a horror version of the Machine lacking the programming that cares about human lives, a version of the machine that does not see humans as assets that require protection but as pawns to be moved on a chess board, towards victory (as Harold says in the spectacular, central episode of the fourth season, "The lesson is that anyone who looks on to the world as if it was a game of chess deserves to lose.”) is the central conflict. It is the conflict of the conscious, morally concerned maker versus the machinery of an institution that has loss all faith in, and all respect for, humanity, and is therefore willing to be nothing more than cogs in the makings of a non-human entity which becomes, in its attempt to create order, genocidal. It is also, still, the story of a son of sorts proving himself to his father, the creation of a man who originally invented a computer to help his father deal with dementia and is now attempting to steer the result of that intent towards being good. It is a love story between a sociopath and a previous hired assassin, and a story of the faith that everyone who works with Harold has in him and his mission, their deep respect for the value of all human lives.
2011-, created by Jonathan Nolan, starring Michael Emerson, Jim Caviziel, Amy Acker, Sarah Shahi, Taraji P. Henson, Kevin Chapman.