Written by David Tebbutt, PC Dealer 06/89 item 01 - scanned
Advice-giving programs will soon affect everyone, directly or indirectly, according to a recently published hook. This probably means that dealers will end up implementing them and having to confront the profound practical and moral issues which they entail. The book is called Benefits and Risks of Knowledge-Based Systems' and it is published by Oxford University Press. Its authors were a working party organised by the Council for Science and Society.
The book looks at what's involved in capturing and encapsulating someone else's expert knowledge. It makes you think about when, if ever, you let an expert system act autonomously on its decisions.
Is it when it thinks missiles have been unleashed on your country by another? Is it when it believes a nuclear reactor is in danger of a meltdown? In these cases, time is short, perhaps too short for a human to assimilate the necessary information in time to act.
The book also looks at more down to earth uses of expert systems; in finance, law, education and health. It looks at the potential benefits and dangers of knowledge-based systems. And it tries to examine the issues from all points of view.
The authors set out to be calm and to present the potential for good along with the potential for harm. The danger of this approach is that, although there may be hundreds of good uses, just one of the harmful ones could annihilate the planet.
The worst kind of expert system is one which is allowed to control equipment without human intervention. This includes missiles, power stations, unmanned battlefield vehicles and, if you're in the money markets, computers which automatically trade with each other.
Imagine if all the trading computers suddenly decided to issue sell orders but none wanted to buy. In the words of the report 'the entire market would sink like a rock'.
Staying with the financial markets, the report suggests that organisations with the best-programmed computers would clean up. Good programs would be able to spot ever more subtle trends from which to make a quick buck. In the end, the computers would be spotting trends which 'will not be generated by real economic movements, but by the buying and selling activities of other computer programs'.
On the positive side, expert systems can make admirable assistants, especially when consulted by an expert in the same field. A DHSS employee trying to help someone establish their entitlements, could be steered round the maze of regulations. A Citizen's Advice worker might welcome an expert system which stimulates them into asking the right questions when trying to identify a client's problems. Any good expert system is one which guides and assists humans without removing their autonomy and responsibility.
This can be very beneficial. The report identifies the areas where a knowledge-based system can fall down. To paraphrase these: the information in the system could be wrong; the rules used to process the information could be unreliable; the system's meaning of a term may differ from the user's understanding of the same term; the program may go awry when it receives an unexpected input; the values, or criteria, on which the system is based, may not be shared by those affected by its conclusions.
Those responsible for building and installing expert systems will have a lot to answer for. It is not like developing, for example, a freight forwarding application, in which the documents and their rules for processing are fairly straightforward. We are dealing here with capturing and encapsulating human knowledge and making it available for exploitation by third parties.
Knowledge-based systems is an area of great potential growth and income for companies who are successful in the field.
If you are halfway serious about entering this area of computing, invest a paltry £6.95 in this book. It will give you a good understanding of the issues and it suggests a sound moral framework within which to operate.