X-Message-Number: 10118
Date: Sat, 25 Jul 1998 16:39:36 -0400
From: <> (Jeffrey Soreff)
Subject: Re: Y2K

Perry E. Metzger wrote:
>> From: Peter Merel <>
>>   * 1/1/00 itself will produce global disruptions in electrical generation
>>     and distribution networks,

>How? Where do you think the average electrical generator stores the date?

Here is an example of date-driven problems in an electrical plant taken
from http://www.euy2k.com/embedded.htm

>A midwestern US fossil facility was testing a boiler feedwater control
>loop for date rollover to Year 2000. The control console date was set
>in a fashion similar to testing a PC - it was changed to 12/31/99,
>23:58, and then powered down. A few minutes later, it was powered back
>up - with the only resultant problem being the year shown as 1980 (a
>typical older BIOS response). The logic loop (PLC and other
>instrumentation) continued to function normally. Boiler levels were
>simulated up and down to drive feedwater regulating valves; again, no
>problem. Then, the technicians reset the console clock to 12/31/99,
>23:58, and did NOT power down.  When the clock rolled over to
>01/01/2000, there was no problem. The technicians powered down the
>console and then restarted it - and guess what happened? The console
>rebooted with a date of 01/04/80, the downstream PLC (which had not
>been powered down) apparently saw this as a significant mismatch with
>it's own clock (time as a function of integers rather than actual
>date), and interpreted this condition as a gross control failure. The
>feedwater regulating valves were driven shut, and the boiler trip
>logic was initiated (the 'fail safe' condition for the boiler). In a
>'live' situation, the plant would have tripped.

To answer your question explicitly: Both the PLC and the control console
stored the date.

Other examples of date related failures in embedded systems are covered in
http://www.xs4all.nl/~zooko/Y2k-real-life.html and in
http://www.auto2000.ndirect.co.uk/intro02.htm

Perry, I sympathize with your skepticism.  Until January of 1997, I
assumed that process control computers measured voltage, pressure,
temperature and so on and controlled voltage, pressure, temperature
and so on.  Why would anyone expect a process control computer to know
about or care about dates???  Unfortunately, this is incorrect.  On
Jan 1, 1997, the Tiwai Pt aluminum smelter demonstrated the date
sensitivity of process computers dramatically: (from
http://www.granite.ab.ca/year2000/incidents.htm)

>A computer glitch at the Tiwai Pt [place in South Island of New
>Zealand] aluminium smelter at midnight on New Year's Eve has left a
>repair bill of more than $1 million [New Zealand Dollars]. Production
>in all the smelting potlines ground to a halt at the stroke of
>midnight when the computers shut down simultaneously and without
>warning. New Zealand Aluminium Smelters' general manager David Brewer
>said the failure was traced to a faulty computer software program,
>which failed to account for 1996 being a leap year. The computer was
>not programmed to handle the 366th day of the year, he said. "Each of
>the 660 process control computers hung up simultaneously at midnight,"
>Mr. Brewer said.

>The same problem occurred two hours later at Comalco's Bell Bay
>smelter, in Tasmania [Australia]. New Zealand is two hours ahead of
>Tasmania. Both smelters use the same program, which was written by
>Comalco computer staff.

>Mr. Brewer said the cause was difficult to trace and it was not till a
>telephone call in the morning from Bell Bay that the leap year link
>was made. "It was a complicated problem and it took quite some time to
>find out just what caused it."

>Tiwai staff rallied through the night to operate the potlines manually
>and try to find the cause. The glitch was fixed and normal production
>restored by midafternoon. However, by then, the damage has been
>done. Without the computers to regulate temperatures inside the pot
>cells, five cells over-heated and were damaged beyond
>repair. Mr. Brewer said they would have to be replaced at a cost of
>more than $1 million.

Robert Ettinger wrote:
>Second, there are many obvious partial and makeshift interim solutions. For
>example, if a power company relies on computers to control its output(s), it
>could--instead of making minute-by-minute computer-controlled decisions based
>on actual supply and demand--make hourly decisions based on averages.
>Brownouts maybe and money lost, but no disaster.

This appears to be plausible - after all, industrial processes used to
operate without computers, and it seems reasonable that they could again
do so.  Unfortunately, note that in the incident above the staff *DID*
attempt to control temperatures manually, but were not able to avoid
physical damage.  Intuitively, it seems reasonable to think that one
can always choose to unplug the computer.  After all, we had a perfectly
viable infrastructure in the 1960s, before extensive use of embedded
computers, and one might imagine that Y2K failures will just force us
to operate as we did in the 1960s, which would imply rather modest
degradations in our industrial processes.

Unfortunately, there are a number of reasons why this type of fallback
won't work.

1) Some equipment has been designed with faster time constants
   than human reflexes can control.  An extreme example is an
   aerodynamically unstable aircraft which requires constant
   active control.  You _cannot_ replace a computer with a human
   in the control loop in these cases.  Analog controls can work,
   but the time required to design and install them is probably
   longer than the time needed to fix the digital programs.
   Switching from minute-by-minute control to hour-by-hour control
   doesn't give you degraded performance in this kind of case.
   It gives you parameters going beyond safe limits, and does
   indeed yield a disaster.

2) Some control points that would have allowed humans to bypass
   electronic controls have been physically removed.  I've read
   descriptions of the physical removal of manual rail switches.
   I've read of physical removal of banks of manual overrides to
   process computers.

3) In some cases, the physical places at which humans worked in
   pre-computer operations no longer exist.  For example, the
   telephone network used to require vastly more operators than
   it now has.  Entire buildings that used to house these
   operators have been adapted to other uses or torn down.

Linda Chamberlain wrote:
>A Y2K (Year 2000) Committee has been formed within the Alcor Board of
>Directors to look into potential problems Alcor could encountered due
>to the Millenium Bug (real or imagined).   The Committee is made up of
>Linda Chamberlain, Chair, Ralph Merkle, Carlos Mondragon, Mark
>Voelker.

I commend you on your efforts.  Thank you very much for the information,
thanks to all of the members of your committee on their consideration of
safeguards, and thank you for visiting Air Products and investigating
their Y2K efforts.


In summary, I have read a great deal of evidence that date-dependent
computer systems are heavily used in life-critical facilities such
as electrical utilities, water supplies and food distribution.  I've
read a great deal of evidence that somewhere between 2% and 50% of
this equipment will fail on 1/1/2000.  If the rollover were today, I
believe that it would be a multi-megadeath event.  Fortunately,
electrical utilities and other industries are working to fix their
date-dependent systems.  Unfortunately, based on the volume of work
that I've seen, and based on the number of embedded computers which
are known to exist, I expect the bulk of the defective systems to
remain in place on 1/1/2000.  In my view, the mortality of the rollover
will depend mostly on two factors:
1) Is the Y2K repair work which is now underway well focussed
   on life-critical systems?
2) What fraction of failures in life-critical systems will require
   very simple actions (like rebooting systems), what fraction will
   require complex debugging, and what fraction will require physical
   repair (as in Tiwai Pt)?
                                              Best wishes,
                                              -Jeffrey Soreff
                                              

standard disclaimer: I do not speak for my employer.

Rate This Message: http://www.cryonet.org/cgi-bin/rate.cgi?msg=10118