X-Message-Number: 15945
Date: Sun, 25 Mar 2001 09:28:49 -0800
From: Lee Corbin <>
Subject: Re: Trust In All-Powerful Lords

Eliezer S. Yudkowsky, by way of Sabine Atkins, wrote

> I ask again:  Can you name something that a peaceful entity
> wants to do, and should be able to do, that can't be done
> from inside the Sysop region?  I think you are confusing
> the potential for interference with actual interference, like
> the philosopher's god who was omnipotent on the condition that
> it never exercise its power.

Sorry, I missed out on a lot of the discussion about exactly
what the Sysop is.  Forgive me, but there are so many
fascinating, diverse things to look into these days :-)

But I will do my best to frankly answer.  If I ever run some
sort of Sysop within my own dominion, in order to keep my own
creations from being really, really cruel in their simulations,
then I might not allow historical simulations.  For example,
if in my piece of computronium, an entity that I created
wanted to do research on World War II, or Mao's Great Leap 
Forward, and the only way to answer certain questions included
the recreation all the incredible suffering that took place,
then conceivably I might not permit it.

If you do conquer me, shall I be allowed to conduct historical
recreations?

>> Fourth:  So, in short, are you asking us to just
>> "trust you, and everything will be all right"?

> No.  I would sort of appreciate if you were to assume that we
> are not idiots, and that everything you can see, we can see...
> Failing that, you can try to get a feel for Eliezer Yudkowsky
> and figure out whether he's the sort of person who probably
> follows protocols that avoid the need for trust.

Sorry.  (And from everything I've read, E.Y. is a very good
fellow indeed.)  When, in the distant future, your AI comes
on line---supposing that Microsoft didn't write one first
under the same benevolent impulses and has already "got" to
both you and me---then we shall indeed study the protocols and
determine whether one of me is going to live under it or not.

(Not that it sounds as though it's going to be in any way
voluntary.)

Lee Corbin

Rate This Message: http://www.cryonet.org/cgi-bin/rate.cgi?msg=15945