Rationality – From AI to Zombies by Eliezer Yudkowsky

Rationality From AI to Zombies
by Eliezer Yudkowsky

Buy for free or more!

Released under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license
⃝cc BY-NC-SA 3.0
http://creativecommons.org/licenses/by-nc-sa/3.0/

Contents v
Preface
Biases: An Introduction xxiii

Book I 
Map and Territory

A Predictably Wrong
. 1  What Do I Mean By “Rationality”? 7
. 2  Feeling Rational 12
. 3  WhyTruth?And… 15
. 4  . . . What’s a Bias, Again? 19
. 5  Availability 23
. 6  Burdensome Details 26
. 7  Planning Fallacy 30
. 8  Illusion of Transparency: Why No One Understands You 34
. 9  Expecting Short Inferential Distances 37
. 10  The Lens That Sees Its Own Flaws 40

B Fake Beliefs
. 11  Making Beliefs Pay Rent (in Anticipated Experiences) 45
. 12  A Fable of Science and Politics 49
. 13  Belief in Belief 54
. 14  Bayesian Judo 59
. 15  Pretending to be Wise 61
. 16  Religion’s Claim to be Non-Disprovable 65
. 17  Professing and Cheering 69
. 18  Belief as Attire 72
. 19  Applause Lights 74

C Noticing Confusion
. 20  Focus Your Uncertainty 81
. 21  What Is Evidence? 84
. 22  Scientific Evidence, Legal Evidence, Rational Evidence 88
. 23  How Much Evidence Does It Take? 91
. 24  Einstein’s Arrogance 95
. 25  Occam’s Razor 98
. 26  Your Strength as a Rationalist 103
. 27  Absence of Evidence Is Evidence of Absence 106
. 28  Conservation of Expected Evidence 109
. 29  Hindsight Devalues Science 112

D Mysterious Answers
. 30  Fake Explanations 117
. 31  Guessing the Teacher’s Password 120
. 32  Science as Attire 124
. 33  Fake Causality 127
. 34  Semantic Stopsigns 132
. 35  Mysterious Answers to Mysterious Questions 136
. 36  The Futility of Emergence 140
. 37  Say Not “Complexity” 143
. 38  Positive Bias: Look into the Dark 147
. 39  Lawful Uncertainty 150
. 40  My Wild and Reckless Youth 154
. 41  Failing to Learn from History 158
. 42  Making History Available 160
. 43  Explain/Worship/Ignore? 163
. 44  “Science” as Curiosity-Stopper 165
. 45  Truly Part of You 168

Interlude: The Simple Truth 173

Book II 
How to Actually Change Your Mind

Rationality: An Introduction 199

E Overly Convenient Excuses
. 46  The Proper Use of Humility 211
. 47  The Third Alternative 216
. 48  Lotteries: A Waste of Hope 219
. 49  New Improved Lottery 221
. 50  But There’s Still a Chance, Right? 224
. 51  The Fallacy of Gray 227
. 52  Absolute Authority 232
. 53  HowtoConvinceMeThat2+2=3 239
. 54  Infinite Certainty 242
. 55  0 And 1 Are Not Probabilities 246
. 56  Your Rationality Is My Business 250

F Politics and Rationality
. 57  Politics is the Mind-Killer 255
. 58  Policy Debates Should Not Appear One-Sided 257
. 59  The Scales of Justice, the Notebook of Rationality 261
. 60  Correspondence Bias 264
. 61  Are Your Enemies Innately Evil? 267
. 62  Reversed Stupidity Is Not Intelligence 270
. 63  Argument Screens Off Authority 274
. 64  Hug the Query 280
. 65  Rationality and the English Language 282
. 66  Human Evil and Muddled Thinking 286

G Against Rationalization
. 67  Knowing About Biases Can Hurt People 293
. 68  Update Yourself Incrementally 296
. 69  One Argument Against An Army 299
. 70  The Bottom Line 302
. 71  What Evidence Filtered Evidence? 305
. 72  Rationalization 309
. 73  A Rational Argument 312
. 74  Avoiding Your Belief ’s Real Weak Points 315
. 75  Motivated Stopping and Motivated Continuation 320
. 76  Fake Justification 323
. 77  Is That Your True Rejection? 326
. 78  Entangled Truths, Contagious Lies 330
. 79  Of Lies and Black Swan Blowups 334
. 80  Dark Side Epistemology 335

H Against Doublethink
. 81  Singlethink 343
. 82  Doublethink (Choosing to be Biased) 345
. 83  No, Really, I’ve Deceived Myself 349
. 84  Belief in Self-Deception 351
. 85  Moore’s Paradox 356
. 86  Don’t Believe You’ll Self-Deceive 359
. I Seeing with Fresh Eyes
. 87 Anchoring and Adjustment 365
. 88 Priming and Contamination 368 Do
. 89 We Believe Everything We’re Told? 371
. 90 Cached Thoughts 374
. 91 The “Outside the Box” Box 377
. 92 Original Seeing 380
. 93 Stranger than History 383
. 94 The Logical Fallacy of Generalization from Fictional Evidence 385
. 95 The Virtue of Narrowness 391
. 96 How to Seem (and Be) Deep 395
. 97 We Change Our Minds Less Often Than We Think 399
. 98 Hold Off On Proposing Solutions 401
. 99 The Genetic Fallacy 404

J Death Spirals
. 100  The Affect Heuristic 409
. 101  Evaluability (and Cheap Holiday Shopping) 413
. 102  Unbounded Scales, Huge Jury Awards, and Futurism 418
. 103  The Halo Effect 422
. 104  Superhero Bias 426
. 105  Mere Messiahs 430
. 106  Affective Death Spirals 434
. 107  Resist the Happy Death Spiral 437
. 108  Uncritical Supercriticality 443
. 109  Evaporative Cooling of Group Beliefs 447
. 110  When None Dare Urge Restraint 451
. 111  The Robbers Cave Experiment 454
. 112  Every Cause Wants to Be a Cult 458
. 113  Guardians of the Truth 461
. 114  Guardians of the Gene Pool 466
. 115  Guardians of Ayn Rand 468
. 116  Two Cult Koans 473
. 117  Asch’s Conformity Experiment 476
. 118  On Expressing Your Concerns 480
. 119  Lonely Dissent 483
. 120  Cultish Countercultishness 487

K Letting Go
. 121  The Importance of Saying “Oops” 497
. 122  The Crackpot Offer 500
. 123  Just Lose Hope Already 503
. 124  The Proper Use of Doubt 505
. 125  You Can Face Reality 508
. 126  The Meditation on Curiosity 509
. 127  No One Can Exempt You From Rationality’s Laws 513
. 128  Leave a Line of Retreat 516
. 129  Crisis of Faith 520
. 130  The Ritual 528

Book III
 The Machine in the Ghost

Minds: An Introduction 539

Interlude: The Power of Intelligence 547

L The Simple Math of Evolution
. 131  An Alien God 553
. 132  The Wonder of Evolution 561
. 133  Evolutions Are Stupid (But Work Anyway) 565
. 134  No Evolutions for Corporations or Nanodevices 570
. 135  Evolving to Extinction 576
. 136  The Tragedy of Group Selectionism 582
. 137  Fake Optimization Criteria 588
. 138  Adaptation-Executers, Not Fitness-Maximizers 591
. 139  Evolutionary Psychology 595
. 140  An Especially Elegant Evolutionary Psychology Experiment 600
. 141  Superstimuli and the Collapse of Western Civilization 604
. 142  Thou Art Godshatter 608

M Fragile Purposes
. 143  Belief in Intelligence 617
. 144  Humans in Funny Suits 621
. 145  Optimization and the Intelligence Explosion 627
. 146  Ghosts in the Machine 634
. 147  Artificial Addition 639
. 148  Terminal Values and Instrumental Values 645
. 149  Leaky Generalizations 654
. 150  The Hidden Complexity of Wishes 658
. 151  Anthropomorphic Optimism 665
. 152  Lost Purposes 669

N A Human’s Guide to Words
. 153 The Parable of the Dagger 677
. 154  The Parable of Hemlock 679
. 155  Words as Hidden Inferences 683
. 156  Extensions and Intensions 686
. 157  Similarity Clusters 690
. 158  Typicality and Asymmetrical Similarity 692
. 159  The Cluster Structure of Thingspace 695
. 160  Disguised Queries 699
. 161  Neural Categories 704
. 162  How An Algorithm Feels From Inside 710
. 163  Disputing Definitions 716
. 164  Feel the Meaning 721
. 165  The Argument from Common Usage 725
. 166  Empty Labels 730
. 167  Taboo Your Words 733
. 168  Replace the Symbol with the Substance 737
. 169  Fallacies of Compression 742
. 170  Categorizing Has Consequences 746
. 171  Sneaking in Connotations 749
. 172  Arguing “By Definition” 753
. 173  Where to Draw the Boundary? 757
. 174  Entropy, and Short Codes 761
. 175  Mutual Information, and Density in Thingspace 765
. 176  Superexponential Conceptspace, and Simple Words 772
. 177  Conditional Independence, and Naive Bayes 779
. 178  Words as Mental Paintbrush Handles 787
. 179  Variable Question Fallacies 790
. 180  37 Ways That Words Can Be Wrong 793

Interlude: An Intuitive Explanation of Bayes’s Theorem 803

Book IV Mere Reality

The World: An Introduction 834

O Lawful Truth
. 181  Universal Fire 843
. 182  Universal Law 847
. 183  Is Reality Ugly? 850
. 184  Beautiful Probability 855
. 185  Outside the Laboratory 861
. 186  The Second Law of Thermodynamics, and Engines of Cognition 867
. 187  Perpetual Motion Beliefs 875
. 188  Searching for Bayes-Structure 879

P Reductionism 101
. 189  Dissolving the Question 887
. 190  Wrong Questions 892
. 191  Righting a Wrong Question 895
. 192  Mind Projection Fallacy 899
. 193  Probability is in the Mind 901
. 194  The Quotation is Not the Referent 906
. 195  Qualitatively Confused 909
. 196  Think Like Reality 913
. 197  Chaotic Inversion 916
. 198  Reductionism 918
. 199  Explaining vs. Explaining Away 923
. 200  Fake Reductionism 928
. 201  Savannah Poets 931

Q Joy in the Merely Real
. 202  Joy in the Merely Real 939
. 203  Joy in Discovery 942
. 204  Bind Yourself to Reality 946
. 205  If You Demand Magic, Magic Won’t Help 949
. 206  Mundane Magic 953
. 207  The Beauty of Settled Science 957
. 208  Amazing Breakthrough Day: April 1st 959
. 209  Is Humanism a Religion Substitute? 962
. 210  Scarcity 965
. 211  The Sacred Mundane 968
. 212  To Spread Science, Keep It Secret 972
. 213  Initiation Ceremony 976

R Physicalism 201
. 214  Hand vs. Fingers 983
. 215  Angry Atoms 986
. 216  Heat vs. Motion 991
. 217  Brain Breakthrough! It’s Made of Neurons! 996
. 218  When Anthropomorphism Became Stupid 998
. 219  A Priori 1002
. 220  Reductive Reference 1006
. 221  Zombies! Zombies? 1012
. 222  Zombie Responses 1032
. 223  The Generalized Anti-Zombie Principle 1040
. 224  GAZP vs. GLUT 1050
. 225  Belief in the Implied Invisible 1058
. 226  Zombies: The Movie 1063
. 227  Excluding the Supernatural 1068
. 228  Psychic Powers 1075

S Quantum Physics and Many Worlds
. 229  Quantum Explanations 1081
. 230  Configurations and Amplitude 1087
. 231  Joint Configurations 1098
. 232  Distinct Configurations 1103
. 233  Collapse Postulates 1111
. 234  Decoherence is Simple 1115
. 235  Decoherence is Falsifiable and Testable 1125
. 236  Privileging the Hypothesis 1134
. 237  Living in Many Worlds 1139
. 238  Quantum Non-Realism 1145
. 239  If Many-Worlds Had Come First 1154
. 240  Where Philosophy Meets Science 1163
. 241  Thou Art Physics 1168
. 242  Many Worlds, One Best Guess 1173

T Science and Rationality
. 243  The Failures of Eld Science 1187
. 244  The Dilemma: Science or Bayes? 1196
. 245  Science Doesn’t Trust Your Rationality 1201
. 246  When Science Can’t Help 1205
. 247  Science Isn’t Strict Enough 1210
. 248  Do Scientists Already Know This Stuff ? 1215
. 249  No Safe Defense, Not Even Science 1221
. 250  Changing the Definition of Science 1227
. 251  Faster Than Science 1229
. 252  Einstein’s Speed 1233
. 253  That Alien Message 1241
. 254  My Childhood Role Model 1250
. 255  Einstein’s Superpowers 1256
. 256  Class Project 1261

Interlude: A Technical Explanation of Technical Explanation 1267

Book V Mere Goodness

Ends: An Introduction 1321

U Fake Preferences
. 257  Not for the Sake of Happiness (Alone) 1329
. 258  Fake Selfishness 1333
. 259  Fake Morality 1335
. 260  Fake Utility Functions 1338
. 261  Detached Lever Fallacy 1342
. 262  Dreams of AI Design 1348
. 263  The Design Space of Minds-in-General 1353

V Value Theory
. 264 Where Recursive Justification Hits Bottom 1359
. 265 My Kind of Reflection 1368
. 266 No Universally Compelling Arguments 1372
. 267 Created Already in Motion 1377
. 268 Sorting Pebbles into Correct Heaps 1380
. 269 2-Place and 1-Place Words 1384
. 270 What Would You Do Without Morality? 1389
. 271  Changing Your Metaethics 1391
. 272  Could Anything Be Right? 1395
. 273  Morality as Fixed Computation 1400
. 274  Magical Categories 1405
. 275  The True Prisoner’s Dilemma 1414
. 276  Sympathetic Minds 1419
. 277  High Challenge 1424
. 278  Serious Stories 1428
. 279  Value is Fragile 1437
. 280  The Gift We Give to Tomorrow 1443

W Quantified Humanism
. 281  Scope Insensitivity 1453
. 282  One Life Against the World 1456
. 283  The Allais Paradox 1459
. 284  Zut Allais! 1462
. 285  Feeling Moral 1469
. 286  The “Intuitions” Behind “Utilitarianism” 1472
. 287  Ends Don’t Justify Means (Among Humans) 1480
. 288  Ethical Injunctions 1485
. 289  Something to Protect 1493
. 290  When (Not) to Use Probabilities 1499
. 291  Newcomb’s Problem and Regret of Rationality 1505

Interlude: The Twelve Virtues of Rationality 1516

Book VI Becoming Stronger

Beginnings: An Introduction 1527

X Yudkowsky’s Coming of Age
. 292  My Childhood Death Spiral 1535
. 293  My Best and Worst Mistake 1539
. 294  Raised in Technophilia 1544
. 295  A Prodigy of Refutation 1550
. 296  The Sheer Folly of Callow Youth 1554
. 297  That Tiny Note of Discord 1561
. 298  Fighting a Rearguard Action Against the Truth 1567
. 299  My Naturalistic Awakening 1571
. 300  The Level Above Mine 1576
. 301  The Magnitude of His Own Folly 1580
. 302  Beyond the Reach of God 1586
. 303  My Bayesian Enlightenment 1595

Y Challenging the Difficult
. 304  Tsuyoku Naritai! (I Want to Become Stronger) 1605
. 305  Tsuyoku vs. the Egalitarian Instinct 1608
. 306  Trying to Try 1610
. 307  Use the Try Harder, Luke 1614
. 308  On Doing the Impossible 1617
. 309  Make an Extraordinary Effort 1624
. 310  Shut Up and Do the Impossible! 1629
. 311  Final Words 1639

Z The Craft and the Community
. 312 Raising the Sanity Waterline 1651
. 313 A Sense That More Is Possible 1655
. 314 Epistemic Viciousness 1660
. 315 Schools Proliferating Without Evidence 1663
. 316 Three Levels of Rationality Verification 1666
. 317 Why Our Kind Can’t Cooperate 1670
. 318 Tolerate Tolerance 1678
. 319 Your Price for Joining 1681
. 320 Can Humanism Match Religion’s Output? 1686
. 321 Church vs. Taskforce 1691
. 322 Rationality: Common Interest of Many Causes 1696
. 323 Helpless Individuals 1700
. 324 Money: The Unit of Caring 1703
. 325 Purchase Fuzzies and Utilons Separately 1707
. 326 Bystander Apathy 1712
. 327 Collective Apathy and the Internet 1716
. 328 Incremental Progress and the Valley 1719
. 329  Bayesians vs. Barbarians 1724
. 330  Beware of Other-Optimizing 1731
. 331  Practical Advice Backed by Deep Theories 1736
. 332 The Sin of Underconfidence 1740
. 333  Go Forth and Create the Art! 1746

Bibliography

Author: krabat

digter, forlægger, oversætter, admin på kunstnerhotellet menneske.dk

Leave a Reply

Your email address will not be published. Required fields are marked *

Note: Commenter is allowed to use '@User+blank' to automatically notify your reply to other commenter. e.g, if ABC is one of commenter of this post, then write '@ABC '(exclude ') will automatically send your comment to ABC. Using '@all ' to notify all previous commenters. Be sure that the value of User should exactly match with commenter's name (case sensitive).

This site uses Akismet to reduce spam. Learn how your comment data is processed.