Rationality – From AI to Zombies by Eliezer Yudkowsky

Rationality From AI to Zombies
by Eliezer Yudkowsky

Buy for free or more!

Released under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license
⃝cc BY-NC-SA 3.0
http://creativecommons.org/licenses/by-nc-sa/3.0/

Contents v
Preface
Biases: An Introduction xxiii

Book I 
Map and Territory

A Predictably Wrong
. 1  What Do I Mean By “Rationality”? 7
. 2  Feeling Rational 12
. 3  WhyTruth?And… 15
. 4  . . . What’s a Bias, Again? 19
. 5  Availability 23
. 6  Burdensome Details 26
. 7  Planning Fallacy 30
. 8  Illusion of Transparency: Why No One Understands You 34
. 9  Expecting Short Inferential Distances 37
. 10  The Lens That Sees Its Own Flaws 40

B Fake Beliefs
. 11  Making Beliefs Pay Rent (in Anticipated Experiences) 45
. 12  A Fable of Science and Politics 49
. 13  Belief in Belief 54
. 14  Bayesian Judo 59
. 15  Pretending to be Wise 61
. 16  Religion’s Claim to be Non-Disprovable 65
. 17  Professing and Cheering 69
. 18  Belief as Attire 72
. 19  Applause Lights 74

C Noticing Confusion
. 20  Focus Your Uncertainty 81
. 21  What Is Evidence? 84
. 22  Scientific Evidence, Legal Evidence, Rational Evidence 88
. 23  How Much Evidence Does It Take? 91
. 24  Einstein’s Arrogance 95
. 25  Occam’s Razor 98
. 26  Your Strength as a Rationalist 103
. 27  Absence of Evidence Is Evidence of Absence 106
. 28  Conservation of Expected Evidence 109
. 29  Hindsight Devalues Science 112

D Mysterious Answers
. 30  Fake Explanations 117
. 31  Guessing the Teacher’s Password 120
. 32  Science as Attire 124
. 33  Fake Causality 127
. 34  Semantic Stopsigns 132
. 35  Mysterious Answers to Mysterious Questions 136
. 36  The Futility of Emergence 140
. 37  Say Not “Complexity” 143
. 38  Positive Bias: Look into the Dark 147
. 39  Lawful Uncertainty 150
. 40  My Wild and Reckless Youth 154
. 41  Failing to Learn from History 158
. 42  Making History Available 160
. 43  Explain/Worship/Ignore? 163
. 44  “Science” as Curiosity-Stopper 165
. 45  Truly Part of You 168

Interlude: The Simple Truth 173

Book II 
How to Actually Change Your Mind

Rationality: An Introduction 199

E Overly Convenient Excuses
. 46  The Proper Use of Humility 211
. 47  The Third Alternative 216
. 48  Lotteries: A Waste of Hope 219
. 49  New Improved Lottery 221
. 50  But There’s Still a Chance, Right? 224
. 51  The Fallacy of Gray 227
. 52  Absolute Authority 232
. 53  HowtoConvinceMeThat2+2=3 239
. 54  Infinite Certainty 242
. 55  0 And 1 Are Not Probabilities 246
. 56  Your Rationality Is My Business 250

F Politics and Rationality
. 57  Politics is the Mind-Killer 255
. 58  Policy Debates Should Not Appear One-Sided 257
. 59  The Scales of Justice, the Notebook of Rationality 261
. 60  Correspondence Bias 264
. 61  Are Your Enemies Innately Evil? 267
. 62  Reversed Stupidity Is Not Intelligence 270
. 63  Argument Screens Off Authority 274
. 64  Hug the Query 280
. 65  Rationality and the English Language 282
. 66  Human Evil and Muddled Thinking 286

G Against Rationalization
. 67  Knowing About Biases Can Hurt People 293
. 68  Update Yourself Incrementally 296
. 69  One Argument Against An Army 299
. 70  The Bottom Line 302
. 71  What Evidence Filtered Evidence? 305
. 72  Rationalization 309
. 73  A Rational Argument 312
. 74  Avoiding Your Belief ’s Real Weak Points 315
. 75  Motivated Stopping and Motivated Continuation 320
. 76  Fake Justification 323
. 77  Is That Your True Rejection? 326
. 78  Entangled Truths, Contagious Lies 330
. 79  Of Lies and Black Swan Blowups 334
. 80  Dark Side Epistemology 335

H Against Doublethink
. 81  Singlethink 343
. 82  Doublethink (Choosing to be Biased) 345
. 83  No, Really, I’ve Deceived Myself 349
. 84  Belief in Self-Deception 351
. 85  Moore’s Paradox 356
. 86  Don’t Believe You’ll Self-Deceive 359
. I Seeing with Fresh Eyes
. 87 Anchoring and Adjustment 365
. 88 Priming and Contamination 368 Do
. 89 We Believe Everything We’re Told? 371
. 90 Cached Thoughts 374
. 91 The “Outside the Box” Box 377
. 92 Original Seeing 380
. 93 Stranger than History 383
. 94 The Logical Fallacy of Generalization from Fictional Evidence 385
. 95 The Virtue of Narrowness 391
. 96 How to Seem (and Be) Deep 395
. 97 We Change Our Minds Less Often Than We Think 399
. 98 Hold Off On Proposing Solutions 401
. 99 The Genetic Fallacy 404

J Death Spirals
. 100  The Affect Heuristic 409
. 101  Evaluability (and Cheap Holiday Shopping) 413
. 102  Unbounded Scales, Huge Jury Awards, and Futurism 418
. 103  The Halo Effect 422
. 104  Superhero Bias 426
. 105  Mere Messiahs 430
. 106  Affective Death Spirals 434
. 107  Resist the Happy Death Spiral 437
. 108  Uncritical Supercriticality 443
. 109  Evaporative Cooling of Group Beliefs 447
. 110  When None Dare Urge Restraint 451
. 111  The Robbers Cave Experiment 454
. 112  Every Cause Wants to Be a Cult 458
. 113  Guardians of the Truth 461
. 114  Guardians of the Gene Pool 466
. 115  Guardians of Ayn Rand 468
. 116  Two Cult Koans 473
. 117  Asch’s Conformity Experiment 476
. 118  On Expressing Your Concerns 480
. 119  Lonely Dissent 483
. 120  Cultish Countercultishness 487

K Letting Go
. 121  The Importance of Saying “Oops” 497
. 122  The Crackpot Offer 500
. 123  Just Lose Hope Already 503
. 124  The Proper Use of Doubt 505
. 125  You Can Face Reality 508
. 126  The Meditation on Curiosity 509
. 127  No One Can Exempt You From Rationality’s Laws 513
. 128  Leave a Line of Retreat 516
. 129  Crisis of Faith 520
. 130  The Ritual 528

Book III
 The Machine in the Ghost

Minds: An Introduction 539

Interlude: The Power of Intelligence 547

L The Simple Math of Evolution
. 131  An Alien God 553
. 132  The Wonder of Evolution 561
. 133  Evolutions Are Stupid (But Work Anyway) 565
. 134  No Evolutions for Corporations or Nanodevices 570
. 135  Evolving to Extinction 576
. 136  The Tragedy of Group Selectionism 582
. 137  Fake Optimization Criteria 588
. 138  Adaptation-Executers, Not Fitness-Maximizers 591
. 139  Evolutionary Psychology 595
. 140  An Especially Elegant Evolutionary Psychology Experiment 600
. 141  Superstimuli and the Collapse of Western Civilization 604
. 142  Thou Art Godshatter 608

M Fragile Purposes
. 143  Belief in Intelligence 617
. 144  Humans in Funny Suits 621
. 145  Optimization and the Intelligence Explosion 627
. 146  Ghosts in the Machine 634
. 147  Artificial Addition 639
. 148  Terminal Values and Instrumental Values 645
. 149  Leaky Generalizations 654
. 150  The Hidden Complexity of Wishes 658
. 151  Anthropomorphic Optimism 665
. 152  Lost Purposes 669

N A Human’s Guide to Words
. 153 The Parable of the Dagger 677
. 154  The Parable of Hemlock 679
. 155  Words as Hidden Inferences 683
. 156  Extensions and Intensions 686
. 157  Similarity Clusters 690
. 158  Typicality and Asymmetrical Similarity 692
. 159  The Cluster Structure of Thingspace 695
. 160  Disguised Queries 699
. 161  Neural Categories 704
. 162  How An Algorithm Feels From Inside 710
. 163  Disputing Definitions 716
. 164  Feel the Meaning 721
. 165  The Argument from Common Usage 725
. 166  Empty Labels 730
. 167  Taboo Your Words 733
. 168  Replace the Symbol with the Substance 737
. 169  Fallacies of Compression 742
. 170  Categorizing Has Consequences 746
. 171  Sneaking in Connotations 749
. 172  Arguing “By Definition” 753
. 173  Where to Draw the Boundary? 757
. 174  Entropy, and Short Codes 761
. 175  Mutual Information, and Density in Thingspace 765
. 176  Superexponential Conceptspace, and Simple Words 772
. 177  Conditional Independence, and Naive Bayes 779
. 178  Words as Mental Paintbrush Handles 787
. 179  Variable Question Fallacies 790
. 180  37 Ways That Words Can Be Wrong 793

Interlude: An Intuitive Explanation of Bayes’s Theorem 803

Book IV Mere Reality

The World: An Introduction 834

O Lawful Truth
. 181  Universal Fire 843
. 182  Universal Law 847
. 183  Is Reality Ugly? 850
. 184  Beautiful Probability 855
. 185  Outside the Laboratory 861
. 186  The Second Law of Thermodynamics, and Engines of Cognition 867
. 187  Perpetual Motion Beliefs 875
. 188  Searching for Bayes-Structure 879

P Reductionism 101
. 189  Dissolving the Question 887
. 190  Wrong Questions 892
. 191  Righting a Wrong Question 895
. 192  Mind Projection Fallacy 899
. 193  Probability is in the Mind 901
. 194  The Quotation is Not the Referent 906
. 195  Qualitatively Confused 909
. 196  Think Like Reality 913
. 197  Chaotic Inversion 916
. 198  Reductionism 918
. 199  Explaining vs. Explaining Away 923
. 200  Fake Reductionism 928
. 201  Savannah Poets 931

Q Joy in the Merely Real
. 202  Joy in the Merely Real 939
. 203  Joy in Discovery 942
. 204  Bind Yourself to Reality 946
. 205  If You Demand Magic, Magic Won’t Help 949
. 206  Mundane Magic 953
. 207  The Beauty of Settled Science 957
. 208  Amazing Breakthrough Day: April 1st 959
. 209  Is Humanism a Religion Substitute? 962
. 210  Scarcity 965
. 211  The Sacred Mundane 968
. 212  To Spread Science, Keep It Secret 972
. 213  Initiation Ceremony 976

R Physicalism 201
. 214  Hand vs. Fingers 983
. 215  Angry Atoms 986
. 216  Heat vs. Motion 991
. 217  Brain Breakthrough! It’s Made of Neurons! 996
. 218  When Anthropomorphism Became Stupid 998
. 219  A Priori 1002
. 220  Reductive Reference 1006
. 221  Zombies! Zombies? 1012
. 222  Zombie Responses 1032
. 223  The Generalized Anti-Zombie Principle 1040
. 224  GAZP vs. GLUT 1050
. 225  Belief in the Implied Invisible 1058
. 226  Zombies: The Movie 1063
. 227  Excluding the Supernatural 1068
. 228  Psychic Powers 1075

S Quantum Physics and Many Worlds
. 229  Quantum Explanations 1081
. 230  Configurations and Amplitude 1087
. 231  Joint Configurations 1098
. 232  Distinct Configurations 1103
. 233  Collapse Postulates 1111
. 234  Decoherence is Simple 1115
. 235  Decoherence is Falsifiable and Testable 1125
. 236  Privileging the Hypothesis 1134
. 237  Living in Many Worlds 1139
. 238  Quantum Non-Realism 1145
. 239  If Many-Worlds Had Come First 1154
. 240  Where Philosophy Meets Science 1163
. 241  Thou Art Physics 1168
. 242  Many Worlds, One Best Guess 1173

T Science and Rationality
. 243  The Failures of Eld Science 1187
. 244  The Dilemma: Science or Bayes? 1196
. 245  Science Doesn’t Trust Your Rationality 1201
. 246  When Science Can’t Help 1205
. 247  Science Isn’t Strict Enough 1210
. 248  Do Scientists Already Know This Stuff ? 1215
. 249  No Safe Defense, Not Even Science 1221
. 250  Changing the Definition of Science 1227
. 251  Faster Than Science 1229
. 252  Einstein’s Speed 1233
. 253  That Alien Message 1241
. 254  My Childhood Role Model 1250
. 255  Einstein’s Superpowers 1256
. 256  Class Project 1261

Interlude: A Technical Explanation of Technical Explanation 1267

Book V Mere Goodness

Ends: An Introduction 1321

U Fake Preferences
. 257  Not for the Sake of Happiness (Alone) 1329
. 258  Fake Selfishness 1333
. 259  Fake Morality 1335
. 260  Fake Utility Functions 1338
. 261  Detached Lever Fallacy 1342
. 262  Dreams of AI Design 1348
. 263  The Design Space of Minds-in-General 1353

V Value Theory
. 264 Where Recursive Justification Hits Bottom 1359
. 265 My Kind of Reflection 1368
. 266 No Universally Compelling Arguments 1372
. 267 Created Already in Motion 1377
. 268 Sorting Pebbles into Correct Heaps 1380
. 269 2-Place and 1-Place Words 1384
. 270 What Would You Do Without Morality? 1389
. 271  Changing Your Metaethics 1391
. 272  Could Anything Be Right? 1395
. 273  Morality as Fixed Computation 1400
. 274  Magical Categories 1405
. 275  The True Prisoner’s Dilemma 1414
. 276  Sympathetic Minds 1419
. 277  High Challenge 1424
. 278  Serious Stories 1428
. 279  Value is Fragile 1437
. 280  The Gift We Give to Tomorrow 1443

W Quantified Humanism
. 281  Scope Insensitivity 1453
. 282  One Life Against the World 1456
. 283  The Allais Paradox 1459
. 284  Zut Allais! 1462
. 285  Feeling Moral 1469
. 286  The “Intuitions” Behind “Utilitarianism” 1472
. 287  Ends Don’t Justify Means (Among Humans) 1480
. 288  Ethical Injunctions 1485
. 289  Something to Protect 1493
. 290  When (Not) to Use Probabilities 1499
. 291  Newcomb’s Problem and Regret of Rationality 1505

Interlude: The Twelve Virtues of Rationality 1516

Book VI Becoming Stronger

Beginnings: An Introduction 1527

X Yudkowsky’s Coming of Age
. 292  My Childhood Death Spiral 1535
. 293  My Best and Worst Mistake 1539
. 294  Raised in Technophilia 1544
. 295  A Prodigy of Refutation 1550
. 296  The Sheer Folly of Callow Youth 1554
. 297  That Tiny Note of Discord 1561
. 298  Fighting a Rearguard Action Against the Truth 1567
. 299  My Naturalistic Awakening 1571
. 300  The Level Above Mine 1576
. 301  The Magnitude of His Own Folly 1580
. 302  Beyond the Reach of God 1586
. 303  My Bayesian Enlightenment 1595

Y Challenging the Difficult
. 304  Tsuyoku Naritai! (I Want to Become Stronger) 1605
. 305  Tsuyoku vs. the Egalitarian Instinct 1608
. 306  Trying to Try 1610
. 307  Use the Try Harder, Luke 1614
. 308  On Doing the Impossible 1617
. 309  Make an Extraordinary Effort 1624
. 310  Shut Up and Do the Impossible! 1629
. 311  Final Words 1639

Z The Craft and the Community
. 312 Raising the Sanity Waterline 1651
. 313 A Sense That More Is Possible 1655
. 314 Epistemic Viciousness 1660
. 315 Schools Proliferating Without Evidence 1663
. 316 Three Levels of Rationality Verification 1666
. 317 Why Our Kind Can’t Cooperate 1670
. 318 Tolerate Tolerance 1678
. 319 Your Price for Joining 1681
. 320 Can Humanism Match Religion’s Output? 1686
. 321 Church vs. Taskforce 1691
. 322 Rationality: Common Interest of Many Causes 1696
. 323 Helpless Individuals 1700
. 324 Money: The Unit of Caring 1703
. 325 Purchase Fuzzies and Utilons Separately 1707
. 326 Bystander Apathy 1712
. 327 Collective Apathy and the Internet 1716
. 328 Incremental Progress and the Valley 1719
. 329  Bayesians vs. Barbarians 1724
. 330  Beware of Other-Optimizing 1731
. 331  Practical Advice Backed by Deep Theories 1736
. 332 The Sin of Underconfidence 1740
. 333  Go Forth and Create the Art! 1746

Bibliography

Forfatter: krabat

digter. webmaster på menneske.dk. oversætter. EPUB-kritiker.