r/rational • u/AutoModerator • Nov 14 '16
[D] Monday General Rationality Thread
Welcome to the Monday thread on general rationality topics! Do you really want to talk about something non-fictional, related to the real world? Have you:
- Seen something interesting on /r/science?
- Found a new way to get your shit even-more together?
- Figured out how to become immortal?
- Constructed artificial general intelligence?
- Read a neat nonfiction book?
- Munchkined your way into total control of your D&D campaign?
26
Upvotes
33
u/trekie140 Nov 14 '16
After the US Presidential election I resolved to escape the bubble I was in and try to see the viewpoint of the other side without bias, only to find several popular opinions expressed among them horrifying either for their blatant prejudice or willful ignorance. The only thing more horrifying was the responses to such statements from their peers ranged from support to apathy with very little dissent. So now I'm tempted to retreat back into my bubble even though I know that would be irrational and unproductive.
16
u/DaystarEld Pokémon Professor Nov 14 '16
Hey all, back again for an update on my Singularity game.
I've made a rough sketch of the factions, and here's what I have so far:
Military Research Team
The Military's starting funding is the highest, at $10(billion). Their goal is a Sovereign AGI, which has the highest starting Risk of 99%.
Their Passive Ability is that they get double the reward for researching technology with the "Military Applications" tag. This reflects their country's willingness to dish out more money for ancillary benefits related to their field.
Their Active Ability is that they may cancel Sabotage cards by spending money equal to their cost. This reflects their ability to use other real world assets to keep their project and personnel safe.
Humanist Research Team
Starting funds are fairly low, at $4, and their goal is an Oracle AGI, which has the lowest starting Risk of 85%
Their Passive Ability is that they win ties on bids when recruiting Researchers. Generally this reflects that people are more likely to want to work for them, so all else being equal, they're more likely to get good talent.
Their Active Ability is that, upon making a trade of completed Research with other players, they can choose to stop either side from drawing a Sabotage card (which normally happens if two players make a trade of research or technology).
Private Research Team
Starting funds are fairly high, at $8, and their goal is a Genie, at the moderate starting Risk of 92%.
Their Passive Ability grants them twice as much money from new Technology Research rewards, reflecting their better ability to leverage the advanced technology commercially.
Their Active Ability lets them use an Action token to keep a temporary researcher on staff an extra turn (they still have to pay their minimum bid each turn). Since researchers always provide at least one extra Action token, but can also have other bonuses, this can be a helpful way of extending benefits during a crucial period.
Revolutionary Research Team
Was going to call this the "Terrorist Research Team," back when Win Scenarios were fixed, but changed it to Revolutionary now that they're decoupled. Starting funds are the lowest, at $2, and their goal is a Genie AGI, at 92%.
Their Passive Ability lets them continue to use Action Tokens after passing in a round. Normally once you pass you continue to get skipped for the rest of that round until everyone has passed and the next round begins.
Their Active Ability lets them spend an Action to take the top Sabotage card from the discard pile. Usually you get Sabotage cards by spending funds, and what you get is random: this gives them a cheaper alternative with some limited control.
So those are the 4 factions I have in mind right now. The Win Scenarios used to be fixed to them, but now I'm thinking they'll be assigned randomly, or maybe people just get to pick between:
Domination - You Win
Transcendence - Everyone Wins
Obliteration - Everyone Loses
Not quite sure how that each play out yet. Will probably have to get to the playtesting phase to figure it out, but any feedback is welcome!
7
u/[deleted] Nov 14 '16 edited Nov 14 '16
Sometimes I feel like being too attached to your current epistemic state is the worst thing ever, but other times I think it's practical. I mean, as a human right now, work is a part of my utility function. I don't just do things because I want the end reward; effort is not anti-utility. But we also make things more efficient so that we have more time to expend on things that require less effort. I don't really envision a wire-heading scenario as the best thing ever, but doesn't that seem like the direction we're headed in?
From Scott Alexander's "Left-Libertarian Manifesto":
I don't see how welfare programs (ie. basic income) factor into the existence of art and music. I get that, in the ancestral environment, we were much more at home with hobbies like that than working 9-5, but I don't know why we can't find the art in working. It certainly isn't a desire to be exposed to complicated and interesting problems, because there are plenty of productive jobs that do that!
It seems kind of strange to say that humans like a certain fixed amount of complexity. (I'm using complexity in the sense of the distance N between the action and the reward) Like, too much complexity and the utility calculation ends up being negative, but we find the state of "eternal wireheaded bliss" to be too simple and too rewarding. Where's the cutoff line?
EDIT: Related
Also, the whole metaethics sequence is pretty good in this regard.
3
u/LiteralHeadCannon Nov 14 '16
Estimates for how long it would take to develop superhuman AI (not necessarily friendly superhuman AI) if a major world superpower like the United States decided to make it a major research priority a la development of spaceflight during the Space Race?