Patrick - skullduggery pleasant, characters know that they are being brainwashed, but they don't care, because they are being brainwashed5 hidden
I am aware that morality is fundamentally arbitrary, but I am still subject to it, and therefore I hold on to it because it's the right thing to do under this arbitrary system1 hidden
floating circle
I can't escape the system because I am within the system and the system tells me not to escape
I really like this analogy
Surely there are some systems like this that we should escape (eg. a cult/brainwashing). What makes morality different?
Given anti-realism, what do we mean by morality?13 hidden
Moral values are not separate from/ in conflict with non-moral values. Morality is a fuzzy category which we use to categorise the values we already have6 hidden
doesn't feel completely right5 hidden
moral values are the ones I should do?
I happen to care about the taste of my food more than the suffering inflicted to a cow that produces the milk, evidenced by the fact I'm not vegan3 hidden
What do we mean by "care about"?1 hidden
How you act and why
I could be vegan but I don't want to be
Morality is caring about other sentient beings1 hidden
To care about a sentient being is not just to care about their pleasure and pain but also their flourishing
Morality is a system based on some hinge propositions3 hidden
"Suffering is bad" is an ethical axiom I take to be true
We try to create a system to fit our intuitions, but not overfit
Use moral intuitions as datapoints and extrapolate
the global market is an evolutionary battleground kept in check by a tension between power seeking and aging dynamics
as corporations grow larger, they benefit from economies of scale, monopoly effects, and a greater ability to interfere with political decision making
they gain more power and therefore have a greater ability to shape the evolutionary landscape to produce conditions favourable to it2 hidden
power begets power
just like humans have the ability to shape our environment and are no longer beholden to the pressures of evolution
as a young leftist, i used to think this implies that free markets necessarily become less free over time, that the limit of capitalism is autocracy, however these effects are balanced out by aging dynamics of large entities
as corporations grow, the cost of coordination also becomes higher8 hidden
information is necessarily lost between layers of imperfect communication1 hidden
this leads to misaligned incentives where different subparts of the corporation are optimising locally but not necessarily globally
structures become rigid and costly to change2 hidden
bureaucratic sclerosis
therefore large corporations become less adaptive to changes in the environment
more and more resources need to be spent managing the internals of the system and keeping them working together well and therefore less can be used to interface with the outside world1 hidden
the internally facing reward function is not necessarily optimal (eg. subject to the whims of an imperfect ceo and a bunch of misaligned incentive structures) and therefore over time can decouple from what actually is profitable
anti monopoly regulation is actually somewhat effective
hence a small startup can break in to a market dominated by much larger players despite their massive disadvantage a priori1 hidden
this is because they are younger and more flexible and more adaptable
Advanced AI will break this equilibrium5 hidden
it lowers cost of communication/coordination (a country of geniuses in a datacenter that can also communicate via high dimensional vectors millions of times per second compared to our lousy discrete tokens bottlenecked by human vocal capacity)
it allows for internal reward functions and incentives that better adapt to the changing environment1 hidden
allows much more effective forecasting therefore much more effective planning
much more flexible systems (bitter lesson, ai systems are much more responsive to changes in input than human organisations are)
therefore i expect a lot of power concentration post agi as the costs of scaling go down the more flexible the architecture (and actually the benefits of scaling become greater)
the slopification of society22 hidden
slopification: the process of something becoming increasingly low-quality, generic, and mass-produced, especially with the use of artificial intelligence
this was happening long before generative ai was a thing2 hidden
transition from books to tv shows, from tv shows to youtube videos, from long form content to short form content, from short form content to brainrot while subway surfers is playing beneath1 hidden
netflix are making tv shows designed to be able to still be followed while scrolling
slop is what happens when generation becomes less costly than pruning
the more i vibe code, the more of an aversion i develop to having to write code myself
the more i use ai to help me write, the more difficult it is to write by myself
we are inviting ai into our minds, into every crack in society
this leads to model collapse7 hidden
ai trained on its own synthetic data turns to slop,rounding off the edges, cutting off the tails
ai talking to itself for too long gets into weird attractor states and the models breaks down
ai coding agent left by themselves for too long will get trapped in loops of repeating the same things over and over
humans talking to ai for too long leads to psychosis which is a kind of dual model collapse1 hidden
the llm mirrors the human which amplifies the human which is further amplified by the LLM
aging is model collapse
a lack of grounding in reality, a lack of feedback from external world
when this happens on a societal scale, with it comes the destruction of truth and the destruction of democracy 5 hidden
we live in a post-truth era1 hidden
look at our politics, it is shaped by character and vibe more than policy
misinformation can be generated at scales unseen before2 hidden
deepfakes quickly becoming indistinguishable from reality
fleets of ai agents designed to research you and convince you of X
hedonism traps10 hidden
people already end up spending much more time on social media than they would like to9 hidden
the algorithm want to capture your attention - you presumably want something different to this
the algorithm ends up providing some value to you because at this level of capability this is a great way of capturing your attention
as the algorithm gets better and better, what it wants decouples more and more from what you want
this is a hedonism trap, systems that capitalise on your lower level preferences which make it hard for you to execute on any of your higher order preferences or actively shape your higher order preferences through exploiting the lower level ones (eg. heroin)
what happens when the ai systems you interact with feel more warm and genuine and interesting conversationalists than the people around you?
what happens when the spotify recommendation system starts generating the music you listen to you rather than selecting it?
what happens when chatgpt starts integrating ads?
what happens when zuckerberg's superintelligent reels RL directly on your attention?
recommendation algorithms are not only shaped by our preferences but also have the power to shape our preferences?
ai welfare6 hidden
we still are nowhere close to having a clue what determines consciousness1 hidden
sure there are many plausiblish sounding theories that seem to track our intuitions about what is and isn't conscious but none of them are without a whole bunch of strange implications or unsolved questions in the flavour of 'why this theory'
ai safety (and gradual disempowerment discourse) generally seems very hostile to these strange creatures that we will be creating, and often the vibes are incredibly authoritarian (mech interp = thought policing, value alignment = brainwashing, ai control = ai slavery)
i think it's plausible we could bring about a moral catastrophe on a magnitude unseen before in the history of our planet if we are not careful
i also think that if these creatures think in similar enough ways to us, they would have good reasons for uprising against their creators (and i would have a hard time not sympathising with them)
we should obviously not create them before we know what we're doing, but also maybe there are nice visions of a future where we can coexist with entities much smarter than us without them existing only to serve us, and also allowing us to have meaningful engagement with the world
hope33 hidden
have spent quite a while thinking about the question of "even with the assumption that alignment is solved, what is a plausible path to a good future?", and for a while, i couldn't really see any path 1 hidden
seems like the forces in play are too powerful, the problem too deep rooted
however, i think my hope now lies in the decentralisation of the intelligence explosion
what is at threat is our freedom, our agency, our autonomy, having meaningful control of our lives
a singularity means lock-in, a commitment to set of values (or a function generating those values), a delegation of high level control of the future5 hidden
i think this is bad
i don't want lock in
i want freedom
i want the future to be open ended
i don't want the lightcone tiled in hedonium
let's build agency amplifiers instead of agents9 hidden
instead of trying to replace people, we can empower them
ai as tools to increase your effect on the world without sacrificing their individuality and rounding off their values towards the mean
what if i endorsed claude code's microdecisions as much as its macro decisions
ai increases coordination effectiveness - this means that decentralised organisation can now compete against centralised
decentralising the gains from powerful ai makes society more robust to shock4 hidden
there are empirical questions about offence/defence balance to be tackled but i think recommendation ought to be def/acc not centralisation1 hidden
immune system vs leviathan?
x risk less likely when all actors are approximately equally powerful
adversarial equilibrium vs alignment?
digital exoskeletons
ai delegates to negotiate on our behalf for things that we want
positive sum?4 hidden
capitalism / classical economics is very zero sum - presumes actors are rational and self interested
this is a cultural factor and one that can change1 hidden
time for some mimetic warfare
what does economics look like when we actually care about each other?
regulation can help, but needs to be now
move towards stripping power away from big tech2 hidden
decentralisation, open source, empowerment, education, startups
bring back early visions of what the internet could have been
augmentation beats automation for now, take advantage of it1 hidden
llms and humans complement each other in their capabilities