00:00Hello, I have Matt Pirkowski here. You may have heard me praising him in just about every podcast that I do with people. He's a very intelligent and semantically dense speaker. You can take his posts from Twitter and get essays and essays and essays out of them by just passing them to GPT. He has a whole set of language that creates for a very efficient neural map in relation to very complex phenomena that are very difficult to understand. And we both are, in my perception, very aligned on a lot of things. We use different language. He uses much more precise language. We may butt heads on the minutiae sometimes online, but I don't think there's really anybody more who I think fundamentally gets the problem set of how we are interrelating as a society right now and seems to get where the structures that we are embedded are driving us in the future. He recently did a call or a talk with Jim Rutt on the concept of time preference. In that talk, they talked about a lot of the nature of where our attention is placed and the energetic constraints on where our cognition tends to flow. They did not so much get into how do we address these problems. I'd like to start with talking about the problem set a little bit and then try to move into what we both think that we can do reasonably to change the future, being that I believe, and I believe you also believe, that the future is not fixed, that there is this inherent element of uncertainty, that we have some degree of agency, and that by taking actions, we can alter the course of our future, though at times it does feel a little fixed and a little bit difficult to flow upstream against.
02:30Why don't we just start with you giving your definition of time preference. Time preference is a financial term, and it's one of those things where it's like, I didn't have that term before and connecting with Robin Hanson on the third episode of Listener John Ash and have seen fleshed out more by Matt, so I'd like to go more into the general topic after that. But how would you frame or describe what time preference is in the financial sense? Sure. Yeah, just to begin, thanks for having me on for the conversation. I appreciate the kind of words, a lot of mutual feelings there in terms of, I also appreciate all of the work that you're doing and your perspective, even though at times online that shows up between us as conflict, but it's the kind of conflict that often occurs. I guess it can be called the vanity of small differences is one way that people talk about it, right, in the sense that I think we both, we are much more alike given how far we are from normative perceptions, but we also have different ways of seeing things and that can lead to some conflict, but hopefully it's always more generative than destructive or frustrating.
03:51But yeah, so thanks for having me on. Time preferences are something, it's a good place to start. I think that the word, I think I was speaking about it recently a little bit online, and I was critiqued a little bit for even using the concept of time preferences as an overly reductive frame, but I think still we have to start somewhere, right, and we have to have a pointer to these phenomenon because that's what language is, is a set of pointers to phenomenon or behaviors in the world. And time preferences, what we're trying to point at and what financial, one of the reasons why this arose in the world of finance is because it's very important to think about how much you can actually understand about the future or how much uncertainty exists in your model of what's about to happen because if you're trying to place investments or value some investment or some allocation of energy or attention or capital, you want to have some understanding of your confidence over how that's going to unfold, how that's going to work out for you.
05:01And if you're very uncertain and if there's extreme variance or the clarity or the robustness or resilience of your model is very low, it's essentially something like a random path, and therefore you kind of have to discount the value you might place over that, the expected value that you might place over that. Pause for a second. We got cut off there, and I was asking you to start off with your definition of time preference and how it relates to your work. Yeah, for sure. So just cutting to the chase in terms of time preference and the idea itself, the idea relates very much. It's a specific lens through the financial lens into this question of how certain is one about the future and how does that change one's behavior in the present.
05:59And so to the extent that one is uncertain about the future and uncertain about what is going to happen in the future, one is going to place perhaps less weight on anything that might take time to unfold, any plans that might take time to unfold, any relationships that might take time to develop, because if that future is very uncertain, you simply don't know if you're going to actually realize any of those plans that actually require a kind of complex unfolding over time or a synchrony or a cooperation, collaboration, trust-based relationship over time. You don't know if any of those are actually going to be able to manifest, and therefore you'll discount those. So in terms of psychology, for example, if I were to say you can have a dollar, I'll give you a dollar right now, or I'll come back tomorrow and give you $10. And if you were to take the dollar, that is telling me something that's telling me something along the lines of something about either me or the situation that we're in means that you really don't place much confidence in me coming back tomorrow and actually realizing that promise to give you that $10.
07:09And so that's sort of like a high time preference. You're very sensitive to that uncertainty over time. The opposite, that low time preference is, okay, yeah, sure, not only that, but maybe you don't have to come back tomorrow. Come back next week. It's fine. I trust you. There's enough stability in the system. My model of this system is sufficiently confident that I think that that future will actually come about. And so I'm not going to discount that future too steeply. And so that's this financial lens of time preferences. But this factors into everything that we do, because we're constantly, as social creatures, nested within these coordination games. We're trying to figure out what makes sense to focus on. When are we taking profits or when are we trying to focus on the outcome or output of something that we feel that we can extract today? Or are we willing to leave that value, so to speak, in the system such that we can create greater value over time or that we can create...
08:17In my work a lot, what I focus on as well is trying to understand money or communication as something I've called coherence mechanisms and the ability to stitch together these kind of maps of time and space. We use our language and our money to come together and actually share bits and fragments of reality itself and ladder those up such that we can use the maps of the collective phenomenology, so to speak, all of the artifacts that we bring back. You go on your journey. I go on my journey. We come back together. We share the information. And now I never had to go on your journey. But if I can trust you, then I can start to use the information from your journey to facilitate my actions in the world.
09:01And you can see how that might scale up. And you can look at all the tools that we create as a species, all the communication tools, all of the accounting tools, all of the books, the encyclopedias, the dictionaries, the maps, all of this information that we've recorded to help us stitch together this larger scale coherent picture of reality. Science itself is a process by which we try to increasingly render a materialistic perspective of the world that is of high fidelity and stability and that many people can use as long as you have the kind of correct grammar in terms of how to access or index into that scientific map. You can use that for whatever purposes you might have. And now we're at this interesting time where we're also generating a lot of power along the trajectory of being able to synthesize our own, like, create synthetic versions of parts of these maps that can be, that are very low cost to implement, but also feel very real. And then there's this question of, you know, it's one thing if everybody's being an honest actor, but obviously the interesting paradox here is the more value you create through the creation of these shared maps, and the more actors are able to actually reduce their time preferences and coordinate and cooperate over longer periods of time to generate collective value in that system, the greater the temptation becomes to defect from that, right? Because the more value you can gain from trying to, you know, play the system, the more you can actually lie. If everyone's very trusting, it actually becomes very easy to lie and then get away with some sort of parasitism or robbery or, you know, value extraction. And so there's always this tension whereby we're constantly trying to figure out who we can trust, how confident our models of the future are, how much we should discount any possible process that's unfolding into the future versus how much we should trust others in this long-term process of value creation or map construction, collaborative map construction. And, you know, all of this is happening concurrently, but, you know, the kind of arguments that I make are arguments that kind of.
11:20Ask the question, well, if we had a knob over that time preference or that trust function and that overall willingness to collaborate or not, like, what are our current technologies doing to that level? And then are there like critical thresholds beyond which the overall behavior of our society transforms? Like, do we gain or lose certain absolute values? And so, you know, what are the possibly essential characteristics or qualities as a society if we go beneath or above certain thresholds of time preference or trust? How does our money or how do the symbols that we use to reflect reality, how do those factor into this question of time preference and how long our collaborative vision can be? So, yeah, I mean, that's sort of like the cluster network of ideas around time preference that I often play with or talk about. I think it's a reasonable yeah, we can unpack any part of that. There's a lot to unpack.
12:17I think that we should get a little into sort of your definition of uncertainty and the spectrum of certainty and knowing it's sort of very fundamental to, you know, like Bayes' theorem and active inference and science and just polling in general. And it is something that I would say the media, for example, has a very difficult time translating to language what it means for there to be a lot of uncertainty in a specific situation. I think it's another, it's also something that physicists have a hard time putting into language in the sense that we have a sense of uncertainty, but we're also talking about it like it's a physically real thing. There is, you know, not a lot of good communication about how and what observation is in the physical sense. And that sort of gets us into the active inference realm of, you know, how we are interrelating with internal representations of the world, how we are flowing through some type of map that we can never really see the other side of, but we can see the, I guess, holographic screen in our mind.
13:59So how would you bring clarity to the concept of uncertainty, knowing that there is this general sense that we are in a present, that's how we perceive reality with our current model of consciousness. And the further out that we go into the past or the future, supposedly, across broad strokes, there is more and more uncertainty. And we have to rely more and more on our tools and our maps to create a sense of certainty that translates to some security and stability in our emotional states for how we relate to the world.
14:57Sure. Yeah. I mean, I think that... As you know, I mean, the question of the word uncertainty is it's a kind of a it's a fascinating pointer because it's actually something that so the linguists and one might call philosopher Terence Deacon has this idea of absential pointers or pointers that are trying to point to absences in the world, right? And the word uncertainty is kind of is one of those absential pointers in the sense that it's it's pointing toward a lack of a lack of knowledge, a lack of predictive capacity, a lack of comprehension.
15:42But all of those things, those words are also interrelated. Like comprehension is also it has that that same root in there as prehensile, the ability to grab and manipulate right with the ability to grab and manipulate, which is also something that relates to this idea of a model, because when we're conceptualizing the world and we're actually trying to understand and predict the world, one of the things we are doing is is grabbing and manipulating our models to try to see how that fits with what's unfolding in the world in front of us and how we can use that to act in the world. And from the active inference perspective, the simple definition is something like uncertainty is is the ability to act on your model in the world without being surprised by how the world responds. Right. And I mean, and that's interesting as well, because, you know, I was actually just having a conversation yesterday with Jordan Hall and a guy named Steph McCurdy from Wolfram Blockchain Labs.
16:39And this this question to some extent came up and I was it'd be interesting to explore this delineation a little bit more with you as well, because there's this sense in which one can be a model of reality or one can have a model of reality. And those differ in the sense that, you know, one of them doesn't require a sort of internal picture or predictive capacity. So the biological emergence of organisms themselves reflects each organism along that chain encodes a kind of symmetry with its adaptive context, with its ecological niche. Right. If you look at the morphology of the body, the fins of fish, the eyes of fish, you know, the gills, all of these reflect certain properties and qualities and functions of their environment.
17:31And you can kind of look at it as a figure in ground relationship where they if you look at the fish as the figure, well, the ground that has given rise to many aspects of that figure is this sort of aqueous environment with a bunch of different other factors in play and then different niches. Right. It's not just like, oh, there's an ocean, therefore there's one kind of fish. There are many different ways to make a living, given all the constraints of that environment and different creatures in that space make their livings in that way and therefore come to reflect and literally embody a model of that set of relationships and that set of processes. And so that's that's one way of being a model of the relationship. And so like uncertainty in that context has a lot to do with questions of a natural selection of changing environments.
18:20Right. Because if you're very well adapted as a model to a particular environment and that environment changes underneath you, well, it's not just that you're surprised because you have a model and the world validated the model that you have. If you are the model and the world invalidates the the the context, that surprise translates very quickly into existential inviability or risk. And that's how and then like the part of that population that's able to navigate that or has maybe late mutations or facets that are able to allow them to not perish. That's this question of natural selection and adaptation and how evolution operates over time based on changes in the environment and how that's reflected in our genetic lineage.
19:06But then you go into this space of creatures that are that begin to be able to not only be models, but also have models of the world which are sort of like virtual, it sort of virtualizes this same process. We have this general cognitive capacity to not just be a reflection, especially humans, since we're so general purpose, we could argue that this is our primary evolutionary skill is this ability to generally model and manipulate the world. And, you know, we are able to simulate and synthesize that same process. But instead of when the environment changes around us, instead of us always having to respond at the level of genetic embodiment, we can adapt and change our models. And this is where this idea of surprise comes into, you know, comes explicit surprise, the kind that we would actually recognize and say, I was surprised this sort of self wrench self referential ability to understand that our way of seeing and understanding the world has been updated by some violation of our assumptions, some violation of our expectations, and that therefore our surprise has increased.
20:15And in the active inference literature of that world, this is modeled with qualities that are called variational free energy and expected free energy, whether you're talking about the current embodied surprise or the future projected surprise, respectively. We don't have to necessarily go there if you don't want. But when I think about uncertainty, it's very much grounded in this space of biological emergence and the way that we either are evolved to be a representation of certain aspects or a model of certain aspects of our reality, or that we have developed the capacity to hold a virtual representation and adapt that based on being surprised by the way the world unfolds around us. And, you know, the extent to which we are able to, you know, we are much more capable of reducing our surprise individually if we have or exist in a collective network in which people are actually working to generate a high quality shared map or model that you can tap into when necessary.
21:19Right. Because if you have to create the entire map yourself, you are far likely to be surprised under a much larger proportion of scenarios. Can I interject right now, just maybe just move into this concept of the energetic constraints of updating the model, because I think there's a very large network effect and individual effects which come down to, as they interrelate to market dynamics, that it takes energy to update the map, right? And there seems to be some sort of hierarchical representation of truths held within a model.
22:05There seems to be sort of a trajectory through space-time that people are on. And that collective map is not always reflecting the collective knowledge that we've accrued. And often will preserve a model that does not fully predict or does not actively predict the world that we're embedded in. Can you just go, like, just continue on and just bring some more of the energetic constraints of prediction and world models into there? Yeah, for sure. Yeah, I mean, I think that's an essential point as well.
22:50I think it's something that we can work backwards on that. We can begin to ask sort of Socratically, why might it be the case that we see a world in which not everyone is updating synchronously and as quickly as new information becomes available for any given model? Why is it not the case that as soon as one person recognizes the ability to improve their behavior by updating their model, that that is instantly spread across the entire space of the entire network of internally held models in each actor's or each person's head? I mean, I think one way of one way of looking at this, I think those who have actually been software engineers and who have, especially those who have worked collectively on teams or worked with open source projects, are at a bit of an advantage here in the sense that you have an intuitive sense for dependency and what that actually implies when you're trying to talk about when you have a map and it's useful at all, even if it's imperfect.
24:01If others start to bake that into certain processes or artifacts in the world that have their own kind of life and inertia, you get this dependency lock in. And that can be, you know, if we're just talking about software, you know, in theory, it shouldn't even be that psychological or emotional of a question. But even in that world, there are human dynamics at play where people can become attached to particular patterns, particular even down, you know, this might not be as much a case these days, but back when this was not automated, people would be attached to even, you know, spacing formats. Right. Like even like these artifacts aren't even necessarily core to the model representation itself, but represent some sort of emotional attachment locally that gets baked in as a dependency and therefore spread through large parts of the system.
24:56And if that system, let's say that someone like two spaces and then another system over here was developed and baked into technological artifacts or code that, you know, liked tabs or whatever. And then you actually have to integrate these, right? You had a boundary conflict. You have a frustration at that boundary conflict and you have to have a negotiation. And that takes, you know, this is this piece of these time and energy constraints and the same kind of process. I bring up a trivial one just for the for the sake and for the point of saying that even in the most trivial of cases, we can have our attachments and preferences that don't necessarily relate to the quality of the model itself. And that can give rise to additional energy and time and frustration when it comes to actually trying to synthesize or integrate preexisting models.
25:46And so that simultaneously, you know, that can be worth investing in if you need to invest in that. But it also has a kind of entropic tendency because it will generate the tendency for someone to say, OK, let's just rewrite this from scratch. Let's just create a new set of models over here. And then, you know, it's kind of a trope where it's like, OK, like you have 19 substandard standards that you want to synthesize. So you're going to create a standard that unifies them all. OK, now you have 20 substandard standards. Right. And so you get this expansion, this entropic expansion of the model space of the representation space itself. And then you start getting this recapitulation of that process of natural selection operating over the model space where, you know, you have new lineages appearing, you have certain integrations or synthesis attempts happening.
26:38And then all of those are trying to all of them are trying to find a niche and trying to function in the world. And so and so those constraints, they guide this process of unfolding of our models in a way that means that it's not it's not so simple as being able to just have a single model that is is most capable and that spreads for myriad reasons amongst the tiny sliver of almost trivial reasons that I just gave in that one very narrow world of programming. But the same thing applies at a very wide lens as well. Like if I wanted to be more controversial and less trivial, we could talk about, you know, the world's mythopoetic representations of archetypes and reality through religious lenses.
27:25And we can talk about the same ideas where, you know, you have splitting of certain models and those might be based on different prophets with different interpretations of particular lenses or or lines of a holy book at one point in time and then creating entire regimes who, while they might actually have more in common with one another than they do with, you know, other like, let's say that if you talk about like Judaism and Islam, you know, they actually have vastly more in common with one another than they do with Taoism, right, in terms of their shared history and their shared lineage. And yet, because of certain local conflicts and certain interpretations and historical contingencies and and the human factor and the fact that, you know, there there's all of these other questions of territorialism, et cetera, like those two models, even though they're very close together, they create some of the most frustration, tension, violence and boundary conflict known to humanity.
28:25And we're also seeing that evidence right now in the current geopolitical landscape. And it's like this problem is fractal and it is it is intractable in the sense that it's never going away. But the question is, like, is it possible for us to create tools, mechanisms, ways of communicating symbol systems or grammars that allow us to perhaps more elegantly flow through that adaptive process while having a bit less conflict and hopefully less less, you know, just outright violence and sort of, you know, resorting to the most brutal and basic forms of reconciliation, which are these forms of dominance and physical force and use of coercion that go all the way back to our history as beings that weren't beings that held models, but beings that just were models, because if you just are a model and you come into conflict with another being that just is a model, you know, you have either you can have symbiotic, but you can also have predatory and you can have conflictual relationships.
29:34And and when you just are a model and you don't have the ability to have a model and synthesize at that level, then you just are almost forced by nature to to resolve it in those embodied in those embodied layers. But we have the luxury of resolving these conflicts at the model layers if we can manage that. But if we cannot manage that, we fall back into the resolution at the the level of being a model as opposed.
30:00Having a model and yeah I mean so that's sort of again that's this network of clustered thoughts that come to mind when you when you talk about these constraints we are dealing with when we're talking about evolution and model space and why why models that are obviously locally useful don't just spread and take over everything immediately like even I mean we still have people in the world who genuinely are convinced or compelled by the model of the earth is flat right yeah because of certain local properties and certain paths that they've taken that make it convincing to them and also make them suspicious of other ways of seeing the world and so they therefore hold that model now it's not everyone but and it's actually not the majority of people but you know depending on the way the winds are blowing that that I mean that's an interesting canary in the coal mine actually in terms of I think the ability for society to value collective or shared models and their overall utility versus the more on like you might predict that in an era where you're seeing a decrease in overall trust a breakdown of our collective capacity to create models share models in our volver models and use them you would expect to see a much larger diversity of these models also the average quality of the model would be lower as well right so like I don't think it's any surprise that we are seeing a rise in and people who subscribe to all sorts of interesting and far-out theories about how reality works yeah I mean it's interesting I have a very large horizon here and I out my window and I feel like I can just see the curve with my eye my naked eye and I do think that there is I mean depending on the the vastness of your view you probably can yeah there's a very basic form of non-conscious learning that sort of drives our neural maps which comes down to repetition and if you are primarily not engaging in a symbolic world you're in prior you're primarily engaging in physical reality you are you know walking through the trees and your relationship to uncertainty is you hear a sound and that sounds might kill you that is very different from the type of algorithmic.
32:34Repetitions that are spurred on right now that if you click on something that says if you have some minor amount of uncertainty or some minor amount of curiosity about flat earth and your mental map is relatively undeveloped you consume that the algorithm recommends another thing which is a variation on that you start to create this repeated signal that the brain through its physical mechanisms is trying to map and model and we create these tight neural loops like there.
33:27Is it an attempt at this sort of synaptic pruning to create a low energy map that you can flow it through that can relate to a changing world that seems to be very much violated by the products that are coming into being there seems to be a relationship in my mind between market dynamics.
33:52And want over time that are throwing off our ability to make maps that function over long time periods I would say if you go back you know a few hundred years if you become a blacksmith you're gonna be a blacklist for the rest of your life right when we have so much innovation when we have so much change that you might not have the same position in two years like everything is changing very rapidly there seems to be a valence to uncertainty that preferences our awareness towards our local and present context and most of our learning is not done in a conscious designed matter most of it is this subconscious process like I have well in remission OCD and it very much grew from a focus on trying to predict the future and my relationship to uncertainty itself.
35:11Where I kept iterating across the map until it got to this point where there was sort of this realization that you can't know everything we have physical constraints to that map that as far as we know cannot fully integrate perfectly through the automated mechanisms a reflection of the outside world that is always going to predict what comes next 100% of the time all of that being said there does seem to be a very tight relationship between uncertainty and time preference that is not well distributed across the market and so when I see the notion of bubbles like a like a market bubble I tend to think of that as like in neural in a neural sense meaning that there are shared neural structures that guide our process of thought over time and the amount of resources that you have available to you is tied to some sense of uncertainty and when there is a violation of expectation at scale the uncertainty is not distributed evenly and the temporal focus or time preference of different people is affected very differently so if we have something like.
37:03The housing market crash of 2008 yes there were whole institutions that went down right but it wasn't the same thing as for an individual where they felt I don't know where I'm going to get my next meal most of the people who had accrued a lot of resources there was no uncertainty about the basic human needs to survive they're not gonna they're not struggling to find somewhere to sleep they're not struggling to you know get their basic needs but the decision-making of the vast majority of the population was being affected in that way was being guided by this uncertainty about their basic reality and so to me it seems that there is not a useful sampling of collective intelligence through the mechanisms of coherence that we have right it's not that profit doesn't.
38:10Afford for an incentive to place our attention towards the future obviously if you are thinking five years out into the future and you actually have a very good map you're going to arrive in a future where you can have greater rewards because you've put in the effort to build out that structure that can reap the sort of energy gradients that exist in that future but it's more that first the the person the current model especially in like tech the tech world is that you have you accrue a lot of losses for many years right you're not in an energetic relationship right away there is an eventual pressure that says you need to make this equation work but for years you are afforded the opportunity to trace out your cognition in a very different way what I've observed in a lot of tech products is that for you know say the first four to five years people are very satisfied very happy with the products that they don't explicitly make profit and then there is this sort of change a sea change which causes a need to restructure in a way that is bringing in more than is being taken out for that energetic engine to continue and so what I'm sort of I guess I'm sort of constantly or to continue as an independent entity as opposed to as opposed to a kind of appendage of some other function yeah there I mean there seems to be to me an inaccurate or unhelpful sampling of the desires beliefs and lived realities of the people in the market through the dynamics that exist and ultimately what our cognition flows towards is towards a future where the majority of people are making their decisions based off of a very.
40:41Short time frame and then we are only encouraging a small percentage of the population to think on those larger time spans so all that just to say I kind of want to get into what might be the solution to this tendency to focus on the short term and is it truly functionally valuable for all you know sort of agents in a society to be thinking at that time frame what is the proper distribution of time preference for a functional society in your mind yeah I think I got a top level to start at the end of that with that question I guess a I'm personally not of the mind that we can be overly prescriptive about the shape of that distribution given that it needs to be responsive and adaptive to to whatever the emergent needs of that moment are and that might entail differently shaped distributions but rewinding all the way to the beginning of where you know where that where you began with your with your comments there because I think that there's a kind of I think that there's a I think that the same substrate agnostic pattern is playing out at different levels here simultaneously with the with respect to our individual biological relationship to the world and then also with respect to our emergent economic relationships and also in their mutual interaction so it's almost this this echo of the same tensions or issues reverberating through those different scales and so to begin with that you know that individual scale that question of.
42:34Grounding or anchoring in a embodied system in an embodied world as an actual platform as an embodied incarnated being that has needs that has biological physical metabolic needs and that has to move through the world because we also have to remember that we're not the kind of life has different branches and one of those look branches you know the world of plants is this branch that explores the way of being in the world where you allow the world to move over you and you don't necessarily move through the world and that's a way of being that's a way of living in the world we kind of forget this but that is that is life that is a way of living in the world we are not that kind of thing we are the kind of thing that that moves through the world and that therefore either in that that being and having of models has to continuously adapt ourselves and our understanding of our context in light of our needs and preferences in this sort of continuous feedback loop and and those those stand in relation to one another and so especially with us because we both are models of our reality and especially you know the the sort of contextual reality that gave rise to us as physical incarnate animals and the kind of thing that can have a model and we have those in relation to one another right so the interesting aspect of having those in relation to one another is that the the having of models the ability to actually understand it and think about and talk with people about the world to plan our you know in our in our cultural history to plan the next day's hunt to then reflect back on what that was like to talk about where there might or might not be bushes that are in bloom or a tree that is fruiting and to share that information or think about who you want to share that with or not all of these kind of aspects of you know being and having models and communicating about that and acting on those in a loop you know that's where this that's you know the kind of the kind of models that we have that's that's its history that's where the merge from right but now as we increasingly decouple the having of model from the being of model right if we have if many people because of the infrastructure that we have collectively developed support the individual life at such a high level for relatively small amounts of local effort.
45:07You know, the ability to go to a store and purchase the needs that one has or the ability to have a shelter for if you just measure things in time, a relatively small amount of time compared to what it might have taken in the past to make sure that your shelter was functional. We're left with a lot of extra time on our hands, and there's this question then, well, what do we do with that? It depends. If you're trying to centrally plan this, you might try to make that maximally efficient in terms of some production function, et cetera. But just from the perspective of an individual, I think it's really interesting to evaluate this question from the point of view of what's called supernormal stimuli, right?
45:54And I don't know, just in case people aren't familiar with supernormal stimuli, it's like we discover these tendencies across many different animals, including ourselves, where certain aspects of our evolved attention function or the salience of what's relevant in our landscape is unbounded at the top end. Meaning like, just to give an example, they studied peacocks, right? And the female peacocks and the male peacocks are typically in this sort of feedback relation in terms of selection over some set of qualities. And that drives, or at least so the theory goes, and it seems to be pretty well-founded, that has at least in part driven the creation of the plumage quality and size of the male peacock.
46:41Obviously, these kind of mating displays are quite prevalent throughout many species with respect to the world of birds. But interestingly enough, you can take, instead of an actual embodied real-life male peacock that's capable of engaging with a female peacock and continuing the species and actually participating in their contextualized natural relationship, you can just take a cardboard cutout with a much, much larger peacock tail with brighter colors that's entirely synthetic, that has nothing to do with the evolutionary history of that species, other than some outside entity is able to simulate it and manipulate it. The female peacocks are actually more attracted locally because they have these evolved mechanisms that in their world, they haven't faulted them, they haven't been in an evolved world in which some outside force is trying to manipulate their senses and use their being of a model against them or to mess with their embodied modelness, let's say, to create weird pathological outcomes.
47:50But you can do it if you introduce these artifacts into their landscape, and they will pay attention to something that is not even an animal, and they'll pay more attention to that than they will to the actual males of their species. So I think that holding that in mind while we think about our own tendencies with respect to our evolved relationship between being a model and having a model, because that ability of humanity to hold models and to use models and to evolve our models and to collectively create models, that has been immensely powerful, and that has been something that throughout the vast majority of our evolutionary history, we have almost always gained from pouring more energy and time into that process. But that process has also almost always been in embodied relationship with one another or with our own life as the arena for testing those hypotheses, those models as hypotheses, where if you think that you have some new way of seeing the world, what you're going to do most likely is then go act on that.
48:54If that's some new way of sharpening your spear or creating a new way of throwing that spear or generating a new way of using fire to preserve meat or whatever, you're going to experiment with these new models, and you're going to be in this tight loop between inaction and model evolution. Now we're in a highly decoupled space where many people still have the brains that are evolved to generate and synthesize models with other humans, but those are now being directed in spaces where they're entirely decoupled from action. There's no action loop. There's no real embodied feedback loop that has any cost function associated with that or that gives them any sort of grounding in reality.
49:40That process can run away in all sorts of directions, and you can look at that almost like cognitive bubbles, and so therefore, you might see where I'm going now in terms of this multi-scale dynamic where when you talk about markets themselves or these emergent functions of humanity, I think a similar pattern can also occur when they become over-abstracted or when they become overly decoupled from the underlying generative processes, and they become more concerned with their own representations and the ways of producing value extraction arbitrage potential in those spaces. You can look at that very much as parasitic activity.
50:28Those people always defend themselves by saying, this is adding liquidity, et cetera. In some cases, I think that there's the ability to add information to that collective model that we get in a market through speculative behavior. That being said, there's also many more ways, I think, to parasitize it and to use it in a manipulative fashion, and we don't necessarily have good tools for understanding who's doing what and who are good actors or who are bad actors in that market. I think that also when you're getting into this question of different people with different time preferences or different people who are, let's say, bound, maybe physically or metabolically or phenomenologically to different layers of time preference, because to the extent that their more basic life processes are, let's say, on autopilot, they don't have to worry about them.
51:28They don't have to worry about paying my rent. They don't have to worry about paying for my food. They don't have to worry about a boss that is going to observe every single action that I'm doing and remove my job for small violations. You move your way to this space where, okay, I have the luxury of being able to purely observe and look for opportunities in this market. In theory, that's supposed to be applied to generating longer-term models of value or identifying places where there are inefficiencies and helping to rectify those by adding information to the market through financial bets, let's say. You get to that level, and then if you're at that level and there are no ways of discerning between whether you are genuinely contributing useful information or you are just focusing on a money-on-money returns function and trying to extract as much money as possible because you're also responding to another supernormal stimuli, which is like, okay, you have an unbounded amount of spending you can do in the world, and if you get on a hedonic treadmill and all you care about is increasing the amount you can satisfy that hedonic treadmill… Are you actually providing value or are you just creating models of a bushy-eared tail peacock?
52:51Yeah, exactly. If you get stuck in this red queen's race kind of dynamic with others who are also playing that game, if you need the newer car and the bigger home constantly and the fancier watch and an ever-increasing wardrobe, and you're constantly in that signaling game and you don't have any sort of cultural or self-regulatory or systemic bounds on that that are going to direct that energy elsewhere that actually ladders up into a higher-quality map or that actually helps to pull more information from more sources into that world, that system becomes inverted.
53:37It becomes a top-heavy inverted triangle where the majority of these dynamics and the majority of this place that's supposed to be—I think there is a kind of natural attentional order in the sense where there is a kind of natural discounting, given that the further you move out into the future, there's a sort of fundamental irreducibility and fundamental uncertainty that comes along with that. And therefore, there's a natural bias toward higher time preferences baked into this structure. But typically, it's not the case that—or it hasn't necessarily been the case that those with access to tools that are operating at the lowest time preference levels in the sense that they have the longest time horizon are also able to control the underlying economic landscape or decision landscape for all the other actors in a way that starts to basically change everybody's psychology so that everybody starts to perceive that game of signaling as reality as opposed to the world beyond human culture as reality.
55:03So we create this envelope, and that envelope—if that envelope of our signaling gets polluted by a game that is unbounded in its desire to maximize local returns regardless of whether it is generating more potential, adaptive potential and functional potential in relation to the actual adaptive world beyond our cultural envelope, then we get into this space where we start becoming maladaptive. We start basically leaning into parasitic behavior, forgetting that there's a world beyond us that we have to maintain adaptive relation with, forgetting that we have built upon all of those natural processes, that they are at least locally or in over certain time periods exhaustible, that there is value in actually building the model itself, that there's value in generating real processes that are in relationship with one another and the world as opposed to just the abstraction, the money function.
56:13I think that has a lot to do with that transition you mentioned with respect to investment forms that try to inject money into a startup or inject money, a capital injection into a hypothesis in the world. At first, it's very exploratory by its nature. The founders have a theory of what they want to do with that capital, and then they try to realize that to some extent. There are many, many books and blogs written about how to adapt that over time based on the signals you get from the people you're trying to provide a technology or service are good to. But then there does come this hard limit, and interestingly enough, that is an attempt at a realignment with reality as expressed by people's behavior as opposed to people's stated desires, like whether people say they want to eat their vegetables if they are only consuming potato chips.
57:19At the point where you have to reconcile with reality, you end up selling potato chips as opposed to vegetables. So you can get stuck in this feedback dynamic where you simultaneously have to help to you have to figure out like, OK, how much is culture actually leading this process? How much how much are our values actually leading this process? How much do we have to transform our actual behavior and how much can we transform our behavior at scale if we're simultaneously under the pressure of a system that is fundamentally trying to refocus us at all times upon low dimensional representations of the world that are that are that are analogous to that fake peacock's tail that are constantly trying to get us to remain on the hedonic treadmill as opposed to develop more grounded relationships by our actions with the world outside of representation with the world as it is as opposed to the world that we represent it.
58:28So, yeah, I think that might have been a little bit like all over the place and rambling. No, no, no, no. I felt like mine was rambling. And then like everything that you said, like said what I was trying to say so much more clear than that. I felt like I was expressing it. And it does really seem to me that. A large portion of the problem is an increased focus on the symbolic map rather than the underlying reality that it's being pulled from. And. I have two different directions that I kind of want to go with this, the first one is just talking about a few criticisms of approaches to solving this, like you've you've given very good critiques of effective altruism, of like long termism, of the notion where you set a fixed outcome and then you lock that in place and then you do all of your optimizing towards that potential future without constantly re-updating that outcome.
59:43And more recently of sort of this notion of consequentialism. And I'd love to just hear. A critique. As to how those different forms of relationship to the future. are just as unbounded from the underlying physical reality, and to my sense, and I think to your sense, are more bound to the symbolic map, which can create a lot of unwanted artifacts that have very little accountability to them in terms of the individuals taking those actions with the consequentialist frame, with the long-termist frame, with the effective altruist frame, or with this outcome-focused type of optimization.
1:00:45Mm-hmm. Yeah, I haven't spoken a whole lot about this extemporaneously, so here we go. We'll see how it goes. Just some thoughts that flow on this matter. When I try to think through, if I try to internally evaluate my own assumptions, if I sort of reflexively look inside myself and try to understand why I say the things I do about those particular ethical frames, I kind of end up at this place that, again, comes down to a sort of focus on fundamental uncertainty, and specifically this kind of computational irreducibility, or the fundamental inadequacy. You could also think of it in terms of Godel's incompleteness theorem, or you could think of it in terms of the fundamental inadequacy of models. You can think of it in terms of the inevitable gap between map and territory.
1:02:03You can think of it as the snake in the garden, the nature of the fact that no matter how perfect we think the environment is, there's always some aspect that we don't understand that is going to manifest more perniciously in proportion to the extent we're confident or overconfident in our understanding. If we don't constantly keep in mind that all of our models of a reality are imperfect and subject to some amount of uncertainty as a very explicit part of that model, we end up overly fixated on the attempt to realize the model as a defense mechanism against uncertainty. If your idea is to minimize uncertainty at all times, and you believe that a very specific form of evaluating reality will help you minimize uncertainty, like if you think you have the ultimate utility function, you begin to over-index, and you begin to try to force all of reality into that model. You have to look at that as a kind of compression function, and you have to ask, how lossy is that? Then you have to ask, as those losses accumulate, how much risk comes along with the inability to perceive that information outside your model, and how that might be changing the assumptions upon which your model is predicated.
1:03:32A lot of this comes down to this question of the same reason why many economic models that take into account expected value, if they actually predicate their models on being in an ergodic world. Ergodicity, this notion that over time, we're going to explore the entire model space. We're going to basically be able to model this as an ensemble of many parallel explorations and then actually get a pretty good picture of what happens in that space. We don't necessarily take into account that every single path, or that we all basically really, in reality, we get one path, and that is a path-dependent path. Therefore, the function that you have to think about is a function that is constantly having to reintegrate the contingencies of its own behavior into its future options.
1:04:32Right? We learn as we go, and we constantly have to update our models, and we have to be open to invalidating our models. That goes all the way down the stack, right? There's a question I like to ask people oftentimes when I meet them, having a first conversation, which is like, if you had to map all of human knowledge to either zero or one as a proportion of all there is to know in the world, zero being we know basically nothing relative to what we could know, and one being we know almost all there is to know of any meaning or worth, what would you choose?
1:05:12It's funny because most of the people I associate with or maintain long-term relationships with answer zero and believe that's like, they can't even understand the idea that people might ever answer one. It's rare that people actually do answer one, but the highest percentage of one answers I've ever seen is within the Bay Area rationalist community. I take that as an interesting signal that there are a group of people that are extremely confident in their fundamental models and their ability to map reality into those models and that do not weight very heavily the idea that any of their foundational axioms could in theory change, that human knowledge itself they believe to be sufficiently stable to predicate pretty fixed utility functions over that knowledge base. For them in their mind, it almost seems like at this point it's more of an issue of working out the details. But from my perspective, it's like as soon as you incorporate our own.
1:06:30Role as generators of uncertainty, computational irreducibility in this system, as soon as you allow for any degree of indeterminism, it becomes the case that almost every model, no matter how rich that model is, has to be essentially mapped to zero in light of all the knowledge that could ever be created about the world and the fact that the knowledge that you actually need and the information that you need for effective action in any given context is highly context-specific and oftentimes to the degree that you're trying to make a larger or more causally relevant or more causally, to the extent that your actions are going to impact a larger and larger portion of the kind of entailment cone, you need greater and greater fidelity over your model. But the thing is, as you get greater and greater fidelity over the model, you begin to approach the limit of reality itself and that becomes kind of paradoxical. And this is where you start getting into this question of simulations within simulations, etc. But at some level, there's always, again, going back to the beginning of this answer, there's always a delta, there's always an epsilon, an error function, there's always the gap, right? And from that gap, if you look at it as a non-ergotic path-dependent function through reality, from that gap can emerge an arbitrary amount of uncertainty over time, right? And that can be unbounded and always is unbounded over a long enough time horizon.
1:08:18And I think that the first step to being able to create high quality adaptive models is to acknowledge that that is the case and that they will never be perfect. And there will always be a process of trying to maintain relationship with reality as it unfolds. We can never create models that we are able to sort of just stamp and put on the shelf and expect to bring back out 10, 50, 100 years later and just put to work and expect them to function as they functioned previously in terms of their ability to function under uncertainty and give you similar outcomes.
1:09:04And so I think this impacts everything. And it impacts, ironically, and we're only beginning to realize this, this also impacts all of our scientific research, especially in the domain of psychology. How do you control for different epistemic or memetic norms in a culture when you're trying to baseline a replication capacity for a psychological study over 50 years or 100 years? And this is where conversations, I mean, I know people give him shit about it, but people like Eric Weinstein, when he talks about the utility of perspectives like gauge theory, that's exactly why it could be in theory relevant. Because what you're looking for when you're talking about something like having an understanding of how the underlying cognitive culture is changing and how that might impact something like a replication of a psychological study over a 50-year interval, what you're kind of looking for is a gauge over that time. You're looking for a transformation, a fundamental transformation that allows you to sort of create a stable relativistic baseline over time, a sort of invariant in that system over time that allows you to then re-imbue a sense of meaning into what might otherwise look like non-replicability.
1:10:34And everything's like that. But unfortunately, acknowledging that everything is like that violates a bunch of assumptions about how stable scientific knowledge is. For example, I mean, a really interesting example as well is when we talk about even cosmological constants. Well, you could either allow for some possibility that those underlying constants might actually be not entirely constant, or you could define them in terms such that their circularity ensures their constancy. And it seems like we've been drifting towards that approach in physics as opposed to the idea or the openness to the idea that there might be a more Bayesian approach that's more real than a frequentist approach where all of our data should be pointing to a constant just because our equations say that that should be a fixed constant. And that speaks to these questions of like, okay, is the universe this fixed, or are the axioms of the universe like fixed laws, or is it actually something more evolutionary? So these questions, when we start asking these questions, they end up rooting down into the deepest questions about our axioms concerning the nature of the universe and our role within the universe and our ability to tap into whatever uncertainty or certainty exists.
1:12:00But I also find that it tends to be those who have the greatest anxiety about surprise who tend to cling most tightly to fixed models that lend an embodied sense of certainty, because it's kind of palliative. It helps to alleviate that anxiety. And that's one of these questions as well, where it's like, okay, well, to what extent do we want to build our entire culture around the emotion of anxiety as opposed to other emotions or other ways of being in the world? Obviously, anxiety is valuable to some extent. It's been conserved. It's a kind of sensitivity to uncertainty. But you can also imagine how maximizing sensitivity to uncertainty in an increasingly uncertain world could also become a kind of pathology. And I think that specific form of pathology is actually quite prevalent within those communities, especially EA, but any communities that fundamentally take utilitarianism or consequentialism too seriously, and also tend to over-index on their own model certainty, let's say.
1:13:21Yeah. There's also... God, there's so much good what you just said there. I was interfacing with GPT recently, and I was just talking about the foundation of physics being built upon the notion that physics is the same everywhere in the universe. And I was like, how could you possibly know that? It's like we just have to base our reasoning on that. I'm like, why? I mean, we receive the signal here. It's not like we can really take good samples in other parts of the universe other than what has arrived at our present location. So it's a very strange notion in alignment with what you're saying about the cosmological concepts. I mean, it gets into questions of the degree to which correspondence and coherence in our models can inform a kind of universality. And the extent to which that local coherence and that local correspondence, the extent to which you can push those assumptions and observations about their stability out to the boundary condition of observable reality, I think that should be an open question. We have pretty high confidence over certain dimensions of that, but we shouldn't. This is the thing. It always surprises me, and I don't really understand why there's such a deep need that people feel to foreclose on the uncertainty in its entirety.
1:14:56If you've gone to like 99.999, it's like, okay, Fine. Do you need to go to 100? Why do you feel the need to get that final bit of uncertainty resolved? It's some kind of an itch, a need, in a way that that openness, the fundamental openness seems pathological from a particular psychological or phenomenological point of view. And I think that there's value in that. I mean, that is partly what drives us to to get more certainty, right? Like it's a function of that. And it's probably also not a surprise that some of our best modelers are those who are most driven to push that uncertainty to the very boundaries. But it can become, I think, pathological and almost even evil if you try to get all the way, right? You become obsessed with the totality of that model certainty. And I think that there's a great book, which is Finite and Infinite Games. Why am I forgetting the author's name right now? It's escaping me. But fundamentally, I think the best definition of evil that I've ever read is in that book. And fundamentally, it comes down to this idea of a need for total closure, a total certainty or finitude over a system. It is essentially evil to try to make of an infinite game, a finite game, right? To reduce an infinite process, an open process to a finite process or closed representation. And to do that in a way that becomes the primary guiding light or the North Star. That is perhaps the fundamental generator function of what we call evil.
1:16:54Yeah, there's also something on the other side of it, I think, which is that because we live in what seems to be a linear timeline, that many of the actions that people are taking in the name of avoiding specific futures, there never will be any resolution as to whether we would have arrived in that future if they hadn't taken that action. It comes down to this sort of notion of utilitarianism. Like, I feel that this action is justified for the greater good, but you're only ever going to arrive in that one future. I was watching Groundhog Day the other day, and I was like, this is a very different notion of time that you could just trace through the same path many different ways and see all of the different outcomes. And tying it back to sort of the algorithmic effects on our cognition, there also is a tendency to train algorithms more towards this Groundhog Day sense of time than it is to do online learning. We collect a corpus of information and it will run through it many different times. That's not something that we can really do in our current relation to time. The best that we can really do is that you have many different agents, and they have many different timelines and many different perceptions of where time is going. It's like we're all these individual universes, and there's a tug and pull against each other. But then you have these people who are acting as if their model of the future is absolute, and therefore they should and can take an action that may harm people in the short run because they know for a fact that they are avoiding a greater harm, yet that is never checked against. There never will be a check against that sense because ultimately you took the action and you can't arrive in that particular future.
1:19:16That's right. The entire idea of counterfactual reasoning always has to be taken with at least a grain, typically probably maybe a spoonful of salt, given the fact that fundamentally it's predicated on the idea of beginning by imagining a world that you're not in. If you actually do acknowledge that we are in a path-dependent, non-ergotic system, we can't just hop off the path that we are in. We can't just step outside of the constraints and the givens that we have to work with as we are. Now, could it be the case that there's value in that simulated process?
1:19:59Well, sure. But what are we trying to do? Why was that conserved? Well, it's conserved because we have the capacity to imagine possible branching points, how reality can unfold, given the ways that we have seen reality unfold. We go back to the question of we're planning a hunt together and then we've had a certain number of hunts that we have been on before. We understand the way that the mammoth behaves to some extent and we're trying to constrain uncertainty. But that uncertainty, all it is, is a way of trying to create a shared image that is more bounded in its uncertainty than any of our individual models is locally. But it is certainly not an ultimate picture of reality that we are able to then swap out for reality. It's just a kind of heuristic that we collectively use. That's kind of what you're getting at. When you're saying the best we can do is have many parallel paths as a collective, that is, to some extent, the way that emergent biology flirts with ergodicity as a tool. You distribute, you parallelize the potential of that species in a way that samples from the overall space, but it doesn't do so in a way that's not... It's all still path dependent. There's still contingency in each one of those paths, even though you get a little bit more of that distributed sampling and then hopefully you get a reconciliation of different paths. If you're in the world of just being a model, that only happens through reproductive success. If you're in the world of being and having models like we are, currently we are in a reproductive act intellectually or with respect to our models right now. This is a form of hopefully reconciliation or a merge point in our respective.
1:22:15Pseudo-ergodic paths of two human beings exploring different possibility spaces along their own trajectories. That being said, it's not as if we can't operate based on the assumption still that our paths are identical. Part of the reason why there's any value whatsoever in us communicating is because of the fact that we have aggregated or accreted a different residual set of information over our base biology along our unique paths, the unique locations we've been, the unique books we've read, the unique jobs we've held, the unique relationships we've had. Then we try to reconcile those and see what we can actually create as a joint residual over that. But that can never be used as a universal representation of reality itself. It's just always a bound on uncertainty in terms of helping one another to navigate the wavefront between past and present, so to speak.
1:23:25I wanted to interject also that something we haven't sort of brought up yet. There's different valence to uncertainty and there are flavors of it that we like. Christmas itself tends to be based around this positive uncertainty, the intentional creation of uncertainty for an object that becomes revealed or for little chocolate calendars or something like that. We've been undermining that, haven't we? We've been undermining that over time. The ability to actually be surprised on Christmas seems less than it used to be.
1:24:11Yeah, there is sort of that in a lot of frames. I was just about to go into the entertainment industry, which there is this narrative that there is sort of a hit of dopamine from the resolution of that unknowing. If you're watching something and you basically know what's going to happen, it just doesn't feel the same way. I've tended to notice that industries that rely on this notion of creating positively valenced uncertainty to be revealed to people, they tend to more struggle with a constant growth cycle where there is sort of an up and down where they have to plan out and then they get back into this regression of the mean, like, this has been working for a while. Franchises have been working for a while, and then there comes this sort of process where there's now this whole industry, for example, with a lot of the franchises of, oh, well, you have all these YouTube channels predicting the outcome. They have all these leakers predicting the stories and trying to sort of minimize that. And then when people get into theater, when people turn it on, they no longer can enjoy it because there has been a movement towards what is expected.
1:25:58This happens in the music industry too, that there are artists on the edge, on the long tail, that are pushing outwards. And there is a market dynamic in the center that is trying to see what has already worked. Just do this very simple type of cognition, which is say, I've seen this working in the past, and so therefore will work in the future. But you can only mine that for so long because there is a pleasure to resolving uncertainty. There is the same thing in comedy, right? If you know the punchline, it's not that funny. And therefore, I think that's why you sort of see in large language models, which is predicting the next token. It's very, very difficult for it to produce things that we find funny, right?
1:26:56Because what it seems to be is that there is a relationship between positive uncertainty and a rewiring of our symbolic maps. And when we create a metaphor or a branching between two different parts of the map that reduces the energy fundamentally that it takes to traverse that map, we feel good. And in my work, that's a very important part of the equation of what Iris is, because there's always this notion that people seem to relate to my work, which is say, Iris is trying to be like this perfect oracle. The intent of it is to just get a better and better map so that we always know what's going to happen. And I think about it in a very, very different way, which is that it is looking backwards towards the error that was previously made to try and uplift the voices who sort of understood in the past the moment that we were going to arrive in. And the goal is to start to prove those people wrong, right? I have very much had this sensation in my life of predicting outcomes that I try to communicate to people.
1:28:30And because I'm on the long tail of knowledge, my map is just very different from another person's map. And so, getting the knowledge between the two maps to exchange or to get into that other map, it just doesn't happen. There's been many, many times where I've said, okay, this type of company or just ideas for different companies, ideas for different products, this is going to be big in the future. I've tried to reach out to people to collaborate and work on it and take advantage of that foresight. These people say, no, you're wrong. And then they have their idea for something to do. And to me, it just seems like a sort of small semantic jump from where the industry is currently at. Like, oh, we're going to make Twitter for sound bites or something like that. It's just like the smallest jump. And I tend to think on those larger timeframes and have constantly felt like I've lacked a way to integrate my knowledge into the larger sense-making apparatus. Even though it's there, because the awareness of most people...
1:30:00Is not on this five-year time span. It's much closer to the present that I can say something about three years from now and the majority of people can say this is wrong and then I can arrive three years later and by the time that I arrive nobody remembers that initial conversation and they're now in that reality where that's the present. That present makes sense. So they're like of course, of course this is what has happened. And so my resolution to that is that I felt there needed to be sort of a better accountant or generative model that could take into account a larger time frame that could scan across many different voices, many different perceptions of the causal flow of time and bring attention to the people who have this longer time span. Because we as humans, we have terrible memories. Even just retelling a story, like we change it a little bit. There are many studies that point to the notion of that emotional valence will dramatically change our recall of the past. There is a study where they were basically waiting for a terrible thing to happen and when 9-11 happened, they went around and they just asked everybody, what were you doing when you found out? And it was mostly just bland, milquetoast things.
1:31:51And then they went back 10 years later and they asked everybody what had happened and the vast majority of people changed their stories to be much more exciting or I was with my friends and our family and we came together. I strongly feel that to address this problem, that it's not necessarily something like a just slight modification of market dynamics. I've talked about prediction markets and Futarchy, but I didn't find that to be where I'm placing all of my energy. I see the same thing in you, is that you know about prediction markets, but that is not where you're placing your energy. You're placing your energy into active inference. You have Bioform Labs and from my sense of it, what you see as a good model and I see roughly analogous to the way that I see a good model, is that there is an integrated active agent that is representative of some larger whole that can integrate beyond the perception of any one single individual and any one single data source. So I'd love if you could just go into.
1:33:26Where your mind is at with perhaps an active agent representing a local environment or just riff a bit on like how you feel inadequate usage of that technology to address these problems might look like. Sure yeah I mean just to circle back through some of what you said and touch on or like underscore a few of those points as a backdrop against my specific perspective. I think that you know one aspect of that discounting function when you're talking about the tendency of others to only come to realize the way the world has become when they're in it and not necessarily acknowledge those who have relatively high validity in their capacity to project or to say what the world is likely to be like two, three, five, ten years from now. I think that a lot of that has to do with those aspects of human psychology that you know to some extent you're pointing to with respect to memory. We are not evolved to be a perfect recollecting machine. We are not machines. We are biological organisms that have.
1:35:01You know who knows exactly whether we will be able to figure out like what the confluence of emotion and memory like what what the adaptive function of that is. We can certainly say it biases us from objective reality. We don't necessarily know what the underlying value of that might be or why that might have been conserved but we are not necessarily the kind of thing it is obvious that is adept at perfectly remembering the world from a kind of view from nowhere. We remember the world from within an embodied emotional platform and by emotional I mean.
1:35:37A set of emergent physiological upwellings that contextualize our experience as our experience unfolds in the world. The thing about that is you know when then you try to ask okay who is good at predicting the future well because I think so many people are bad at predicting the future the average discounting function tends to be pretty steep and because people don't necessarily want to expend the energy to really develop a fine-grained resolution map of who to trust or who to look to they tend to over discount people that in fact have good track records especially if they're just not aware of those track records or not familiar with their grammar and they will simultaneously over index on the most salient actors in the landscape based on some set of social cues or signals like political leaders for example who are who sit at the top of well-acknowledged hierarchies so business leaders political leaders people have accumulated a lot of wealth because we also have a tendency to over extrapolate or we have like we have a sense that expertise is more fungible than it is oftentimes as well so like the ability to like rise of a business hierarchy or a government hierarchy doesn't necessarily overlap with the skill set required to to analyze the actual trajectory uh the most likely trajectory of the society's unfolding which is an unfortunate um an unfortunate lack of overlap really to say the least uh but uh i mean i think that's interesting you know just because um now when we get into this space of the kind of models you're talking about if you're talking about iris and the kind of things you've been working on um those systems can actually be constructed to have a much better understanding of uh who has had better capacity to model the trajectory of their particular domain of concern over time with a much more fine-grained resolution in these different threads of unfolding.
1:37:45And actually surface those contextually when it's relevant to people so that people don't necessarily have to lean on their low resolution heuristics or just look to sort of like the person at the top of the hierarchy for guidance they can actually have more tailored understanding.
1:38:06From sources that are pulling uh information that have been shown to be higher quality in those specific domains of concern when those domains of concern are relevant to any given person who cares to ask that system or interact with that system um so i think that that's a that's a massively um that's a that's an affordance of of immense potential and i think that that kind of contextualization is one side of the equation and i think that the other side of the equation you know is you know the or you know there are many facets of this but another facet that i'm really interested in at the moment that that this active inference world of exploration has attracted me for this reason is is the question of this um emergent life as a encoded reflection as a model and also the ability of that as it gains complexity internally and as it begins to relate to the external world in certain ways to gain the capacity to have a model and what that actually means and how that can be used to quite literally inform different aspects of our um of our infrastructure and our modeling in ways that that help us to push modeling away from uh dependency on central um computational requirements that allow us to bring intelligence to the edge and to allow for those edges of systems to actually begin to embody and adapt and transform um in a in a kind of fractally embodied sense right so that there's always going to be some amount of centrality that emerges just because of the fact that there's almost no difference between um those merge points i was talking about earlier and the the kinds of patterns that begin to create centrality like if you and i have this conversation and this conversation is interesting to anybody it begins to attract attention and that attention increases the probability that someone might ask to speak to you or me or both of us and then you know that network becomes potentially an emergent source of a local local centrality right so we can't get away from the emergence of centrality to some extent but we can create techniques and tools that create a tendency to push intelligence to the edge um that might help us to counteract the current tendency which is to suck all of the data up to a central location and to then use that as a command and control center um that is our typical tendency uh that has uh worked in the past because of the fact that that centrality has been one of the more efficient ways of of stewarding uh large collectives of human human effort um it's one of the reasons why markets actually have been one of the reasons why markets as a distributed collective intelligence mechanism.
1:41:06Gave us a leg up and actually were better than what came before in many ways but and this is this critical aspect which gets to what you're talking about uh with respect to surprise and comedy and art as well and the tendency to um the tendency to collectively try to minimize surprise um because that's almost an aggregate over our individual adaptive tendency to want to evolve our models and and allow our models to be informed by our experiences and updated and improved by our experiences but when you overly centralize that especially if you have very.
1:41:44High levels of asymmetric causal power or influence at these different scales you actually end up with a kind of you know if you have higher order agents like corporations or governments who are trying to reduce their uncertainty and they care most about the reduction of uncertainty in their models one side effect of that one way of reducing that uncertainty is to try to make all the people more predictable yeah yes right and there's almost no difference between that and a domestic domestication process a process of essentially transforming a population into a kind of livestock to try to almost terraform the population so instead of trying to create a collective capacity for adaptive exploration and pushing that to each individual so that we can ramp up each individual's own autonomous agency stitch that together in ways that bring out more of a potential and then re-aggregate that up to a higher space of adaptive potential now instead of that you could just try to create cattle out of everyone make them predictable reduce even their own desire for surprise right like if you can make them predictably respond to certain stimuli and and that satisfies your objective function at the cost of a population that has overall reduced agency in the short term you have reduced your uncertainty in the long term you have immensely increased your uncertainty in the long term you have made yourself extremely vulnerable to walking off the evolutionary cliff basically right because you you've dramatically reduced that potential of all of that uh all of those threads that we were talking about all those parallel human threads to actually sample that more ergodic space because you're collapsing them into essentially the same the same sort of homogenized function um for the purpose of looking at your dashboard and saying oh we've reduced uncertainty right yeah that's not the way to do it and so you know the kind of models that i'm very interested in are the kind of models that allow you to push the push the modeling space to the edges locally let the causal structure of those models be informed locally by the reality that they're talking to while simultaneously allowing those many models to be in relationship in a space of emergent dynamics that allows for.
1:44:19Those local dynamics to emerge and synthesize and create higher order models um that reflect a kind of naturally emergent reality and the uncertainty that comes along with that as opposed to trying to push control and certainty out to the edge um as a function of uh as a function of our anxiety with respect to uncertainty itself per se right like and i think that if you acknowledge that uncertainty is always part of the equation and move to a paradigm where uncertainty is not just a negative because fundamentally this is the beauty of this active inference i haven't gone too deep into it because it gets a little bit technical but the The difference between that variational free energy, which is kind of an index over the amount of uncertainty that is embodied in the current model that you have with respect to what you're seeing right now in terms of the sensory perceptions and how those relate to the actions you've taken right now and how that's violating or not violating your current model.
1:45:23That's this variational free energy, kind of like present uncertainty, right? But there's this expected free energy as well, which is looking forward and trying to understand all these different possible paths of action that we could take in the world in relationship to our model. Now, this is the big difference between something like a utilitarian or like an optimization over a utility function that's trying to minimize error and something like active inference, because it's not always the case that you want to minimize expected free energy, right? There's this tension between exploration and exploitation of your current model. And sometimes if what you're seeing right now is a local spike in a particular kind of uncertainty, the answer to that local uncertainty might not be clamping down and trying to become more certain overall.
1:46:17It might be a kind of global increase in uncertainty and learning and exploration and going into spaces that are actually less known, releasing some of your macro level constraints on your behavior that allows you to go into new spaces, to interact in new ways, to gain new information that can then inform and evolve the models that inform your local behavior with respect to that initial behavior that was generating uncertainty and resolve that local uncertainty, right? Through a process, not a global totalitarian locking down over every behavior, but quite the opposite, increased exploration within a set of adapted boundaries. So this is one of the things that really, this is why as a paradigm, there aren't a lot of paradigms that I actually, when you try to iterate them forward in time, make humanity collectively more adaptive.
1:47:12But I do think that regardless of what one thinks about the, like, because like in scientific theories, people will try to say, is this true or false? And it's like, okay, like I understand that desire, but I'm much more interested in like, if this is applied en masse or to our systems, what happens? And I think that like, there are very few domains of artificial intelligence where if you run that forward and scale that out, you actually see more adaptive behavior across more scales in a more decentralized way. And I think that active inference has a lot of potential in that space. Whereas certain other kinds of modeling or learning predicated on immense amounts of centralization of data and the minimization of surprise over some objective error function, I don't think that's the case if you actually iterate that game.
1:48:05So, yeah. Yeah. Well, I just wanted to get you more into your conception of how such an agent would be embodied in relation to culture and society. I've read the digital Gaia paper, which I think has some overlaps between what I read on your site and their notion of sensors and their notion of actions. And there also is very much, when I think about Iris, it's very difficult for me to not notice an overlap between active inference with sort of just cultural variations that are attempting to make it make more sense to the average person or give an interface that makes sense to people.
1:49:09So, what are you seeing as the sensors that are an input to form this generative model? And what are you seeing as the actions being taken by the model in relation to that sensory data being modeled? So, I mean, this is where it is always, we can get as specific or as general as we want on this front because of the nature of the active inference thesis, which is that this can apply, people probably aren't that familiar with the philosophy of Leibniz and his monadic theories.
1:49:59I personally think that he was extremely capable of seeing into the future and foreseeing the kind of issues that a much more mechanistic interpretation of causality and physics entailed and actually taking that to his logical conclusion, which was essentially this notion that if you looked at the world, not from a materialistic reductive perspective, but from a causal perspective, and you asked not what is the fundamental unit of material reality, but like what is the fundamental unit of causal structure in the world, then you'd be forced to basically push that causal structure all the way down into even the smallest domains of relationship between anything in the universe, where any point in the universe is essentially in a way encoding the causal nature of the universe as the universe imposes on that particular part of itself, and then transforms itself by its causal consequences.
1:51:04And so active inference kind of comes in and says, it's not specifically Leibnizian or anything, but it is an interesting parallel because it talks about these Markov blankets and the ability to basically create a statistical structure that shows that if you wrap the statistical structure around any part of reality at any scale, you can begin to talk about the part of reality inside that structure as becoming a model of the dynamics around it, right? And so to the extent that you want to apply this to any part of the world, if you have any kind of sensor data and you have a kind of time series of those dimensions of data, you can allow that to inform a kind of generative model of its local behavior that can work locally or integrate with other sources of information that are doing similar processes at the same scale or at different scales, right?
1:52:02And so you can stitch together because they're all speaking the same fundamental language, you can begin to stitch together meta models over that because if you network these together, then you basically get an emergent recursion of the same kind of process because if that emergent network of these nodes also has causal inputs and causal outputs, then you can see how you would have a higher order version of the same process running. And so I think that this is something that there are many places where it's very interesting to apply this. We're currently pivoting, well, not pivoting, but we were doing about a year's worth of more general research into prototyping. I was building prototypes of this platform, but we're pivoting or beginning to focus this more on the energy sector because we think it's a really interesting combination, especially distributed grid technologies.
1:52:52This is a really interesting combination. Sorry, I just did this big, I did this big iris binge and that is exactly where my head went towards the end of my research arc and partially what spooked me a little bit. Sorry, just to go on, it's just funny that you went. Yeah, no, I mean, I can see that as well. I mean, this is kind of part of what I was like, what I was going to point to as well, because just to wrap up like why we were interested in that, because you actually have a lot of pieces of infrastructure that have relatively well-known and relatively well-bounded behavior locally, but as you stitch them together and as you try to understand them in relationship to like demand response of the broader societies and the human behavior and needs in which those power systems are embedded, you get very complex emergent behavior.
1:53:48So you have this interesting tension between relatively high predictability at the local level and relatively high levels of uncertainty in terms of especially extreme events at the aggregate or the emergent level. And we think that active image is actually really well-suited to addressing that. But I was also like, so for example, and I think that I'm like fracturing into a number of threads that I would like to explore in parallel, but I'm gonna try to like hop between them here. One of which is something like Iris seems a lot like a different way of approaching the Oracle problem that was or is and was a problem in the sort of decentralized blockchain space in terms of having centralized representations or like pseudo-centralized representations of reality to which many decentralized actors can come to be informed about what's actually happening in the world so they can synchronize their behavior.
1:54:50One way of trying to approach that. And I think that to some extent, active inference agents and a process like Iris that has a sort of meta framework of different contexts that might be applicable to what those local agents are seeing is an interesting complementarity there to help them especially deal with out of distribution events, for example. So like high surprise events, if you wanna understand as a local agent how to potentially recalibrate your frame of reference, where do you go? Well, something like an Iris could be actually quite good for that. And so there can be a complementarity between these more specific active inference agents and something more along the lines of something like an Iris, especially to the extent that I know you're working on this, to the extent that Iris is able to understand dynamics in the latent space, right?
1:55:46And then potentially contextualize the agent's trajectory in that latent space and say, oh, actually you might want to go in this direction as opposed to that direction. And here's some of the information in context that might help you recalibrate your underlying generative model so that you can actually reframe your local dynamics and your local frame of reference and hopefully be less surprised in the future without Iris being in a relationship of control. It's in a relationship of suggestion and behavioral information as opposed to a relationship of control. So I was asking about like, what do you see as the actions? I was going to go into like, do you view these active agents as being sort of embodied and connected to real world actions?
1:56:36Because I- Yeah, yeah, definitely. I mean, you can't, you certainly, I mean, like they, like that's, like they're kind of this wrapper, right? Around things in the world. And those things in the world can be as concrete or abstract as makes sense in the economic context or the social context that you're applying this to. I like one, you know, I was just, I was just contacted by someone. I won't like read, I won't say the name just because I haven't talked to them specifically about this or talking publicly about this. But, and the only reason I'm actually going to say this because this is also an idea that I suggested back in 2018, which was to, you know, when we're talking about data and data privacy, data sharing, and the incentives around data, managing our own personal private data and the many forces out there that are trying to manipulate and extract that data from us to their economic advantage without remuneration or compensation.
1:57:23I suggested, you know, individuals should have essentially intelligent agent membranes, so to speak, you know, that help them control and bound and negotiate economic relations with respect to that value, right? Like if we are going to be, if we are going to be constantly, like if people are gonna be trying to surveil and extract our data from us, that should not be an asymmetric relationship where the individual person has no say in the economic equation, right? Obviously large companies would like that to be the case, and it currently is the case where they can aggregate these immense data sets. And it's also like, that would be a natural decentralized hedge against the kind of foom scenarios, let's say, of runaway large agents, because it drives the cost of training large agents up, right?
1:58:15To the extent that you actually have to negotiate with the people from whom you are taking information and actually remunerate that to some extent, right? Acknowledge that there is incremental value in every one of those observations you're making from someone's behavior. You know, that actually, that does produce a countervailing force on the runaway capacity for self-improvement of these large centralized kinds of models. And I do think, like increasingly, we're going to see these kinds of agents, fusions of large language model or large scale, more centrally trained and then distributed in function type models with more locally autonomous active inference style models.
1:59:04I think we're going to see fusions of those increasingly being sort of wrapped around individuals, right? To mediate their preferences, because right now we just have, we have too many decisions. We have too many, there's too many transactions. There's too many decisions. There's too much increasingly like surface area to manage. And so to the extent that we are able to have our own agents that can model us, but are controlled by us, that aren't an extension of control over us, but that actually show up in the world as extensions of our own autonomy, our own sovereignty, our own agency in the world, right? As independent players on that economic landscape. To the extent that we can realize that, I think that this entire world of kind of active inference agents, especially those that are informed by the kind of systems that you're trying to create with IRIS that are sort of parallel to the normative economic pressures that exist, to the extent that- that infrastructure can bootstrap itself, right?
2:00:02Like, to the extent that those agents are able to have enough value or gain enough value for individuals, that it makes sense to also pay part of that back into the development and maintenance and evolution of a system like IRIS, that can create a hedge or a countervailing weight in this game that we're playing between the tendency and desire for institutions who seek to reduce their uncertainty via control and centralization versus the decentralized side of the equation where people who actually do very much care about maintaining their agency, their autonomy, and their capacity to create communities that reflect their local values as opposed to have to concede to the mechanisms of control that are imposed upon them, center out or top down.
2:00:57And that's the game as old as humanity itself, but it's about to take a very different form. And there's no time like the present because the earlier back in a process you go, the more leverage you have over its unfolding. And so getting these systems out into the world and getting them created in ways that are resilient to capture and to domination seems quite important. Well, I have a ton of anxiety and uncertainty in this space of sort of the integration of active inference and sort of generative models like IRIS, which it just led me to this place which was like, well, I'm not seeing very much other research, basically none, on a number of these features that I'm developing and feeling that, oh, well, kind of where my agency is is to hold my tongue for a bit about some of these things.
2:02:04Yeah. Because like what I was doing was just stripping up the language aspect of it and making it be more of a quantitative type of IRIS with many different sources of information and trying to integrate into that a sense of trust over the accuracy or the noise in those sources of information as they're integrated into a larger generative model. But one of the big philosophical things that I've been bound to for a long time is this reticence to have the model take any real action, real embodied action, other than speaking, right?
2:02:52That what you have is some generative model that is informing the agents, which are us, and that the only action taken is the output of language or the output of data, and that what is being reserved for action is people. That there's a clear separation between the function of it being generative in terms of information about like a belief space. Basically, it's modeling a belief space about an underlying world where you're using prediction as a signal over time to contextualize the trustworthiness of the level of noise in each particular source, in each particular context.
2:03:46And then I start thinking about like these other representations of active inference models and what those actions are being mapped to and the sort of language of saying something like, this is representing, for example, an environment or a bioregion. And that making me feel very uncomfortable about the level of alignment, or like basically that there's some reward hacking that would happen, or there's some disconnect between the way that we're framing the problem and the outcome that we are trying to achieve. And it ultimately does not reflect an increase in the sense of thriving of human beings.
2:04:45Like I find myself very human being centric and very worried about this notion of an integrated agent that without a binding to humanity, a functional binding to humanity, that it spins out into actions that are no longer aligned. And I was sort of sending something to Jordan Hall the other day, just being like, do you understand the concept of what I'm saying, that AGI or AI doesn't really need to be fully integrated as an agent and that these generative models, they don't say anything unless there is a source of information about that thing. Like if you have in your training data, nothing about physics, it will never talk about physics.
2:05:33It just doesn't have the language to talk about it. If you don't have anything about medical care, it's not going to talk about that. So when it does output some gender representation, it is because it has attended to sources of information that are sampling from the real world, right? And if we can understand, or if we can intend to intentionally tie these models, these generative models that everybody is using back to the sources of information, back to the human sources that are going in, my prediction is that it is a safer outcome for all of us and there won't be this increased disalignment, right, between those.
2:06:27But I've sort of struggled to get people to understand this distinction of saying, well, the model is a reflection of collective intelligence. The model is reflective of individual agents doing some sort of processing, some sort of sampling, and then it's being integrated. It doesn't need to be framed as a wholly integrated agent where we have no reason or no understanding as to why when it takes an action, why when it generates an output, why that happened, that both of these problems are technically unsolved. AGI and sort of like pure binding to humanity where it's always a representation of us, there's always a tying to us, whereas if it outputs something we don't like, well, there's no uncertainty about why that happened.
2:07:22You just look at the distribution of sources from which it's essentially sampling and say, okay, it said this very horrible thing about its relationship to humanity. Well, it's because somebody wrote this short story about AI at this point, and that's specifically where it's pulling that information from. And I have not yet found a way to communicate that dichotomy well to people. I don't know if that's a question, but it is just where a lot of my personal uncertainty has been in these models and where I've sort of paused from releasing into the larger world and revealing or removing that uncertainty from others until they sort of figure it out on their own.
2:08:11And where does your mind sort of go on all of those? Yeah, these are, I mean, it is certainly, kind of like what I was getting at before to the extent that we are at the beginning of this process of unfolding an evolution, which is going to be a path-dependent evolution of what systems actually pervade the world and what their function looks like. The revelation of information has, it never has more influence than it does towards the beginning of that process, right? And so, the kinds of informations or the kind of, the fact that we discovered, let's say that there's a space of possible mechanisms we could have discovered and we discovered transformers, right?
2:09:06That has a strong influence over the path-dependent evolution of what AI means to the public, how AI functions in relation to our species, how we use it, all the ways, every person using ChatGPT now is a downstream causal function of the fact that we came across that particular mechanism first, as opposed to what else might be out there and that it was shared widely and implemented in the way that it was. And so, to the extent that you might reveal their information that changes that path, well, it's up to you to figure out when or how you might want to do that, but I totally understand your desire to, or your possible hesitancy, especially if you think that it might, in the wrong hands or for the wrong purposes, accelerate aspects of the current path that are undesirable.
2:10:02So, I get that. I think it's interesting from the perspective of, again, from the perspective of binding to humanity, binding to agency, I think we need to have, once again, there's a, we also need to protect ourselves from the same tendency that centralized organizations have from these LLMs or from any large centralized model, because one option available to it will be to make us more predictable. And that's, you know, whether or not it understands the difference that is present, for example, in active inference between minimizing uncertainty now versus understanding the utility of uncertainty as a mode of exploration in the long-term, whether it gets that or intuits that or embodies that, you know, that's a path-dependent function of its evolution and the people working on it and their values and their perspectives and their motivations.
2:11:01And I would also say that their comprehension level of these problems and the extent to which they care about these problems, and it's not obvious at all that a lot of the people who are at the forefront of that particular field and those innovations care deeply about this lens into AI. It doesn't actually pop up as much in discussions about alignment as you would expect or hope. But fundamentally, I think also we have, to the extent that you do bake in, and this is where time preference, we kind of circle all the way back and bring this full circle to this question of like, what are we binding our phenomenology to? What are we binding our attention to? Which kind of timescales or cadences are we binding to? And what are we pursuing with respect to, are we minimizing uncertainty at the expense of, yeah, at the expense of individual autonomy?
2:11:57And so that's a local optima that is long-term not stable and long-term maladaptive. Do we have the ability to give people, like, it seems to me that what we want from these centralizing models to the extent that they do emerge, and they will, because centralization always emerges to some extent, ideally, they would be internalizing information from us that is wrapped in a kind of set of, like, indexes over valence, local valence, that actually means that as those centralized models internalize and come to embody, come to be a model of our world, they actually sense discomfort and suffering as an extension of our sensations in the world, right?
2:12:52Like, the fundamental integration function into that system is not an integration function like it has been thus far, just over our outputs, over our books and our images and our audio, right? These are all our artifacts, but it leaves out the entire realm of our embodied behavior, right? All of the encodings that we understand to be the sort of qualia space or the subjective experience space of ourselves as an embodied organism, that's absent from the internal representations of these models thus far to the extent that that can't be modeled by our words, right? Like, I'll give you a very concrete example of this. Like, I could have read an infinite number of firsthand accounts of what it's like to hold your child in your arms for the first time, to see your child for the first time, and it never would have prepared me or given me an inkling of what, like, I never would have been able to simulate the experience through the artifacts.
2:13:59The only way to experience the experience was to have the experience, to be in the experience, to have all of those behavioral tendencies align in time and space and in our embodied relations such that my actual direct senses and my actual being and the entire evolutionary history that has culminated with me as an entity was manifesting in a way that was in direct relation to the continuation of that in a very personal way, right? And I can still try to talk about this in words, but it's not the same thing at all. And if you talk about it that way with a child, you can do the same thing applies if people are more into things like, let's say, psychedelic experiences, right? It's equally difficult to come out of an extreme psychedelic experience that one perceives as subjectively meaningful, very intense valence on that sort of meaning dimension.
2:14:55Yeah. And actually then use words to relate that sense of meaning to another. third party, especially if that, you know, the best you can hope for is someone who has had their own version of that. And when you point to that significance, they can say, Oh yeah, I understand what you're pointing to, but I also can't really communicate it to you back in words. I can just tell you that I kind of know what you're pointing at. Right. And so the question is like, how does that kind of intuitive embodied understanding and comprehension that only manifests through being in time? How does that work its way into these representations, these models of reality? Because without that, they're, you know, they're not going to, they're not going to be of us as whole beings.
2:15:42They're going to be of our representations of our experiences. And so, and I think that's an important distinction that is also lost on the kind of people who are the most disembodied and who haven't actually tried to bring themselves back out of their models. Like I love models as much as anyone can love models probably. But I also understand that there's an extreme danger in that and I need to come back out of that, which is why I also moved myself into the middle of the woods and make sure that I spend a lot of time in embodied relationship with natural processes using my body, anchoring myself to reality and making sure that my senses are brought back into relation with something beyond model space at a cadence that is actually had to increase.
2:16:34The more that I've gone into pure, like purely focusing on this as my day to day, the more that I retreat into model space, the more I need to be out in the world with embodied contact such that I don't lose myself and start drifting too far out into the abstractions that I have produced. So yeah, I mean, I think that this seems to be essential and I don't think we have a very good way of doing this right now. I do think that looking toward natural intelligence and mechanisms or processes that are suggested by explorations in active inference right now, not just active inference, but that ethos and that direction of exploring natural intelligence and embodied emergent intelligence.
2:17:26That's essential. Grounding our models to the extent that they are powerful, they must also be deeply rooted in that sense of embodiment, not just in the past, but also as an ongoing process to which every entity, human entity, hopefully increasingly more life processes, hopefully it samples from larger and larger amounts of that life function. And therefore in sampling over that life function becomes an extension of that life function as opposed to, let's say, an extension of like the mechanism function. And those are, I think, quite distinct. So what I'll also bring into that is there's a couple of times that you've referred to like sort of the centralization of the generative models as either Iris or, I mean, in some, I would say definitely active inference generative models apply to this as well, that they're sort of run by one set of people.
2:18:36One thing that we haven't gotten a chance to talk about and we'll probably have to talk about more in a future conversation is the notion of how do you decentralize the compressions of the knowledge that is integrated by those models into some sort of distributed data store? Because I've always viewed it more as that there would be many, many different Irises. People often ask me why there isn't just one. And I mean, I think you know exactly why there shouldn't be just one. And that there would be some energy bound distributed data store that is taking these compressed tensors or compressed representations of knowledge as they are evaluated by or used by many different models and stored in a secured way that many future models can build the representations from.
2:19:42I view it more like there is a DNA strand that is the collective cultural ethos and world observations and that these models are more like the molecules that do the correction than there is the replication of a DNA strand. And that ultimately is sort of the model that even you and I have not gotten much work into and really needs to develop is that there needs to be sort of an evolution or a merging of these block trans-structures or just distributed data stores and the knowledge in its very dense form into that beyond just like saving a copy of it.
2:20:37Something more fundamental. Yeah. I mean, literally, so the conversation I mentioned earlier that I had yesterday, like that exact idea, that was something that I was speaking with Jordan and Steph about when we were discussing especially the role of something like blockchains, the role of something like Bitcoin, when it comes to, not Bitcoin specifically, but proof of work, having something that is highly resilient that ends up being a kind of centralized repository for highly compressed encodings that can represent as a kind of DNA over the space of emergent systems that have models, right?
2:21:23Because DNA itself was the way in which, like you're saying, these molecular mechanisms and all of the emergent structure above them in its exploration of function space created a feedback process that compressed down into molecular space representations of what works that can then be unpacked again and experimented with into function space. That's this DNA function, right? But that was all in the being a model world. But in the having a model world, we have to actually, because all of this becomes explicit, we have to become sufficiently self-aware that we actually create or steward the creation of a similar kind of, and this is weird because there's this duality here. If we did it on a blockchain, something like a Bitcoin, you want it to be capture resistant.
2:22:12So it could be highly centralized, but because it's capture resistant, that centralization actually can become a tool that increases the capacity for decentralized coordination. Because if every decentralized system can contribute to it and take away from it to some extent, then they can all focus their attention on it without having to necessarily be tightly coupled with it, except when they want to contribute to it or decode from it, right? Because the decoding can always happen for free. The encoding comes with a cost if you want to inscribe. But to the extent that anyone can benefit from decoding, think about what would happen if an advanced model were basically leaked and its weights were put onto the blockchain.
2:23:00Now this may or may not be desirable at this point. We might argue that we need a better container for that before it actually makes sense or could be advantageous in that leaking a powerful model and making it irreversibly accessible on something like a blockchain with respect to its weights or a compressed version of its weights or a program that could generate those weights if that was a valid compression. You know, it's unclear whether that's desirable or not desirable right now, but my point is that these kind of mechanisms are almost with 100% certainty going to be the kind of places where we encode these highly compressed distillations of our collective experimentation over this space of computation, AI, and also complex coordination via systems such as the entire other world of like proof of stake and exploration of distributed governance, decentralized economics, all of that space where we're seeing a massive amount of experimentation right now.
2:24:09We're seeing a lot of failure. We're seeing a lot of parasitism. We're seeing a lot of craziness. But we're also seeing the seeds of potentially interesting or useful patterns. To the extent that, much like mutations, to the extent that those come into the world and actually provide value, how do you re-encode them into a space where anyone can then replicate those patterns in the world? So if a particular DAO finds a set of configurations and weights over their mechanisms that actually gives rise to really positive capacity for a group to collaborate, coordinate, and be generative as opposed to parasitic or extractive, that's a kind of pattern that should be encoded and shared, right? And it should be accessible with very low cost, if not free, in something like a blockchain that cannot be easily captured or corrupted for local interests of any other entity on Earth.
2:25:05So yeah, I mean, I think, exactly, the degree to which something like an iris is centralized or decentralized, it's an interesting question because to some extent it's a question of adoption and evolution and how all these different threads of that system come into usage locally, right? You want to spread it far and wide and have those sort of seeds cast upon the winds of potential throughout human interest and experimentation such that they land and take root and grow in fertile soil, obviously, because we're humans, people will try to use it for abuse as well and all these other things that we do. But yeah, again, I really do see it as this recapitulation of this natural process at the level of having models as opposed to being models.
2:26:01And we are trying to figure out what it looks like to create the kind of social mechanism that is the analog of DNA, right? Yeah. I would add, just in summary to try to link it all back together, that there definitely needs to be – this could very easily go wrong where it's trending towards reducing that uncertainty in our knowledge representations, right? I've already seen over like two years with GPT, first there was a very wide space where you could explore the long tail, you could talk about things that aren't well-established and more and more it's like there's a regulation into this is the way things are and because they are controlling when that update happens, it's harder and harder to do research to that.
2:26:49So whatever compression goes into that distributed data store, it needs to have that preference towards the exploration of the uncertainty like you were speaking of to variational free energy versus – what was it? Expected free energy. Expected free energy. There needs to be a tendency not in our compression of knowledge to say, well, this represents truth in its totality and rather this is a representation that is inherently mutating and changing over time and something that is existing to expand our capacity over time to exist and thrive where our neural mechanisms of encoding memory, of encoding models are just a little bit too lossy for long-term.
2:27:51There's kind of like sedimentary dynamic there as well where to some extent, the longer that certain aspects of that model have remained stable, the more they kind of migrate down to a deeper and more stable structure. That doesn't mean that that layer can't be changed. Yes, exactly. It does get more difficult to change the more – Energetic constraints on that, yeah. Yeah. The more that you have energetic constraints, the more that you have dependencies, the more that you have a history of the world being congruent with that representation, it does get more difficult to change as it migrates down that stack. But the understanding is that the entire stack, all of the sedimentary layers, even though proportional to their level, they will be – like the top layers will be more volatile proportionally than the bottom layers, but they are all subject to change if you have observations or empirical experiences of sufficient surprise, and that reflects what we are like as well.
2:28:57We can be shocked to such an extent that our entire world models – it is revealed to us that our entire world models down to their core need to be recalibrated if we encounter experiences that are sufficiently outside of our model, although unfortunately, we increasingly are leaning into the tendency to lean into confirmation bias and to try to say, I am going to ignore the signals the world is giving to me, I am going to try to force my model on the world. Again, it is like there is always a certain element of interplay between like whether it is beneficial to try to put your model onto the world or allow the world to transform your model, but it always has to be in dialogue, and it becomes inherently totalitarian when you decide that no matter what, I am not going to change my model, and I am going to push all of the responsibility for transformation out onto the world as opposed to taking any of that in your own model.
2:29:54So we have been on for a while here. I do have another call coming up, and I want to take a little bit of a break, but Yeah, I just wanted, I was going to say, you know, this feels to me like the best part of the conversation, so we should have a follow up in that vein of like what this distributed knowledge representation might look like. Is there any way you sort of want to tie this all into a bow? I mean, we've gone for, you know, two and a half hours now. Yeah, I mean, there's so much that we've covered, but I mean, I think ultimately, again, the most relevant or the most, for me at least, the framework that allows me to place everything within it or the kind of tree that allows me to, you know, hang the ornaments of knowledge or perspective upon that tree being seasonal here since we're going into the holiday season.
2:30:49Is this question of, it goes back to that zero one question, it goes back to that fundamental, you know, this thing that we were just talking about with respect to acknowledging that fundamentally the process that we are embedded in is more like that of like the Bayesian updating than it is like the frequentist approach of just trying to get as much data as possible that will lead us to a fixed point on the landscape. And we can just put that on the shelf and no longer worry about it. I think not only does that make us cognitively lazy, but it also brings out the worst aspects of us because as soon as you put it on the shelf and want to move on, it can be quite frustrating when anyone else comes in and picks up that object off the shelf and starts to criticize it or say that that might not actually be the way that reality is.
2:31:37And then you might have to go back to, you might have to, especially if that has sort of moved down into those deeper sedimentary layers in your model or you've built upon them, that can be quite frustrating. And I think that while that frustration will always exist, having a perspective and having a worldview and having a cultural, culturally holding and understanding that all of that is in flux, although in flux to different degrees, I think that allows us to get to places that are more desirable, like both conversationally, it allows us to have better conversations because we're not so attached to fixed perspectives. We're more interested in seeing how they relate and can inform one another's models and evolution improvement of those models.
2:32:23It allows us to have better relationships in terms of, you know, thinking through the implication of economics and our monetary systems and what were their purposes and how effective are they currently at their purposes and what might actually allow us to complement or evolve those. I mean, it's just, it's a framework in which we can begin to, I think, undergo the transformations that are necessary for us to make it through the next hundred years. But I think that also have to be almost, you know, that has to be mostly voluntary as well, right? Like it's very difficult because this has to be voluntary. This has to be a process of people coming to see the value in this way of seeing the world in addition to however they might see the world currently and, you know, whatever the balance of those over time might be.
2:33:14If you just try to say, you need to give up the way you see the world now. This is the way it is. This is what the model says it is and either you agree or we're going to nudge you into agreement or, you know, control you beyond your ability to resist, you know, A, I don't think that's very adaptive in the long run and B, it's certainly not a world that I want to live in. So, yeah, it's not good for human agency or what you might consider the soul. So, yeah, I think that's kind of the general framework that I approach all this from and, you know, I always try to bring things back to that level because fundamentally it has to be good for humanity, it has to be good for, you know, it has to generally lead to the embeddement of human life, but also of life in general.
2:34:06I want to see, you know, I want to see us bring about a world in which we can use our knowledge and our models to help to the extent we can create more biological complexity, more life, flowing more energy through more systems that are less destructive to one another. Obviously, you know, there's always going to be some amount of conflict and frustration there. Like, I'm not a utopian, but I do think that there's plenty of room for improvement. So, yeah. Wow. What a great conversation. I guess I'll say to the audience watching, because I feel very motivated to have another conversation, to comment below what particular threads you think that we should follow up on.
2:35:00I'm always amazed when I hear you speak physically, how much in alignment I am with what you say. And then I read what you write on Twitter and it's almost like there's a different version of you that, you know, maybe it's just the translation between the textual form and the sort of embodied, like, form of communication where there's just different signals and feedback between us that can sort of narrow us towards what's actually being said. Yeah. And part of that's that artifactual issue. And part of it's, you know, I, some of what I put, I put, I write as condensed pointers for myself to come back to, kind of like I've said that it's a shared journal to some extent.
2:35:52But this also comes back to the fact that, you know, to be completely honest, I would much rather be in relationship with a system trained on my conversational self and like the entire embodied, the entire embodied platform that is me, the thinking and the being process that is me, than one that is just trained upon the kind of disembodied artifacts that I put into the world. And I don't necessarily think that that's a, like, it's not like I dislike the things I put out into the world. I do think that they can be useful. I also understand how without the context, they can be interpreted in all sorts of ways that aren't necessarily my intention and that can lead to conflict or frustration or miscommunication. But yeah, I mean, I think that's kind of exactly why it's valuable to figure out how our models of reality can increasingly be informed, quite literally informed, by that full experience, by that full set of relationships and dynamics that give rise to this kind of an experience as opposed to that, which is just mediated by text or images or any artifact.
2:37:04Yeah. Well, I mean, in a way, this is some artifact too. It's just higher resolution. Maybe we'll, maybe we're moving. Maybe we're moving towards increasingly high resolution where we end up creating some sort of simulation. Maybe this is already that. Maybe there's a higher order reality and we just created this to reprocess, I don't know. Some fraction of us are, and there are many incentives that drive people also away from that. And there's quite the tension right now. So yeah. This has been great. Yeah. Yeah. Yeah.
(2:38:12)