00:00A community's trust graph. Like, imagine that a DAO, like, currently, how does a DAO decide who it's following? Like, what people it's following, what accounts it's following? But the practice of sort of having a sort of community setting the people they're following is already kind of, that's part of the collective attention governance. Like, that is a very simple sort of aspect of it. There's almost like these epistemic, epistemic hills where it's like, this is where most of the belief is, and there's people sort of like little ants going out seeking out new sources of information. [Music]
00:28Hello, uh, talking about attention. And I had just asked you about your background in in general. Yeah, so, uh, final year, basically final month PhD student, um, in Hebrew University in Israel, uh, and my focus was, and, natural language processing. Mhm. And yeah, somewhere in the middle of my PhD, this was also related to covid, I kind of fell down rabbit holes of of how these, um, big tech AI industries are controlling the data, they're controlling the compute, they're controlling the algorithms, uh, centralized social media platforms.
01:22And as I was kind of following the the sort of rabbit hole of of how this control manifests itself and how they exercise control, I felt like I was getting led on to this idea of attention and how attention is manifested in in sort of digital online environments, like the digital traces of human attention, because attention is a very human thing but it also has very specific sort of, uh, technicity, that's this, a word I learned recently, technicity of attention, like how software reduces attention to code.
01:53Yeah, so yeah, so I've been, I'm very interested in that because it feels like it's a very basic building block of like how we interact with information online. Uh, yeah, we can go into more detail, there's a lot to talk about, and I'm also just in your sort of angle on attention too. I'm just looking right now, I just pulled up the Common Sense Makers because I'm familiar with both Brad de Graf and, um, and Daniel from Active Inference Institute. Uh, did you want to talk about that a little bit?
02:30Sure, sure, uh, so, so after I kind of fell down the rabbit hole of of looking at these centralized platforms and and getting the sense that things are really really messed up in sort of online information ecosystems, I started getting interested in, uh, decentralization and kind of web 3 areas, and, um, started connecting with people in the web 3 ecosystem. I started working in a company that did tools for DAOs, like, um, DAOstack, a DAO tooling company. And yeah, through this, this, I was exposed to some of, Brad and Daniel are both kind of active in these liminal spaces of of, you know, decentralized data ownership and, uh, that sort of thing.
03:07Daniel was really instrumental in in in my personal journey, because yeah, he, he's, uh, an insect intelligence researcher, like he, he does research on collective intelligence and ants, and he introduced me to this idea of stigmergy. Which I think was like, if I had to like three concepts, the top three concepts that have changed the course of my PhD and probably even more than my PhD, then I think stigmergy would be one of them.
03:41Yeah, that's the, uh, paper that drew my attention the most, which was From Users to Sensemakers: On the Pivot of, Pivotal Role of Stigmergic Social Annotation in the Quest for Collective Sensemaking. Yeah, that one. So that's, that's a paper with Daniel and, and it was kind of through his, um, through him introducing me to the concept of stigmergy and then us trying to sort of of synthesize that with, you know, going from insect intelligence to what it means for social media and and online information ecosystems.
04:14What's a sort of concise definition of stigmergy? Stigmergy is the idea, it's a coordination mechanism, and it's the idea of indirect large scale coordination. It's mediated through environment modifications. So basically, like, the textbook example is ant pheromone trails. So ants leave pheromone trails, other ants, um, see those trails, react to them, and, uh, it's all very local sort of environment modifications, but you get this global emergent intelligent collective behavior from those local interactions. And stigmergy is, that was, it was coined by, you know, a researcher that was trying to understand how that, uh, how those those signals and other kinds of signals that insects leave drive this kind of emergent collective intelligence.
04:47And even in those systems do, like, we can sort of hack their behavior by modifying chemicals, making ants think they're dead, making them continue to carry themselves to the dead pile, um. Right, there's a spiral of death, I think there's also this famous video that shows how they, yeah. It has failure modes of course, it's not like some kind of fail safe mechanism. It's just a very powerful example of like, when it's functioning normally it leads to, you know, a lot of very interesting behavior.
05:17And I think for me, the sort of aha moment was like, platforms in a sense are the ground, right, the ants like the ground that the ants put the pheromones on. And it just got me to think that the platforms are like messing up our our stigmergy, like the signals are amplified in ways that we don't understand, we don't even, you know, control really what happens with the pheromone. So it'd be almost like if you took an ant colony and started messing around with the properties of the, you know, soil and stuff, like you said, you cause a lot of, you know, crazy failures, and and it feels like we're kind of starting at that place right now.
05:44We try to have to figure out the, what is even healthy stigmergic infrastructure look like for for communications? And it's evolved quite a bit since the beginning of the internet, from, you know, intuition which might be something like a 10 star scale, to just, well we don't have the space on the page for this, and most users are only, you know, doing, uh, the extremes, so let's just go to like, to dislike, to now they're actively removing the dislike.
06:17And, you know, ants can't really design their system, but we have that freedom, but it does seem almost like it's more reactive, uh, on our end to market forces at this point where there isn't a lot of applied intelligence in the design of those signals that we're tracking. And we've had like at least, I think, 5 years since the problems, uh, started to really bubble up and become readily apparent, but there's a sort of desire to have them all be managed in the background passively rather than introducing some, um, some sort of active mechanism.
07:04Um, did you want to talk about your, just, paper in general and what you sort of have found from the signals that we're tracking, and, uh, maybe sort of the differences between like passive tracking of users versus, uh, active tracking which is sort of harder to get users to do but might be more valuable or sometimes even they might not even know their own self that well?
07:44Yeah, um, for sure, you brought up a lot of really sort of interesting points that we can, uh, explore I think. So talking about this kind of passive active distinction, and yeah, maybe you ask about the paper kind of more broadly. So the paper is, is, is a really short paper, it's basically just an extended abstract. It's kind of call to attention, we thought of it like, it's trying to say hey, this is a thing we really need to be looking at, we're not really thinking about our relation to data in this way.
08:16And this is kind of, if I had to really summarize it, I would say we have this kind of metaphor of, you know, data is the new oil, and we talk about how all these, um, social media companies and platforms in general have managed to capture oil, mine our oil and harvest it and extract it and convert it to gold. So there's this kind of pipeline of, you know, data, kind of, it's some kind of oil, and they extract it and they turn it into gold. And this paper is kind of trying to invite an alternative interpretation where data is, it's not oil that we're just kind of, you know, this sort of remains of like dead, you know, animals.
08:51It's more like these are our markers that we, like, data is stigmergy, it kind of invites us to be much more active participants. Like, it invites us to think of each, each, each of us as like a kind of a neuron and a collective brain, and we're, we're part of this collective intelligence network. Uh, so it's a whole other relation to data, it's like you're very, like, you know, ideally right, no one is going to do this in the beginning at least, but you're very mindful of the traces you're leaving online, what am I sharing, with who am I sharing it, how am I sharing it, um.
09:17And and what we talked about in the paper is, is there's kind of two parts to this. One is sort of our individual mindset shift, that like we become, instead of like seeing ourselves as just passive, you know, doom scrolling users, we're more active. We coin the term sensemakers, so it's kind of like, um, we're trying to evoke sort of the hacker ethos where the hackers are like people that take ownership over the code, you know, they're producing and they create their own tools to, you know, create the digital spaces that they want to, uh, live in, basically.
09:48So yeah, sensemakers are also creating their own information ecosystems, and they have ownership over those tools, and it's kind of trying to evoke that active sense of of of ownership over the data you're, you're, uh, sharing. And that's one part, so that's the individual aspect, but we also recognize that individuals are ultimately very limited in what they can do. So even if you decide today I don't want to be a user anymore, I want to be a sensemaker, yeah, you're set up for failure because the infrastructure is so lacking, and it's so, uh, there's such powerful infrastructure arrayed against you that these platforms have, you're very limited in what you can do by yourself.
10:16And fight against, like, just yesterday I went through, for probably the nth time, uh, just going through YouTube and saying don't recommend this, don't recommend this, don't recommend this, but it just leaks in over time, where you get into this passive deterministic cognition kind of mode where it's like, I can't actively be controlling my attention all the time, right. And I really sort of feel that attention is kind of, it's kind of like breathing, in that it can be both active and it can be automatic, and most of the time it's falling into that automatic space, and there are sort of boundaries on the extent to which you can control your attention.
10:52So it really is important that we have these mechanisms of, um, what you, I think you call in the paper or somewhere, like the governance of attention itself, um, because that is defining so much of the public, uh, discourse, is what is going to work itself through those filters. And it seems after all this time, either there's not a motivation to correct for the sensationalism that, uh, guides our attention, or they haven't found a way that still manages to keep people on site, like there's sort of a conflicting optimization plane there, where you have to have people on site, and to do that you have to get people's attention.
11:37And there might be some desires from some of the people working on those teams to align that attention with some better outcomes, but if you aren't going to extract it then TikTok's going to come along with their insanely powerful algorithm, which I can't even leave that installed on my phone because it's, it's so powerful, and the model that they sort of have of me now on different platforms are sort of getting scarily accurate, um, which is, it's just bringing up things that I'm almost actively thinking about at the same time.
12:17It's becoming an extension of ourselves without any control. But we should have some level of rights over the direction and flow of our attention stream. Yes, this is very much so, there's, yeah there's a lot of things happening because like you say the platforms have so much data over you that they can, and you know, so much data collectively that they've amassed from all the users that they track, that they can build really good predictive models.
13:03But in fact those models are kind of, they're they're very dangerous in a way because they they're not models that are optimized for anything other than leaving you on the the site. Like, they're not optimized for example for your long-term intellectual or emotional development or something like that, um. What is, what is a model of sort of attention governance look like that does that? Like, what is a feed algorithm that sort of, or, or, you know, a platform that is trying to promote this kind of, you know, intellectual growth, what does that look like?
13:35It probably looks completely different than than what we have right now. So they're, they have a very certain kind of control that's very effective at the, you know, very short term, and that's basically that's all they need because they're, they're sort of exploiting our our, you know, our our worst weaknesses, uh, to to leave us on on the site, and that's all it interests them. So this is, there's one of the papers we uh drawn heavily in in that paper is, um, I found a paper, it's a book called Stand Out of Our Light, and it's, uh, freedom and resistance in the attention economy, it's by a Google, former Google ethicist, I think his name is James Williams.
14:05And he has this, uh, meme that he calls from attention to intention, and it's about like the kind of, again this kind of passive to active mindset, where these technologies are purposefully sort of leaving us in this passive mindset. They're they're entrenching us, they're digging us deeper into the passive mindset, because we're more predictable in the passive mindset. As active users we're much less predictable, much less controllable, um, they have much less that they can kind of even, you know, help us with, it becomes something completely different, like when the kind of tools we're looking at for example are like tools for thought, those are something that are more close to sort of the active, like, sensemaker tools, and not like a doom scrolling feed like Twitter.
14:38So so they have very good reasons to want to leave us in this kind of passive, uh, domain. So you've mentioned, um, doom scrolling a couple of times, and I wanted to talk a little bit about the role of dopamine and modulating attention. Um, the sort of common way that people think about dopamine is like, it's a reward that you get a little hit of when you get something good, which is not really true. It's, it's more related to anticipation and, um, the modulation of attention over time and and goal oriented behavior.
15:09And, you know, Aza Raskin at the Center for Humane Technology developed the infinite scroll, and there's sort of like this always an anticipation that there's some other piece of novelty below the fold. And so it's not like that we're getting a rush or a massive hit of it, it's just messing up that entire loop where the people who are most, uh, sort of addicted to social media actually have much lower levels of, of, from, I mean from what I've I've read recently of, uh, of dopamine in their system. But it's still that fundamental loop that is modulating and controlling the behavior, um, like individually what do you have to say or what, like thoughts can you, you contribute to the the notion of it at a neurochemical level, um, or at like an individual level?
15:47Yeah, so yeah, the dopamine aspect is quite interesting, and I actually, I'm not an expert on on that, um, on that aspect. Uh, it does, I mean, I feel like there's this kind of general I think reward, you know, reward is much more complex probably than dopamine, and I, I know there's like other signals, I wish I could pull up the the names, but there's all kinds of reward that we get, and you know dopamine is just one of them. And it's kind of, again they're like hacking, hacking a reward system to go after something that is going to sort of reliably keep us engaged but not necessarily good for us.
16:30Just like, you know, sugar is going to keep you like, yeah, make some food sugary it'll keep the person, like it'll keep a kid happy. What does a design where our, our flows of attention are reoriented towards the self look like, where we still have access to community, where we still have access to information, where we still have access to novelty? Because novelty is good, learning things is good, uh, it's just that there's sort of a reduction down to the most base common denominator in a lot of cases, and also the emergence of just hyper-targeted, um, content per individual, which is just like, it's just scratching the perfect thing that you can't fight.
17:16So in your mind what is a better design? Yeah, so I think one sort of overarching principle is diversity. So it's, it's not going to be one feed, it's a lot of different ways to interact with information. And people have a lot of different styles too, that, you know, some people prefer like a visual layout, maybe they want like to see a graph and they kind of would explore a graph, uh, some people it's actually easier, yeah, maybe maybe some people it is actually easier to have the feed, but, um, but having the ability to sort of shift in between different kinds of of modes of interacting with information, I think that is really important to get you out of the sort of rut of doom scrolling.
17:46There's a nice tweet by I think Visa on Twitter, do you know he's got, yeah, Visa, I'm not sure the last name, but talking about sort of this metaphor we used to have, the metaphor of surfing the web, it was like a very active thing, like we used to like explore between different websites and follow the hyperlinks and, um, it was much more sort of this kind of journey, uh, as opposed to this kind of very passive doom scrolling. And I think bringing back some of that energy of like letting the user do more stuff, like, so you might like have a, you're on this on-ramp, like the highway of, you know, the feed is like a highway of information, but like it invites you to leave it actually, and and go off and explore some rabbit hole and and like do some deep reading as well.
18:27Because the feed is ultimately just for snippets of information, but it would send you out to like, um, to off-ramps to like, you know, here's like this enchanted forest of a lot of like deeper content and you like read stuff and yeah. And and and right now I think there's a lot of, there's a lot of value that we can have in in making that kind of, making those parts the deeper content interesting. So I can give you a few examples of of things I'm aware of, um.
18:57One is, uh, like more synchronous interactions. So for example, you're on a web page and you have the ability to sort of see that your friends were on a web page, or say friends friends or some community of people that you're like, you know, respect or are interested in, and they like have an invitation there like a standing invitation for someone to, uh, you know, join like a live session, like to talk about some, you know, if it's a blog post or scientific paper or something like that.
19:26You could also, you could almost imagine like a Tinder over like web pages, like I saw this paper, I swiped right to have like, to indicate that I'd be interested in like doing a, you know, doing a live session over it, and then someone, someone else, you know, goes and swipes it right and then it connects you, um. Obviously you need a little more filtering to, you know, get the noise out, but that kind of sort of drawing us into deeper connection, uh, that's kind of the example I'm trying to bring here.
19:56So you know, every web page can be this kind of invitation for some kind of deeper connection with another human being. And this this sort of, each web page is basically like, I like to think of it as like a Schelling point, you know, that people use, like a meeting place, like we're meeting each other on these web pages. And that's why I'm so interested in the idea of like stigmergy, is these trails we're leaving, because right now we're sort of all navigating the information unmapped.
20:24Like if you're on a web page you have no idea if your friends were there, what they thought of it, if, you know, anyone else was there, you really don't know any of that information. And by bringing that information to the surface and and letting it, um, actually be, you know, leveraged and used for for connection, then it's a whole different experience. Because like yeah, all your, all your surfing, all your, all your actions, you know, all your browsing actions, uh, can sort of contribute to connecting you to people and finding, you know, more interesting information.
20:51So yeah, that's kind of one example, I mean there's there's others that we can talk about but, um, what is sort of an ideal signal, uh, or set of signals that are stigmergic signals that we can optimize in or towards? Um, you know, like the word like, uh, in its semantics is, is very bland and, you know, there might be some people say well we're, we're optimizing towards, you know, well-being perhaps, or something like that, and you could have self-reported well-being.
21:39But even then there's sort of this time preference towards, uh, short term, when, you know, real transformation often takes longer periods of time, and there are troughs that we have to go through often to get to a more stable sort of, uh, plane, uh, of well-being. Um, and so there can be just as many problems from directly optimizing self-reported well-being. So what in your mind are better signals that we can provide to an algorithm to help direct our attention in a healthier way?
22:23Yeah, um, I think there's a few, there's two parts of this question. One part is the sort of what are the signals we're leaving, like, and you, you kind of raised this question about like, is like signal enough, is that, or is that too sort of reducing too much into, you know, one sort of, uh, one dimension. So that's like one aspect, what are the signals we're leaving. And the second aspect is, how are we leveraging those signals with AI or other algorithms to to, you know, guide us through information.
23:07So those are the two parts I would say, and then I think for each part there's, you know, we can talk about it, but for example for the like part, yeah I think totally, I mean there's a lot of room here for innovations and sort of expanding the range of of expression. Right, like, just like Facebook, they decided instead of a like now you can do like five emojis over a post, right, so they give you more more latitude for expression. But again this is not anything that they designed for like improving your epistemic environment, they just, you know, they they figured it's going to help keep people on the site.
23:34So I think actually recently someone introduced me to an idea from the LessWrong, I think they have something called epistemic statuses, it's like, it's emojis basically related to like epistemic status. So you like, this was insightful, this was too complicated, uh, this made me laugh, or whatever, all these different things. So it's basically yeah, inviting people to sort of express their their more full range of emotion, and with the knowledge that that data is actually going to be used later.
24:01And and not in ways that, I think trying to get people more into the loop of like how this data is actually being used. Like, I don't want it to be like today where I don't really have such a good idea of how the like is used. When I make the like on something I don't really understand how it's contributing to the sort of algorithm, I don't have any way to influence it. But you know, imagine like, you could, you're kind of partner with, you know, you're part of, you're creating data, like so you're saying this made me laugh, this made me cry, this was insightful.
24:31And later then you have also the ability to kind of configure the algorithm and tell it, you know, whatever, I'm in the mood for something that make me think right now, or I'm in the mood for something that make me laugh. And just by that intention it kind of shifts you in this mindset of like, okay I have to set an intention, now I'm not just like doom scrolling, but I'm actually like, I have to be more active, I have to think what do I want to see right now. And that already kind of puts you in the more sensemaker space and less in the user space.
24:55Well there's an interesting sort of transformation, um, from I think models built on discrete values to something more less quantitative and more qualitative, um, from the emergence of these large language models. In theory you could write a textual description of what you expect to be filtered out of your stream, and that can be applied at every single step. But even with those large language models, if you dive into the reinforcement learning with human feedback, the signal that they're optimizing towards is something like quality or relevance, and there's a context in which the people are evaluating what is quality or or or relevant.
25:23And part of it is that they're being paid to do that, um, and they have sort of the larger motivation, so they might not be inherently honest. And the second part of that is really, I think, the time preference. I think our our attention spans are getting shorter and shorter, almost sort of flowing into this sort of singularity. And if we don't find a way to stretch that attention out further, uh, we're almost, like it's almost like we're losing our agency to self-direct, um. And so, oh go ahead, do you have a thought in your mind I'd love to hear it?
26:03Uh no, yeah, yeah you finish that, I'll just give you an example of an app that that is kind of trying to lengthen the attention span, but but we can get back to it. What what app is that? Um, this is an app that some colleagues of mine are working on, it's called Gather. And the basic idea is just to create sort of a campfire around some social media, or not around social media, around a media object, like it could be a blog post or, you know, any any any media on the internet, um.
26:43And what does that mean? It just sort of invitation to gather around some kind of, uh, you know, some some kind of media object, and then have a discussion about it that is sort of structured. They have sort of a way for, it's basically I mean a video chat room that has a little more some affordances for, like people can mark sort of, uh, moments in the chat that were insightful, and there's some like structured prompts to get people talking, and there's an AI that helps like summarize what happened in the end.
27:14So you've got to get this kind of summary artifact that that you can go out with. But but the whole thing I mean is just kind of, I see it as an invitation to lengthen your attention span. These things are actually very pleasurable, like I, you know, you spend an hour talking with people and if you curate the right set of people, I mean it's really interesting, and you're going to get a lot of insight and and it, and it's not like, yeah there's so many boring you know Zoom calls we've all been on, but if you do it right then it's actually really interesting.
27:42And and yeah, that's kind of just a practice of widening our attention. I totally agree with you, I think that that, like, super important, and and there's just like so many ways that we need to explore to to start doing that, because you know, every person has their own preference and and there'll be a lot of, again, diversity here. There is like also a disconnect between how we use the platforms and what we expect out of their behavior. The curation of any algorithm takes a lot of work, and often is, like on Facebook it'll be like going down a long list of checkboxes, uh, for categories specifically, and I don't find it to be a particularly useful approach.
28:06Um, I've been posting, uh, in the last few days, um, um, quite a bit about FourThought which is part of this Cognicism framework, and it's sort of a protocol for sensemaking, and it's built around the concept of belief staking, um, where individually staked claims or beliefs are tied to long-term, um, reputation or amplification of your voice. Um, meaning that there would be a distinction online between a passive post which is that you don't particularly expect this to affect you in any way, and a staked post.
28:38And a staked post, you know, will be tied to your reputation, you know, will affect particularly in the context of what you're saying in the future, whether you say that again. For example, if you're, um, a nutritional blogger and you tend to post all of these things which later many scientific studies come out and contradict, that if you're continuing to try and post in that context I don't think it makes a lot of sense for the algorithms to continue to amplify that content just because a lot of people at that moment liked it.
29:37And so there has to be a longer term time preference in the algorithms themselves beyond just what people are doing or thinking right now. Even in a lot of the conflict that's going on right now, the primary thing that I notice is just, even waiting 12 hours there will be better information, and by that time it's already spread through the internet and there's no real mechanism to go back to each of, to the people who might have received that misinformation and grab their attention again and be like, by the way that thing that you just posted to everything, you should be, you should go back and check or correct that, um, because it often, it's not going to always be 12 hours.
30:18You know, sometimes it's going to take three months, sometimes it's going to take three years, sometimes it's going to take a decade before we know the error of where we placed our attention. And sometimes myths about various truths get so baked into the collective zeitgeist that it takes just a really really really long time to break through that. For like example the the dopamine thing, which is where people think that, you know, either dopamine or serotonin are just sort of a reward drug that you just sort of get a hit of, um, whenever something good happens.
30:58And there's like a lot of different old wives' tales that sort of fit our cognition in a way that requires us to have to think less, um. I'm really enamored with Daniel's work with active inference and the the notion of, the brain doesn't want to use a lot of energy. And if you have a hierarchy of beliefs and some piece of information comes along that violates some base foundation of that, there would be a, there would have to be a lot of rewiring to get that flow back into that like a chugging engine, a chugging optimal engine that is, a well, like a person who is happy and and good.
31:40And so it's very easy, I think, for whatever passive mechanism that exists in the brain to reject, um, something, even if it's very logical, even if it's, it's very clearly presented, because you can't just plug a piece of information into it and just like put it on top like a Christmas tree, like there will inherently need to do some rewiring and you'll have to think over a lot of different things, you'll have to re-evaluate and restructure, and that would just take too much effort.
32:23And so there also is this process of sort of synaptic pruning for most people as we age, um, which is trying to whittle down those complexities to get sort of a baked-in model that is going to serve us for, until we're like 35, not until we're like 75. And you really have to actively challenge your mind with a certain type of information flow to keep, to keep the plasticity up. Yeah no I mean, I, me, yeah, you raised like yeah like five different points, uh, that we could dive into, but yeah just this last one I think of the idea of like this cognitive effort and like meeting people sort of where they at, where they're at cognitively.
33:07I mean you would think that this would be something that would be of like supreme interest and importance to like funding agencies or science, but it's like there's very little I feel like actual research being done here. One of the reasons is because the data that you would need, kind of, you know, you would need like a lot of social media data, and YouTube, like YouTube could be a treasure trove if you had access to all the trajectories people are making through the platform, um, over time, like that would be a treasure trove right? But we don't have access to that.
33:49So so and there's, so there's probably like, it's just a, if it's a, it's a blue ocean. There's really not much information I think about like, how, what do those cognitive trajectories look like? What does it look like, you know, the the intellectual growth of someone along with their, you know, emotional growth and all those things? Um Zak Stein is another guy who talks about this a lot, like I don't know if you're familiar with with Zak, but he's got a lot of really interesting sort of research and thinking about like the future of education.
34:26He talks about these kind of trajectories through information and YouTube and other platforms, um. So I totally agree with you, and I kind of feel like, you know, part of what we've been talking about earlier, we talked about these epistemic statuses and the different kind of reactions and expanding that range. So I think part of what that is should be like tailored and designed to do is is really help us sort of record our trajectories through content online.
34:51And and you know, if we're all like, if it was, you know, sort of a normative practice that people would kind of do their thinking out in the open and we'd all be sharing this kind of data, I'm sure you could do really interesting stuff, um, in terms of like tailoring people. Like for example you know, I'm, I'm at this misinformation story, um, but I'm coming at it from I don't know, very whatever, uh, uh, you know, right-wing perspective. So the the next content that, you know, I maybe would want to see is not the same as someone who is coming at the story from, you know, the other side of the political spectrum.
35:18So being able to have those kind of trajectories based on other people, I think, would, you know, allow you to to address some of these issues, um. And I wanted to say something about the earlier part that you talked about, the, um, this idea of kind of, you called it sort of the staking of reputation. Mhm. And I think this is also another really interesting topic that kind of ties into, um, another, this is kind of more my recent research related to scientific publishing.
35:46And, uh, a practice, like there's a practice now in in scientific publishing called nanopublishing, and the idea is to instead of producing, you know, like a paper, like a large PDF as your research output, you can actually start thinking of much smaller units of research, basically you know a tweet size assertion can also be a unit of scientific knowledge. You know that's called nanopublications, and you can think of these nanopublications, in a sense they're almost like, it's just like a tweet but it has a little more structure, and it also comes with like the the staking of reputation that that science brings with it.
36:14So science has already kind of been exploring this way of, like, you know, epistemic communication sort of, right? It's already been doing that kind of thinking about, like, there is some issue of reputation baked into science, there's an idea of retraction. So you mentioned how, you know, when something becomes disproved then there's methods of sort of rolling back that research, so that's the idea of the retraction in science, mhm.
36:49So you know, when you talked about the nutritionist that maybe published stuff and then that turned out to be false, and how would people get notified? So science kind of has a mechanism for doing it, again it's like way way too old school and it needs to be, you know, revamped for like the fast pace of current social media. But you could almost imagine that, you know, if we were sort of adopting some of these practices, you might get like, let's say, you know, you took some, uh, you tagged some piece of misinformation, you said this was really insightful for me, and that would kind of automatically kind of trigger this kind of watch for retractions.
37:15And if there was a retraction on that piece, then you kind of get a notification, and it wouldn't matter if it's, you know, a day after or a month later or a year later. Um, and I totally agree with you that this is the kind of infrastructure we need to be thinking about, like, people interact with information, uh, and currently on social media it's, say, kind of ephemeral stream, you're never going to see that tweet again. But in a new system it should be something that's not ephemeral, it's, it's, you know, if you reacted to it very strongly then we should take that into consideration and account for later.
37:41You can also link that back into sort of the trajectories of thought or trajectories of research, where it's like if something goes out there into the world and bakes into sort of the priors of like, these number of researchers and contributes their work, they're building upon this, you know, first principle, um, and then making a publish, publishing about that. And you can sort of see how that information creates flows of reconstructions in different people's brains.
38:05Because ultimately what happens is there is this, there's this a wiring of the physical structures of our brains that we don't have that much control over, um, and yet it affects so much of how our attention is being guided. If you want to break an addiction, you know, it takes an intense level of application of will to create a new structure that will guide the flow of your attention in, um, in a healthier way, um. Would you say that is like information that you're lacking?
38:32Or sometimes it's not information, sometimes it's actually like embodied practice that you need, or maybe some, it's a combination of both. It's like how the information within yourself is structured and oriented to each other. I mean, two people can believe like 10 separate facts equally and have that same cognitive structure produce a very different belief about the same article, um, because they're not always necessarily relating in the same way. Each, each architecture of mind, uh, is very very different, and we don't really have a lot of good tools for how we construct that, um.
39:14So before we started recording, you, you, you asked a little bit about my background and I, I think I could sort of zoom back to sort of the individual level and talk a little bit about my experience with attention. Um, because when I was five, you know, a teacher said, okay this, this student is having trouble paying attention, and they had read, um, or heard from somebody about the concept of attention deficit. And so for the better part of 30 years on and off I've been on some sort of stimulant, um.
39:56But it's not like there is ever a lack of attention. Our attention is always somewhere, it's whether or not we, the placement of that attention aligns with our identity and aligns with the identity of the community. So I was never, for example, um, asked about my social deficits, I was primarily asked about my attention in school, whether I was able to focus on, you know, math and reading. And a huge aspect that went by the wayside was my capacity to create friendships, to create connections, which is such an important, um, part of being a human.
40:40So I advanced, um, in my grades really quickly, and I was able to, um, learn very quickly, but they didn't really understand that it was not a cognitive failing on my behalf. Like they thought I couldn't read, which was, I, I was just not interested in the things that they were presenting to me. As I grew and as I, I've gotten older, around 25, um, I had sort of, well with the ADD, I, I had a manifestation of like hyperfocus, right, which is very helpful at times, you know, if I need.
41:24Like medication, or was it independent of? I, I, if I am interested in something, if I want to do it, there has always been an intense level of attention. It's when I feel a need to place attention in a particular direction that I just don't want to that it becomes very challenging, um. And so hyperfocus can be very positive, it has, it's very connected with flow for me and and the effortless type of cognition. And then around 25 I developed OCD, which is the opposite type of hyperfocus, which, you'll have a thought which for whatever reason makes you feel uncomfortable or doesn't agree with your identity in some reason, and then you just can't stop thinking about it.
42:18And it's almost gravitational, and and it's, it's almost like magnetism, right, where my consciousness, my awareness, will be drawn physically to thoughts, and then they get into an orbit. And once I'm in that orbit around that thought, all of the cognition happens in relationship to that root, and if I don't have the momentum to escape that trajectory, it's going to start to reshape the fundamental architecture. So my entire life has been in relationship to attention.
43:06And when I look out to a lot of my friends now in this world who are suddenly at, in their mid-30s, getting prescribed Adderall for the first time in their life, I try to explain to them, you don't, we probably don't have ADD, it's just that the information architecture that we have has deconstructed our capacity to self-regulate, and there are expectations about where you are to place your attention that don't necessarily align with your identity. So I just said a lot, so I'll just let you sort of say whatever you think in relation to that.
43:47Yeah, I think it's fascinating, um. I, I have done a little bit of reading about ADD, uh, and something that I remember really caught my attention was the sort of importance of environments. Like if you put a, you know, take a kid with ADD out of the classroom and let's say, I don't know, you know, take them to some like, I don't know, be out in nature playing with animals or something like that, it's like a completely different, you know, they could be completely different, and uh, you wouldn't see any attention problems at all, maybe completely normal.
44:39Um, so it feels almost like there's, and you, you said this piece about you felt the struggle to sort of force your attention where it didn't want to be, mhm, and it feels like, yeah, that feels really like an important piece, because you know, you get the sense in this kind of industrial era we're in, we're in this kind of, uh, you know, culture is the the ratchet of like creating machines out of humans, right? It's like the ratchet is turning, we get the, you know, ever since the industrial era where, you know, we had the manual labors on the production lines.
45:20And more and more we're kind of being coupled more tightly, you know, the gig workers in the Amazon warehouses or the the drivers, or, right, we're kind of becoming part of a machine, and the machine is kind of dictating our attention regime. And that's why as kids, you know, they start early with that, the the sort of indoctrination, that as kids you're expected, you know, read this book, write this thing, answer this, uh, very sort of rigid kind of regimes of attention.
45:53And it seems like, you know, some people maybe are good at that, and it almost feels like there's kind of probably a cultural self-selection, like the people that are good at sort of, um, overriding their attention and kind of, you know, forcing it to do things even if they're not interested in them, probably have an advantage, whereas people that might have like really great focus in their flow state but it's hard for them to, um, to kind of force their attention where it doesn't want to be, are at a disadvantage in this kind of culture.
46:22Uh, yeah, to me it's just like I hear this and I'm like, we need so many different kinds, and you know, such a diversity of like learning environments and not this kind of one monoculture that we have, which seems so wrong and and, uh, does so much damage to people, and makes, you know, makes people feel like they're sick or something, but the environment is so, and we definitely have this culture like blaming the individual as opposed to blaming the system, so it ties into a lot of those things as well.
46:50Um yeah, and I wanted, I wanted to, there's, there's a term that I learned recently and this is a really interesting line of research, uh, called attunement. Like your attunement, your, your the the quality of being attuned to to something, uh, um. It's comes from, like there's there's now a philosophy and ethics of attention I recently learned, so I've been following some research around this. And um, this idea of like where your attention is at is kind of, there's moral vices and virtues associated with that.
47:17Um, and and you know, it's an individual level, but it also is like a societal level, because a society might be ignoring something really important, and an individual might be ignoring things that are really important, um. And we can, we need to start talking about that, especially in an age of, of, you know, information overload, that we all like, you know, uh, drink information, we have to kind of make sense of it. So sort of the attunement to information, to me seems like such a, it's kind of a virtue we never knew was important.
47:48But nowadays like, I would, like, yeah, it's it's going to be, it's becoming so important so quickly, and and we can kind of see how oftentimes I feel like that's the, when I look at people these days I feel like that's one of the, beyond like left and right in politics I feel like there's this quality of, uh, attunement, like who is attuned to like what is relevant in this current moment. Yeah, I've even, the the person that wrote this piece about attunement, she even claims that like the wokeness, woke is basically like she likes, she she characterizes it as a quality of attunement.
48:18So like wokeness is a kind of attunement to certain kinds of, um, you know, um, um, moral, I guess, like, uh, transgressions or or whatever, um, but it's kind of a quality of attention, and it does sound like that. I mean, intuitively we can kind of hear that in the in the word, um. So yeah, and again, these things are sort of like, you know, like no one has heard about any of this stuff, but it feels like, I mean you know we were talking about attention, I I can just see that attention is gathering attention.
48:48You know people are becoming more interested in all these different aspects of attention, so I'm, I think that's like at least hopeful. But but yeah, there's definitely sort of, uh, very little sort of broad, you know, from the mainstream kind of to these topics. I think attention is also deeply connected [Music] to trust and reputation when it comes to sensemaking. If if you have a distribution of potential voices that you could listen to on any particular topic, um, you know, which one are you going to focus into?
49:52And there's also this aspect that, like, attention itself is inherently extractive because you cannot really consume the whole of an information, so it needs to take out the parts, it needs to like, like a little scalpel, cut it into pieces that fit into the existing architecture bit by bit and construct it up into a compressed representation. Of you mean it's like a scarce resource, is that what you're kind of saying? Because yeah, okay. And um, so I think moving forward, if if we want to talk about collaborative sensemaking where it's not just you're setting for for each individual the goal of the algorithm, but you're saying we're part of a group, we're part of a DAO, and we have these particular goals.
50:37And so each individual might have their own attention goals, but there can be a larger set of goals that have been defined collectively that will affect the information stream that each person is getting. And they have the freedom and right to choose to attend to parts of it, and it can adjust accordingly, but I think that we need to have better systems that are akin to trust, or a trust attention for various topics. And if we don't want to place attention to this, this, uh, institutional thing where for example, like I don't have the credentials really, really to get anything that I would write, any research paper I would write into anywhere that could get the type of attention, um, that it needs, even if that research is far ahead of other researchers.
51:17There are plenty of things that I have written about publicly that three or four years later I'll pull up an article and they'll say, we are the first person to ever do this, and I'm like, well how the hell am I supposed to, like what is my recourse here to say, well actually my attention was here far far beyond yours? I do think we have in language models, uh, an incredible capacity to assess at scale the semantics of what people are putting out into the world when those voices, uh, might move against the crowd at first, and then as time moves forward it becomes aligned, right.
52:07There there's almost like these epistemic, epistemic hills where it's like, this is where most of the belief is, and there's people sort of like little ants going out seeking out new sources of information. And over time those hills sort of just move and sit on top of the hill, that ant, and that ant isn't brought to the top of the pile, it's like it's covered up by everybody else. And I think that's sort of the mechanism that we need to move into, which is, well there is the trajectory of the individual ants over time, and there's a trajectory of the larger collective whole.
52:50And if we are measuring the trajectory of thought of research of all of these different voices at each moment, we can sort of shift or distribute attention in the algorithm relative to the past, relative to, well, what was this trajectory of these individual ants? Was this, was this an ant that was just tied up in what everybody else was saying? Are they always at the peak of the bell curve, or are there some voices that are just consistently ahead of the curve, they're also saying things that people don't want to hear at the moment that need to be heard?
53:26And then we can take that as a signal to say, this voice even though it's not getting a lot of likes right now, consistently they're ahead of the curve, so we should inject what they're saying into the data stream even if people aren't liking it. Like that was, yeah, metaphor um. Yeah I really like the sort of the the yeah, the ants and the bell curve. I, we can kind of feel this intuitively again, I think having the data out there would allow us to do a lot of really interesting, there's just there's so much you can do actually.
54:01I mean, you can really turn those intuitions into something practical if you have more data about people's this kind of cognitive trajectories through information, which we don't have right now. But yeah, um, I think we kind of feel it intuitively. And I want to give an example of, um, I think now at least this has been my experience, again we're talking about sort of the right, like the the war right now in Israel-Gaza and in general I think we've seen this in all these kind of world events, that oftentimes there's these people on Twitter that you follow and they will never get it into mainstream news, you just won't see their voice.
54:32And they could be saying things that are incredibly important. I found this from my personal experience that I, I look at news now, but it's definitely, it's only like one part and not even a very significant part of my sort of information intake. Like I much more trust different sources, like personal people that I I follow, and I get information that I would never be exposed to on the mainstream. And it really freaks me out because I figure, like, okay, you know, this person has let's say 300 followers or 400 followers or whatever.
55:01Most of the world is not seeing what this person is saying, I think it's incredibly important, I have no way to sort of convey that, and it's never going to make it into the mainstream. And yeah, collective attention, like you say, is going to be misattuned, it's not going to be, it's going to be missing very important information. We don't have any tools to to sort of account for that, so yeah, it's a huge gap in in our current information infrastructures.
55:28And that's part of the reason, you know, we talked about these kind of the different stigmergic failure modes like the death spirals, all these things, I mean. And we see this you know, mainstream media or people that control platforms can control, when Elon Musk tweets something he appears at the top of the pile, so like, yeah we we totally know that the the information streams that we're currently exposed to are very, um, prone to all kinds of failures.
55:50So yeah, so that was just kind to the point about like the the person on Twitter that tweeted something and and doesn't make it into the mainstream, I think it's interesting. So okay so this is kind of circling back to what you, you said early in the in the beginning about sort of the attention distribution like collective attention distribution and how like a DAO might manage that kind of thing. And um this is some some kind of research I've been working on, um, in Common Sense Makers with Brad de Graf, so this is his idea.
56:17He's got this idea of trust graphs, and and a few people have been talking about this idea. So a community's trust graph, like imagine that a DAO like, currently how does a DAO decide who it's following? Like, what people it's following, what accounts it's following? But the practice of sort of having a sort of community setting the people they're following is already kind of that, that's part of the collective attention governance. Like that is a very simple sort of aspect of it. And so a DAO, yeah, could, right now we don't see this really happening but it's kind of my dream to see sort of a decentralized collectively-controlled social media account.
56:44It would have like the whole API of Twitter you could imagine is like decentralized and like voting or whatever, you know, collective decision mechanism you want to use, so that yeah, your likes, that this thing makes, the retweets, the the follows, all these things would be sort of a collective decision. And then you'd kind of have this ability to sort of share the, yeah, I think it would be doing, I think it'd be an interesting experiment.
57:11To, I've been thinking about this for a while, but but this idea of setting a trust graph I think is part of the part of the collective attention, because yeah, the the when you follow someone you're basically signaling, you're saying my attention is on this person, I want to see more information from them. So it's a very simple act of kind of, uh, uh, of expressing this kind of attention. And I think that, you know, I'm I'm interested in trust a lot, and you know I've looked at a lot of different systems that have some sort of trust layer, and usually it's pretty much a wholesale sort of, this person or this source of information is trustworthy or not.
57:37And that doesn't really track in the real world, like trust, like I trust my doctor for things related to healthcare, but I don't trust them to, you know, fix my car or something. Yeah yeah yeah. Like so there is sort of a contextual aspect that we can get into where if somebody says a particular thing, because it can be placed within a semantic context using, uh, natural language processing, the algorithm can make a decision about whether or not to amplify that voice in that particular context.
58:10And that reputation doesn't need to be tied to a direct trust rating of another person, it can be related to just their trajectory basically, right? And it could be inferred from that trajectory such that you can have people who are not part of an institution, um, who might be saying something ahead of that curve, and over years maybe that institution catches up, and then they start saying okay we are the first people to ever write this, but you can just go back, we have the whole record, we have that whole record of all of those claims over time.
58:55And we can find the earliest representation of an idea, we can find the provenance of cognition where it's like, this is the first little seed, and this seed got into maybe these three minds and it mutated a little bit, and then that seed transferred from from these minds to these minds to these minds, and slowly as it was, as it worked through the collective mind space, um, and started to grow into various different people's minds, until it reaches a critical breaking point where it suddenly goes viral.
59:30And it says oh, this new article has come out that says and proves definitively, um, this new piece of information. We can track back over time and find non-institutional sources of information, those people with only 300 sources, or I mean, 300 followers, and really lift up their voice, but only in the context within which they have proven themselves. And I do think that it's very important for that, a part of that aspect to be to separate passive posting, meaning I'm just, you know, fucking around, with the, I need something stronger to to indicate to the algorithm, I need a mechanism that says hold up, you need to pay attention to this over time and you need to continually assess relative to this post, and maybe I'll discover that I'm wrong down the line, but I don't have that mechanism.
1:00:08I don't have a mechanism other than just repeating content multiple times, and even then like it's based off of my basic understanding of how these algorithms work and not that I actually have access to how they really work. I don't know particularly how the Twitter algorithm distributes information. I know that when I post something, if it's like a schizo Buddhist non-dualist post, like very quickly those people, the people who are associated with that are alerted to that.
1:00:59But is is that not just a bubble of misinformation, where it's only presenting the people who are already in the similar part of the semantic field? How do we push beyond that and give people the tools to say okay this is, this is a smellier scent marker, and I'm choosing to leave this smellier scent marker, and you should smell this smell because it's very important right now, and it won't just get covered up, it won't just get buried in like the millions of grains of sand of other posts that are there, and it can actually bubble to the top, bubble to the top of of the, of, I guess, the collective mind space when it is actually necessary.
1:01:39Yeah. I mentioned earlier the FourThought dialectic which is part of the Cognicism toolset, and each sort of claim that you would stake has two spectrums along which they're associated with vote, which is, I guess you could call it verity and valence. And it's, verity is like the spectrum of certainty, with a full confidence that a piece of information is false on one end, and the center just complete uncertainty, and on the other end a full confidence that the piece of information is true.
1:02:17And then there's valence which is, uh, is it, is it good or bad or neutral, with the same type of spectrum within it. And the notion being that we need to balance between hearing things that are true that we don't want to hear that may not be good, um, and we do want to minimize sort of the misinformation that we get even if it makes us feel good over time. So there's sort of a balance in that vector of which direction we want to go in society. And I think markets for example really lean into the valence of want, short-term want, does this make me feel good, and not so much on, is this true, um?
1:02:59Or, and in particular not, we don't have a society that leads us into those degrees or leads us into uncertainty. Um, the last thing that I want to bring up is maybe not directly semantically related but it's just bubbling around in my head, which is the degrees of freedom that we have in our attention, um. Because I, I kind of think that our cognition is sort of in this semantic field, and I know from my experience that I can run from thoughts a little bit, but they're still somewhere out there, and eventually they'll come or they'll bubble back up and they'll have to to deal with whatever that thought is trying to tell me until I let it sit in the center of my attention for a while, and just let it, I call it like, the thought needs to feel heard, uh, or it won't stop.
1:03:40Even if it's like a crazy thought I just have to sit with it and be like, okay. And so there is like, I would say that we have at least the the ability to move away from a thought, so there's sort of a directionality to the lens of attention in our mind. But the very underrated other aspect of attention is the widening or the closing of the lens. And I really feel like that's what's lacking in a lot of the discourse right now, that a lot of what people are saying, if they just zoomed out and took a broader picture and then sort of zoomed back in again, it would alleviate a lot of the emotional based posting that the algorithm seems to love.
1:04:20There's a tight relationship with our personal relationship with attention and how we are able to modulate it, and and the algorithmic aspect of it. So we both need to be able to work on the larger algorithmic aspect of our attention modulation, and gain the knowledge about how we can strengthen the muscle that allows us to direct our attention within our own minds. There's also this big problem that, like, I think when I look at Elon's behavior and how he's sort of reconstructing Twitter, is that he maps attention very closely to physical energy.
1:04:56And so a lot of the design decisions that he's making are simply from creating an energetic bubble where it feeds back into itself, like some of the first things that he did was like take away links outside, like you, they tried to penalize people having a link to other platforms. In the top of the thing and now, yeah yeah, because they want to keep you inside. And when you only have such a short type of content you're going to keep that attention smaller, but you want to have circumstances where it can sort of widen and open to consume in more detail, um, information.
1:05:41And maybe that, that, that's going to come in the future, maybe he's a 37-dimensional chess master and it's all going to work out in the end, but. And there has to be a sort of energy that comes back to get them the profit that allows the platform to continue existing and allows the, um, you know, maintenance of the site, and so they're always sort of intrinsically connected. And I I sense, I sense that we're at a sort of point similar to, you know, when America came into being, like Declaration of Independence and the Constitution.
1:06:30Of, there is, there is evolving the replacement, or the, a new fundamental type of system that was once what governance was doing, except for now it's happening every second, and we've built it based off of corporate incentives. And there needs to be something like a big moment or a big document or a big, a sense that it is fundamentally of a greater weight and importance to our society than just what it is, which is fun little social networks. And people need to understand that these systems are actually guiding democracy more than democracy is guiding them anymore.
1:07:09So I think that's a pretty good, uh, place to put a pin in it, especially if we lost 17 minutes of this, um, but this was a really good [Music] conversation. [Music] No don't.