Brass Tacks

A peek into Saul Munn’s thoughts.

View / . / previews.

  1. 2025 8 min
    Getting much less slow at basic mental arithmetic

    [Note: this piece is still very much a draft, but (a) I thought it might be useful enough to the occasional person that it was worth posting, and more proximately, (b) I needed to post something for a blog-a-thon I was hosting. However, as such, it’s not quite finished and not up to my usual par. Enjoy!]

    I think this piece may be especially useful for people who have always just been unexplainably “bad” or “slow” at math, but who also seem to sometimes just “get” some math conceptual stuff rock solidly well. This piece is least likely to be least helpful for people who have always been “good” at math.

    I. I’m slow at math

    I was always slow at math — the sort of math you do in middle- & high-school, and early college. Slow enough that I’d take twice (!!) as long on homeworks as my friends. Slow enough that practice problems were a horrible, draining, tensed process. Slow enough that studying for tests would involve reviewing the general ideas, not just doing some example problems to get the gist.

    This was especially annoying because I otherwise enjoyed and was reasonably good at the abstract stuff — at grasping the underlying ideas, working through their implications, and deriving new-to-me characterizations. (In my Calc 1 class, I turned in none of the homeworks the entire semester, and studied just by explaining the concepts to my friends, then letting them work through practice problems themselves — but I aced every exam and got an A+ in the class. And when I took first-order logic, I again did almost none of the homeworks, skipped half the Iectures, skimmed a couple of the slides, and aced all of the exams.)

    A couple years ago, a friend showed me Zetamac. It’s a “fast-paced speed drill where you are given two minutes to solve as many arithmetic problems as you can.” And it’s surprisingly fun — you’re shown questions like 15+87 or 224÷7, and just have to solve them as quickly as possible. I found doing Zetamac fun, like doing typing speed tests.

    At this point in the post, I encourage you to try doing one or two rounds of Zetamac. My guess is that it’ll help you get a sense of the things I discuss below, especially if you happen to be as bad at it is as I was.


    With Zetamac’s default settings, I’d typically get a score of 8 or 9. On a great run, I’d get 12 or 14. My best score of ~50 trials over a week or two was 20. My friends were consistently getting 25-30, and occasionally up to 40 or higher.

    I knew I was slow, but this was, somehow, simple enough to be clarifying: there was something they had that I didn’t.

    II. …but maybe I don’t have to be slow

    It was early 2025 that I started reading some of Justin Skycak’s writing. His tone is at times frustratingly self-help-y, condescending, and un-empathetic. And also, at different times, he’s inspiring and enlightening. For examples, look at these pieces.

    However: I found his piece on having automaticity in basic arithmetic for learning algebra pretty raw. The piece is short, so consider just giving it a skim. For me, the feeling of some of his characters resonated — that of painfully working through the details of some problem, getting bottlenecked on the slowness of particular subskills (in this case, basic mental arithmetic), giving up in frustration, or just generically not being able to "get" it. That feeling bounced around in my head, and swelled ‘til it hurt.

    So in May 2025, I oriented with greater agency to the fact that being slow at math is a problem I could try to solve. What can I do to get faster? Much faster? (Or alternatively: in a future where I become much faster at math, what must have happened?)

    Here’s what I did, and how it worked.

    III. Straight to the goal

    I asked myself “what about just trying to get way faster at mental arithmetic?" For which the straightforward method is just to grind Zetamac, the thing which I was using as a metric. Maybe Goodharting works!

    But after ~100 trials, my score was basically not improving. Hmm. Maybe Goodharting doesn’t work after all.

  2. 6 min
    Interview: What it’s like to be a bat

    For the purposes of this transcript, some high-pitched clicking sounds have been removed. The below is an otherwise unedited transcript of an interview between Dwarkesh Patel and a bat.

    DWARKESH: Thanks for coming onto the podcast. It’s great to have you—

    BAT: Thanks for having me. Yeah.

    DWARKESH: You can hear me okay? I mean, uh, all the equip—

    BAT: Yep, I can hear you.

    DWARKESH: Great, great. So—

    BAT: I can hear your voice, too.

    BAT: If that’s what you were asking.

    DWARKESH: What? No, I was—

    BAT: Oh, “hear” probably isn’t the right word, I guess. “Sense”? No, it’s not “see.” The translation suggestion thing isn’t right.

    BAT: I can [inaudible] you. It’s still so weird to me how humans echolocate through your eyes.

    DWARKESH: Er, sorry, I was asking—

    BAT: Yeah, I can also hear your voice.

    DWARKESH: Uh, great. Okay.

    DWARKESH: So, the question we’ve all been waiting for, haha: what is it like to be a bat?

    BAT: Oh, sure. Yeah, that’s been everyone’s first question. I dunno, what’s it like to be a human? Haha.

    BAT: No, but — I mean, it’s not like I’ve ever felt your internal experience. How should I know which details of my phenomenology are relevant to you, and which aren’t?

    DWARKESH: Oh, interesting. I guess that’s fair. Do you feel like you have a good grasp of what it would be like for you to be another bat? Or is it, like, just a mystery whether—

    BAT: I have as much a grasp on what my fellow bats’ consciousnesses feel like as you have on your species-mates’ consciousnesses. Actually, no. I have much worse of a grasp of what it would be like for me to be a different bat than you do of what it would be like to be a different human from you.

    DWARKESH: Oh, really? Why, uh — why would that be the case?

    BAT: We can’t — couldn’t — communicate with each other with nearly the precision nor fidelity that humans can. We haven’t built epistemic institutions that are curious about philosophy of mind, nor societal traditions that cause our young ones to rigorously reason on their natural empathy, nor neurological technology that let us peer into others’ minds, nor psychiatric practices that carefully catalogue and study every ontology of mind-state. We don’t — didn’t — have the intellectual capabilities of mapping each others’ phenomenology, let alone the physical and social technologies necessary to create such maps with detail.

    DWARKESH: But it doesn’t seem like we’ve made much progress, right? We being humans, sorry.

    [pause]

    DWARKESH: Like, we don’t have a great grasp of what stuff consciousness is made of, or what the fuck is going on with psychedelics. There’s so much in our phenomenology that’s just completely bizarre. Synesthesia, aphantasia — I mean, I have no idea what it’d be like to not have visual imagery. Like—

    BAT: Oh, sure, but you were saying that humans haven’t made much progress. I don’t think that’s true at all.

    DWARKESH: Yeah, explain what—

    BAT: You know what synesthesia is. Humans in the 1700s didn’t. Again with aphantasia. I mean, until the 1950s people didn’t know what LSD was. You have made strides — serious, important strides — in neuroscience, in psychology, in cognitive science, in philosophy of mind. Even just in the last 50 years. I could go on. You get my point.

    DWARKESH: Okay, I see what you’re saying. Yeah, I agree we’ve definitely made progress, but it still doesn’t feel like we’re actually getting anywhere closer to knowing what it’s like to have consciousness. I mean, obviously I know what it’s like to have consciousness, right, like I’m not a p-zombie, but I don’t know what Trump’s internal experience is, or—

    BAT: Or what it’s like to be a bat?

    [laughter]

    DWARKESH: Yeah, exactly. Or what it’s like to be a bat. Or a bird, or a floor tile, or a Balrog. Or the United States, or an organization. Like, we’re nowhere closer to knowing what the internal experiences of minds very different from our own— actually, we’re not even close to knowing what it’s like to be something really similar to us. Not from the inside.

  3. 10 min
    Recap of Memoria 2025

    Memoria 2025 was a one-day, ~120-person unconference/festival on spaced repetition, incremental reading, and memory systems, held at Lighthaven in Berkeley, CA, on September 22, 2025, and organized by Saul Munn and Raj Thimmiah.

    The goal of this post is to provide context on the event — what caused it to exist, what happened during the event, etc. I (Saul Munn) wrote this from my own perspective.

    Our goals going into the event

    Roughly ordered, by priority:

    1. Throw down a bat-signal for people with years of experience using and tinkering with memory systems like Anki, Supermemo, Remnote, Mochi, etc, to meet other folks who were attracted by the same bat-signal. Foster durable connections, start long-term friendships & partnerships, etc.
    2. Get a more & better projects started in the space.*
    3. Push the frontier of memory systems usage forward.*
    4. Raj & I enjoy ourselves fun.

    *Within 2 & 3, Raj was especially-but-not-exclusively interested in incremental reading projects/frontiers.

    But on a deeper level, I just really wanted to attend something like Memoria — and it seemed to me that the only way that was going to happen was if I organized it.

    What went well

    • Lot of people: ~120 tickets purchased, ~120 total attendees (attrition from ticket-buyers cancelled out folks to whom we sent free tickets, like guests of honor), ~100 people max concurrently
      • (Side-note, but basically every major event that I run, I’m shocked at how many people register at the absolute last minute. We got ~half of our tickets in the four days before the event. Nuts!)
    • Seemingly high proportion of the memory systems community — at least, many of the people I was hoping could make it ended up coming
    • Incredible density/signal of people
      • Raj & I were aiming for Memoria to be more of a place for experienced, non-beginner memory systems users to come together. I think we basically succeeded — e.g. Michael Nielsen had planned for his opening talk to be something like “bringing everyone up to speed,” but when he did a raise-your-hand poll at the beginning of his talk, such a majority of people already had so much experience with memory systems that he just pivoted his whole talk to an extended Q&A (which I think went wonderfully).
      • Also many people from adjacent areas, e.g. neuroscience
    • On the whole, attendees enjoyed the event (more analysis at a later section):
    • The sessions were pretty great. Some examples of talks/workshops:
      • 10 Important (and Underappreciated) Observations, with Michael Nielsen
      • Augmenting Attention, with Andy Matuschak & Taylor Rogalski describing their in-development app
      • Deliberate Practice for Software Engineers, with Sameer Ismail
      • Why can LLMs win the IMIO but can’t write good flashcards?, with Ozzie Kirkby and Andy Matuschak
      • Memory Systems need Genuine Interest, with Soren Bjornstad
      • SuperMemo Team Q&A, with Piotr Wozniak and the rest of the SuperMemo team
      • Homemade Tools Show & Tell, hosted by Raj Thimmiah
      • And also some smaller side-sessions:
        • What is worth remembering?, with Karson Elmgren
        • Are short-term memory systems possible?, with Alexis Gallagher
        • Content-Aware Spaced Repetition, with Giacomo Randazzo
        • Fermi Estimation with Spaced Repetition, with Drake Thomas
        • Anki for Math, with Lydia Nottingham
        • Making Memory Systems Widespread, with yours truly (Saul Munn)
      • (We’ll have a videos up on YouTube of most of the talks soon — will update the text of this post when they’re here!)
    • Net profitable (!)
      • ~$9k in total costs, ~$12k in revenue, ~$3k in profits
      • We intend to plow all of the profits back into events like Memoria in the future. (If you’re interested in drawing from this war chest to host meetups in your city, let me know!)
    • No ghastly operational failures (e.g. a literal fire a publicly-visible, largely disruptive, literal fire).
    • Raj & I had a good time! In particular, we were both a bit worried about Memoria eating up all of our time/energy, and I don’t think this happened — we ended up at a pretty great 80/20.
  4. 15 min
    Mistakes often made by inexperienced event organizers

    (this is geared toward events of size 50-5000, and probably a lot less useful for e.g. a casual gathering with 5-10 friends than e.g. a 100-person weekend conference.)

    • do before anything else:
      • goals, date, and venue. why figure out goals, pick a date, and pick a venue before all else?
        • so many other tasks are blocked on those. try doing marketing for an event without knowing where/when/what/why it actually is!
        • once you’ve figured those out, you’ve committed yourself in a way that will mentally prepare you for the fiddly details (i.e. the rest of this doc)
      • figure out your goals, broadly speaking. for ex: write a one-pager, send it to a couple friends, think for 20-60 minutes
        • this should be real, not fake — there’s a reason you’re doing this first. if you come out of it realizing that an event is the right thing to do, run an event; if not, don’t!
        • leading questions:
          • what do you want?
            • seriously, what do you want?
          • why are you running an event?
            • what are other things you could do that would satisfy those goals? what about doing those other things instead?
              • what reasons do you have to think that running an event would be more effective toward those goals than some other thing you could do?
          • what do you want the event to accomplish?
            • why do you have those goals?
              • why again?
                • (ad infinitum)
            • how will you know whether you accomplished those goals?
      • pick a date. ask a few potential attendees if they have any conflicts. some common mistakes:
        • don’t pick a date that coincides with another event that your attendees will want to go to. i once picked a date for an OPTIC forecasting tournament that was the same date as the putnam, and we lost ~ 15%-20% of possible attendees.
        • try to avoid coinciding with national/religious holidays — many people schedule things on top of jewish holidays, which forces me to pick between being a good jew vs going to their awesome event.
        • pick a date/time that generally works for the people you want to attract. e.g. don’t pick a weekday at 3pm for working folks, or a work event for jews on a saturday, or summertime for students.
      • pick a venue.
        • finding a good venue as early as possible in the process is so ridiculously important and so often undervalued. you do not want to be in the unfortunate position of having venue issues, you do not want to be scrambling to find a venue last-minute. this is one of the few things that can totally fuck up your event before it starts.
        • (sidenote: if you’re considering running an event in the SF bay area, check out Lighthaven. i have literally never encountered a venue in which i would rather run an event — it’s stunningly beautiful, wonderful for parallel small-group conversations, incredibly customizable for different types of events, and about a hundred other things you want in a venue. strong recommend!)
    • food matters WAY more than you think it does
      • why?
        • people don’t enjoy themselves when they’re hungry. they’re cranky, they get more upset, they don’t have patience, etc.
        • eating together is an incredibly bonding experience: to the extent that “bonding” is one of your central goals for the event, food is good at this, one of the most effective things you can do
          • from austin chen:

            one of my favorite business books is “Never Eat Alone”. I’ve never read it but the title says it all

            (I don’t think you should literally practice that but — yeah humans are like hardwired to like the people they share meals with)

            ross rheingans-yoo responds:

            Having read/listened to about 70% of it, I think the title doesn’t actually say all or even most of the content of the book, which has a large amount of “how to organize your social life as a systematizing autist / sociopath”.

            That being said, the title is on its own a good thing to have in mind.

        • in particular, many organizers realize that “food is important,” but don’t quite realize exactly how fucking important it is. i doubt you need convincing that food is important, but i hope i can just repeat it to you enough that you should make your food situation really really good
          • in particular, consider spending:
            • $10-$25 /person /meal, depending on e.g. how fancy you want to get, whether you’re doing buffet style or individually packed, etc
            • 1/4-3/4 of the budget of your event on food
      • if you’re not serving food, strongly consider serving food
      • if you’re serving poor-quality food, or small portions, strongly consider serving better quality food with bigger portions. food does cost money — consider shifting more of your budget to an actually-good food-experience, or feeling less bad about spending $x on food
      • if you’re not quadruple checking that your food is coming on-time, that it’s high-quality, that it’s what you ordered & as much as you ordered, that it’s the right kind, that it’s as vegan/vegetarian/gluten-free/allergen-free/etc as you need it to be, that it’s hot/cold/whatever temperature you need it to be — strongly consider doing so
      • serve good drinks
      • serve good snacks in between meals
      • probably err on the side of buying too much > too little
      • don’t just think about the food itself, but all of the structures around the food:
        • where you’re serving the food; how long it’ll be out for; what the lines look like; etc
        • where people are eating the food; how long they’re eating it for; what sorts of groups will form, whether you can make the groups better by their own lights; etc
        • remember to think about the things you actually care about! don’t just optimize for efficiency!
          • 3-5mins of lines to get food might be a lot better than 0 mins — it gives people an opportunity to find folks in line with whom they can chat and (potentially) sit down to eat
          • having people sit on the ground might be better than having formal tables — the ground is a little uncomfy, but encourages more natural groups to form + is easier to socialize
      • downsides:
        • food is quite expensive, sometimes the biggest line-item budget. it also scales linearly with the # of attendees, which can make planning hard
        • food is pretty annoying to handle. there are a bunch of regulations, people can get sick, it has to be kept hot/cold, it goes bad quickly, etc.
    • the one secret tip most conference organizers don’t want you to know: you don’t have to do the boring stuff. you can think carefully about the experience you want your attendees to have, then optimize around that.
      • misha glouberman’s post “Everything you did to make your conference better actually made it worse” is such a great post on this topic, with some great concrete examples.

        Say you had 200 people in a room and you wanted to stop them from talking to each other. A great way to do this would be to give one of them a microphone and put the other 199 in chairs facing forward.

      • examples of boring stuff you can probably cut/refactor better:

        • endless 1-on-1s
        • talks from bad/boring speakers
          • maybe just talks at all?
        • “everyone go around and say your name, how you first heard about this event, and a fun fact”
          • in general most icebreakers are fake. why are you doing it? what’s your goal here?
            • okay, but like what’s your real goal here?
              • sure, but if that wasn’t your goal, what would the goal actually be?
        • overly long intro/closing speeches. just get into the good stuff!
        • panel discussions are often really hard to get right, and when they’re wrong they’re pretty bad
        • scheduled breaks (can be good, but often executed poorly/leave a lot of value on the table)
        • mandatory, contrived team-building exercises
        • awards ceremonies that drag on with speeches for minor recognitions
      • examples of non-boring stuff:

        • if you want attendees to make new friends, you can do stuff like speed-friending (thread on how to run this well)
        • if you want attendees to learn some concrete skill, you can ask a practitioner of that concrete skill who’s demonstrated that they’re competent at teaching it extremely well (e.g. at previous events, or on YouTube/Twitch) to give a workshop
    • you should be constantly modeling your attendees, constantly checking that model against reality (e.g. by asking your users), and constantly updating that model. some common mistakes:
      • when you’re modeling an attendee, you should think of them as being mentally exhausted, physically tired, overworked, overstressed, overanxious, and a bit of an idiot not caring as much about your event as you, the organizer, does. they have vastly less context. they’re probably only the first 5. implications of this abound, see e.g. “don’t use a map” (below) or “make your signs like 5x bigger” (also below)
      • many people don’t really understand their preferences well at events. knowing peoples’ preferences better than they do is super useful. examples:
        • people constantly complain about loud music at parties, but it fills an important social role (sewing together gaps in conversations, providing something to talk about, giving people an excuse to dance, breaking up big conversations into smaller conversations)
        • at conferences, people often feel like they should go to a big talk, when really they should probably meet with people 1-on-1 or in smaller conversations.
    • small pushes on key coordination can go a long way.
      • austin chen:

        probably the whole job of an organizer is to push on coordination, otherwise they wouldn’t need the event in the first place

      • examples:

        • people will by-default just get dinner at a restaurant or something. if you do a tiny bit of coordination for them — like handing them a list of restaurants nearby, or making a channel in your slack/discord/whatever for “#dinner-plans” — you’ll unlock a lot of value. sometimes you can literally just round everyone up, shout “if you want thai food, go to this corner and meet all the other people getting thai food; if you want italian, go to that corner; if you want chinese, go to that corner” → let everyone go to the cuisine of their choice and make some new friends.
          • leila clark:

            Oh this reminds me of one of my favourite unconference techniques which is to have people put ideas on a board and then have others gather around the things they want to do, then have them break out.

        • having some sort of attendee database with contacts, emails, and how people can help them/how they can help others
        • having some sort of central communication system, like a slack or discord or groupchat
        • setting up default points where people can just hang out/cowork/vibe/eat/meet each other
    • people don’t know where anything is. people will never know where anything is, no matter what you do. but there are a few things you can do to prevent the worst of the worst:
      • don’t have a big map — this requires people to figure out where they are, where they want to go, and how to get from one place to another. that’s a lot!

        • “are we the red star, or the yellow exclamation point…?”
        • “which room was the talk in? 45b or 54d? or was it the Skyroom?”
        • “do i go up the stairs and to the left? or do i have to take the elevator first? which one? to what floor?”
      • have a lot of signs that say very little on each sign, and place them EXACTLY where someone would be looking for them. examples:

        • when do you want to know where the bathroom is? when you’re walking into the venue, when you’re walking out of the venue, or when you’re walking out of any session space.

        • when do you want to know where lunch is? at like noonish: don’t put up signage telling people where lunch is until then

        • at intersections, or right before someone goes down a long hallway — it’s costly for them to make a decision about where to go, so having an answer right in front of them goes a long way

        • a lot of this comes down to modeling the attendee experience really, really well

        • joel becker (below quote is completely taken out of context, but IMO applies here):

          you should be iterating + attempting to model attendees/getting better at this over time

      • make your signs like 5x bigger, with minimal info

        • bigger

          • people hate having to squint at signs!
          • i consistently see people think “that’s gotta be big enough, right?” → it’s not even close to big enough
          • think HUGE signs with HUGE font
            • bigger than what you’re currently thinking
              • yup, still bigger
                • okay that’s probably about right
        • minimal info

          • model your attendees, try to figure out what they want when they’re about to look at your sign, then give them exactly the info they want and no more.
          • like half of the things that people want to find are bathrooms and food. make these ones so ridiculously clear that i’d be difficult to avoid knowing where they are.
        • negative examples:

          • too much info, not nearly big enough:

        • right amount of info, but way too small, and still some distracting visual elements:

      • positive example:

        • it’s HUGE, it’s exactly what the people need, they don’t need to think at all, it’s no additional info, it’s a chef’s kiss
      • but even if you print your sign HUGE with HUGE font and minimal info — people don’t read. they never read. expect this ahead of time, see “modelling an attendee well” above

      • joel becker, against this section:

        i feel like i did absolutely none of this and everything was fine, because there are bigger wins elsewhere. (in particular people)

    • premortem the whole event, many times, in every stage of the process, and for many sub-processes
      • premortems are so fucking awesome. they’re just fantastic. i love them.
      • i put the premortem guide that i use below:
        • meta
          • i’d recommend reading through the whole doc before actually doing one yourself
          • i’d recommend writing your premortem out, and/or recording a conversation with your team
        • first: imagine the event has just finished. close your eyes, really put yourself in that headspace.
        • now imagine that things went…
          • really, really badly. worst-case. shit hit the fan. pause for a moment.
            • what happened?
            • why did it go really badly? list as many scenarios as you can think of
            • what factors led to those scenarios?
            • what can you do to make those scenarios less likely to happen?
            • what can you do to make those scenarios less bad if they were to happen?
          • fine/okay/meh, but you lost a lot of potentially value. pause for a moment.
            • what happened?
            • why did it go really meh? list as many scenarios as you can think of
            • what value wasn’t captured? why not?
            • what can you do to capture make it more likely that you’ll capture that value?
            • what can you do to make any value you do capture much better?
          • really, really well. everything went smoothly, everyone had a great time. pause for a moment.
            • what happened?
            • why did it go really well? list as many scenarios as you can think of
            • what factors led to those scenarios?
            • what can you do to make those scenarios more likely to happen?
            • what can you do to make those scenarios even better if they do happen?
      • for the “produce solutions” part of premorteming, i found it useful to reference my notes on some methods ive used for producing solutions
      • why are premortems useful?
        • they push you to notice the things that you’ve been flinching from
        • they give teams a space to talk about things that might’ve been uncomfortable or hard to talk about
        • they let you pick up on things that are harder to pick up on otherwise
        • they give you a record of things that you’re concerned about/eager to do more of
        • they let you prioritize between how important different tasks are
        • against:
          • joel becker: “these reasons didn’t strike me as very useful. i’m like just be less anxious + put things into high prio vs low prio”
          • austin chen: “I’ve never put that much stock in premortems either, sorry”
    • take pictures
      • this is useful for stuff like advertising, social proof, doing posts on social media, etc

      • by default, you’ll probably take too few pictures, because it’s not viscerally intuitive while the event is going on why you’d want a bunch of pictures. take more!

      • probably take a quick video or two, but i usually don’t find them nearly so useful as good pics

        • your phone is definitely sufficient, but having a good camera (and someone who knows how to use it) does make a difference
      • group photos are particularly good — they seem a little annoying when you’re doing them, but are typically the main picture you end up using anywhere

      • austin chen:

        bonus points if you can get a photographer to come and bring their fancy camera. you can hire for this too ofc, but it’s great when you have photographers like Misha or Rachel in your community

    • naming (h/t conflux)
      • names are really sticky & costly to change. it’s important to at least not use a really really bad name, but also note that a really really good name can be substantially better
        • positive example: “Manifest” is itself a word, is a pun on “manifold festival” (which is also descriptive of the thing), and is also coherently within the world of manifold
        • positive example: “Mox” has a number of good connotations (e.g. “moxie,” or “mana rocks”), but also doesn’t specifically bring some particular thing to mind, which also means it doesn’t limit the organization to something
      • this post is great, and this post is pretty good (including the comments, which add some nuance-via-disagreement). this post is also directionally correct, especially when considering writing your slogan.
      • ask an LLM for feedback on your name — in particular, ask it for reasons it might not work/be bad
        • austin chen: “fun fact, I was deciding between 3 names and Claude said like ‘Mox is obviously the best choice’”
      • ask friends for thoughts on names you have, but don’t just give them a potential name and ask for thoughts — give them a list of names, then ask which they think are particularly good/bad and why.
    • relax, have fun, lean into it :) (h/t WS)
      • quoted from WS:

        One of the biggest things I pay attention to while facilitating an event is to persistently keep in mind what the participants are thinking and feeling. Some questions I ask myself are:

        As a participant…

        • How socially accepted would I feel in this environment? Do I feel like I can speak up and contribute to conversations? Do I feel welcomed and included?
        • Something I do as an organizer to get these questions to "yes":
          • Try to have an easygoing and accepting demeanor.
          • Talk with attendees during breaks, and try to connect with them on something their interested in, to build rapport.

        Some mistakes I made (and still make) in vibes:

        • Acting visibly stressed and anxious.
          • I was pretty anxious during my first few workshops. I now try to premortem what will make me stressed and reduce that before the workshop.
          • I figured it was fine if I was acting high-strung because I associated "x person acting high-strung about an event" with "x person caring a lot about their work". But I think one of the main mental states to be in when presenting yourself to attendees as a group is being easygoing, accepting, and generally sort of chill, especially when people are already coming into something feeling anxious and wondering if they fit in. [Saul notes — Joe Carlsmith wrote an excellent piece on this topic, “On clinging.”]
            • Note that in individual conversations, this is less applicable. [Saul notes — I disagree on this point.]
      • also from WS:

      • concrete methods i use:

        • picture someone you know who’s particularly charismatic, charming, social butterfly, outgoing person you know. try to act a bit more like them. (h/t nathan young)
        • as WS said, premortem things that you expect to cause you stress/anxiety explicitly to make it more likely that you won’t act stressed/anxious
        • set up systems that will explicitly remind you to calm down (e.g. set a reminder on your phone, or set up a TAP that each time you walk through a particular doorway, you take a breath and release tension in your abs/shoulders/etc)
        • (if legal) have half a beer
    • i haven’t read “the art of gathering,” but most of the people who looked at drafts of this recommended i either read it or recommend it here. quick summary of the book here, full book here.
  1. 2024 4 min
    Active Recall and Spaced Repetition are Different Things

    Epistemic status: splitting hairs. Originally published as a shortform; thanks @Arjun Panickssery for telling me to publish this as a full post.

    There’s been a lot of recent work on memory. This is great, but popular communication of that progress consistently mixes up active recall and spaced repetition. That consistently bugged me — hence this piece.

    If you already have a good understanding of active recall and spaced repetition, skim sections I and II, then skip to section III.

    Note: this piece doesn’t meticulously cite sources, and will probably be slightly out of date in a few years. I link to some great posts that have far more technical substance at the end, if you’re interested in learning more & actually reading the literature.

    I. Active Recall

    When you want to learn some new topic, or review something you’ve previously learned, you have different strategies at your disposal. Some examples:

    • Watch a YouTube video on the topic.
    • Do practice problems.
    • Review notes you’d previously taken.
    • Try to explain the topic to a friend.
    • etc

    Some of these boil down to “stuff the information into your head” (YouTube video, reviewing notes) and others boil down to “do stuff that requires you to use/remember the information” (doing practice problems, explaining to a friend). Broadly speaking, the second category — doing stuff that requires you to actively recall the information — is way, way more effective.

    That’s called “active recall.”

    II. (Efficiently) Spaced Repetition

    After you learn something, you’re likely to forget it pretty quickly:

    Fortunately, reviewing the thing you learned pushes you back up to 100% retention, and this happens each time you “repeat” a review:

    That’s a lot better!

    …but that’s also a lot of work. You have to review the thing you learned in intervals, which takes time/effort. So, how can you do the least the number of repetitions to keep your retention as high as possible? In other words — what should be the size of the intervals? Should you space them out every day? Every week? Should you change the size of the spaces between repetitions? How?

    As it turns out, efficiently spacing out repetitions of reviews is a pretty well-studied problem. The answer is “riiiight before you’re about to forget it:”

    Generally speaking, you should do a review right before it crosses some threshold for retention. What that threshold actually is depends on some fiddly details, but the central idea remains the same: repeating a review riiight before you hit that threshold is the most efficient spacing possible.

    This is called (efficiently) spaced repetition. Systems that use spaced repetitions — software, methods, etc — are called “spaced repetition systems” or “SRS.”

    III. The difference

    Active recall and spaced repetition are independent strategies. One of them (active recall) is a method for reviewing material; the other (effective spaced repetition) is a method for how to best time reviews. You can use one, the other, or both:

    Examples of their independence:

    • You could listen to a lecture on a topic once now, and again a year from now (not active recall, very inefficiently spaced repetition)
    • You could watch YouTube videos on a topic in efficiently spaced intervals (not active recall, yes spaced repetition)
    • You could quiz yourself with flashcards once, then never again (yes active recall, no spaced repetition)
    • You could do flashcards on something in efficiently spaced intervals (both spaced repetition and active recall).

    IV. Implications

    Why does this matter?

    Mostly, it doesn’t, and I’m just splitting hairs. But occasionally, it does matter — for instance, it's sometimes prohibitively difficult to use spaced repetition and active recall, still quite possible to use just one of the two. In these cases, folks sometimes throw up their hands. But the right response is to do use the method that works nicely!

    For example, you can do a bit of efficiently spaced repetition when learning people’s names, by saying their name aloud:

  2. 4 min
    Generic advice caveats

    You were (probably) linked here from some advice. Unfortunately, that advice has some caveats. See below:

    • There exist some people who should not do the advice.
    • Moreover, people are different.
    • More moreover, situations are different. What worked there/then might not work here/now.
    • Some of the advice is missing context, contradictory, inaccurate, misleading, won’t replicate, or is downright false or useless.
    • Consider reversing the advice.
    • The author’s incentives might be misaligned with your goals. The author might say something like “these are affiliate links” or “I’m investing in this company,” but sometimes they just get more clicks/upvotes/karma/likes/whatever from overconfidence or exaggeration. Or they might just get a kick out of giving advice.
    • You might not be the target audience for the advice.
    • The author of the advice might be explaining the advice poorly, even if it’s a good idea. (Corellary: the author might be explaining the advice really convincingly, but it still might not be good advice.)
    • Reading the advice selects for advice that you’re liable to read; hearing the advice selects for advice that you’re liable to hear; etc. You’ll read a lot more blog posts telling you to start a blog than ones telling you not to, because the people who have good takes on why you shouldn’t write a blog aren’t writing a blog posts about it.
    • The advice might not help at all. It might even make your problem worse. Even if the advice helps, it might not totally solve your problem, or it might create new problems you didn’t expect.
    • The advice is for entertainment value only, and not professional advice. The author is not a [lawyer, doctor, whatever], and if they are, they’re not acting in a professional capacity.
    • Following the advice might relevantly change your frame/outlook/goals/etc in life. This could, for instance, cause your success criteria to change such that it’s impossible for the advice to ever “succeed” by your current self’s version of “success” — only by past versions.
    • The advice might assume certain capabilities, knowledge, connections, or other resources that you don’t have.
    • The advice might have second, third, or nth order effects. Even the sign of these can be extremely difficult to predict, let alone their general shape, let alone specifics.
    • Stolen from dynomight's essay on advice, and bastardized to fit the context of this list:
      • The advice might be incomplete without relevant lived experience. Some things make no sense when you first read it, but make a lot of sense after having experienced something like it.
      • You might not understand the advice, even if you think you do. Even slight misunderstandings can cascade into completely different things.
      • You might feel like it won’t work, even if you intellectually think it will, which could affect your ability to actually do it, so that when you “try” it, you’re not actually trying it.
      • You might not be able to act on the advice for the same reason you need the advice in the first place. (See image below.)
    • Stolen from page 5 of the CFAR Handbook, lightly edited to make sense in this context:
      • Remain firmly grounded in your own experiences, in your own judgment, in what you care about, in your existing ways of doing things. As you come across new concepts, hold them up against your own experiences. If something seems like a bad idea, don’t do it. If you do try something out, pay attention to how useful it seems (according to however you already judge whether something seems useful) and whether anything about it seems “off” (according to however you already judge that). If you wind up getting something useful out of the advice, it is likely to come from you tinkering around with your existing ways of doing things (while taking some inspiration from the advice).
      • What happens when someone reads the advice? The guy who gave you the advice doesn’t know. Information from folks who read the advice doesn’t make it back to the advice-giver reliably. To take a guess, most of the time, not all that much happens. Reading about how to swing a tennis racket probably doesn’t have much effect on one’s tennis game.
      • You might expect that tennis analogy to lead into exhortations to actually try out the techniques and practice them, but (to reiterate) the advice-giver doesn’t really know what will happen if you actually try out the techniques and practice them guided only by the advice. If someone reads the words that’ve been written about the numbered steps to the advice, and forms an interpretation about what those words mean, and tries to do the thing with their mind that matches their interpretation of those words, and practices again and again… the advice-giver might be surprised to see what they actually wind up doing. Maybe it’ll be something useful, maybe not.
    • [From niplav’s Life Advice.]
      • There is value-laden and value-agnostic advice. Most advice is value-laden. Before you carry out some advice, check whether it actually corresponds to your values.
        • This also applies to all advice in this post (yes, even self-referentially)
  3. 8 min
    Rowing vs steering

    Alex Lawsen used a great metaphor on the 80k After Hours podcast:

    [1:38:14] …you’re rowing a boat on your own, and you’re trying to get somewhere, and you’ve got some map that you need to look at to see where you’re going, I imagine like a map and compass. […] When you’re rowing, you’re facing back; you can’t see where you’re going. You’ve just got to sit there and pull both of the oars, and do that a bunch of times, and then the boat goes forwards. […] You steer [… by pulling] harder with one side, something like that.

    I can imagine […] you sitting forwards in the boat, and trying to hold the map with your left hand while it’s gripping one oar, and trying to hold the compass with your right hand while it’s gripping the other; pushing them rather than pulling them while looking at where you’re going; so you’re always precisely on track, but my guess is you’re just going to go super slowly, because that’s not how to row a boat.

    Whereas you can imagine someone else, maybe someone that’s racing you, who is going to point the boat in pretty much the right direction — they’re not exactly sure it’s the right direction, and they might go a bit off course. And then they go, “Cool. I’m going to row hard for a minute, and then I’m going to stop and check I’m pointing in the right direction, and then I’m going to row hard for another minute.”

    [1:37:56] The metaphor is trying to point at … the strategy, [which] is pretty clear: gather some information, make a decision with that information, stick to that decision for some period of time that you’ve planned in advance, and then reevaluate, gather some more information, and then make a new decision.

    [1:35:58] … you [should] stick to some policy, which is like: “I’m going to look at a bunch of things, I’m going to actually seriously consider my options. And then, with all of the information I have, I’m going to make a decision. And I’m going to make the decision to do the thing that seems best for some fixed period of time. At the end of that fixed period of time, then I will consider other options.”

    [1:47:43] … if you think expected value is a reasonable framework to use, … then I do actually want to say: I think having this kind of policy is actually the thing that seems best in expectation.

    [1:41:21] … I think some people … they start doing a thing, and then they’re so worried about whether it’s the best, that they’re just miserable, and they never find out if it is the best thing for them because they’re not putting all of their effort in, because they’ve got one foot out of the door because they think something else could be better.

    When you’re in a rowboat, you don’t want to be constantly rowing (and never steering), nor constantly steering (and never rowing). But there’s also an in-between state that’s still a failure mode, where you’re trying to half-row and half-steer all at the same time. You’d be way better off by purely rowing for a bit, then purely steering for a bit, then back and forth again, but it causes anxiety to purely row without steering (“what if I’m rowing in the wrong direction!”), and it causes less forward progress to purely steer with no rowing (“I’m not even moving!”). So Alex’s solution is to set a policy that looks something like: “For the next minute, I’m going to row hard. After sixty seconds, I’ll turn around and steer. But for the next sixty seconds, I’m not even going to consider that I’m rowing in the wrong direction, because I’m in rowing mode, not steering mode.

    And importantly, having the knowledge that you’ll be correcting your course sixty seconds from now makes it so much less anxiety-inducing to purely row for sixty seconds straight.

    I’ve used this in situations where it’s costly to be thinking about how best to do something while you’re in the process of doing it. A career is a great example of a rowboat: it’s draining to be constantly searching for opportunities while you have a job. It takes up cycles in the back of your brain, and it prevents you from committing hard in a way that has important and nonlinear effects.

  4. 15 min
    quotes 2

    when i come across good quotes, i write them down. the below are those i collected between my last quotes post and the day i published this — november 19, 2023 through august 5, 2024.


    [T]he work that needs to be done is not a finite list of tasks, it is a neverending stream. Clothes are always getting worn down, food is always getting eaten, code is always in motion. The goal is not to finish all the work before you; for that is impossible. The goal is simply to move through the work. Instead of struggling to reach the end of the stream, simply focus on moving along it.

    — Nate Soares, Rest in Motion


    [You’re] looking for ... 1) someone that you will find fascinating to talk to after you’ve talked for 20,000 hours, 2) you feel comfortable with them talking through the hardest and most painful decisions you will face in your life, and 3) the conversation is wildly generative for both of you, in that it brings you out, helps you become.

    — Henrik Karlsson, Looking for Alice


    [S]how the inside of your head in public, so people can see if they would like to live in there.

    — Henrik Karlsson, Looking for Alice


    A mid cameraman finds the sheer magnitude of angles, scenes, props, and lighting decisions impossibly complex, and necessarily conveys camerawork through such a lens. The very best, on the other hand, finds it all very natural, and indeed is able to convey camerawork as such to intelligent inquirers.

    — Tanner Hoke, Really knowing things


    [M]aking Anki cards is an act of understanding in itself. That is, figuring out good questions to ask, and good answers, is part of what it means to understand a new subject well.

    — Michael Nielson, Augmenting Long-term Memory


    [M]assive leverage can be had when you're surrounded by people who are locked in to normal incentives, and you get to play by other rules.

    — Andrew Connor, Playing with incentives


    At some point, I realized there are certain games you can never win, and I resolved to stop playing those games.”

    — Cate Hall, h/t Things you learn dating Cate Hall


    If you’re a real philosopher, you don’t need privacy, because you’re a living embodiment of your theory at every moment, even in your sleep, even in your dreams.

    — Agnes Callard, tweet — h/t Agnes Callard's Marriage of the Minds


    All you have to do is find the action that best brings about the stuff you care about, and then do it.

    — Ronny Fernandez, Think on Purpose


    The great benefit that I experienced from thinking of integrity as a virtue, is that it encourages me to build accurate models of my own mind and motivations.

    — Oliver Habryka, Integrity and accountability are core parts of rationality


    [P]eople who care a lot about making the best decision often neglect the implicit costs of the decision making process such as time and money.

    — Duncan Sabien, CFAR Handbook, section entitled “Units of Exchange”


    [P]eople make the mistake of thinking that rationality is the process of muting those primitive, intuitive processes and just relying on System 2. It’s an understandable mistake—after all, those are the “higher brain” functions, the ones that allow us to do things animals can’t, like writing and philosophy and math and science. But turning off or ignoring large parts of your brain is rarely helpful, and applied rationality is about using every tool in your possession.”

    — Duncan Sabien, CFAR Handbook, section entitled “What is ‘Applied Rationality’?”


    I have come to believe that people's ability to come to correct opinions about important questions is in large part a result of whether their social and monetary incentives reward them when they have accurate models in a specific domain.”

    — Oliver Habryka, Integrity and accountability are core parts of rationality


    [H]umans instinctively execute good game theory because evolution selected for it, even if the human executing just feels a wordless pull to that kind of behavior.”

    — Ruby, Friendship is transactional, unconditional friendship is insurance


    [I]n the presence of anthropic effects, you can still reason and receive evidence about the latent variables and mechanistic factors which affect those anthropic effects.”

  5. 5 min
    Come to Manifest 2024 (June 7–9 in Berkeley)

    This is a cross-post of a piece I wrote for the Manifold Substack; here's a link to the original post. I'm helping to run this, and I would be delighted to see you there!


    TLDR

    Manifold is hosting a festival for prediction markets: Manifest 2024! We’ll have serious talks, attendee-run workshops, and fun side events over the weekend. Chat with special guests like Nate Silver, Scott Alexander, Robin Hanson, Dwarkesh Patel, Cate Hall, and more at this second in-person gathering of the forecasting & prediction market community!

    Tickets & more info: manifest.is

    WHEN: June 7-9, 2024, with LessOnline and Summer Camp starting May 31

    WHERE: Lighthaven, Berkeley, CA

    WHO: Hundreds of folks, interested in forecasting, rationality, EA, economics, journalism, tech and more. If you’re reading this, you’re invited!

    People

    Manifest is an event for the forecasting & prediction market community, and everyone else who’s interested. We’re aiming for about 500-700 attendees (you can check the markets here!).

    Current speakers & special guests include:

    Content

    Everything’s optional, and there’ll always be a bunch of sessions running concurrently. Like last year, we’ll host a mix of talks from experts, attendee-run workshops, and fun side events. We’re especially excited about attendee-run sessions — people loved hosting their own and attending their friends’. Topics range from:

    • forecasting in journalism & government
    • legalizing prediction markets
    • AI forecasting (& AIs making forecasts)
    • the case against forecasting/prediction markets
    • adjacent experimental stuff (impact certificates, quadratic voting, etc)
    • …and much, much more!

    Since Manifest is forecasted to be bigger this year, we’re putting extra effort into making it easier for the right people to connect. We’ll have interest-specific meetups for politics, journalism, AI, mechanism design, etc — and plenty of infrastructure for you to run your own sessions on topics you’re excited about.

    Last year’s content

    To give you a sense of what to expect, here were some talks from last year:

    • Prediction Markets in Journalism — Scott Alexander (ACX) and Dylan Matthews (Vox)
    • Fireside Chat — Nate Silver (538)
    • Biggest Questions in AI Forecasting — Matthew Barnett (Epoch AI)
    • The Future of Trust and Evidence in the Age of AI — Emmett Shear (Twitch, YC, OpenAI)
    • Improving AI Benchmarks — Isabel Juniewicz (Open Philanthropy)
    • Revolution Strategies — Robin Hanson (GMU, Mercatus Center)
    • Prediction Markets vs. Financial Markets — Byrne Hobart (The Diff)
    • Genetic Enhancement: Prediction Markets for Future People — Jonathan Anomaly

    And some side events from last year:

    • Poker tournament with ex-pros (and live betting, ofc)
    • Spicy live polling (and live betting, ofc)
    • Wrestling in the park (and live betting, ofc)
    • Magic: the Gathering tournament (and live… you get the idea)
    • Night market
    • Jazz Jam (BYO Instrument)
    • Speedfriending
    • Lightning talks
    • Descent into Dance Hell, run by Aella

    And a whole lot of smaller, attendee-run events too! We’re looking forward to hosting even more of them this summer.

    Testimonials Festimonials

    Manifest 2023 was pretty awesome. But don’t take our word for it — the median response on the feedback form was 10/10 (n=152). Here were some of our favorite festimonials:

    The vibes were immaculate. I was actively excited by easily 70% of the talks and side conversations. Fantastic.

    Kudos to the whole team. Definitely worth crossing an ocean and a continent to attend. Thanks for everything 💙

    The venue, the vibe, the weather, the speakers, the size, the weekend, the food, the fun, the laughs, the markets, the mayhem, the everything

    Variety of events, fun things as well as serious. High density of interesting people to talk to. Great food and site.

    You can look through more festimonials here, or read the New York Times’ piece here.

    Place

    Just like last year, Manifest will be held at Lighthaven. We credit much of the great vibes of Manifest 2023 to the space — it’s beautiful and cozy. It has literal fires for literal fireside chats, infinite nooks for one on one conversations, spaces for small workshops & big talks, and (gorgeous) overnight accommodations.

  6. 6 min
    Things You’re Allowed to Do: University Edition

    This post is not titled “Things You Should Do,” because these aren’t (necessarily) things you should do. Many people should not do many of the items on this list, and some of the items are exclusive, contradictory, or downright the reverse of what you should do. If your reaction to something is “I think that’s a bad idea,” then it probably is, and you probably shouldn’t do it.

    • classes & professors
      • attend classes you haven’t signed up for because you find them interesting
      • attend classes even if the waitlist is full
      • ask the professor to waive a prerequisite
      • ask the professor to join a class even if its full
      • drop a class that you don’t like
      • take a class because you really liked the professor, even if you’re not sure about the content of the class
      • cold email professors you don’t know, just asking to chat
      • show up to office hours for classes you aren’t a part of, just to chat with the professor
      • ask the professor questions about the things you’re not sure of
      • skip class(es) for great opportunities elsewhere
      • ask the professor if you can help them with anything in the class (grading, setting up assignments, editing papers, etc). professors have a long list of tasks, are perpetually behind, and encounter fairly correlated problems; if you track what problems your professors have, you can quite quickly become unreasonably useful for them
      • ask professors at the beginning of the semester what things would be most important to memorize, then throw their answers into an Anki deck
      • take non-credit courses or workshops in things like pottery, coding, or creative writing
    • studying
      • at places outside of your university:
        • coffeeshops
        • public libraries
        • coworking spaces
        • random offices, cold email them
      • start a study group for the class
        • ask the professor if you can announce that you’re starting a study group for the class in the class
        • start a group chat to ask questions about the class. this is one that everyone loves to be added to, and sometimes it just… doesn’t happen, because nobody took the initiative to create it
      • use Anki to study the things your professor said would be most important to memorize after you asked them at the beginning of the semester
      • learn the content of a class by using materials that the professor doesn’t point you toward (e.g. online textbooks/videos/tutors/etc)
      • hire a tutor
        • hire multiple tutors
        • hire a tutor purely so that you have to study for some class you hate — you might not need help, but if you're paying someone $x/h for their time, you'd better be studying
      • become a tutor in a subject you want to brush up on
      • use ChatGPT as a tutor
      • cowork
        • with random people
        • with me
    • clubs
      • join clubs
        • join many clubs
        • join many different types of clubs. shortlist: sports clubs (even intramural), art clubs, research clubs, project-based clubs, religious/cultural clubs, community service clubs, pre-professional clubs, music clubs
      • show up at a club’s meeting that you’re not a part of
      • stop going to a club's meetings
        • completely stop without telling anyone
        • tell the club leaders why you’re stopping, and what changes would make you stay
        • tell the club leaders you’re considering stopping, and what changes would make you leave or stay
      • ask if you can help out at the next club event
        • ask this multiple times in a row
        • ask what’s preventing them from letting you help out yet
      • start your own club. notably, schools will often throw hundreds or even thousands of dollars of funding at you to start a club with a few friends, and you can do a lot of cool things by saying “hey, I run [x] club, could you [ask]?” (h/t Joey)
    • career capital
      • evaluate not just “will this be good for my career” but “is this among the best options given the limited resources (time, money, energy, etc) that i have” — and also “is there something else i can do with these resources that’d give me more career capital” or alternatively “is doing this in line with following rules that i endorse upon reflection?”
      • actually utilize the alumni center — you can find alumni in ~any industry, and most major companies, and many are happy & eager to chat with you
      • find events oriented to the career you want to go into
        • attend them
        • volunteer for them
        • offer to run or help out at the next one
      • organize events for undergrads interested in your career — the bar for “casual meetup for pre-____ students!” is pretty low, and you can probably get some money from the relevant department for food & drink
    • money
      • apply to random grants and fellowship programs (1, 2)
      • get a job
        • get a weekend job
        • get a part time job
        • get a job that means you rub shoulders with the types of people you want to be rubbing shoulders with — e.g. working at a golf course, or at the registration desk of a google office
        • ask the people doing the job that you want to do if you can also do that job right now
      • get a paid internship
    • friends
      • stay in your room 24/7 and make no friends
      • make friends with the first people you meet, even if you don’t like them, and then never find new friends (h/t Joey)
      • call your old friends out of the blue, especially the ones from high school that you haven’t talked to for a while. imagine if they called you out of the blue, you’d love it. you can just… do that to them.
      • have 1-1s with friends
      • join a frat
      • don’t join a frat
      • offer a friend to swap dorms for a weekend
      • offer a friend who goes to a different school to swap dorms for a weekend
      • offer a friend who goes to a different school to have them stay at your dorm for one weekend, then you’ll stay at their dorm for another
    • misc
      • optimize for:
        • your degree
        • making friends
        • finding (a) partner(s)
        • career capital
      • take time off
        • seriously, you can just… take a semester off, or a year off, or more. this is much more common than you realize, since there’s a huge amount of selection bias: you never see the students who take time off, because they’re not going to be campus.
        • drop out entirely
      • decorate your dorm rationally
      • use the gym — there will be ~no other time in your life during which you’ll have free access to a great gym whenever you want
      • intentionally & rapidly try out tons of various life improvements — you have a fairly regimented environment that, by default, controls for a number of confounding variables
      • leave campus (h/t Joey)
      • stay on campus
      • create art — this is one of the few times in your life you have access to kilns, or high-quality paints, or a glass-blowing shop, etc (h/t Joey)
      • write a thesis under an advisor (h/t Joey)
      • get involved with school admin — not just student union, but you can, e.g., do informal, independent research and make recommendations about dining, sustainability, etc. also, there are sometimes grants within schools, like sustainability grants from the administration. (h/t Joey)
      • most universities have pretty great art available for free
      • travel to random places on a weekend, stay with a friend/relative
      • make a personal website
      • start a blog
  7. 4 min
    Explaining Impact Markets

    Let’s say you’re a billionaire. You want to have a flibbleflop, so you post a prize:

    Make a working flibbleflop — $1 billion.

    There begins a global effort to build working flibbleflops, and you see some teams of brilliant people starting to work on flibbleflop engineering. But it doesn’t take long for you to notice that the teams keep running into one specific problem: they need money to start (buy flobble juice, hire deeblers, etc), money they don’t have.

    So, the people who want to build the flibbleflop go and pitch to investors. They offer investors a chunk of their prize money if they end up winning, in exchange for cold hard cash right now to get started building. If the investors think that the team is likely to build a successful flibbleflop and win the billion dollar prize, they invest. If not, not.

    If you squint, you could replace "flibbleflop" with highly capable LLMs, quantum computers, or any number of cool and potentially lucrative technologies. But if you stop squinting, and instead add the adjective "altruistic" before "billionaire," you could replace “flibbleflop” with “malaria vaccine." Let's see what happens:

    Make a working malaria vaccine — $1 billion.

    There begins a global effort to build working malaria vaccines, and you see some teams of brilliant people starting to work on vaccine engineering. But it doesn’t take long for you to notice that the teams keep running into one specific problem: they need money to start (buy lab equipment, hire researchers, etc), money they don’t have.

    So, what should they do?

    Obviously, the people who want to build the vaccine should go and pitch to investors. They should offer investors a chunk of their prize money if they end up winning, in exchange for cold hard cash right now to get started building. If the investors think that the team is likely to build a successful malaria vaccine and win the billion dollar prize, they should invest. If not, not.

    The prize part of this is how a lot of philanthropy is done. An altruistic billionaire notices a problem and makes a prize for the solution. But the investing part of it is pretty unique, and doesn’t happen too often.

    Why is this whole setup good? Why would you want the investing thing on the side? Mostly, because it resolves the problem that some teams will be wonderfully capable but horribly underfunded. In exchange for a chunk of their (possible) future winnings, they get to be both wonderfully capable and wonderfully funded. This is how it already works for AI or quantum computing or any other potentially lucrative tech that has high barriers to entry; we can solve the same problem in the same way for the things that altruistic billionaires care about, too.

    But backing up a bit, why would an altruistic billionaire want to do this as a prize in the first place? Why not use grants, like how most philanthropy works?

    1. Prizes reward results, not promises. With a prize, you know for a fact that you’re getting what you paid for; when you hand out grants, you get a boatload of promises and sometimes results.
    2. The investors care a lot about not losing their money. They’re also very good at picking which teams are going to win — after all, investors only get rewarded for picking good teams if the teams end up winning.
    3. The issue of figuring out which people are best to work on a problem is totally different from the issue of figuring out which problems to solve. Using a prize system means that you, as a lazy-but-altruistic billionaire, don’t have to solve both issues — just the second one. Investors do the work of figuring out who the good teams are; you just need to figure out what problems they should solve.

    If you do this often enough — set up prizes for solutions to problems you care about, then let people build those solutions and get the prizes — you can start making prizes for more and more things, with investors who profit from picking the right teams to work on the right problems. You can also get more and more vague, expecting that teams (and investors) will figure out your preferences as they go. And in the extreme, you can just say “make stuff I value, and I’ll give you a prize to the extent that I value it.” When those things that you’re giving out prizes for are valuable to just you, people call it “capitalism.” When the things you’re giving out prizes for are valuable for the world, we call it an “impact market.”

  8. 1 min
    Map of the prediction market and forecasting community

    Hello!

    No big essay or anything. I built a map of the prediction market and forecasting ecosystem, inspired by similar maps of related fields. I want a good reference for people who want to get into the prediction market & forecasting community, so I made one.

    Take a peek: predictionmarketmap.com

  1. 2023 3 min
    Link Collection: Impact Markets

    0. Readme (or don't)

    • This is not a literature review. I'll vouch for links with an associated archived link, author name, and summary, but not the others.
    • Last updated: Dec 2023

    1. Overviews

    • Impact Markets: The Annoying Details (a), Scott Alexander
      • Comprehensive description of impact markets, reasoning from first principles. Extremely well-written, and a great introduction to the details of impact markets. Gets technical, but in an easy-to-follow way.
    • Toward Impact Markets (a), Dawn Drescher
      • Comprehensive description of the benefits, risks (& proposed solutions), and current work on impact markets. Fairly technical.
    • Impact certificates and Impact Markets - Owen Cotton-Barratt
    • Impact Markets: A Funding Mechanism for Speculative Work
    • A Fresh FAQ on Impact Markets
    • Impact Certificates on a Blockchain
    • Will "impact certificates" value only impact?
    • Hypercerts: A new primitive for public goods funding
    • Crypto loves impact markets: Notes from Schelling Point Bogotá
    • Impact Certificates | Evan Miyazono, Head of Research at Protocol Labs | Green Pill #21

    2. Subtopics

    • Altruistic equity allocation (a), Paul Christiano
      • Original proposal of allocating altruistic equity. Somewhat technical.
    • Certificates of impact (a), Paul Christiano
      • Original proposal of impact certificates. Somewhat out-of-date with current work. Somewhat technical.
    • Impact markets may incentivize predictably net-negative projects (a), Ofer and Owen Cotton-Barratt
      • Describes how impact markets can incentivize funding some types of projects that have clear negative expected impact. Argues that impact markets may exhibit the behavior of those types of projects, and therefore that impact markets should never be funded on impact markets.

    3. Implementations

    Last updated: December 2023

    • Manifund*, run by Rachel & Austin
      • all of Manifund's internal docs are publicly available
      • includes the ACX Forecasting Impact Mini-Grants round, the Open Philanthropy AI Worldviews Contest, and the leftovers of ACX Grants 2024
    • AI Safety GiveWiki, previously Impact Markets, [?possibly previously something else]
    • Hypercerts
    • Gitcoin (not an impact market, but they do retroactive quadratic public goods funding, which is pretty damn close)
    • NPX Advisors (also in the "close-to-an-impact-market-but-not-quite" category. NPX recently shut down.)
    • The Impact Purchase
    • Experiment in Retroactive Funding: An EA Forum Prize Contest
    • Plan for Impact Certificate MVP
    • Retroactive Public Goods Funding (a), Vitalik Buterin
    • The Retroactive Funding Landscape
    • The Retrofunder’s Dilemma
    • Experiment in Retroactive Funding
    • Chaining Retroactive Funders
    • Accelerating Academic Research with Impact Certificates

    5. Other resources

    Places that curate content on impact markets

    • EA Forum's "Impact Markets" Tag
    • ?others, I'd be particularly interested in readers' ideas

    People

    Note: the following people haven't (necessarily) consented to being contacted nor placed on this list; I've compiled this list myself.

    Note 2: if you want to get ahold of any of these people but for some reason can't, contact me.

    • Rachel & Austin
    • Scott Alexander
    • Paul Christiano
    • Dawn Drescher
    • Dony Christie
    • those who participated in the ACX Mini-Grants Forecasting Impact Market, including:
      • me, Tom, and Jingyi
      • Max Morawski
      • William Howard
      • everyone else listed here
    • if you think you ought to be listed here but you aren't (or if you are listed here but you don't want to be), please let me know.

    This post originally came from a comment I left on a bounty for charity money to help someone out with an undergrad paper.

    *COI: I’ve done some work for, might do some more work for, and own a tiny bit of equity in Manifold, the parent company (?) of Manifund. I'm writing this independent of any work I'm currently doing or planning to do for Manifold or for any other entity. I just think the ideas are cool.

  2. 5 min
    Solving Two-Sided Adverse Selection with Prediction Market Matchmaking

    0: Navigation

    I’m aiming for a reader who knows what “prediction markets” and “adverse selection” are, who likes the first and not the second, and who enjoys systems that have neatly aligned incentives. For a primer on prediction markets, read Scott Alexander’s FAQ. If you’re already familiar with Manifold Love, skip to section 2.

    COI: I’ve done some work for, might do some more work for, and own a tiny bit of equity in Manifold. I'm writing this independent of any work I'm currently doing or planning to do for Manifold or for any other entity. I just think the ideas are cool.

    1: Love

    Manifold, a play-money prediction market platform, recently released a prediction market dating app called “Manifold Love.” Users — those seeking love — sign up and fill out their profile like a regular dating app. Matchmakers — some of whom are users of the app themselves — pair users up. After a matchmaker makes a match, prediction markets are automatically created on various benchmarks of the pair’s (potential) relationship.

    People make bets (using play-money) on whether the pair will go on a first date; conditional on that first date happening, a second date; conditional on the second date happening, a third; conditional on the third date happening, a 6 month relationship. The idea is that you should go on dates with matches who have the highest chance of leading to subsequent dates, and eventually turning into a relationship. Instead of guessing who that'll be, you can just check the prediction markets.

    2: Generalizing

    Importantly, Manifold Love’s setup isn’t limited to solving problems in the dating market. It solves (or a least, has the potential to solve) problems in all scenarios in which there’s two-sided adverse selection, where each side of a two-sided market selects for something disfavorable by the other side’s lights. In the words of Groucho Marx, “I don’t want to belong to any [country] club that would accept me as a member.” If a club wants Groucho Marx, it’s probably not a very good club, and vice versa — if Groucho Marx wants to be in a particular club, he's probably not a very desirable member.

    More generally, the setup of "run a bunch of conditional prediction markets on a bunch of key benchmarks for potential pairs between two sides that are normally caught in adverse selection" seems like it could work pretty well.

    One of the classic cases of two-sided adverse selection is the labor market, so here's how a sort of “Manifold Jobs” might play out. The platform has three entities:

    1. Prospective employees
    2. Prospective employers
    3. Headhunters

    The platform makes a bunch of conditional prediction markets on key benchmarks for each of the first two entities. For example:

    • Conditional on being hired by [prospective employer], will [prospective employee]:
      • still be at their job in [timeframe]?
      • have a higher weekly average life satisfaction rating in [timeframe] than they do now?
      • etc.
    • Conditional on hiring [prospective employee], will [prospective employer]:
      • still be employing [employee] in [timeframe]?
      • have a higher weekly average employee rating in [timeframe] than the previous employee did?
      • etc.

    If Manifold actually makes this, I'm sure it'll look somewhat different, much better, and far more fleshed out. But importantly, the dating and labor markets aren't the only two landscapes of two-sided adverse selection.

    3: Insert Two-Sided Adverse Selection Here

    There are a bunch of other landscapes of two-sided adverse selection where the conditional prediction market setup from Manifold Love could potentially solve a lot of problems:

    • dating (Manifold Love)
    • friendships
    • gym partners
    • grantmaking
    • college applications/decisions
    • grad school apps/decisions (e.g. med school, law school, business school, etc)
    • labor/hiring/talent-seeking/jobs (as in section 2)
    • cofounders
    • seed- and preseed-stage investing
    • child adoption
    • internship applications/decisions
    • used car sales
    • tutoring
    • events/venue spaces
    • therapists/patients
    • selling & buying insurance
    • credit/lending
    • residential & commercial real estate
    • students picking classes at college
  3. 7 min
    quotes

    when i come across good quotes, i write them down. the below are those i collected between october 2022 and november 2023.

    It is much, much easier to pick out a way in which a system is sub-optimal, than it is to implement or run that system at anything like its current level of optimization.

    (zvi, Leaders of Men)

    Prediction markets are a truth generator, powered by the invisible hand.

    (oliver roeder, Financial Times)

    127. know what you want

    (alexey guzey, lifehacks (also see "questions to ask people" by me))

    friendships are not ledgers of obligations.

    (me, here)

    Definition: Slack. The absence of binding constraints on behavior.

    Poor is the person without Slack. Lack of Slack compounds and traps.

    Slack means margin for error. You can relax.

    Slack allows pursuing opportunities. You can explore. You can trade.

    Slack prevents desperation. You can avoid bad trades and wait for better spots. You can be efficient.

    Slack permits planning for the long term. You can invest.

    Slack enables doing things for your own amusement. You can play games. You can have fun.

    Slack enables doing the right thing. Stand by your friends. Reward the worthy. Punish the wicked. You can have a code.

    Slack presents things as they are without concern for how things look or what others think. You can be honest.

    You can do some of these things, and choose not to do others. Because you don’t have to.

    Only with slack can one be a righteous dude.

    Slack is life.

    (zvi, Slack)

    My teachers used to say that I could do great things if only I applied myself. I used to tell them that if they wanted me to apply more effort, they would need to invent higher letter grades.

    (nate soares, Half-Assing it with everything you’ve got)

    The power of reason lies in its ability to generate highly detailed maps. Where direct experience gives us rough sketches (“things fall”, “sun is bright”), reason gives us mathematical precision. Reason showed us the path to advanced medicine and computing. Reason has mapped entire worlds that were previously hidden behind tiny points of light painted on the night sky.

    But on reason’s map, there are also huge swaths of territory labeled: “here be dragons”.

    These are spaces where reason’s cartographic skills fail. Try falling in love rationally. Try reasoning with a conspiracy theorist, or with a jealous partner. Try navigating the dream world with reason, and watch Morpheus laugh. No amount of studying psychology, sociology, and neurology is going to tell you what it feels like to be an Evangelical Christian, or prepare you for your first psychedelic trip.

    In these spaces, intuition and empathy are better guides than logic.

    [cont.] Rationalism deals with these spaces by ignoring them (“dreams are just random noise!”) or by quixotically trying to conquer them (“our proprietary algorithm will help you find your soulmate!”). Both approaches reinforce the imagined superiority of rational thought, while providing very little value.

    Rationalism, no matter how powerful and successful, is limited in scope. It can never give us the full picture.

    (max goodbird, You Don't Always Have to be Rational)

    being good at something doesn’t feel like you’re good at it. it feels like everyone else is terrible, and you’re just not terrible.

    (paraphrased from Rob Miles, who himself forgot from where he got it)

    you sorta realize that, under the hood, it’s all just people.

    (paraphrased from austin chen, after he saw me interact behind the scenes with “famous people” at manifest.)

    Some people are obsessed with using brute force to warp their world into what they want it to be. Others lie dormant, waiting for their desires to materialize. I think somewhere in between the two lies the ideal strategy: you need to set things in motion, but momentum takes a life of its own.

    (nix, falling into life)

    a lot of life is about avoiding absolutely catastrophic risks.

    (david rapperport. this feels like kelly betting with life?)

    Tyler Cowen: Uncertainty should not paralyze you. Try to do your best, pursue maximum expected value, and just avoid the moral nervousness. Be a little Straussian out about it. Like, here's a rule, on average it’s a good rule, we're all gonna follow it. Bravo, go to the next thing. Be a builder.

    Joe Walker [interviewer]: Get on with it?

    TC: Yes. Because ultimately, the kind of nervous Nellies, they're not philosophically sophisticated, they’re overindulging in their own neuroticism when you get right down to it. So it's not like there's some kind of brute ‘let’s be a builder’ view and then in contrast there's some deeper wisdom that the real philosophers pursue. I think it’s: you be a builder or you’re a nervous Nelly. Take your pick. I say be a builder.

    (tyler cowen, Joe Walker Podcast #104, 12:06-12:58)

  4. 1 min
    questions to ask people
    • of all the things that you do, what do you think you do best? what do you think you do worst?
      • of all the things that i do, what am i worst at?
    • where do you go to find people who think well?
    • what do you want?
      • what do you want?
    • what are you most wrong about?
    • what are some of the most important problems in your field? are you working on them? why/why not? (the hamming question)
    • if you were put into a room with everyone you’ve ever known, who would you seek out first? (h/t blake)
    • what something weird or unusual you did early on in life?
    • given what we've talked about,
      • what questions should i be asking you?
      • what questions would you ask yourself, if you were in my position?
      • who else should i talk to? could you make an intro?
    • what have i done worst in this conversation? what's been most rude, or annoying, or unkind, or idiotic, etc etc?
    • untested ones below — i can't personally vouch that they work well. half are stolen from tyler cowen's "talent"
      • what blogs or sub Reddit do you read?
      • what tabs do you have open on your browser right now?
      • if we became friends, and then 3 to 6 months later, we stopped being friends, why would that be?
      • pet projects you’re working on?
      • what is something esoteric that you do?
      • the world is a market, so what is your edge?
      • what part of your life are you most unhappy about?
      • what thoughts do you flinch from?
      • of the various nonconsensus opinions you probably have, what's the most important one?
  5. 1 min
    Memorize Translations Between Odds and Percentages

    Edit 4/1/2024: I no longer use this deck. Since publishing this, I think I've gotten better at learning how to use Anki effectively — I endorse some better version of this deck, whereby you memorize translations between odds and percentages, but I don't endorse this deck.

    Quickly translating between odds and percentages can be pretty helpful for intuitive forecasting. I made a flashcard deck to improve my translation skills; you can download the CSV file below, then import it to Anki, Quizlet, or whatever you use for flashcards.

    common_odds_probscommon odds & probabilities, to be downloaded then used in a flashcard deckcommon_odds_probs.csv · 498 Bytes
  6. 2 min
    A Better Multiple-Choice Test

    Multiple-choice test scores are meant to reflect a student’s knowledge. Students estimate the right answer for each question; students who understand the content estimate the right answer more often than students who don’t. But multiple-choice tests force students to be 100% confident in one answer and 0% confident in the rest, when (a) it’s impossible to be 100% or 0% confident in anything, and (b) it doesn’t map to how people think — nobody takes a test and has perfect confidence in all of their answers. There’s always some uncertainty, and often quite a bit.

    Example: you’re split between A and B, leaning toward A. You’re pretty sure it’s not C or D. Something like this:

    A — 50% confident (1:1 odds) B — 40% confident (2:3 odds) C — 5% confident (1:19 odds) D — 5% confident (1:19 odds)

    The conventional advice is to just guess A. You'll get half-credit in expectation, but you'll never actually get half credit — you'll only ever get full credit or no credit. And we don't test students nearly enough on the same types of problems for the law of large numbers to kick in, and for the expected values to converge onto reality. There's a lot of variance that doesn't get accounted for.

    Instead, we want a system for students to answer questions (and for teachers to score those answers) that would, for the above example, have the following properties: the student is rewarded with some credit if the answer is A, a little less credit if the answer is B, and a tiny amount of credit if the answer is C or D.

    The system which matches those properties is:

    1. Have students give their confidence in each answer choice in percentages (1%,100%), where the sum of their percentages for all of the answer choices is 100%. (I.e. explicitly state the confidences from the example.)
    2. Use a proper scoring rule to award students credit.

    I set up a playground on Google Sheets that you can experiment with — check it out here. It uses two common types of proper scoring (log & Brier) to evaluate an example multiple-choice test question; this can be straightforwardly extrapolated to a full multiple-choice test by summing the scores for each question.

    I think stats teachers should strongly consider using this in their classroom. It better reflects students’ knowledge, and it also integrates stats concepts into everyday teaching. The main downside is that it’s a lot more confusing for students than just giving one answer — that’s why I’d expect it to work best in stats-heavy classrooms, where students are already learning a lot of the requisite concepts.

    Edit: looks like this (sorta) exists! See Bayes-Up.

  7. 3 min
    methods for producing solutions
    1. ask chatgpt

    2. google it, see if someone else has already solved the problem

      1. if it’s something a research paper might have solved, try asking elicit
    3. set a 5-minute timer, and think about the problem for 5 minutes. don’t think for 1-2 mins then go do the dishes, actually set a timer and actually do nothing else but think for a full 5 minutes.

    4. do the obvious things

      1. “I continue to wonder how powerful a person could become, if they simply managed to do all the obvious things in pursuit of their goals.”
    5. ask the duck

    6. ask people

      1. ask a friend/parent/sibling/etc
      2. don’t have any available? close your eyes, imagine they’re in front of you, ask them, imagine what they would say. seriously!
      3. don’t have any at all? close your eyes, imagine a wise/smart/strategic/good fictional character, imagine they’re in front of you, ask them, imagine what they would say. seriously!
      4. ask multiple friends/parents/siblings/etc
      5. don’t know any fictional characters? ask yourself from 1y ago, 5y ago, etc. ask yourself, out loud.
      6. ask friends/parents/siblings to go through other items on this list, how they would do them
      7. find whomever’s most publicly knowledgeable about the problem, and send them a cold email (unless you already know them).
      8. ask me
    7. post a bounty on manifold and see if someone else can solve your problem

    8. write the problem down, completely.

    9. seriously consider ignoring the problem entirely. how bad would it be to just let the problem be? could you tank them? could you hedge them?

    10. wait until you’re in a totally different mood, then tackle the problem from the exact same angle

    11. it’s 5 minutes/hours/weeks/years/etc from now, and you solved the problem. what happened?

    12. wait

      1. set a 5 minute timer, don’t think about it, then check on it afterwards
      2. set yourself a 24h timer, and don’t think about it for a day. when the timer goes off, see if you’ve come up with any solutions
      3. work out, and check on it afterwards
      4. go to sleep, and check on it afterwards. edit: from Bertrand Russet (pseudonym): "this doesn't [even] have to be overnight. By the end of my undergrad math degree, I was doing problem sets by just taking naps and writing down the answers on waking. Not for all types of problems! But good when it works."
    13. ask reddit

    14. ask twitter

    15. try diagramming out the problem, even if it’s not a visual problem. works best with an actual pen & paper.

    16. stimulants (+ obligatory "use with caution")

      1. caffeine
      2. adderall (can’t personally confirm, friends have though)
      3. others (?)
    17. interface with reality — try some random bullshit and focus on having the results help you generate better solutions.

    18. backchain:

      1. what do you want? what is the desired outcome? what’s the goal, the objective?
      2. work backward from that goal to where you are now. don’t start where you are and work forward.
    19. untested ones — i can’t personally vouch for them, but they seem pretty good/others have said they’re good. if you try them and they work, let me know.

      1. analogize to other fields/communities/subcultures/people/etc
        1. look at the overall shape of your problem. define it in as general a sense as possible, with as little context as necessary. someone who reads this definition shouldn’t be able to tell what field you’re in.
        2. has another field/community/subculture/person/etc already solved that shape of problem?
          1. if so: could you apply parts of that solution to your problem?
          2. if not: SWOP (seems wrong on priors). most problems have already been solved, just in different fields from what you think.
      2. ask yourself “what happen if i had zero constraints?” then, figure out how to make those solutions possible, given your constraints.
    20. go through this list again and try all the things you said “hahaha that won’t work for MY problem.” odds are, you’re wrong.

    edit: added these ones later.

    • get someone on fiverr to do it for you
    • babble out all of the questions you have, especially the cruxy ones. after you finish writing the questions, start answering them.
    • imagine the current problem but you cannot exit the situation (h/t miraculous cake)
    • what would a better, more capable, kinder, smarter, more virtuous version of yourself do? (h/t miraculous cake)