2024-12-27

Read for Speed#

I spend a large portion of my life reading - for pleasure, pain, and pursuit of knowledge. Reading faster would give me more life to live. Unfortunately that's not particularly easy. At around 360 WPM (words per minute), I'm already pretty fast for a 'normal' reader. To improve, I need to properly pick up speed-reading.

Skip the post and try the thing#

Word Count: 0 | Estimated Reading Time: 00:00

Ready

Speed: 800 WPM

The How and The What#

Two key elements of speedreading are:

Structured scanning
Removing subvocalisation

Structured Scanning#

All text has structure. Headers, multiple different callouts, graphs, lists, sections - there's a gazillion ways of organising and segmenting information. Textbooks are extremely explicit, but even with raw text you'll have introductions, conclusions, topic sentences, oxford lists.[1]

Textbook Page

I always read the main column text of my textbooks top to bottom, then went through every callout in order. Apparently this is not the best way to do it. Taking advantage of the structure will speed things up, prepping your brain for the content and knowing where to skim and where to stop, how to order things to best understand what's going on.

Subvocalization#

Reading is generally done with subvocalization, or 'speaking aloud' the text to yourself. This is apparently not actually a requirement. Instead, just rawdog text in the OG form, downloading the visual data straight to your cortex, zero parsing required.

Subvocalization puts a hard limit on reading speed; you can never go faster than you can internally narrate. Generally, the max speed for subvocalization seems to hover around 400 wpm for the fastest readers, with the average being around 250.

Getting rid of subvocalization gets rid of this somewhat-debateably-artifical limit. But how?

Force the Fast#

A recommended tool to remove subvocalisation is to remove the alternative, by flashing words by at a higher speed and seeing how you do. Try it here with my Context Poisoning conclusion and see how high you can push the WPM while still being able to read:

I call this Context Poisoning: where LLMs get trapped into continuing the train of thought that already exists in the context.

This is also why it can be difficult to get LLMs to correct their own mistakes; they find it hard to break from the existing conversation thread. There's many examples where it becomes impossible to get LLMs to debug their own code, because the only solution is a rearchitecture. Claude, ChatGPT, Qwen - even the best models get too caught up in the existing context to continue.

Leading questions are a good example of context poisoning, and can honestly be difficult to break from. It takes thought to engineer your prompt in such a way that the LLM actually thinks for itself.

Putting that effort in, however, is genuinely useful. Once I got Claude to think about the flow freely, it pointed out a genuine issue with the code that I otherwise hadn't thought of.

Context Poisoning is the negative expression of a LLM trait that's usually incredibly useful. LLM's attachment and obedience to context is the exact same reason that N-shot prompting is such a successful prompting strategy. While that can be used to great effect, it means we need to be very aware of everything that is being put into the context. Start a new chat when the LLM starts walking up the red tree, and take care with the context you inject as the user.

That line between poison and prompt can be thin. Just this example shows that; initially I poisoned the context with a nonexistent error, but then later followed it up with a useful prompt about whether the session tokens would be unique or not. In one case I was wrong, and in the other I was right. That was the main difference.

Proper coding experience makes such a difference here. After many years of software, I'm reasonably good at making sure the suggestions and ideas I provide are legitimate. I can also tell when the conversation goes off the rails, and rewind the conversation to a place before the context got poisoned. This is a massive reason why development experience is still an edge.

Without that experience, you're left to trust the LLM. For all their hallucinations, they are pretty trustworthy - provided you give them the space to think. Avoid leading questions. Give them freedom. Ask the same questions in several ways, in new chats.

It's a nice bot. Don't poison it.

Word Count: 411 | Estimated Reading Time: 00:36

Ready

Speed: 800 WPM

Practice makes perfect, etcetera, ad nauseum. Pushing yourself to read without any chance of subvocalization will remove this hindrance in everyday reading as well.

Force Fed#

While this kind of tool is very useful for the subvocalization aspect, it also completely prevents you from scanning the structure. This is well enough for my blog text that's intended to be read start to finish, but becomes troubling when the material does come with structure. Using this with an excerpt from Artificial Intelligence, Scientific Discovery, and Product Innovation by Aidan Toner-Rodgers quickly illustrates this:

6 Scientist-AI Collaboration

To summarize, I have established three facts. First, AI substantially increases the average rate of materials discovery. Second, it disproportionately benefits researchers with high initial productivity. Third, this heterogeneity is driven almost entirely by differences in judgment. To understand the mechanisms behind these results, I investigate the dynamics of human-AI collaboration in science.

I begin by documenting a dramatic change in the content of scientists’ work (Section 6.1). AI automates a majority of idea generation tasks, reallocating researchers to the new task of evaluating model-produced candidate materials. Next, I develop a simple prioritized search framework to analyze the implications of this shift (Section 6.2). Taking the model to the data, I find that scientists with strong judgment learn to prioritize promising AI suggestions, while others waste significant resources testing false positives. The resulting gap in discovery rates explains the tool’s heterogeneous impact. Turning to the survey, I present evidence suggesting that differences in judgment are driven by domain knowledge. Finally, I show that the lab responds to the changing research process by adjusting its employment practices (Section 6.3). After the conclusion of the experiment, it redesigned its hiring and firing criteria to favor scientists with strong judgment. My estimates may therefore understate AI’s longer-run impact due to organizational adaptation.

6.1 Tasks, Automation, and Reallocation

Figure 8, Panel A plots the share of scientists’ research hours dedicated to idea generation, judgment, and experimentation tasks, revealing a substantial reorganization of the discovery process. These measures come from logs of scientist activities, employing the text classification procedure described in Section 3. Without AI, scientists allocate 39% of their time to idea generation tasks. This drops to less than 16% after the model’s introduction. In contrast, while judgment tasks initially take up 23% of research hours, they consume 40% by the end of the sample. Finally, the fraction of time dedicated to experimentation tasks increases from 37 to 44 percent. The total number of research hours remains unchanged. Appendix Figure A4 reports event study estimates for these task shares, showing an identical pattern.

Panel B reports the change in task composition for scientists in the top and bottom quartiles of comparative advantage in judgment, γˆ j s /γˆ g s . While all researchers significantly adjust their time allocation, those especially skilled in evaluation display a 46% larger shift from idea generation to judgment tasks. Additionally, these scientists see a smaller increase in time spent testing materials. In Appendix D, I show that these patterns are consistent with a task framework where scientists are assigned to roles based on their comparative advantage and AI automates a fraction of idea generation tasks (Acemoglu and Autor, 2011; Acemoglu and Restrepo, 2022).

I corroborate these results using the survey of scientists. Consistent with the task allocation measures, scientists report initially allocating 36% of their time to idea generation, 26% to judgment, and 42% to experimentation. After the introduction of AI, these shares become 13%, 35%, and 47%, respectively. Matching survey responses to the empirical estimates, I confirm that the shift from idea generation to judgment tasks is increasing in ˆγ j s / ˆγ g s .

These results provide granular evidence on automation and reallocation in science. The tool automates a majority of material design tasks, while scientists are reallocated to the new task of evaluating model-generated compounds. AI is thus labor-replacing in the specific activity of creating candidate materials, but labor-augmenting in the broader discovery process due to its complementarity with judgment tasks. Indeed, scientists’ total research hours are unchanged after the introduction of AI, revealing that demand for labor remains strong. While the equilibrium implications for employment and earnings are beyond the scope of this paper, my findings highlight the importance of considering task reallocation when assessing the implications of AI for workers.

6.2 The Central Role of Human Judgment

Motivated by these changes to the research process, this section develops a simple framework to study the effect of partially-automated compound design on the rate of materials discovery. I begin by outlining the model and defining the key objects that I measure empirically. Taking the model to the data, I show that scientists’ differential ability to evaluate AI-generated compounds explains the tool’s heterogeneous impact. Finally, I present survey evidence suggesting that these differences in judgment are driven by domain knowledge.

6.2.1 A Model of Materials Discovery as Prioritized Search

Following Agrawal et al. (2023), I conceptualize materials discovery as a prioritized sequential search problem. The research process requires three steps:

1. Idea Generation: Generate a set of candidate materials M = {m1 , . . . , mM}. Each candidate has binary quality θm ∈ {0, 1}, which depends on research inputs but is unobserved by the scientist. Let θ¯ = 1 / M ∑ M m=1 θm denote the share of high-quality candidates.

2. Judgment: Evaluate candidate materials and rank them in descending order of predicted quality. Predictions improve with the scientists’ judgment, bringing the ranking r : M → {1, ..., M} closer to the true ordering.

3. Experimentation: Conduct costly tests to reveal candidate materials’ true quality, where each test requires cost c regardless of its result.

Let θ˜ k represent the probability that a material in position k of the ranking is high-quality. θ˜ k depends on two factors — the overall share of viable materials and the accuracy of the ranking. Shown in Panel A of Figure 9, the sum of θ˜ k across all materials is θ¯M, but its shape reflects prioritization. With no prioritization, tests are conducted at random, resulting in the flat curve θ˜ k = θ¯. Perfect judgment, in contrast, produces a step function, with all viable materials tested first. Intermediate judgment yields a curve between these extremes.

Given θ˜ k , optimal search takes a simple form: test candidate materials in descending order of predicted quality, until the cost of an additional test exceeds the expected benefit (Weitzman, 1979; Agrawal et al., 2023). Therefore, scientists test every material such that θ˜ k ≥ c. Let k ∗ denote the ranking of the last material above this cutoff. The total number of expected discoveries is then:

E (Discoveries) = k ∗ ∑ k=1 θ˜ k

I call θ˜ k the “discovery curve,” as it fully pins down the expected rate of materials discovery.

In this framework, AI could impact materials discovery through two channels. First, the partial automation of compound design may change the average quality of candidates, affecting the area under the discovery curve. Second, AI-generated candidates may prove differentially challenging to evaluate, shifting the curve’s slope. Figure 9, Panel B illustrates one potential scenario where AI increases θ¯ but makes candidates more difficult to prioritize. As a result, the optimal number of tests rises from k ∗ to k ∗∗. The effect on the expected rate of materials discovery — defined as the ratio of discoveries to tests — is ambiguous, reflecting the competing forces of higher average quality and worse prioritization.

6.2.2 The Empirical Discovery Curve

Leveraging the granularity of my data, I construct an empirical analog of the discovery curve — the share of high-quality materials at each position in the sequence of tested candidates. The empirical discovery curve, denoted θˆ k , is defined as:

θˆ k = 1 / J J ∑ j =1 θk( j )

where θk( j ) ∈ {0, 1} represents the realized quality of a candidate at position k in the sequence of tested compounds for research cycle j . Each research cycle represents a complete process of idea generation, judgment, and experimentation, and J denotes the total number of cycles. To aggregate across materials with different baseline discovery rates, I normalize θˆ k by the average viability of each material category. I then group k into bins, adjusting the bin widths to represent a fixed share of total tested candidates within each category.

Figure 10, Panel A plots θˆ k before and after the introduction of AI, revealing a shift along two dimensions. First, the tool boosts the overall share of viable compounds, θ¯, captured by the increased area under the curve. Second, AI-generated compounds prove more challenging to evaluate, evidenced by the shallower slope in the treatment group. As I show in the next section, this reflects the difficulty of evaluating compounds scientists did not themselves create. The combination of these effects results in a higher average discovery rate and a greater number of materials tested overall. Additionally, the nearly identical values of θˆ k for the last tested candidate in each curve lend support to the prioritized search framework, as the theory predicts that search should stop precisely when the cost of an additional test exceeds the expected benefit.

Panels B and C compare pre- and post-treatment discovery curves for researchers in the top and bottom quartiles of judgment, revealing a widening gap in their ability to evaluate candidates. While top-quartile scientists experience only a modest decline in prioritization, those in the bottom quartile see a marked decrease. Strikingly, the post-treatment curve for bottom-quartile researchers is essentially flat — their ranking of AI-generated candidates is no better than random chance. Consequently, scientists with strong judgment test significantly fewer candidates but discover more viable compounds. Regressing observed discovery rates in the post-treatment period on the share of tests yielding a high-quality material, I find that differences in prioritization explain over three-quarters of the variation in research productivity. These results explain the central role of judgment in shaping the heterogeneous impact of AI.

Next, I demonstrate that differences in scientists’ abilities to evaluate AI suggestions increase over time. Figure 11 presents post-treatment discovery curves relative to time since adoption. For scientists in the bottom quartile of judgment, θˆ k remains flat across all periods, indicating no improvement in prioritization. For those in the top quartile, the curve is also flat during the first five months after adoption. However, it becomes increasingly steep over the post-treatment period, reflecting rapid improvement. This shows that top evaluators learn to apply their expertise to the new problem of assessing AI-generated compounds, while others fail to do so.

The prioritized search framework also rationalizes the results on task composition presented in Section 6.1. First, the combination of partially-automated compound design and more challenging evaluation shifts research effort from idea generation to judgment tasks. Second, the rise in the number of tested candidates explains the increase in time spent on experimentation tasks. Finally, superior prioritization causes scientists with strong judgment to see a smaller change in effort allocated to experimentation.

6.2.3 Domain Knowledge and Judgment

Why are some scientists so much better at judging AI-suggested compounds? To investigate this question, I turn to the survey of the lab’s researchers. I ask scientists three questions about their evaluation process:

Q1: Compared to previous methods, how would you rate the difficulty of evaluating AI-generated candidate materials (much easier, somewhat easier, about the same, somewhat harder, much harder)?

Q2: How frequently can you identify AI-suggested candidate compounds as false positives without synthesizing the material (never/rarely/sometimes/often)?

Q3: On a scale of 1–10, how useful are each of the following in evaluating AI-suggested candidate materials (scientific training, experience with similar materials, intuition or gut feeling, and experience with similar tools)?

Consistent with my empirical results, 88% of scientists find AI-generated compounds more challenging to evaluate. The reason is simple: designing a material provides information about its quality. However, the difficulty is not uniform across researchers. Compared to the bottom quartile, those in the top quartile of judgment are nearly twice as likely to report frequently ruling out false positives without testing (62% vs. 34%, p < 0.00).

This disparity suggests that top scientists possess some form of knowledge that allows them to effectively assess AI suggestions. Figure 12 investigates four potential sources of their expertise. Panels A and B show that researchers in the top half of judgment place greater value on their scientific training and experience with similar materials when evaluating model-generated candidates. The utility of “intuition or gut feeling” — a proxy for tacit knowledge — is also positively correlated with judgment (Panel C). In contrast, familiarity with similar AI technologies does not explain the observed differences, as all scientists report minimal prior exposure (Panel D). This is consistent with the fact that the prioritization gap emerges over time. Supporting the importance of domain knowledge, I find that scientists in the top quartile of judgment are 3.4 times more likely to have published an academic article on their material of focus.

I rule out three alternative explanations for differences in prioritization, shown in Appendix Table A10. First, they are not due to confusion about how to use the tool. Second, they are not explained by scientists’ differential trust in the model’s suggestions. Finally, they do not reflect variation in material of focus, as the relationships in Figure 12 hold within compound type and intended application.

These results demonstrate the value of domain knowledge for assessing AI suggestions. From a machine learning perspective, this shows that top scientists observe certain features of the materials design problem not captured by the model. Consequently, integrating human feedback into algorithmic predictions may be a promising approach to scientific discovery (Hassabis, 2022; Alur et al., 2023, 2024). From an economic perspective, these findings highlight the complementarity between algorithms and expertise in the innovative process. In particular, they emphasize the growing importance of a new research skill — judging model suggestions — that augments AI technologies.

Some have speculated that big data and machine learning will render domain knowledge obsolete (Anderson, 2008; Klein et al., 2017; Gennatas et al., 2020). In the context of materials science, this appears not to be the case. Instead, only researchers with sufficient expertise can harness the power of the technology. The fact that the tool is ineffective without skilled scientists suggests that demand for human researchers may remain strong, even as AI transforms scientific discovery. This finding aligns with recent firm-level evidence showing that pharmaceutical companies with greater domain knowledge benefit more from data-driven drug discovery (Tranchero, 2023).

6.3 Organizational Adaptation and Longer-Run Effects

AI changes the returns to specific skills, increasing the value of scientists’ judgment while diminishing the importance of idea generation. Therefore, adjusting employment practices to prioritize researchers with strong judgment implies significant productivity gains.

In the final month of my sample — excluded from the primary analysis — the firm restructured its research teams. The lab fired 3% of its researchers. At the same time, it more than offset these departures through increased hiring, expanding its workforce on net. While I do not observe the abilities of the new hires, those dismissed were significantly more likely to have weak judgment. Figure 13 shows the percent fired or reassigned by quartile of γˆ j . Scientists in the top three quartiles faced less than a 2% chance of being let go, while those in the bottom quartile had nearly a 10% chance.

The lab’s response exemplifies the LeChatelier principle: it reacted more strongly to the tool over time because it could re-optimize a broader set of inputs (Samuelson, 1947; Milgrom and Roberts, 1996). The resulting change in the composition of researchers suggests that my estimates may understate the technology’s longer-run impact.

Word Count: 2535 | Estimated Reading Time: 03:36

Ready

Speed: 800 WPM

Here the section headers are as indistinct as any other text, bulletpoints become useless, and citations are given with the same focus as constructive text.[2] Switch to the Fiction text, and you'll see another side of it - force feeding speed reading can make dialogue frustrating, without pinned quotation marks to delineate speech.

Comprehending the Controversy#

The primary argument against speed reading is comprehension; the faster people read, the less they understand. The line between speed and skimming is very, very thin. This is why I use longer[3] examples; without a pause to consolidate, the information has no time to settle in your brain. The counter is that you'll have more time to re-read, and know which interesting parts to focus your attention on.

Personally I also dislike speedreading for pleasure. Much of my joy of reading comes from beautiful turns of phrase and particularly clever sentence structure, which come across much better when subvocalized. The best stories are poetry, which should be read aloud for proper delight in the way words and expectations are played with. Speed reading novels destroys the entire point of reading them at all, personally. Still, speed reading is useful for the large amount of text I need to injest for the content without care for the construction.

In your own words#

You can use this tool for anything you like:

Word Count: 0 | Estimated Reading Time: 00:00

Ready

Speed: 800 WPM

👁️‍🗨️ Tip

This has some additional slider past the normal wpm controls. You can give yourself a longer break at the end of a paragraph (recommended as otherwise you have no time to blink). I've also givin a minimum characters per word-flash cutoff, so instead of seeing 'a' 'word', you'll see 'a word' as a single flash. Personally I found this a lot easier. Finally I added a slider that will give you extra time on particularly long words. This is entirely a reaction to my attempt as speedreading scientific papers.

1: Novel structure is far more implicit, but the vibes definitely exist. Any prolific fiction reader can probably tell when the next five pages are going to be a shopping expedition, an action fight (and who the victor will be), or a detailed disquisition of Van Eck phreaking . ↩

2: That's not even getting into the mess of the math. ↩

3: At least longer than the usual speed reading challenges you might see on tiktok ↩