Robots, Pencils, and Keyboards

ChatGPT’s deficiencies as a writer have been one of its most-derided qualities, a marker of verbiage critics call “AI slop.” Astra has improved its writing significantly.

In this exercise, I take a typical quantitative academic paper on machine-learning investment strategies, use ChatGPT 5.6 Sol to summarize it, and then rewrite it myself, explaining my changes and the principles behind them. The judgments I used for organization, logic, rhetoric, language, phrasing, metaphors, humor, and artistry could all be taught to the model, or could be presented as stylistic options through a user interface. Writing isn’t rocket science, which in a way makes it harder to teach. Nontheless, I’ve learned from the best writing instructors and taught writing for decades; LLM writing could be so much better.

I instructed ChatGPT to find academic papers about machine learning in investing with text that could be improved upon. It returned four choices but had to be steered back on course before finding an appropriate one, below.

Ronen Israel, Bryan Kelly, and Tobias Moskowitz, "Can Machines 'Learn' Finance?," Journal of Investment Management 18, no. 2 (2020): 23–36.  (Excerpt pp. 24-25). 

So why have we moved to using new terminology to describe old ideas? Above and beyond the marketing angle—machine learning is a sexy name that carries the connotation of bleeding edge Silicon Valley technology—there are at least three substantive reasons for this shift. First, the historical practical usage of statistics was frequently confined to “small” models—those with a handful of input predictor variables (or “features” in ML terminology) and simple, often linear, association rules between those inputs and the output (i.e., dependent variable) of interest. The term “machine learning” has come to serve as a shorthand to signal an explicit interest in “large” models, those with many input variables and/or those allowing for complex nonlinear associations between the inputs and output. 

This idea is captured by part (i) of the GXK definition above. In order to learn through experience, the machine needs a representation of what it is trying to learn, which requires a research choice. Machine learning brings an open-mindedness for statistical representations that are richly parameterized and often nonlinear. Such models are of course not new to statistics, so it would be misleading to describe this as a contrast with “traditional” statistics. But it is fair to say that machine learning specializes in this sophisticated end of the model spectrum. Small models are rigid and oversimplified, but have the virtue that they can be used with small data sets. They are also “robust” in the sense that their behavior can be relatively insensitive to reasonable changes in the data. Large and sophisticated models are much more flexible, but can also suffer from poor outof-sample performance when they overfit noise in the system. Researchers turn to models like these when they believe the benefits from more accurately describing the complexities of real-world phenomena outweigh the costs of potential overfit. Part (i) of this definition also points out that the primary objective of machine learning is to generate accurate predictions. As emphasized by Breiman et al. (2001), its focus on maximizing prediction accuracy in the face of an unknown data model is the central differentiating feature of machine learning from the traditional statistical objective of estimating a known data-generating model and conducting hypothesis tests.

Second, machine learning seeks to choose a preferred model from a “diverse collection” of candidate models. Again, this idea has a long history in statistics under the heading of “model selection” and therefore is not a new contribution of machine learning. But the process of searching through many models to find the best performer is characteristic of essentially all machine learning methods—it is closely connected with what machine learners call model “tuning.” Of course, looking at multiple models and selecting the top performers in-sample mechanically leads to overfit and poor out-of-sample performance. Because of this, the model search process is always accompanied by so-called “regularization” techniques and methods for identifying models that are likely to perform best out-of-sample. Regularization is a blanket term for constraining the size of a model. An optimal model is a “Goldilocks” model. It is large enough so that it can reliably identify the true and potentially complex predictive relationships in the data, but not so flexible that it overfits and suffers out-of-sample. Regularization methods encourage smaller models, and make sure that a richer model only gets selected if it is likely to give a genuine boost to out-of-sample prediction accuracy. A cornerstone method in the model selection process is cross-validation, in which the researcher simulates out-of-sample tests in historical data and picks models that would have performed best in these “as-if” out-of-sample scenarios. Element (ii) of our machine learning definition describes refinements in implementation that emphasize reliable out-of-sample performance in order to explicitly guard against overfit.

Note that this version is greatly condensed and uses much simpler language. It is also re-sequenced. However, it is still quite long and robotic; clearly written by AI. I could have given it instructions to write at a more advanced level but since I would rewrite everything anyway that would just complicate matters; simpler was better.

Why call it machine learning?

The term machine learning describes ideas that overlap substantially with traditional statistics, so why has it become a distinct field? Part of the answer is marketing: the phrase evokes cutting-edge Silicon Valley technology. But the distinction reflects several meaningful changes in how researchers build predictive models.

The first difference concerns scale. Traditional statistical models were often designed around a small number of predictors and relatively simple relationships between those inputs and the outcome being predicted. Machine learning expands this approach by allowing researchers to analyze much larger sets of variables and more complex, nonlinear relationships.

This does not mean machine learning replaces statistics. Many of its techniques have deep roots in statistical research. Rather, machine learning extends statistics toward the more flexible end of the modeling spectrum. Its central advantage is that it allows researchers to search for complex patterns without specifying those relationships in advance.

That flexibility comes with a tradeoff. Smaller models require stronger assumptions, but they are often more stable when data are limited. Larger models can capture more complicated relationships, but they also risk identifying patterns that exist only in historical data rather than in the underlying system. This problem, known as overfitting, occurs when a model learns noise instead of meaningful signals.

Researchers therefore seek a balance: models flexible enough to identify genuine predictive relationships but constrained enough to perform well on new data. Machine learning emphasizes this balance by focusing not only on developing models but also on testing and selecting among many possible models.

A second defining feature of machine learning is systematic model selection. Traditional statistics has long included methods for comparing alternative models, but machine learning places greater emphasis on searching across large collections of possible models and identifying those that perform best. Because searching among many models can itself create misleading results, machine-learning methods rely heavily on techniques such as regularization and cross-validation.

Regularization limits unnecessary complexity, preventing models from becoming more elaborate than the data can support. Cross-validation tests whether a model that performs well on historical data is likely to remain effective on new information. Together, these methods help researchers distinguish genuine predictive power from patterns that appear meaningful only by chance.

Investing with Stats and Machine Learning

A Summary of an excerpt of: Ronen Israel, Bryan Kelly, and Tobias Moskowitz, "Can Machines 'Learn' Finance?," Journal of Investment Management 18, no. 2 (2020): 23–36.

by Jacob S. Dorman, Ph.D.

Many machine-learning investment techniques rely on statistical methods; they do not replace them. To the contrary, they use statistics to find complex patterns in mountains of financial data. In other words, they don’t just find the needle in the haystack; they analyze the position of every piece of hay. Using machine learning to expand statistical techniques enables analysis of much larger and more complex datasets in a greater variety of ways. It enables statistics to operate with greater flexibility, allowing researchers to let the AI choose methods and patterns rather than determining those choices in advance.

Frankly, marketing is another consideration when we speak of “machine learning” or “AI”: those phrases are punchier and have greater commercial cachet than “computer-assisted advanced statistics.”

But applying stats and machine learning to investing is not without peril; searching for models can itself produce misleading results, so researchers use model-tuning techniques known as “regularization” and “cross-validation”—kind of like using various methods to “check the math” of LLM-generated computer code. Regularization limits unnecessary complexity, keeping models from becoming more elaborate than the data supports. Cross-validation checks whether a model that performs well on past data will still be effective on new information. Together, these methods help researchers distinguish useful patterns from those that might seem predictive but are actually just the result of chance events. The goal is to find the “Goldilocks” model: just right.

After all, not all machine learning investment models are created equal. Smaller models require more assumptions, but they are often preferable to large models when there isn’t much data. On the other hand, larger models can find more complex patterns in the data, but they also risk “overfitting,” which happens when a model cannot distinguish between “noise” and meaningful patterns. As a result, researchers must balance the size, sophistication, and flexibility of their models to find genuinely predictive patterns. In other words, models need to analyze a bale of hay and adapt so they don’t go haywire when they try to make sense of a barnful of data. As a result, researchers carefully develop and test many possible models. Humans need to be in the loop—the hay is not going to bale itself!

Throughout this process, I delegated the intermediate tasks that AI performs efficiently—finding articles, summarizing them, and generating alternative drafts—while making all substantive editorial decisions myself.

The finished piece differs substantially from the AI drafts. I reorganized the material into a more logical structure, removed repetition, condensed 619 words into 366, and relied on rhetorical techniques that remain difficult for LLMs to produce consistently: extended metaphor, alliteration, subtle humor, rhythm, and carefully sequenced paragraphs. I also restored several effective metaphors from the original text that the LLM had discarded because I judged their communicative value to outweigh their complexity.

I revised the passage repeatedly, and then used Grammarly and Microsoft Word as final quality-control tools. If OpenAI deemed it important, it could replicate and improve on such tools. Grammarly was valuable for catching grammatical issues and overlooked typos, but I accepted relatively few of its stylistic suggestions, making far more substantive revisions on my own. It is easy to think of ways that ChatGPT could improve on existing AI tools.

This project reinforced my view that AI works best as an accelerator rather than a replacement for experienced editors. Currently, it can dramatically speed research, brainstorming, and early drafting, but effective writing still depends on judgment, structure, voice, lived experience, and rhetorical craftsmanship developed over decades.

But such matters of taste can be learned; they are far less complicated than solving the Navier–Stokes Millennium Prize Problem. But such problems will not be solved by accident or by engineers, but by writers.