"AI Isn’t Coming for Tech Jobs—Yet" In Asterisk Magazine

extra notes on the counterfactual scenario I didn't have space for in the article

In my recent article in Asterisk Magazine, “AI Isn't Coming for Tech Jobs—Yet”, I predicted the impact of large language models (LLMs) on the tech job market by 2025. While I gave a brief justification, due to space constraints, I had to keep that section short, so I wanted to post more about the counterfactual scenario - what the tech job market might look like in the absence of LLMs.

The counterfactual scenario is a crucial part of any forecast. It helps us establish a baseline, a depiction of the world as it would be without the factor we're studying - in our case, a world without LLMs. This approach aligns with the concept of the "outside view" in forecasting. The "outside view" involves making predictions based on general statistical or historical information, rather than focusing on the specifics of the instance at hand.

By exploring the counterfactual scenario, we're essentially taking this "outside view". We're looking at broader trends and historical data to understand how the tech job market might evolve without the influence of LLMs.

Counterfactual Analysis

When constructing a counterfactual scenario, historical trends serve as a valuable guide. In the context of the tech labor market, it's important to remember that for the past ~20 years - the period for which we have reliable data - large language models (LLMs) were not a significant factor influencing job market fluctuations. In fact, these fluctuations have been relatively minimal. This historical perspective allows us to imagine a future tech job market continuing on its current trajectory, unaffected by the advent of LLMs.

Tech Employment History

Source: Bureau of Labor Statistics

Monte Carlo

When the situation allows, running a Monte Carlo simulation can provide an empirically-grounded outside view since the patterns observed in the past can be used to account for a range of possibilities and uncertainties. 

For those who are unfamiliar with the Monte Carlo method, it uses historical changes from a supplied data set to generate random future scenarios. After generating thousands of these random simulations, a distribution pattern emerges which is generally predictive of future outcomes as long as future events continue to follow a similar distribution pattern to historical events. One thing to be careful about when using this method is that it reflects a naive independence assumption (e.g. if one year is an extreme outlier, it doesn’t increase the chance that the next year continues the same pattern, but instead assumes something closer to a random walk) meaning that it can lead to an underestimation of extreme events.

Normally, the hard part is in determining the right contextual historical range, but, in this case, due to the limited data set I was able to collect from the BLS, it made sense to use everything I could find.

Because I was dealing with percentage changes, I used the geometric mean and standard deviation since minor differences in the growth rate can lead to major changes over time.

Tech Employment in 2025

After 100,000 trials, the Monte Carlo simulation showed a distribution with a median estimate of 5.2M tech jobs and a standard deviation of about 0.3M tech jobs.

Considering the limitations of the Monte Carlo method mentioned previously, it’s important to recognize that it may not capture the full extent of potential uncertainties.

Consequently, I increased my uncertainty as per my other estimates to reflect the potential range of outcomes resulting from LLM adoption, economic factors, and other unforeseen events.

Based on these results however,  history suggests we’re in for a period of continued growth and, as per the remainder of my analysis from the article, there are reasons to expect this might be true.

Conclusion

The counterfactual scenario serves as a baseline, a world where the tech job market evolves unaffected by LLMs. Yet, as we know, technology is anything but static. The advent of LLMs introduces new dynamics into the tech job market, accelerating growth in certain areas while leading to job displacement in others.

Moving forward, it's important to continue monitoring these trends, adjusting as new evidence comes to light, and preparing for a range of possible outcomes. The future of the tech job market, like the technology it revolves around, is a complex, evolving landscape, full of both challenges and opportunities.