The AI scientist?

I've decided we're at something more like the cyborg scientist.

What do I mean? People are wondering if you can just do science with AI. Can AI build the next iteration of itself? Can AI be used to come up with math conjectures or at least prove or disprove conjectures that we have? People certainly think so, including Terrence Tao of Fields Medal renown and some other major mathematicians whom I will not name for privacy reasons.

I wanted to do an experiment. Could I get AI to write a decent paper for me? I had tried in the past to get it to come up with a good paper idea, and it had failed terribly. AI lacks creativity even now. But math is filled with rules than AI could memorize, and coding is filled with rules that AI has memorized, so I attempted to get AI (in particular, Claude) to write an entire paper with only about 15 prompts.

My past relationship with AI has been that it helps, but does not supplant. Information theory identities needed to be double-checked and corrected due to major hallucinations for a paper on bacterial chemotaxis. Factors of two had to be checked for a theoretical connection between memory and regularization when training neural networks. So I assumed that actually, AI would need a lot of handholding and would be unable to produce a decent paper all by itself.

To give it its best shot, I chose an easier research project. The application was novel (pharmaceutical manufacturing), but the theoretical ideas were well-known. Basically, it had to do with turning the control theory patent that I mentioned in a prior blog post into a paper that was readable. All I had to show was that closed-loop control beat open-loop control in theory and sometimes in practice, something that is straightforward and well-known but just not well-known in pharmaceutical manufacturing.

Not only was this an experiment about what AI could do these days; I was also just too tired to write. I prompted Claude to write the paper, with prompts that ranged from requests for code and figures and LaTeX files to genuine asks of the AI chatbot to produce results. In about 15 prompts, I had everything, including a bibliography, with minimal effort. Don't get me wrong-- I still have to check over the math and code. And the writing is, at times, atrocious. There are run-on sentences everywhere. I don't think AI could have come up with the paper idea, a permanent weakness of AI (as it lacks creativity). But the AI-assisted paper is totally fine, and only a few modifications were made in getting it to its final form.

Actually, the original abstract raised some questions because it didn't tough on AI enough, but a slightly modified abstract (the cyborg scientist abstract) was given a 50% discount on open access publication fees.

It is insane what AI can do nowadays. This paper is as close as you can get to AI slop without being AI slop.

If this paper gets in, it will make me very scared for the future of science and humanity. In the future, will a math PhD simply involve a year of prompting an AI chatbot? How close will future graduate students and researchers come to AI scientists who simply sit there and monitor AI chatbots doing all our work? It is already uncomfortable to be at this level of cyborg scientist, where AI is so involved in writing the paper that it seems like the paper is Claude's rather than mine.

I don't know if the paper will get accepted or if it will get the discount that I'm asking for (so I don't have to scrape together funds to publish an AI paper) but I will let you know how things progress.

Right now, I feel like we're fucked as researchers. In 100 years, someone with my job will just be asking AI to produce a paper that they check. This is the Golden Age of AI, despite its gloom, before we slip into the years in which AI replaces our brains. All of our brains.