Self-studying mathematics

I’m starting this blog as a way to document progress on a long-standing goal of mine — teaching myself mathematics (with a bit of physics/CS too). I’ve been wanting to do this ever since I joined a theoretical neuroscience lab in my bachelor’s, but for a long time I was a) afraid of failure and b) dealing with some chronic health issues, which have been (mostly) resolved now.

Why am I doing this? #

In early 2021, in the midst of COVID, I had the realization I would end up working either on psychopharmacology or theoretical neuroscience in my career. So I did a master’s in cognitive neuroscience and psychopharmacology — and hated it — but I was fortunate to do some research that gradually led me toward what I now consider deeper questions in neuroscience and AI. Now, I feel deeply enamored by questions around the relationship of substrate and cognition, how networks learn, learning rules in general, and perhaps most importantly, how biological neural networks give rise to consciousness, or more precisely, subjective experience. It is not just a scientific question, but a philosophical one, that might be solved through the slow regression of the ’easy questions’ (see: The Real Problem) on the harder problem. Right now, the link between a network architecture and the model it might build, or the inductive biases it incorporates, feels a lot like the hard problem. Maybe the brilliant engineers and scientists working on these models feel that way, too, but then I understand even less than they do.

So, it’s time to learn math. I’ve decided to build this website and document my journey as a way of motivating myself and building a portfolio along the way. There are many brilliant mathematicians and engineers out there, and if I hope to compete, I will need to show my work. In a few months, I’ll attend the Gatsby Bridging Programme, but until then I’m going to speed-run as much math as I can stomach.

A syllabus of sorts #

I’ve cobbled together a learning plan based on MIT’s excellent OpenCourseWare, some textbooks I’ve been recommended, and the Open Source Society University’s math course. I’ve been following the team at Math Academy for a while now and have decided to make MA my primary form of learning, using textbooks and courses as supplementary (especially while traveling or when I’m without access to a laptop + iPad to study from). Their team has clearly thought through their method and it’s worked well for me so far. I’m not sure why there isn’t more competition for this kind of approach. Reversing the decline of genius is one of the most exciting use-cases for AI (and specifically chatbots). Their approach is based on feedback, using AI to solve the sigma-2 problem — that is, students tutored one-on-one perform, on average, two standard deviations (98% in a normal distribution!) better than students learning via conventional methods. The problem being that one-on-one tutoring is incredibly expensive.

Of course, one has to believe that the feedback that chatbots can provide is effective in principle or in practice (see a counterargument). I’m not convinced by the idea that most teachers outperform chatbots in identifying the source of human error and the emotion involved right now, let alone in a few years. From my own educational experience, it’s clearly not the case — the learning I do with LLMs is significantly more effective.

In the two months I have until the Gatsby programme, I’ll be trying to speed-run the following:

graph BT
MF2[Mathematical<br>Foundations II] --> MF3[Mathematical<br>Foundations III]
MF3 --> LA[Linear Algebra]
MF3 --> ML[Mathematics for<br>Machine Learning]
LA --> MVC[Multivariable<br>Calculus]
MVC --> PNS[Probability &<br>Statistics]
MVC --> DE[Differential<br>Equations]

I plan to finish through Mathematical Foundations II/III, which is mostly a review of calculus, trigonometric identities, basic vector math, etc., and then move on to the other available courses that MA has posted. At linear algebra and beyond, I’ll work with some textbooks and lecture series, too:

The thing about learning a new subject is that you often don’t know what you don’t know. Mentors provide wonderful roads to follow, but as I progress, I’ll inevitably start to focus on one area or another. The above diagram will likely not change much, if at all. When writing this post, a friend told me, “the mission should be to learn enough math to ditch this diagram ASAP and come up with a better one.” I share the below diagram as proof of my innocence:

graph BT

%% Edges
F[Current foundation] --> DM[Discrete mathematics]
F --> AA
AA---DL
DL---RL
DL---O
RL---O
F --> O[Optimization]
F --> DL[Deep learning]
F --> RL[Reinforcement learning]
F --> MP
F --> A[Analysis]
F --> TN[Theoretical Neuroscience]
F --> LO
LO[Logic] --> A
MP[Methods of Proof] --> AA
MP --> A
AA[Abstract<br>Algebra]
DM --> A
AA --> T[Topology]
A --> T
AA --> CT[Category Theory]
T---DL
T---RL
CT---DL

This one is… a bit more of a mess. At this point, there are aspects of these fields I’m really interested in — for example, I worked briefly on category-theoretic approaches in structural models of consciousness (see Qstr-IIT Summer School) — and I’d like to learn how category theory might relate to deep learning. I’m increasingly interested in neural manifolds and studying topology might help a lot. But we’ll see where the actual mathematics takes me.

Showing my work #

The major issues with a self-studied curriculum are that there’s little feedback, both to ensure that one is really learning the material, and there are no grades to prove to you, the world, that I actually know what I’m doing. So I’ll be doing a few things — I’ll be writing essays, lecture notes, posting the answers to problem sets with explanations of how and why I did what I did, and working on projects that are relevant for each stage of my learning. These projects will be worked on under the supervision of mentors in my field and outside it.

Conclusion #

I’ve wanted to do this for a long time. Publishing this plan is my way of making it real — of turning an intention into action. If you’re interested in doing something similar, you can subscribe to the newsletter or follow the RSS feed. But really, I’m excited. I’ve been looking for this kind of focus, calm, and single-mindedness for a long time. I might end up going back to school to do a proper degree, or maybe I’ll get into a PhD program first. We’ll see. Either way, this is for now.