Upper Division math for Computer Science Major

[2ck]

I'm a sophomore selecting my courses for the fall semester and my advisor suggests that I take my required upper-division math this fall. I don't know which I should take though.

I think I want to do machine learning in terms of CS, but I'm not really sure. I've done calculus, introductory linear algebra, and a couple of classes where we did formal proofs, but I don't know what would be most helpful at this point.

What course would be most helpful for 2nd year CS major?

[Meem1029]

I'd say probably Graph Theory.

[freakish777]

I'm a sophomore selecting my courses for the fall semester and my advisor suggests that I take my required upper-division math this fall. I don't know which I should take though.

I think I want to do machine learning in terms of CS, but I'm not really sure. I've done calculus, introductory linear algebra, and a couple of classes where we did formal proofs, but I don't know what would be most helpful at this point.

What course would be most helpful for 2nd year CS major?

A list of available classes would be helpful.

A class that covers Bayes Theorem would (likely) be very helpful (hence the Graph Theory suggestion from Meem1029). A Statistics class would be also useful for the topic of Probability.

If there's a Fuzzy Logic class at your school, that would also be incredibly helpful.

[gorcee]

Statistics isn't going to cover probability the way you need. Take actual probability. You don't really use statistics (the way it's taught in university) to do machine learning (although statistical concepts are used). It's just not a great use of time learning Chi squared tests and whatnot, since those aren't the meat and potatoes of what you need to know.

Machine learning has a large amount of overlap with signal processing. Many of these classes, depending on the university, find themselves in the Computer Engineering or Electrical Engineering departments. Machine learning isn't an abstract, theoretical space that you can solve by just being clever. Much of it has to do with model-based algorithm development, which is pretty firmly in the domain of electrical engineering, and has been for about 40 years.

That said, there is an abstract component that goes beyond the pragmatic study of these systems. So I'd focus on the following topics:

Differential Equations (useless to do machine learning if you can't understand the model of the world that you're trying to teach the machine)
Variational Calculus
Discrete Math
Probability
Digital Signal Processing
Computer Vision
(some) Abstract Algebra
If available: automatic classification

Neural Networks are a sort of canonical example of machine learning. However, there's nothing special about them. They're just a network of input-output nodes, with a number of nodes in-between. All each node does is compute a number in some way, and passes that number on to the next node(s). Where Neural Networks get interesting is the training of them. Well, what do you need to know to train them? You need to understand data analysis/signal processing. You need to know how to select training data. You need to understand linear and nonlinear optimization. You need to understand how to validate the network.

Part of my work involves developing neural network models for automatic classification. Here's general outline of what I'd do to approach such a problem:

1.) Develop a feature extraction method from the raw data (frequency components? Image histograms? Edge detections? Means?)
2.) Specify a network structure
3.) Select a representative subset of the data
4.) Compute the hidden layer nodal weights by training the network on the data
5.) Cross-validate the network on the full data set, perhaps using k-fold cross validation, or some other metric
6.) Deploy the network, or networks, in a test environment against virgin data
7.) Evaluate ROC curves and confusion matrices

At each of these steps, I need math learned in different classes. Step 1 is open-ended and depends on the particular problem. Step 2 relies on some EE knowledge and signal processing background. Step 3 involves statistics and probability. Step 4 involves quite a bit of linear algebra, calculus, variational calculus, or differential equations. Step 5 involves statistics and data mining (often a EE course). Step 6 is just business level stuff. Step 7 is again signal processing and probability.

So even a simple, well-known machine learning application involves math from many different fields. Of course, different folks specialize in different aspects of this math. I don't need to go through the nodal weight optimization step by hand. I have software written that does that automatically. But there are still parameters to specify, and there are still edge conditions that can mess up the result. So I do need to understand how the algorithm works, so that I can analyze whether the network I've made is garbage or not.

Short version: look into the options that the university offers in not just the math department, but also the EE and CompEng departments.

[Yakk]

Bayes Theorem being taught in a Graph Theory course? Wut?

Machine learning is going to depend on knowledge of statistics and probability theory.

Fourier analysis (and related) is heavily used in signal processing, and if doing anything like machine vision will need a good grounding here.

Graph Theory is quite related to a lot of CS theory, in that both are branches of "discrete math" in some curriculum.

There are parts of CS itself that could easily be taught from a Mathematics perspective... I'd be leery of a CS program that doesn't have a proof-based "theory of computation" requirement (or at least option!) -- that could easily be a mathematics course. Similarly, complexity theory (a branch of CS) could easily be a mathematics course. Information theory is even closer to being a branch of mathematics.