Compilation AI Resources from Linear Algebra to GNN, RL or NeuroScience

Every topic about AI is widely studied. It becomes difficult for noobs (and I guess for everyone) to find and select the proper resources to start learning about some of them. That’s why I have made this compilation of materials to learn about different topics of AI.

The idea arise because, lately, I am gathering some amazing resources to start learning about GNN’s, and I found very helpful to gather all of them in a same place. But this compilation wants to cover all the trajectory of the Machine Learning learning path: Starting from Maths, Linear Algebra and Calculus, going through Statistics, specific and also broad Machine and Deep Learning Courses and finally leading to cutting-edge areas like Graph Neural Networks, Reinforcement Learning or Neuroscience.

Adaptation of G. Stang (MIT) Diagram of Deep Learning learning path.

Therefore, I have gathered every useful resource I found through many platforms. There are MOOCs, Courses, Video Lectures, Slides, Code, Books, PDFs, Papers… And, the best thing it’s that most of them are free. Courses usually have videos, slides, code and/or books attached to the main page. The code is mostly stored in GitHub repositories and coded in Python or R. You will see that some resources are starting point and another requires a higher level of knowledge of the topic, but, with all of them you can build an awesome own track which will cover all the learning path.

This is mi little piece of help in the trip of discovering amazing teaching and academic material. I know there are tons and tons of outstanding resources online and other compilations about ML or CS. But this is my personal selection. However, I always encourage to discover more books, courses (MOOCs or actually university open courses), blog posts or Githubs or papers.

The best way to get to the cutting-edge level of any topic is being aware of the latest papers and academic publications of the researches of that topic. Following top researchers on Twitter are very helpful, they promote papers and research as soon as they publish them. However, this list of resources tries to ease this exciting path.

Resources

Linear Algebra and Calculus

Learn Linear Algebra and Calculus to understand and play with all the roots of Statistics and Deep Learning

• | Course - Videos - Slides | G. Stang crash course (MIT): A 2020 Vision of Linear Algebra
• | Course - Book1 - Book2 - Videos | G. Stang Course MIT 18.06 and books for Linear Algebra and its relationship with DL: Linear Algebra
• | Book - Videos - Code | Steve Burton book: Data Driven Science & Engineering: Videos and code also available
• | Course - VideosLA - VideosCA | Imperial College London: Mathematics for Machine Learning: Linear Algebra and Multivariate Calculus: Coursera Course and Youtube videos

Statistics

Learn the most pure statistics to understand and analyze every model

• | Code - Web - Web2 | Kevin Patrick Murphy Books and Code: Probabilistic Machine Learning
• | Book | T. Hastie et. al. book: Elements of Statistical Learning
• | Course | EDx Hardvard Course: Statistics and R
• | Course | C. E. Rasmussen University of Cambridge 4f13 Course: Probabilistic Machine Learning
• | Code - Slides | Francisco Rodríguez-Sánchez: Introduction to Linear, Generalized, and Mixed/Multilevel models with R

Machine and Deep Learning

General Courses of Machine and Deep Learning. These courses cover broad topics of ML and DL. Some of them deepen more in maths, other focus in CNN, other in GNN…

• | Course - Code | Yann LeCun & Alfredo Canziani course of DL: web, slides, videos and code explaining broad topics of DL: Maths behind methods, Graph Neural Networks (GNN), Energy Based Models (EBM)…
• | Course | University of Amsterdam Course Deep Learning: general DL topics, EBM, GNN…
• | Compilation | Semi-Supervised learning compilation of papers and code
• | Course | UC Berkeley Course: Full Stack Deep Learning
• | Course | MIT 6.S191: Introduction to Deep Learning: Ongoing Course
• | Course | Standford CS231n: Convolutional Neural Networks for Visual Recognition
• | Course | Oxford Course: Machine Learning: From Linear prediction to LSTM or RL
• | Course | Microsoft Research Course: Geometry of DL
• | Code | Tensorflow in 30 days
• | Book&Code | FastAI book: Fastai and Pytorch course and code

Reinforcement Learning

Train an agent through an observation-action-reward cycle to maximize the reward. From DeepBlue to MuZero they are children of this field.

$$P(S_{t+1}\mid S_t) = P(S_{t+1}\mid S_1, …, S_N)$$

• | Course - YT Videos | David Silver (DeepMind) course: Introduction to Reinforcement learning
• | Web | OpenAI Gym
• | Book - Code by Shangtong Zhang | Sutton & Barto’s book: Reinforcement Learning: An Introduction (2nd Edition)
• | Course - YT Videos | Stanford CS 234 Course: Reinforcement Learning
• | Course - YT Videos | UC Berkeley CS 285 Course: Deep Reinforcement Learning
• | Code - Book | Maxim Lapan: Deep Reinforcement Learning Hands-On
• | Code | Denny Britz Code Examples: taken from Barto book and Silver course
• | Code | Repo for the Deep Reinforcement Learning Nanodegree program
• | Compilation | Wide Compilation of Codes, Books, Papers, Platforms etc. by Hyun Kim

Network science

Graph and networks theory. While it doesn’t deepen into GNNs, it is an excellent resource to get strong foundations for operating on graphs

• | Book | Network Science Book by Albert-László Barabási

Graph Neural Networks a.k.a. Graph Machine Learning a.k.a. Graph Representation Learning a.k.a. Geometric Deep Learning

Combination of graph theory (NS) and ML-DL to boost the potential flexibility applications of AI. DL dealing with graph data.

Lately, I am gathering some amazing resources to start learning about GNN's [@williamleif book, courses: @jure | @misovalko | @mmbronstein | @alfcnz-@ylecun, @Graph_NN...]. Also, @gordic_aleksa just made this hub of explanations and resources. Just awesome https://t.co/8pk1jIFXO5

— Adrián Arnaiz (@Arnaiztech) February 8, 2021

• | Book | William Hamilton Book: Graph Representation Learning
• | Course - Notes | Jure Leskovec Stanford CS224W Course: Machine Learning with Graphs
• | Course - Course | M. Valko (DeepMind-Inria) Course: Graphs in Machine Learning
• | Blog | Aleska Gordic Blog: How to get started with Graph Machine Learning
• | Blog - Video1 - Video2 - Video3 | Michael Bronstein Blog
• | Course soon | Cambridge Course of ML and GNN: resources not available yet
• Aforementioned:
  • | Course - Code | Yann LeCun & Alfredo Canziani course of DL
  • | Course | University of Amsterdam DL Course: DL, EBM, GNN
• | Code | Pytorch Geometric
• | Paper | Review paper, Yu Zhou et. al.: Graph Neural Networks: Taxonomy, Advances and Trends
• | Paper | Review paper, Zonghan Wu et. al. A Comprehensive Survey on Graph Neural Networks
• | Paper | Review paper, Ines Chami et. al.: Machine Learning on Graphs: A Model and Comprehensive Taxonomy
• | Compilation | Compilation of papers (2.8k ⭐)
  • Ordered by topic and venue
• | Compilation | Compilation of GNN papers by Jie Zhou, Ganqu Cui, Zhengyan Zhang and Yushi Bai (9.7k ⭐)
  • Ordered by Surveys, Type of models and Applications
• | Papers&Code | Papers and code in PapersWithCode

NeuroScience

“We draw upon research in cognitive and systems neuroscience to take advantage of what is known about how humans communicate and solve problems in order to design advanced artificial neural network architectures” Thomas Dean

• | Course - Notes | Stanford CS 379C Course: CS379C: Computational Models of the Neocortex
• | Course - Code | Computational Cognitive Neuroscience: CLPS1492

ML-Ops

How should be the pipeline of ML models ins production? Have you evere thought about all the technical implications around a ML model in production?

• Awesome MLOPS by Visenger
• ML in lifecycle - EthicalML
• Code docker by Chris albon
• More in Elvis Medium Post