CompSciProgram
This repository contains exercises and projects on computational science and AI for the CompSci program
First project:
Machine learning with linear and non-linear regression, logistic
regression and support vector machines as well as Bayesian linear
regression. This involves linear algebra (matrix inversion,
determinants, eigenvalues, SVD and more from FYS4150), convex
optimization problem (gradient descent, steepest descent, stochastic
gradient descent, iterative solvers) and several central
(deterministic) ML methods. Calculation-oriented statistics with
Bayes’ theorem and MCMC sampling can also be included. Bayesian linear
regression can be omitted.
Workload: 6 ECTS.
Datasets you study can be adapted to your research field, whether it
is astro, physics, chemistry, bioscience, geoscience or mathematics.
Planned finished end January 2023
Second project:
Deep learning: standard neural networks, convolution and neural
networks (CNN), recursive neural networks, Boltzmann machines, various autoencoders and possibly general adversial networks. Reduction
of dimensionality in scientific problems. Possible topic to work with:
solution of ordinary and partial differential equations. Here we can
take this from a deep learning perspective and a traditional final
difference form taught in FYS4150. But we can also focus on classification problems.
Datasets can again be adapted to the field.
Workload: 7 ECTS. Planned finished end March/begin April 2023
Third project:
See project description here: https://raw.githubusercontent.com/CompPhysics/CompSciProgram/main/doc/Projects/2022/Project3/pdf/Project3.pdf
Workload: 7 ECTS. Deadline: June 7 2023
In total 20 ECTS.
Lectures
October 26
- Intro to machine learning and linear regression, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week42/html/week42-reveal.html
- Video of Lecture at https://youtu.be/C8dL1pLUJ3A
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesOct262022.pdf
- See also notes on derivatives of matrices and vectors at https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2022/NotesExercise5Week452022.pdf
November 2
- From ordinary least squares to Ridge and Lasso regression, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week43/html/week43-reveal.html
- Video of Lecture at https://youtu.be/EiO7WOm_DLs
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesNov22022.pdf
November 9
- Video of Lecture at https://youtu.be/XIdtY2_9rgk
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesNov92022.pdf
November 23
- Video of Lecture at https://youtu.be/HvBSIGAemvE
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesNov232022.pdf
December 12
- Logistic regression and intro to optimization problems, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week45/html/week45-reveal.html
- Video of Lecture at https://youtu.be/me_tglaPvI0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesDec122022.pdf
December 13
- Gradient methods, from simple gradient descent to stochastic gradient descent, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week46/html/week46-reveal.html
- Video of Lecture at https://youtu.be/8WoA-MQt_8U
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesDec132022.pdf
December 14
- Stochastic gradient descent, from momentum to adaptive methods and discussions of project 1. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week46/html/week46-reveal.html
- Video of Lecture at https://youtu.be/EFNlvtjhpMo
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesDec142022.pdf
January 17
- Start Deep Learning, basics of neural networks, mathematics and architecture. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week3/html/week3-reveal.html
- Video of Lecture at https://youtu.be/JY-7F6TXK7U
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesJan172023.pdf
January 24
- Deep learning, back propagation algoritm and automatic differentiation, part 1. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week4/html/week4-reveal.html
- Video of Lecture at https://youtu.be/KaL0W1jaRmQ
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesJan242023.pdf
January 31
- Deep learning, back propagation algoritm and automatic differentiation and code for neural networks, part 2. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week5/html/week5-reveal.html
- Video of Lecture at https://youtu.be/cWCebuNKrA8
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesJan312023.pdf
February 7
- Deep learning and codes for neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week4/html/week4-reveal.html
- Video of Lecture at https://youtu.be/_pOf4__oN28
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesFeb72023.pdf
February 21
- Discussion of project 2 and start discussion of Convolutional neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week7/html/week7-reveal.html
- Video of Lecture at https://youtu.be/leazXTMbRCM
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesFeb212023.pdf
February 28
- Convolutional neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week9/html/week9-reveal.html
- Video of Lecture at https://youtu.be/C8Pj7Wq_7fw
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesFeb282023.pdf
March 7
- Summary of CNNs and start discussion of Recurrent neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week10/html/week10-reveal.html
- Video of lecture https://youtu.be/VvgiK3TUdxg
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMarch72023.pdf
March 14
- Summary of RNNs and work on project 2
- Video of lecture at https://youtu.be/TD63ggQbkTk. No whiteboard notes.
March 21
March 28
- Intro to part 3 of the course (Bayesian methods)
- Video of lecture https://www.dropbox.com/s/u5nmoodg3v7vc2b/2023_March_28.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMar282023.pdf
April 25
- Aspects of Bayesian statistics
- Video of lecture https://www.dropbox.com/s/5upi56c54e92kxy/2023_April_25.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesApr252023.pdf
May 2
- Wrap up intro to Bayesian statistics
- Nested sampling
- Video of lecture https://www.dropbox.com/s/jajawr636pdksx4/2023_May_2.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMay022023.pdf
May 9
- Wrap up discussion of nested sampling
- Video (from 2022): https://www.dropbox.com/s/dtnivfgna3kcy00/2023_May_9__nested_sampling_from_2022.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMay022023.pdf
- Gaussian processes
- Video (from 2022): https://www.dropbox.com/s/7zf9l931nwgu9pb/2023_May_9__GP_regression_from_2022.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMay092023.pdf
May 16
- Gaussian processes, cont.
- Video (from 2022): https://www.dropbox.com/s/yl7uwpx6fc1pv72/2023_May_16__more_GP_regression_from_2022.mp4?dl=0
- Handwritten notes: same as previous lecture