UINA500 Deep Learning

Faculty of Philosophy and Science in Opava
Winter 2023
Extent and Intensity
2/2/0. 6 credit(s). Type of Completion: zk (examination).
Teacher(s)
doc. Ing. Petr Sosík, Dr. (lecturer)
Mgr. Tomáš Filip (seminar tutor)
doc. Ing. Petr Sosík, Dr. (seminar tutor)
Guaranteed by
doc. Ing. Petr Sosík, Dr.
Institute of Computer Science – Faculty of Philosophy and Science in Opava
Timetable
Tue 14:45–16:20 B3b
  • Timetable of Seminar Groups:
UINA500/A: Tue 16:25–18:00 B3b, P. Sosík
Course Enrolment Limitations
The course is also offered to the students of the fields other than those the course is directly associated with.
fields of study / plans the course is directly associated with
Course objectives
This classical branch of Artificial Intelligence covers a range of machine learning algorithm, typically benefiting from gradient-based learning methods. The most typical learning model is the artificial neural net with many efficient algorithms capable to learn from examples, to generalize knowledge and to search for approximate solutions of intractable problems. These algorithms can be run on special parallel machines but also on classical computers.
Learning outcomes
The student will get acquainted with the basic mathematical and structural principles of deep learning. Will be able to design and test deep learning networks for a variety of tasks such as classification, image analysis, comprehension and text generation, or strategic decision making.
Syllabus
  • 1. The structure of biological neuron, mathematical model of a simple neuron and a multi-layer neural network. Advantages and applications of artificial neural nets in deep learning. 2. Active, adaptive and organization dynamics, neural training schemes (supervised/unsupervised/reinforcement). Training and testing sets, training process, the overfitting problem. 3. The perceptron and its training algorithm. Implementation of simple logic functions. Limited capabilities of single-layer perceptron. 4. Multilayer networks and the Backpropagation (BP) algorithm. Modifications and improvements of the BP algorithm, conjugate-gradient methods, resilient propagation, further training methods. 5. Deep feedforward networks, architecture design, regularization methods. 6. Optimization of deep learning, parameter initialization methods, adaptive learning rates, meta-algorithms. 7. Deep learning in recurrent networks, topology and training algorithms, recursive networks. Problem of long dependencies, LSTM networks and related models. 8. Radial Basis Function networks, organization and active dynamics. Three phases of training, properties, applications, a comparison with multilayer perceptron. 9. Competitive networks and vector quantization problem, Lloyd's algorithm. The Kohonen training rule, the UCL, DCL a SCL algorithms. Self-organizing maps – SOM. The ART networks, principles and properties.
Literature
    required literature
  • Chollet, Francois. Deep learning with Python. Simon and Schuster, 2017.
  • GOODFELLOW, Ian, BENGIO, Yoshua, COURVILLE, Aaron. Deep Learning. Cambridge, Massachusetts: MIT Press, 2016
    recommended literature
  • MIRJALILI, Seyedali. Evolutionary Algorithms and Neural Networks: Theory and Applications. New York, NY: Springer International Publishing, 2018.
Teaching methods
Interactive lecture Lecture with a video analysis
Assessment methods
Individual projects and exercises for solutions at home.
Language of instruction
English
Further Comments
Study Materials
The course is also listed under the following terms Winter 2021, Winter 2022, Winter 2024.
  • Enrolment Statistics (Winter 2023, recent)
  • Permalink: https://is.slu.cz/course/fpf/winter2023/UINA500