Modern Deep Learning Design and Application Development

Chapter 1: "A Deep Dive Into Keras"Chapter Goal: To give a structured yet deep overview of Keras and to lay the groundwork for implementations in future chapters.Number of Pages: ~30Subtopics1.Why Keras? Versatility and simplicity.2.Steps needed to create a Keras model: define architecture, compile, fit.a.Compile: discuss TensorFlow optimizers, losses, and metrics.b.Fit: discuss callbacks.3.Sequential model + example.4.Functional model + example.5.Visualizing Keras models.6.Data: using NumPy arrays, Keras Image Data Generator, and TensorFlow datasets.7.Hardware: using and accessing CPU, GPU, and TPU.

Chapter 2: Pre-training Strategies and Transfer LearningChapter Goal: To understand the importance of transfer learning and to use a variety of transfer learning methods to solve deep learning problems efficiently.Number of Pages: ~30Subtopics1.Transfer learning theory, practical tips and tricks.2.Accessing and using Keras and TensorFlow pretrained models.a.Bonus: converting PyTorch models (PyTorch has a wider variety) into Keras models for greater access to pretrained networks.3.Manipulating pretrained models with other network elements.4.Layer freezing.5.Self-supervised learning methods.

Chapter 3: "The Versatility of Autoencoders"Chapter Goal: To understand the versatility of autoencoders and to be able to use them in a wide variety of problem scenarios.Number of Pages: ~30Subtopics1.Autoencoder theory.2.One-dimensional data autoencoder implementation, tips and tricks.3.Convolutional autoencoder implementation, tips and tricks, special concerns.4.Using autoencoders for pretraining.a.Example case study: TabNet.5.Using autoencoders for feature reduction.6.Variational autoencoders for data generation.

Chapter 4: "Model Compression for Practical Deployment"Chapter Goal: To understand pruning theory, implement pruning for effective model compression, and to recognize the important role of pruning in modern deep learning research.Number of Pages: ~20Subtopics1.Pruning theory.2.Pruning Keras models with TensorFlow.3.Exciting implications of pruning - the Lottery Ticket Hypothesis.a.Example case-study: no-training neural networks.b.Example case-study: extreme learning machines.

Chapter 5: "Automating Model Design with Meta-Optimization"Chapter Goal: To understand what meta-optimization is and to be able to use it to effectively automate the design of neural networks.Number of Pages: ~20Subtopics1.Meta-optimization theory.2.Demonstration of meta-optimization using HyperOpt on Keras.3.Demonstration of Auto-ML and Neural Architecture Search.

Chapter 6: "Successful Neural Network Architecture Design"Chapter Goal: To gain an understanding of principles in successful neural network architecture design through three case studies.Number of Pages: ~25Subtopics1.Diversity of neural network designs and the need to design specific architectures for particular problems.2.Theory and implementation of block/cell/module design and considerations.a.Example case study: Inception model.3.Theory and implementation of "Normal" and "extreme" usages of skip connections.a.Parallel towers and cardinalityb.Example case study: UMAP model.4.Neural network scaling.a.Example case study: EfficientNet.