Why Is AT The Heart Of Deep Learning?

Why Is At at the heart of deep learning? Discover the pivotal role of the General Matrix to Matrix Multiplication (GEMM) in neural networks. WHY.EDU.VN unveils the secrets to optimizing deep learning performance. Explore the GEMM algorithm and matrix multiplication techniques for AI acceleration.

1. Understanding the Significance of “Why Is AT” in Deep Learning

The field of deep learning has revolutionized various aspects of technology, from image recognition to natural language processing. At the core of these advancements lies a fundamental operation known as GEMM (GEneral Matrix to Matrix Multiplication). To appreciate why GEMM, represented abstractly by “AT,” holds such significance, it’s essential to delve into its role within neural networks and the broader context of computational efficiency. This section will explore the fundamental importance of AT in deep learning, setting the stage for a deeper understanding of its applications and optimization techniques, revealing insights that you can further explore on WHY.EDU.VN.

1.1 The Central Role of GEMM in Neural Networks

Neural networks, the backbone of deep learning, consist of interconnected layers of nodes (neurons). These networks learn complex patterns by adjusting the connections (weights) between these neurons. A crucial step in this learning process involves matrix multiplications, where input data is transformed through weight matrices to produce outputs. GEMM is the workhorse that performs these matrix multiplications efficiently.

• Matrix Operations: The fundamental operations within neural networks rely on matrix multiplications.
• Weight Matrices: These operations involve transforming input data using weight matrices.
• Efficient Computation: GEMM enables these computations to occur rapidly and efficiently, making it essential for training and inference.

1.2 Addressing Computational Bottlenecks

Deep learning models often involve large amounts of data and complex network architectures. This leads to significant computational demands, and matrix multiplications can become a bottleneck. Optimizing GEMM operations is crucial for reducing training times and enabling real-time applications.

• Large Datasets: Deep learning models deal with massive volumes of data.
• Complex Architectures: These models employ complex network architectures.
• Optimization: Optimizing GEMM is essential for alleviating computational bottlenecks, a topic you can delve deeper into at WHY.EDU.VN.

1.3 GEMM as a Building Block

GEMM isn’t just about multiplying matrices; it’s a fundamental building block for many other operations in deep learning. Convolutions, fully connected layers, and recurrent neural networks all rely on GEMM to perform their core computations. Improving GEMM’s performance directly translates to improved performance across the entire deep learning pipeline.

• Convolutional Layers: GEMM supports crucial operations like convolutions.
• Fully Connected Layers: It is fundamental to computations in fully connected layers.
• Recurrent Neural Networks: GEMM plays a role in the functionality of recurrent neural networks.

2. The Mathematical Foundation of GEMM

To truly understand the significance of GEMM, it’s crucial to explore its mathematical underpinnings. GEMM, or GEneral Matrix to Matrix Multiplication, is a fundamental operation in linear algebra and serves as a core building block for numerous computational tasks, especially within the realm of deep learning. In this section, we will dissect the mathematical intricacies of GEMM, examining its formulas, properties, and the nuances of its implementation, offering a deeper understanding that you can use to formulate further questions on WHY.EDU.VN.

2.1 Defining Matrix Multiplication

At its core, GEMM involves multiplying two matrices, A and B, to produce a resulting matrix, C. The dimensions of these matrices play a crucial role in defining the multiplication process.

• Dimensions: If matrix A has dimensions ( m times k ) and matrix B has dimensions ( k times n ), the resulting matrix C will have dimensions ( m times n ).
• Element-wise Calculation: Each element ( c_{ij} ) in matrix C is calculated as the dot product of the ( i )-th row of matrix A and the ( j )-th column of matrix B.

Mathematically, this can be expressed as:

[
c{ij} = sum{l=1}^{k} a{il} cdot b{lj}
]

Where:

• ( c_{ij} ) is the element in the ( i )-th row and ( j )-th column of matrix C.
• ( a_{il} ) is the element in the ( i )-th row and ( l )-th column of matrix A.
• ( b_{lj} ) is the element in the ( l )-th row and ( j )-th column of matrix B.

2.2 Properties of Matrix Multiplication

Matrix multiplication possesses several key properties that influence its application and optimization within deep learning models.

• Non-Commutative: Unlike scalar multiplication, matrix multiplication is generally non-commutative. That is, ( A cdot B ) is not necessarily equal to ( B cdot A ).
• Associative: Matrix multiplication is associative, meaning that ( (A cdot B) cdot C = A cdot (B cdot C) ). This property is essential for optimizing sequences of matrix multiplications.
• Distributive: Matrix multiplication is distributive over addition, i.e., ( A cdot (B + C) = A cdot B + A cdot C ).

2.3 GEMM in Practice

In practice, GEMM is implemented using highly optimized libraries and algorithms. These implementations take advantage of hardware-specific features, such as SIMD (Single Instruction, Multiple Data) instructions and cache hierarchies, to maximize performance.

• Optimized Libraries: Libraries like BLAS (Basic Linear Algebra Subprograms) and cuBLAS (for NVIDIA GPUs) provide optimized GEMM routines.
• Hardware Acceleration: Modern CPUs and GPUs include specialized hardware units designed to accelerate matrix multiplications.

3. GEMM’s Role in Fully Connected Layers

Fully connected (FC) layers are foundational components of neural networks, integral for processing information across various layers. GEMM (General Matrix to Matrix Multiplication) plays a pivotal role in these layers, facilitating the complex computations required to transform input data into meaningful outputs. This section elucidates how GEMM is employed within fully connected layers, providing a detailed explanation and enhancing your understanding, encouraging you to explore more on WHY.EDU.VN.

3.1 Understanding Fully Connected Layers

In a fully connected layer, each neuron is connected to every neuron in the preceding layer. This structure allows the network to capture complex relationships between the input features.

• Dense Connections: Every neuron in one layer is connected to every neuron in the adjacent layer.
• Feature Extraction: This enables the network to learn intricate patterns from the input data.

3.2 GEMM Operation in FC Layers

The primary operation in a fully connected layer involves multiplying the input data (represented as a matrix) by a weight matrix and adding a bias vector. GEMM is used to perform this matrix multiplication efficiently.

• Input Data: The input is typically represented as a matrix ( X ) with dimensions ( m times k ), where ( m ) is the number of samples and ( k ) is the number of input features.
• Weight Matrix: The weight matrix ( W ) has dimensions ( k times n ), where ( n ) is the number of neurons in the FC layer.
• Bias Vector: A bias vector ( b ) of dimension ( 1 times n ) is added to the result.

The output ( Y ) of the FC layer is computed as:

[
Y = X cdot W + b
]

Here, ( X cdot W ) is a matrix multiplication that can be efficiently performed using GEMM.

3.3 Mathematical Explanation

The matrix multiplication ( X cdot W ) involves computing the dot product of each row of ( X ) with each column of ( W ). Specifically, the element ( y_{ij} ) in the output matrix ( Y ) is calculated as:

[
y{ij} = sum{l=1}^{k} x{il} cdot w{lj} + b_j
]

Where:

• ( x_{il} ) is the element in the ( i )-th row and ( l )-th column of matrix ( X ).
• ( w_{lj} ) is the element in the ( l )-th row and ( j )-th column of matrix ( W ).
• ( b_j ) is the ( j )-th element of the bias vector ( b ).

3.4 Optimization Considerations

Optimizing GEMM operations within FC layers is crucial for improving the overall performance of neural networks. Various techniques can be employed:

• BLAS Libraries: Using optimized BLAS libraries can significantly speed up matrix multiplications.
• Hardware Acceleration: Leveraging GPUs and specialized hardware accelerators can further enhance performance.
• Matrix Reordering: Reordering matrices to improve cache utilization can also yield performance gains.

4. GEMM’s Role in Convolutional Layers

Convolutional Neural Networks (CNNs) have revolutionized image recognition and processing tasks. While the convolution operation might seem distinct from matrix multiplication, GEMM (General Matrix to Matrix Multiplication) plays an integral role in implementing convolutional layers efficiently. This section details how GEMM is adapted to perform convolutions, elucidating the methods and benefits involved, and encouraging further exploration on WHY.EDU.VN.

4.1 The Convolutional Operation

Convolutional layers apply a set of learnable filters (kernels) to input images (or feature maps) to detect patterns.

• Kernels: Small matrices of weights that are convolved with the input data.
• Feature Maps: The result of applying kernels across the input, producing a set of output feature maps.

4.2 Transforming Convolution into Matrix Multiplication

To leverage GEMM for convolutions, a transformation known as “im2col” (image-to-columns) is commonly used.

• Im2col: This transformation rearranges overlapping patches of the input image into columns of a matrix.
• Weight Matrix: The convolution kernels are reshaped into rows of another matrix.

4.3 The Im2col Transformation

The im2col transformation involves extracting overlapping patches from the input image and arranging them into columns of a matrix.

• Patch Extraction: For each position where the convolution kernel is applied, a corresponding patch is extracted from the input image.
• Column Arrangement: These patches are then arranged into columns of a matrix, creating a new representation of the input data.

4.4 GEMM Application

After applying im2col, the convolution operation can be expressed as a matrix multiplication.

• Input Matrix: The im2col transformation produces an input matrix ( X ) with dimensions ( (k times k times c) times (H times W) ), where ( k ) is the kernel size, ( c ) is the number of input channels, and ( H ) and ( W ) are the height and width of the output feature map.
• Weight Matrix: The convolution kernels are reshaped into a weight matrix ( W ) with dimensions ( n times (k times k times c) ), where ( n ) is the number of kernels.

The convolution operation is then computed as:

[
Y = W cdot X
]

Here, ( Y ) is the output matrix with dimensions ( n times (H times W) ), which is then reshaped into the output feature map.

4.5 Mathematical Explanation

Each element ( y_{ij} ) in the output matrix ( Y ) is calculated as:

[
y{ij} = sum{l=1}^{k times k times c} w{il} cdot x{lj}
]

Where:

• ( w_{il} ) is the element in the ( i )-th row and ( l )-th column of the weight matrix ( W ).
• ( x_{lj} ) is the element in the ( l )-th row and ( j )-th column of the input matrix ( X ).

4.6 Advantages of Using GEMM

Using GEMM for convolutions offers several advantages:

• Optimized Implementations: GEMM has highly optimized implementations in BLAS libraries and hardware accelerators.
• Efficiency: By transforming convolution into matrix multiplication, these optimizations can be leveraged to speed up convolutional layers.

5. Optimizing GEMM for Performance

Optimizing GEMM (General Matrix to Matrix Multiplication) is crucial for enhancing the performance of deep learning models. Given its central role in both fully connected and convolutional layers, even marginal improvements in GEMM efficiency can lead to significant reductions in training time and increased inference speed. This section explores various strategies and techniques for optimizing GEMM, encouraging further learning and inquiry on WHY.EDU.VN.

5.1 Leveraging BLAS Libraries

BLAS (Basic Linear Algebra Subprograms) libraries provide highly optimized routines for performing basic linear algebra operations, including GEMM.

• Optimized Routines: BLAS libraries are meticulously tuned for specific hardware architectures, ensuring maximum performance.
• Hardware-Specific Implementations: Implementations such as cuBLAS (for NVIDIA GPUs) and MKL (for Intel CPUs) are tailored for their respective hardware.

5.2 Techniques for Optimizing GEMM

Several techniques can be employed to optimize GEMM performance.

• Loop Reordering: Reordering the loops in the GEMM algorithm can improve cache utilization and reduce memory access latency.
• Tiling: Dividing the matrices into smaller blocks (tiles) and performing GEMM on these tiles can improve cache locality.
• Vectorization: Utilizing SIMD (Single Instruction, Multiple Data) instructions to perform multiple operations in parallel can significantly speed up GEMM.
• Strassen’s Algorithm: For very large matrices, Strassen’s algorithm can reduce the number of multiplications required, though it may introduce additional overhead.

5.3 Loop Reordering

Loop reordering involves rearranging the order in which the loops are executed in the GEMM algorithm.

• Cache Utilization: Different loop orderings can result in different memory access patterns, affecting cache utilization.
• Optimal Ordering: The optimal loop ordering depends on the hardware architecture and the dimensions of the matrices.

5.4 Tiling

Tiling involves dividing the matrices into smaller blocks and performing GEMM on these blocks.

• Cache Locality: This improves cache locality, as the smaller blocks can fit entirely within the cache.
• Reduced Memory Access: Reduces the number of accesses to main memory, which is much slower than cache.

5.5 Vectorization

Vectorization utilizes SIMD instructions to perform multiple operations in parallel.

• SIMD Instructions: These instructions allow a single instruction to operate on multiple data elements simultaneously.
• Parallel Computation: Vectorization can significantly speed up GEMM by performing multiple multiplications and additions in parallel.

5.6 Strassen’s Algorithm

Strassen’s algorithm is a divide-and-conquer algorithm that can reduce the number of multiplications required for large matrices.

• Reduced Multiplications: Traditional GEMM requires ( O(n^3) ) multiplications, while Strassen’s algorithm requires ( O(n^{2.81}) ) multiplications.
• Overhead: Strassen’s algorithm introduces additional overhead, so it is only beneficial for very large matrices.

6. Hardware Acceleration for GEMM

To meet the increasing computational demands of deep learning, hardware acceleration has become essential. Accelerating GEMM (General Matrix to Matrix Multiplication) operations using specialized hardware can significantly improve the performance of deep learning models. This section examines the hardware accelerators used for GEMM, detailing their functionalities and advantages, encouraging further study and exploration on WHY.EDU.VN.

6.1 GPUs (Graphics Processing Units)

GPUs are widely used for accelerating deep learning tasks due to their massively parallel architecture.

• Parallel Processing: GPUs consist of thousands of cores that can perform computations in parallel.
• High Throughput: This makes them well-suited for matrix multiplications, which involve a large number of independent operations.
• cuBLAS: NVIDIA provides cuBLAS, a highly optimized BLAS library for their GPUs.

6.2 TPUs (Tensor Processing Units)

TPUs are custom-designed hardware accelerators developed by Google specifically for deep learning workloads.

• Custom Architecture: TPUs are designed from the ground up to accelerate matrix multiplications and other deep learning operations.
• High Performance: They offer significantly higher performance compared to CPUs and GPUs for certain deep learning tasks.

6.3 FPGAs (Field-Programmable Gate Arrays)

FPGAs are reconfigurable hardware devices that can be customized to accelerate specific computations.

• Reconfigurability: FPGAs can be programmed to implement custom GEMM accelerators.
• Flexibility: This allows for fine-grained control over the hardware architecture, enabling highly efficient implementations.

6.4 ASICs (Application-Specific Integrated Circuits)

ASICs are custom-designed chips that are optimized for a specific application.

• Application-Specific: ASICs can be designed to implement highly efficient GEMM accelerators.
• Maximum Performance: They offer the highest performance but lack the flexibility of FPGAs.

6.5 Advantages of Hardware Acceleration

Hardware acceleration offers several advantages for GEMM:

• Increased Performance: Specialized hardware can perform GEMM operations much faster than CPUs.
• Reduced Power Consumption: Hardware accelerators are often more power-efficient than CPUs.
• Scalability: Hardware acceleration enables deep learning models to scale to larger sizes and datasets.

7. GEMM in Different Deep Learning Frameworks

Deep learning frameworks provide high-level APIs for building and training neural networks. These frameworks rely on optimized GEMM (General Matrix to Matrix Multiplication) implementations to efficiently perform the underlying computations. This section examines how GEMM is utilized within popular deep learning frameworks, explaining the underlying mechanisms and optimization strategies, prompting further research and questions on WHY.EDU.VN.

7.1 TensorFlow

TensorFlow is a widely used open-source deep learning framework developed by Google.

• Computational Graph: TensorFlow represents computations as a computational graph, where nodes represent operations and edges represent data flow.
• GEMM Implementation: TensorFlow uses optimized GEMM implementations from libraries such as Eigen and cuBLAS (for GPUs).
• Automatic Optimization: TensorFlow automatically optimizes the computational graph to improve performance.

7.2 PyTorch

PyTorch is another popular open-source deep learning framework known for its flexibility and ease of use.

• Dynamic Computation Graph: PyTorch uses a dynamic computation graph, which allows for more flexibility in defining and modifying the network architecture.
• GEMM Implementation: PyTorch also relies on optimized GEMM implementations from libraries such as BLAS and cuBLAS.
• Custom Operators: PyTorch allows users to define custom operators, which can be used to implement specialized GEMM routines.

7.3 MXNet

MXNet is a flexible and efficient deep learning framework that supports multiple programming languages.

• Hybrid Approach: MXNet uses a hybrid approach that combines symbolic and imperative programming.
• GEMM Implementation: MXNet also leverages optimized GEMM implementations from libraries such as BLAS and cuBLAS.
• Scalability: MXNet is designed for scalability and can be used to train large models on distributed systems.

7.4 How Frameworks Utilize GEMM

Deep learning frameworks utilize GEMM in several ways:

• Layer Implementation: GEMM is used to implement the core computations in fully connected and convolutional layers.
• Optimization: Frameworks automatically optimize GEMM operations by selecting the best implementation and applying various optimization techniques.
• Hardware Acceleration: Frameworks leverage hardware acceleration by using specialized GEMM routines for GPUs and other accelerators.

8. Future Trends in GEMM Research

As deep learning continues to evolve, the demand for more efficient and faster GEMM (General Matrix to Matrix Multiplication) implementations will only increase. Researchers are actively exploring new techniques and approaches to further optimize GEMM performance. This section highlights some of the future trends in GEMM research, encouraging further exploration and learning at WHY.EDU.VN.

8.1 Sparsity

Sparsity is a technique that involves reducing the number of non-zero elements in a matrix.

• Sparse Matrices: By representing matrices as sparse matrices, the amount of computation required for GEMM can be significantly reduced.
• Sparse GEMM: Researchers are developing specialized algorithms and hardware to efficiently perform GEMM on sparse matrices.

8.2 Quantization

Quantization involves reducing the precision of the data used in GEMM.

• Reduced Precision: By using lower precision data types (e.g., 8-bit integers instead of 32-bit floating-point numbers), the memory footprint and computational cost of GEMM can be reduced.
• Quantized GEMM: Researchers are exploring techniques for performing GEMM with quantized data while maintaining accuracy.

8.3 Approximate Computing

Approximate computing involves trading off accuracy for performance.

• Approximate GEMM: By allowing for small errors in the GEMM computation, the performance can be significantly improved.
• Error Tolerance: This is particularly useful for applications where some level of error is tolerable.

8.4 Novel Hardware Architectures

Researchers are also exploring novel hardware architectures for accelerating GEMM.

• In-Memory Computing: In-memory computing involves performing computations directly within the memory, reducing the need to move data between memory and processing units.
• Analog Computing: Analog computing uses analog circuits to perform computations, which can be much faster and more power-efficient than digital computing.

8.5 Potential Future Improvements

Future improvements in GEMM research could lead to:

• Faster Training Times: More efficient GEMM implementations can significantly reduce the time required to train deep learning models.
• Improved Inference Speed: Faster GEMM can also improve the speed of inference, enabling real-time applications.
• Reduced Power Consumption: Lower power GEMM implementations can enable deep learning on mobile and embedded devices.

9. Practical Applications of Optimized GEMM

The optimization of GEMM (General Matrix to Matrix Multiplication) has far-reaching implications across various practical applications, particularly in the field of deep learning. Enhanced GEMM performance translates directly into improved efficiency and capabilities in numerous real-world scenarios. This section explores some of the practical applications of optimized GEMM, encouraging readers to pose further questions and seek deeper insights at WHY.EDU.VN.

9.1 Image Recognition

Image recognition is a fundamental application of deep learning, used in tasks such as object detection, image classification, and facial recognition.

• Faster Processing: Optimized GEMM enables faster processing of images, allowing for real-time object detection and recognition.
• Improved Accuracy: Efficient GEMM can support larger and more complex models, leading to improved accuracy.

9.2 Natural Language Processing (NLP)

NLP involves the processing and understanding of human language, used in applications such as machine translation, sentiment analysis, and chatbots.

• Faster Training: Optimized GEMM can significantly reduce the training time for NLP models.
• Real-Time Translation: Efficient GEMM can enable real-time translation and other NLP tasks.

9.3 Speech Recognition

Speech recognition involves converting spoken language into text, used in applications such as voice assistants and transcription services.

• Improved Responsiveness: Optimized GEMM can improve the responsiveness of speech recognition systems.
• Higher Accuracy: Efficient GEMM can support more complex models, leading to higher accuracy.

9.4 Recommendation Systems

Recommendation systems are used to predict the preferences of users and provide personalized recommendations, used in applications such as e-commerce and streaming services.

• Real-Time Recommendations: Optimized GEMM can enable real-time recommendations based on user behavior.
• Improved Personalization: Efficient GEMM can support more complex models, leading to improved personalization.

9.5 Scientific Computing

Scientific computing involves using computers to solve complex scientific and engineering problems.

• Faster Simulations: Optimized GEMM can speed up scientific simulations, allowing researchers to explore more complex scenarios.
• Improved Accuracy: Efficient GEMM can support larger and more complex models, leading to improved accuracy.

10. Conclusion: Why GEMM Optimization Matters

In conclusion, the optimization of GEMM (General Matrix to Matrix Multiplication) is of paramount importance in the field of deep learning. As the core computational building block for both fully connected and convolutional layers, GEMM directly impacts the performance, efficiency, and scalability of deep learning models. By optimizing GEMM, we can unlock significant improvements in training times, inference speeds, and power consumption, enabling a wide range of applications across various domains. To explore these concepts further and find answers to your specific questions, visit WHY.EDU.VN.

10.1 Key Takeaways

• Fundamental Operation: GEMM is a fundamental operation in deep learning, serving as the core computational building block for both fully connected and convolutional layers.
• Performance Impact: Optimizing GEMM directly impacts the performance, efficiency, and scalability of deep learning models.
• Wide Range of Applications: Improved GEMM performance enables a wide range of applications across various domains, including image recognition, NLP, and scientific computing.

10.2 Future Outlook

As deep learning continues to evolve, the demand for more efficient and faster GEMM implementations will only increase. Researchers are actively exploring new techniques and approaches to further optimize GEMM performance, including sparsity, quantization, approximate computing, and novel hardware architectures.

10.3 Final Thoughts

The journey to optimize GEMM is an ongoing endeavor, driven by the ever-increasing demands of deep learning. By understanding the importance of GEMM and exploring the various optimization techniques, we can unlock the full potential of deep learning and create more powerful and efficient models for a wide range of applications. For more in-depth information and to have your questions answered by experts, visit WHY.EDU.VN.

FAQ: Frequently Asked Questions About GEMM

1. What exactly does GEMM stand for?
GEMM stands for General Matrix to Matrix Multiplication. It’s a fundamental operation in linear algebra that multiplies two matrices together.

2. Why is GEMM so important in deep learning?
GEMM is essential because it forms the backbone of computations in neural networks, especially in fully connected and convolutional layers. Optimizing GEMM directly improves the performance of deep learning models.

3. How does GEMM work in fully connected layers?
In fully connected layers, GEMM is used to multiply the input data (represented as a matrix) by a weight matrix. This operation transforms the input into the output of the layer, allowing the network to learn complex patterns.

4. How is GEMM used in convolutional layers?
Convolutional layers use a transformation called im2col to convert the convolution operation into a matrix multiplication that GEMM can perform efficiently.

5. What are some techniques for optimizing GEMM?
Techniques include leveraging BLAS libraries, loop reordering, tiling, vectorization, and Strassen’s algorithm. These methods improve cache utilization, reduce memory access latency, and speed up computations.

6. What hardware accelerators are used for GEMM?
GPUs (Graphics Processing Units), TPUs (Tensor Processing Units), FPGAs (Field-Programmable Gate Arrays), and ASICs (Application-Specific Integrated Circuits) are used to accelerate GEMM operations.

7. How do deep learning frameworks utilize GEMM?
Frameworks like TensorFlow, PyTorch, and MXNet rely on optimized GEMM implementations to perform core computations in layers. They also automatically optimize GEMM operations using hardware acceleration.

8. What are some future trends in GEMM research?
Future trends include exploring sparsity, quantization, approximate computing, and novel hardware architectures to further optimize GEMM performance.

9. What are the practical applications of optimized GEMM?
Optimized GEMM has applications in image recognition, natural language processing, speech recognition, recommendation systems, and scientific computing, among others.

10. Where can I find more information and get my questions answered about GEMM?
Visit WHY.EDU.VN to explore these concepts further and find answers to your specific questions from our team of experts.

Unlock the full potential of deep learning! Do you have more questions about GEMM or other AI topics? Visit WHY.EDU.VN today and connect with our experts to get the answers you need. Our dedicated team is ready to provide clear, reliable, and in-depth explanations. Contact us at 101 Curiosity Lane, Answer Town, CA 90210, United States. Whatsapp: +1 (213) 555-0101. Website: why.edu.vn.

Deep learning convolutional kernels visualized as RGB images, showing primitive patterns the network looks for.