Goal
The primary intellectual merit of this proposal is to enhance learning algorithms for complex deep learning systems by accelerating the training process and improving accuracy. To achieve this, we will apply Quantum Annealing and Classical methods to Deep Boltzmann Machines, showcasing the effectiveness of quantum-based approaches on two particularly challenging machine learning tasks. First, we will tackle seizure detection from EEG signals, which requires mastering intricate temporal and spatial dependencies. Second, we will focus on the automatic interpretation of digital pathology images, involving the analysis of vast high-resolution images (50K x 50K pixels) through the integration of local and global dependencies. By addressing these tasks, we aim to demonstrate significant advancements in the capabilities of machine learning systems.
Prior Research and Findings
In our preliminary work, we compared classical Gibbs sampling and simulated annealing (SA) with D-Wave QA, revealing the presence of unfamiliar states in the configuration space. These states exhibit a high probability of meeting certain training criteria, yet they remain difficult to sample classically. Our findings indicate that QA can rapidly uncover many LVs that are often overlooked by prolonged classical searches, including those associated with high probabilities and significant barriers to escape. This ability to identify such LVs presents a promising opportunity to enhance deep learning systems.
The Importance of Basin of Attraction
The existence of a large Basin of Attraction (BoA) is crucial for classical SA to effectively locate specific LVs. In our investigations, we noted that thermal jumps coexist with quantum tunneling in QA due to finite operating temperatures. Interestingly, the absence of a large BoA does not hinder QA's ability to find LVs. This realization is particularly important, as it highlights the potential of QA to reveal significant LVs that have previously gone unnoticed. While the formation of spurious LVs during training is a recognized phenomenon, our discovery of many important LVs remains a surprising and critical aspect of our research goals.
Advantages of QA-Based Sampling
Our preliminary results suggest that QA-based sampling avoids several key limitations faced by classical techniques. By enriching the sample from the model distribution with essential regions of the configuration space, QA captures rare events—what we refer to as "needles in a haystack." These events, which convey significant information during training, are often missed by classical methods. The ability of QA to target these rare events provides a unique advantage that could lead to more robust model training.