Effect of Entanglement on Model Training

This blog post is written as part of the QOSF(Quanutm Open Source Foundation) Mentorship Program.

Student: Syed Farhan Ahmad
Mentor: Amira Abbas

Project Description

Classical neural networks encode higher dimensional vectors(inputs) to lower dimensional vectors(outputs), but the reverse is not possible. Recent research has shown that scrambling of information from a smaller subsystem to a larger one is feasible. In our QOSF project, we analyse the effect of entanglement as a variational circuit trains and we also study the role of various entropies to characterize entanglement.

Entropy to quantify entanglement

Research has shown that entropy(von-Neumann entropy) can be used to quantify entanglement in a quantum circuit. We extended our work on this by studying various entropies and choosing the most relevant ones that impact entanglement in a circuit:

• von-Neumann entropy
• Meyer-Wallach entropy

Entanglement Metrics Used

von-Neumann Entropy measure

One of the most studied and frequently used entropy functions is the von Neumann entropy, which is defined as follows:

For a density operator $\rho\in\mathcal{D}(\mathcal{H})$ von-Neumann entropy is:

$S(\rho)=-\mathrm{Tr}(\rho \log\rho)$

This entropy is a quantum generalization of the classical Shannon entropy. If \(\{p_i\}_i\) are the eigenvalues of a density operator , then the von Neumann entropy equals the Shannon entropy of a random variable X(\(\rho\)) with same probability distribution :

Meyer Wallach Entropy measure

It is a single scalar measure of pure-state entanglement. The Meyer–Wallach (MW) measure written in the Brennen form is:

where $\rho_k$ is the one-qubit reduced density matrix of the kth qubit after tracing out the rest.

Implementation: Meyer Wallach entanglement measure is implemented in libraries/meyer_wallach_measure.py.

Toy Model under study

The Quantum Circuit automatically learns the parameters to generate a Bell State. We have used a very simple parameterized circuit and have manually coded an SGD optimizer with MSE Cost function to analyse the effect of entanglement as the model trains and optimizes its parameter to return to the bell state.

Variational Quantum Classifier Setup

Specifications

• Quantum Embedding: ZZFeatureMap
• Quantum Variational Circuit: RealAmplitudes
• Optimizer: ADAM
• Cost Function: Sigmoid estimation cost function

  Quantum Circuit

Datsets used

• Ad Hoc dataset from qiskit.ml.datasets.
• Synthetic data generated from sklearn’s make_blobs() with varying standard deviations in the data.

Results & Observations

Toy Model(Parameterized bell state with custom optimizer)

• Let’s begin by plotting the Loss function vs the number of epochs and expect a quadratic graph.
  
• The von-Neumann entropy increases as the circuit trains, reaches it’s peak and then decreases again.
  
• Observing from right to left, we see that the von-Neumann entropy increases as the Loss function decreses, which is expected from von-Neumann entropy’s equation since we are approaching a pure state from a mixed state.
  

  Therefore, we can conclude that the von-entropy measure under consideration behaves as expected(after testing on the Toy Model) and we are now ready to test it on our actual VQC experiment.

Variational Quantum Classifier

• Let’s again begin by plotting the Loss function vs the number of epochs and expect a step-like plot(since the Sigmoid estimation loss was used).
  
• von-Neumann entropy follows a pattern as the model trains and the peak values in the plot gradually decrease. Further analysis is required to understand the correlation.
  
• Meyer Wallach measure decreases exponentially as the model trains for both Ad-Hoc and synthetic datasets.
  
• This follws the same trend, i.e, von-Neumann entropy’s peaks slighly decrese as the cost values decrease(from right to left).
  
• We see that Meyer Wallach measure decreses as the cost function decreases(from right to left).
  

Code

The code has been open sourced and can be accessed on GitHub.

Conclusion

• Based on the results obtained, we can safely conclude that Meyer Wallach entanglement measure decreases as the model trains and as the cost function’s value decreases.
• We can also find a pattern in the von-Neumann entropy measure for every 10 epochs where the entropy value peaks at almost 10% and these peaks decrease as the cost function decreases.
• We are looking for more insights from the controlled synthetic data that is being generated. Please check back soon for updates!

Congrats👏! You have finally completed reading the blog. Stay tuned for more updates on this project as this isn’t the end!

References

1. Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms, Sukin Sim, Peter D. Johnson, Alan Aspuru-Guzik [Link]
2. Information Scrambling in Quantum Neural Networks, Huitao Shen, Pengfei Zhang, Yi-Zhuang You, Hui Zhai [Link]
3. An observable measure of entanglement for pure states of multi-qubit systems, Gavin K. Brennen [Link]
4. A characterization of global entanglement, Peter J. Love, Alec Maassen van den Brink, A. Yu. Smirnov, M. H. S. Amin, M. Grajcar, E. Il’ichev, A. Izmalkov, A. M. Zagoskin [Link]

Acknowledgements

I would like to thank my mentor Amira Abbas for her constant support and guidance without who this project wouldn’t have taken shape. I would also like to thank the QOSF team for giving me this fantastic opportunity of being a part of an awesome mentorship program.

If you’re interested in following me on my journey, connect with me on Linkedin or follow me on Twitter.
For any queries, feel free to contact me on Linkedin, on twitter or by email.