1) Quantized neural networks to reduce memory footprint
2) Equilibrium propagation for analog hardware
3) Culture evolution method to reduce sync and comm wrt SGD
Read more at: https://syncedreview.com/2018/07/31/synchronized-sgd-outdated-yoshua-bengio-on-hardware-friendly-dl/