Performant Spectral Graph Partitioning in Python?
<p>I'm working on partitioning a sparse graph I have, and while I'm happy with the results I've seen, it's currently far too slow. The graph has around 200,000 nodes and scipy.sparse.linalg.eigsh takes on the order of hours. I've tried using shift-invert mode and various initializations, as well as a linear spectral shift, but it still takes many hours to run on my i7-4770k. Are there any tricks for initialization or alternate algorithms that are faster for sparse graphs of certain structure?</p>

<p>Part of my hope for this improvement on performance is that nVidia has benchmarks for some of their GPU approaches for similar graphs running in the order of seconds: 
<a href="https://devblogs.nvidia.com/parallelforall/fast-spectral-graph-partitioning-gpus/" rel="nofollow">https://devblogs.nvidia.com/parallelforall/fast-spectral-graph-partitioning-gpus/</a></p>

<p>Of course that's on a GPU and not on a CPU like I have now, but that seems like a massive gap for something that's not especially parallelizable (matrix-vector products). Maybe that's all it is, though - I'm trying to avoid adding a GPU dependency for the moment, but I can if that's what it takes.</p>

<p>I found scipy.sparse.linalg.lobpcg to give much better results relatively quickly, so if anyone else is trying this in the future I suggest starting there. I also used a <a href="https://en.wikipedia.org/wiki/Preconditioner#Jacobi_.28or_diagonal.29_preconditioner" rel="nofollow">Jacobi diagonal preconditioner</a> based on my matrix, which I would recommend, though I didn't try it without.</p>

python algorithm performance linear-algebra linear