Threads spawned in Graph init

Usage
1

Calling Graph() spawns 11 threads.
In order to avoid spawning these threads, I found that calling GraphBase() does not spawn them.

Should I expect any degradation in performance if I use GraphBase rather than Graph? The only APIs I’m using are add_edges, add_vertices, and topological_sorting`

Thanks

In [1]: import psutil
   ...: p = psutil.Process()

In [2]: p.num_threads()
Out[2]: 2

In [3]: from igraph import Graph, GraphBase

In [4]: g = GraphBase()

In [5]: p.num_threads()
Out[5]: 2

In [6]: g = Graph()

In [7]: p.num_threads()
Out[7]: 13
2

Why are you concerned about the number of threads in the Python process?

I believe these are due to loading of external libraries and their initialization.

The normal way to use igraph is to operate with Graph(), just as the documentation states. GraphBase is essentially an implementation detail.

We might be able to give a better answer if you ask about the actual problem you are having, in the spirit of https://xyproblem.info/ Are you looking to control the number of active threads in the few parallelized igraph functions? Note that that is not the same as what you see with p.num_threads().

3

@szhorvat , Thanks for the quick response.

It was raised as a concern in an internal review, and it got me wondering. I tend to agree that the effect if negligible.

Investigating a bit further, it seems like the extra threads are coming from numpy.

In [1]: import psutil

In [2]: p = psutil.Process()

In [3]: p.num_threads()
Out[3]: 2

In [4]: from numpy import ndarray, matrix

In [5]: p.num_threads()
Out[5]: 13
1 Like
4

Most of igraph is not parallelized yet.

The exceptions are:

  • PageRank with the PRPACK method uses OpenMP, and so does the power law fitting. OpenMP must be enabled at compilation time, which will not be the case for all binary distributions. As I recall, it is enabled for the PyPI binaries. You can control the number of threads with the OMP_NUM_THREADS environment variable.
  • The BLAS implementation that igraph is linked to may be parallelized. How to control it differs between BLAS implementations. The python-igraph on PyPI does not currently use a parallelized BLAS.
5

A post was split to a new topic: How to tell if igraph was compiled with OpenMP?