I have a large number of graphs—up to millions of multigraphs.
The issue is that they consume too much memory, which likely causes slowdowns.
I noticed that if I save only the edge lists instead of full graphs using EdgeTaggedGraph, the memory usage is significantly reduced.
However, the inconvenience of this method is that every time I need to work with a graph, I first have to convert the edge list into a graph, perform operations, and then convert the result back into an edge list for output.
I’m wondering if there is any function in IGraphM that allows for more memory-efficient storage of multigraphs and provides a fast way to encode or decode between formats?
Thank you. I see the ‘New Topic’ button now — I was probably looking in the wrong place.
IGraph/M won’t currently help with this as it is designed to work with Mathematica’s native Graph expressions. I do have plans to provide direct access to the internal native igraph data format, but note that the memory usage is still larger than that of a basic edge list, and implementing this is some time away still.
The short answer is that IGraph/M won’t help with this for the time being.
Thank you for the info.
I just came across the function IGIndexEdgeList mentioned in the documentation, and I see that it returns a packed array and is faster than EdgeList. So it seems like it could work in my case, where I need to save each graph out of millions, by using IGIndexEdgeList instead of EdgeList.
Is this what you meant by the internal native graph format, or is that something different?
(But I now see an issue—it cannot save the edge labels.)