I would like to know how would the igraph community feel about dropping vendored library dependencies from the igraph source tree.
Right now igraph depends on lots of third-party libraries, some of which are essential (and their source is vendored in the igraph source tree), while others provide optional extra functionality that gets compiled only if the corresponding third-party library is detected at compilation time. An example of a vendored library is CXSparse, which is used for sparse matrix operations. The igraph source tree also contains a vendored copy of BLAS / LAPACK / ARPACK, converted from Fortran using f2c (which also means that we need to vendor f2c itself). This is getting harder and harder to maintain, and since most people use igraph from higher-level interfaces where a precompiled package is already provided (this is the case for Python for sure), I’m leaning more and more towards saying that we should drop the vendored libraries from the source tree and rely on the user to install them before compiling igraph. This is not too complicated on Linux systems as the third-party dependencies are packaged in all major Linux distributions; however, it could be a problem for Windows, and I have zero experience in C programming in Windows so I have no idea whether it is indeed a huge barrier on Windows or not.
Perhaps this could be done gradually. We do not need to unbundle all these libraries at the same time.
I would suggest starting with those which are easy to build on Windows. It was once suggested to me to look at vcpkg for easy access to C/C++ libraries on Windows. vcpkg works on Mac/Linux as well, so it is easy to try even for those of us who do not use Windows. It is centred around cmake, which is a plus if igraph migrates to cmake too.
I just installed it, and searched for some of igraph’s dependencies. It has OpenBLAS, clapack, suitesparse, gmp/mpir. It does not have GLPK and I could not find ARPACK either, although I may have been searching for that incorrectly.
Indeed, support on Windows is probably most difficult to guarantee. Most proper package management systems on *nix will be able to provide most of the packages, but until Windows has a proper package management system, this will remain a challenge. In addition to vcpkg, it seems that Microsoft is busy working on winget, which might be a good possibility in the future. I agree that when the switch to CMake is done, it would be nice to integrate igraph in vcpkg. However, vcpkg does not provide binaries, but only source code, as far as I understand. As far as CI-requirements go, we would then have to make sure to setup some caching of build packages.
However, I think the most important question is: how can we guarantee proper support for the higher-level interfaces using external libraries? Preferably, almost nobody should ever have to compile igraph themselves, especially not for the higher-level interfaces, unless they want to do development or have some special architecture.
For Python we can provide binary wheels, and we should be able to install the dependencies using the Anaconda supplied binary libraries. Anaconda provides (in the conda-forge channel) GLPK, suitesparse, openBLAS, LAPACK and ARPACK. It also provides libxml2. All these packages are supported on all major platforms: linux, macOS and Windows (although some are limited to 64-bits only). I am quite sure that we would be able to link against those libraries and simply provide binary Python wheels for all major platforms. When building igraph and python-igraph itself in conda-forge, we would be able to simply link against these libraries.
I don’t know enough about the R interface and compilation, but I think it should be easier than for Python, because it is using mingw, so it should make it easier to provide all necessary packages.
For Mathematica @szhorvat, you are probably best placed to comment. Does it need to be compiled with MSVC? That would complicate things. If it can be build using mingw I think it should be easier.
I think most CI systems support some kind of a build cache so this shouldn’t be a problem. But it’s good to be aware of.
As for the C core on Windows – maybe we could also provide pre-compiled DLLs and LIBs for the Windows platform so people who want to start with igraph development on Windows in C would not have to get and compile all the dependencies themselves. Is it feasible on Windows? I don’t have enough experience with development on Windows to judge that.
Yes, I think we should also try to do that. To be clear, with “development” I meant contribute to the igraph core themselves. I assume almost anybody who actually does that would use Linux/macOS anyway, so they should have less problems compiling against external libraries.
There is one possible problem though and that is possible linking problems due to version differences. In package systems this should be nicely addressed, but with pre-compiled packages we should then use static-linking, and different libraries can then not be mixed.
By the way, we may find that some of our tests give different result on different versions of the external packages, so that is probably also something that we should pay attention to.