Tussling with vcpkg

Collecting and installing all the dependencies for a codebase worth a million lines is tough and using C++ doesn't help either. Unlike Python or  JavaScript, C++ doesn't have a de-facto package manager, even after existing for almost 3 decades. I didn't count, but I am sure the number of direct dependencies Krita has, would be something between 20 and 30. And those dependencies mostly, also have dependencies of their own.

Why is this a problem now? Krita has existed for over a couple of decades and it doesn't seem that the project would be dead soon. Everyday some new face is hacking on it. Thankfully, that "some" is growing day by day kudos to our discourse forum which include mostly artists. Open source gives those artists the chance to mould their software of choice according to them and why would they miss it? But there is a problem, everyone has a workflow and any changes in their workflow would definitely hurt their productivity.

We don't want to discourage people from hacking on Krita. And building Krita on Windows is no joy. And most of the time if you don't know what you are doing, it could cost you dearly. Though thanks to Agata's scripts it is pretty easy to start hacking on Krita on Windows but that too has some caveats. Everyone's setups differ and those scripts are susceptible those differences if someone fails to make appropriate changes to them.

My recent switch to Windows has poked me with the same problem, thankfully building Krita in my case was straight forward since I didn't have many things to change, can't say the same for most though.  What is the solution? Easy, use a package manager, but which one? C++ has a bunch of package managers and none of them are great. Last summer Dmitry tried to play with conan, one of the popular ones but then again it didn't work out quite well. I didn't like conan for the most part, it requires you mess with the project CMake files which I don't like to. The only other option was vcpkg, cause in terms of proper documentation and resources the others sucked bad compared to these two.

Counting with my fingers vcpkg lacks around 15 of the direct dependencies of Krita and porting them is a no easy task, at least when you are on Windows. The few caveats of using vcpkg were,

• No proper versioning support, you need to checkout specific commits to get the right versions.
• No pre-built binaries, every library would be compiled on your machine.
• No server where you can host the libraries you built.
• Leaking targets? The approach would probably leak targets, but I am not sure though, not a CMake guru by any means.
• It only supports CMake.
• MinGW support is flaky as well as dynamic builds in Linux.

And here are some of the pros,

• Pretty easy to hack on the packages and the package manager itself, you are encouraged to do so.
• Since it only supports CMake, it supports it very well compared to conan.
• You don't need to touch your CMake files.

Ouch, seems like a bad package manager, but the prospect of hacking on it makes it worth giving a shot, in my opinion.

Note: I wasn't aware of most of the downfalls of vcpkg, before trying it out.

Starting, the first challenge was to learn how to package libraries for it, that was easy cause all it required was a CONTROL file and a portfile.cmake. The CONTROL file just describes the package and it's build dependencies but it has a bunch more parameters to help categorize packages easily.

The portfile.cmake is the one that fetches the source code, determines its integrity, and then goes ahead to build & install it. Thankfully vcpkg provides some prebaked functions which take care of the heavy lifting. Anyway, still, I had no idea how to port stuff onto it. So, the first step was to look for some examples. And fortunately, some good people already tried to port the Kate dependencies to vcpkg which but they couldn't make it to the upstream. So as a starting point I just utilized their work and went on fixing the stuff that didn't work. Within a couple of hours, I was able to port the required KDE Frameworks dependencies at least on Windows 10. Mostly it was trial and error as the error messages didn't make much sense to me.

For the reference, I am talking about this[1][2] 2 PRs. By the end of the night, I almost had all the mandatory dependencies, except Quazip, I don't what is the problem with this library but it seems like it doesn't like Windows at all. And Windows isn't my piece of cake after all, but on the other hand, Linux is. It was a pleasure to port the packages on Linux and now that I was a bit more familiar with the errors, I was able to port most of the dependencies, with a bunch of hiccups in the middle thanks to this bug.

Till, I am successful, time to check whether this was a facade or not. Turns out this was a facade, Krita doesn't like static libraries, ohh, I just need to rebuild the libraries dynamically, not a big deal. Boy, I was wrong, vcpkg on Linux could do dynamic builds but it doesn't that officially. Bugs with the ability to induce headaches start to pop out one by one, this being the first one. What do you do when you find a bug in open-source software, you fix it. Patching starts, first with an environment variable workaround for the library path, then onto the portfile.cmake, downloaded source code, and then on the installed CMake configs. This can't be worse, right? Unfortunately more was yet to come.  The folks who packaged OpenColorIO decided that it should use the dependencies from the system instead of the ones which came with it. It doesn't stop here, turns out OpenColorIO only builds with C++98, come-on who still uses std::auto_ptr, and it doesn't even like ninja either. Fuck my life!!

Patching like these went on for a couple of days, though it did end on the high note. And I also upstreamed a lot of those changes, ending with 4 different PRs. One is closed since they don't support dynamic builds on Linux officially so this is not required for now. One is approved, another requires a bit of discussion and the last one requires some changes from my side.

Some notes,

• Not all deps could be packaged, Poppler and SIP are missing, if anyone can help, feel free to drop a message to me.
• lcms2, fftw3 and a couple other which I am not able to recall have different target names, have to manually point them while configuring.
• Quazip on windows is broken, it builds fine but then while compiling Krita I get a corrupted dll error, it is fine Linux though.
• The system libraries needed to build qt properly is not listed properly, it requires much more than said.
• The prl parser was backported from Qt 5.14 to Qt 5.12 so it doesn't work properly either.
• GCC assumes a lot of things which MSVC doesn't, leading to loads of errors while compiling Krita with the latter.

And I think that will be it. Hopefully, sometime in the coming months, we could build Krita in Windows with MSVC and vcpkg.