Replying to Avatar ChipTuner

Curious what you might consider a long software build time? My vnlib projects are starting to creep up on 4-5 minutes on an 8 core vm. Incremental stuff during development is nearly instant so it doesn't matter, but a full publish be hammering my hardware.

Just running all vnlib builds and testing at one time can keep a 64 core machine very busy. If I wasn't running my own hardware I don't think I could afford to run a cloud build server, albeit with slightly newer hardware. Builds would probably take all day on a 2-4 vcore cloud vm.

I've never used GH actions, but do you have to pay for CPU time?
Avatar
Diacone Frost 9mo ago

our sw takes ~1,5h with cold caches, with downloading deps and with parallelization where possible on a recent 32 core machines with 120gb of ram for fs caching 😂

Reply to this note

Please Login to reply.

Discussion

Avatar
Diacone Frost 9mo ago

and thats without any testing whatsover...

Avatar
ChipTuner 9mo ago

Lol good to know. Is this corporate software or something public?

I've spent way too much time trying to parallelize and optimize my pipeline, I'm sure I could tweak some more but, diminishing returns. I already build my own build "orchestration" software to help with this.

https://www.vaughnnugent.com/resources/software/modules/vnbuild

Avatar
Diacone Frost 9mo ago

It's a corporate sw.

with populated chaches it goes to 8-10 minutes when it does basically nothing then assembling the project parts. The average build with real code takes under 20 minutes.

Avatar
Diacone Frost 9mo ago

I'd give gradle a try. C is not really a first class citizen, but it worked surprisgly well even for larger C projects for me. Batteries included (parallelization, compiling avoidance, local and thing remote caching)

Avatar
ChipTuner 9mo ago

Will take a look thanks!

Yeah compiling C is the bulk of the CPU time. The rest is probably in tar+gz and copying files around. My projects are mono repos with anywhere from 4-30 libraries that all have their own "independent" packages. The C# takes a while to but not as bad as the C libraries.