100k loc without a reasonable architecture is… problematic.
I have run into problems with C++ linker/optimizers which will choke on big (50k loc) .cpp files which are auto-generated from definitions, they get slow like that, build times around 20 minutes for the worst of them. Keep the files more like 10k loc and down if you don’t want this kind of issue.
Nowhere did I say there’s no reasonable architecture. Please don’t put words in my mouth. What I said, is that I haven’t found a good way to get quick compiles end to end. And already I don’t have any files more than 3-4k loc, most are under 1k.
Yeah I doubt it. I’ve worked in large Rust codebases too. I’d do some benchmarking. Cargo has some nice flags that let you see exactly what it’s doing. You can open the compilation report up in a browser and zoom in on different steps of the process and see everything.
Make sure you’re using a fast linker. Although I think lld is the default now, so that should be less of a concern these days. But see if mold or wild would help.
More relevantly, you should be using cranelift for codegen in dev builds.
If you have codegen-units = 1 in your release profile, make sure you have a separate dev profile that doesn’t.
If you still experience decapitating slowness (doubtful), and it’s actually the rust compiler being slow (super doubtful), you can try the parallel frontend. Apparently there are almost™ no bugs left in its implementation (It’s one of the project priorities now).
Needless to say, but you should be checking your --timings to really know what’s going on.
It would be a cool if you report back in a separate post if you discovered anything interesting. It’s not like we’re overwhelmed by too much activity here.
Also, while I think it should be obvious, but I forgot to clarify that if you’re inheriting from a profile with codegen-units = 1, then you need to set that explicitly to a higher value in the inheriting profile.
No, I’m not using --release every time of course. And sometimes it’s minutes even with the cache.
That’s highly abnormal to have minute long debug build times.
for 100k loc?
100k loc without a reasonable architecture is… problematic.
I have run into problems with C++ linker/optimizers which will choke on big (50k loc) .cpp files which are auto-generated from definitions, they get slow like that, build times around 20 minutes for the worst of them. Keep the files more like 10k loc and down if you don’t want this kind of issue.
Nowhere did I say there’s no reasonable architecture. Please don’t put words in my mouth. What I said, is that I haven’t found a good way to get quick compiles end to end. And already I don’t have any files more than 3-4k loc, most are under 1k.
Yeah I doubt it. I’ve worked in large Rust codebases too. I’d do some benchmarking. Cargo has some nice flags that let you see exactly what it’s doing. You can open the compilation report up in a browser and zoom in on different steps of the process and see everything.
I’ve tried looking before and couldn’t get a significant improvement. I’ll have to investigate it again I guess.
Make sure you’re using a fast linker. Although I think
lldis the default now, so that should be less of a concern these days. But see ifmoldorwildwould help.More relevantly, you should be using cranelift for codegen in dev builds.
If you have
codegen-units = 1in yourreleaseprofile, make sure you have a separate dev profile that doesn’t.If you still experience decapitating slowness (doubtful), and it’s actually the rust compiler being slow (super doubtful), you can try the parallel frontend. Apparently there are almost™ no bugs left in its implementation (It’s one of the project priorities now).
Needless to say, but you should be checking your
--timingsto really know what’s going on.thanks will see what happens
It would be a cool if you report back in a separate post if you discovered anything interesting. It’s not like we’re overwhelmed by too much activity here.
Also, while I think it should be obvious, but I forgot to clarify that if you’re inheriting from a profile with
codegen-units = 1, then you need to set that explicitly to a higher value in the inheriting profile.The various linkers do make a difference, but all of them choke, eventually, when individual files get too massive.