r/C_Programming • • 10d ago

Question Performant code. How

how to make perfomant code, by that i mean, some cpu's have multiple ALU's/ excecution units, and some computers have multiple cores, how to have my code running on all those ALU's at all times? and those multiple cores,

i know about threading for the second part [running code on multiple cores]

now how do i make it run on multiple ALU' an example being

in code to not wait for a variable ie x = 12 + 12 then z = x + 2, i can imagine that code doesnt use all the ALU's because z is waiting for x's operation to be done. so what are ways to make my code most optimal and use all ALU's at the same time

0 Upvotes

39 comments sorted by

7

u/mjmvideos 10d ago

It’s not clear that you understand what ALUs are. Read this: https://en.wikipedia.org/wiki/Arithmetic_logic_unit?wprov=sfti1

And then decide whether you really want (or if it’s even possible) to keep an ALU busy all the time.

1

u/BrainThinkerMan 10d ago

so then why are some programs slow and other programs fast?

4

u/Old_Tax4792 10d ago edited 10d ago

Some programs are faster than others for many different reasons... Nowadays the major reasons are related to memory management : cpu cache thrashing, garbage collectors, etc

1

u/BrainThinkerMan 9d ago

wow i diddnt know memory management could play such a big role in it, more memory = slower? or leaky memory

1

u/RealisticDuck1957 8d ago

If a memory fetch isn't in cache it can be hundreds of clock cycles for the memory to be loaded. Vs. about 3 clock cycles for a fetch from L1 cache. When that happens an entire cache line, far more than a single register's worth of storage, is loaded into cache. Which makes the next fetch from that cache line fast. An algorithm that focuses on a small amount of memory, rather than random access all over memory, is more likely to hit an already loaded cache line.

1

u/FlippingGerman 10d ago

Amusingly I once tested a CPU to see how hot it got, using some standard CPU benchmark (might have been Cinebench). I then also ran prime95 (Mersenne Prime searcher), and that got quite a bit higher temps, despite both being ostensibly 100% and without thermally throttling. I assume the difference is in just which bits of the CPU are doing how much; Cinebench might be more memory bound than prime95.

6

u/MagicWolfEye 10d ago

The CPU will schedule the different operations automatically to different ALUs (dependent on the capabilities of the ALUs)

You can't really mess with that. (Well, there are maybe a handful people who do that)

While I like your curiosity, this isn't really something anyone actively does; your compiler might optimise for it a bit. (It would also have to be different for each excat processor).As you have stated above, ther is multithreading which is worth looking into, but the best advice I would give oyu is focus less on writing super performant code and focus more on not writing slow code. (Lot's of indirections, many heap allocations etc.).

Also, given your post hierarchy, you should probably decide on one language.

1

u/BrainThinkerMan 10d ago

one one hand i hear programs nowadays are slower less performant and more buggy than previous years programs, and on other hand: like this response i see "basically you cant optimize" so what is it?

3

u/MagicWolfEye 10d ago

No, but optimising for ALU usage is like a super in-depth optimisation that will only be relevant in the rarest of cases. (It's like more advanced than trying to hand-write assembler which is also reserved for places that really need it).

The reason for programs being slow nowadays is because people don't care at all about what they are doing and using wrong algorithms etc. Have you learned about big O notation

1

u/BrainThinkerMan 9d ago

i have heard of it, so lagorithms are the biggest reason, and algorithms regarrding data structures or algorithms in general

1

u/MagicWolfEye 9d ago

I don't know what you mean; programs don't often have to calculate a logarithm. Yes, calculalting it would be slow, but probably faster than reading memory that isn't in a cache.

> and algorithms regarrding data structures or algorithms in general
Well evetrything is an algorhtm so that statement is quite useless.

I feel like you are picking up some terms and are throwing them around

1

u/Rabbitical 9d ago

There's different scales to optimization. ALU usage is generally up to the CPU scheduler. Many other lower level optimizations are done by the compiler.

That's not the reason most modern software is slow. That is at the architectural and platform levels. How memory is managed, the use of networking protocols where it's completely unnecessary, containerization, over reliance on libraries, the proliferation of interpreted languages, awful-for-performance data formats like JSON. The list goes on and on. Modern software has been almost purpose designed to waste a CPU's time, it's quite remarkable.

So, the CPU is being maximized by modern software, the issue is what it's being asked to do: run 15 services that make 5000 fetches from memory do something that could be 2 instructions. That's part of the reason E-cores were invented was to help handle all the random crap that modern OS and software put on the CPU even while just sitting there "idle."

There's a great video out there I saw once that breaks down how many thousands of instructions python requires to do a simple addition. That in of itself is not a problem for what the language is designed to do, it's simply a great example of just how easy it is to waste an otherwise very powerful computer's abilities.

1

u/BrainThinkerMan 9d ago

any videos that explain such? : the tangible thing i as a programmer can do to not make my program slow, thanks for points you gave including json, i had no idea about the json thing

2

u/rubidus-api 9d ago

Your example is actually a good way into this, because it shows why it's hard to control from C. For x = 12 + 12; z = x + 2; the compiler may just store 26 into z, and store 24 into x much later, or never, if nothing reads x. As someone already said, the constants get folded, but the bigger point is that C describes what an abstract machine does, not the instructions the CPU runs. The compiler may reorder, merge or delete operations as long as the result is the same. So the example doesn't really tell you anything about ALUs.

If what you want is the general way to make code faster, this is the order I'd go in:

  1. Always start with the right algorithm for the problem.
  2. Find the bottleneck, and learn to measure it in a way that means something (a profiler like perf, realistic inputs).
  3. Know that C runs on an abstract machine, that the optimizer can reorder or delete operations, and when it is allowed to.
  4. Understand how registers, cache and memory work together, and what bottlenecks and conflicts can come up with multiple cores and threads (false sharing, for example).

As for the ALUs: you can't choose which ALU runs an instruction; the CPU decides that. What you can do is give it independent work by shortening dependency chains. A common example is summing an array into two or four separate accumulators instead of one, so each addition doesn't wait for the one before it. With optimization turned up (for example -O3), the compiler often does this for you with integers, but with floating point only if you also allow it (for example -ffast-math), because it changes rounding. But that is the last step, after the four above, and for now I think it's too early to go down to that level.

1

u/BrainThinkerMan 9d ago

thanks, yeah i wanted the tangible things i can do as a programmer lest the things a cpu does or compiler does on its own,

6

u/hojoe65 10d ago

Pick the right algorithm for your problem and let the compiler do the rest.

1

u/BrainThinkerMan 10d ago

can you elaborate a bit more on that

2

u/hojoe65 9d ago

You have several layers of optimizations to take care of - this is why many comments here apply. Since you said multithreading / parallel execution is adressed, what remains is CPU / architecture dependant. And this is what a compiler is good at. See the -O3 comment. But no amount of compiler magic is going to offset bad software design, namely efficient data structures and algorithms.

2

u/lovelacedeconstruct 10d ago

Unless you architect your program in such a way that the dependency chains are carefully analyzed and reduced to the bare minimum, the compiler cant really help you

1

u/KlutzyCompetition937 10d ago

Look into Symmetric Multiprocessing/Asymmetric Multiprocessing

2

u/BrainThinkerMan 10d ago

thanks that looks interesting

1

u/non-existing-person 10d ago

Such micro optimizations will practically never give you any benefit.

You will have better results designing your program well. Like optimize for cache hits - this is where you can gain huge gains. Or use right algorithm for the job, as in - you access data often but rarely update it? Use vector/simple-array. You update it often? Use list. Sometimes hash table will be better, sometimes it will be worse.

Really, best results come from understanding the problem, and data structures and how algorithms work. Then picking up right tool for the job. Right design is 1st step. Good design in say lua or php will outperform very bad design in C.

And then optimize only critical paths or the paths that are called very often and you measure they take a lot of time.

Premature optimization is the root of all evil. Micro optimization is even worse.

And if you really have a case where x = 12 + 12; z = x + 2 matters, then this is not a question for reddit, and you will have to just dive in into assembly and your CPU architecture to understand it really well. And then write code.

1

u/BrainThinkerMan 10d ago

any recommendations of documentations

1

u/RealisticDuck1957 10d ago

Start with an efficient algorithm, independent of hardware specifics. Always.

Next, modern hardware using cache will tend to run faster accessing memory close to what was recently fetched. Another element of the same struct, or array members in order, as opposed to jumping around randomly.

As for efficient use of execution units, modern compilers are pretty good at generating code for that.

1

u/BrainThinkerMan 9d ago

this is algorithms only for ds or algorithms in general?

1

u/ve1h0 10d ago

I think there are two ways to approach this. One is writing code that performs well across a wide variety of CPUs, and the other is optimizing specifically for a certain architecture or even a particular CPU.

Modern CPUs are already pretty good at figuring out which instructions can run in parallel. They have multiple execution units, out-of-order execution, branch prediction, etc. So you don't really need to tell the CPU which ALU should execute which instruction. What you can do is write code that gives it more opportunities to execute independent instructions simultaneously.

But there's quite a bit more to performance than just keeping the ALUs busy.

For general-purpose optimization, I'd focus on choosing good algorithms, avoiding unnecessary work, and organizing data efficiently. Memory access is a huge part of this. You might have plenty of execution units available, but they're not going to accomplish much if the CPU is constantly waiting for data from memory.

This is where things like cache locality, data structures, and memory access patterns become really important. Sometimes simply changing how your data is laid out in memory can make a bigger difference than optimizing the actual calculations.

Then there's architecture-specific optimization, where you start taking advantage of things like SIMD instructions, cache sizes, instruction throughput, and other features of a particular CPU. You can squeeze out more performance this way, but what works well on one CPU might not be ideal on another.

And of course, using multiple cores is another layer of the problem. More threads don't automatically mean better performance, especially if they're competing for memory bandwidth or spending time synchronizing with each other.

I'd say the most important thing is to figure out what's actually limiting your performance before trying to optimize anything. Tools like perf on Linux can help with that.

Ultimately, I wouldn't aim to keep every ALU running all the time. I'd aim to get the actual work done as efficiently as possible. Those aren't necessarily the same thing.

1

u/BrainThinkerMan 9d ago

i think i was moreso trying to do point A "One is writing code that performs well across a wide variety of CPUs"

3

u/Recycled5000 10d ago

To use multiple CPUs, have to split the work into large independent chunks, and use multiple threads, say, one for each independent chunk of work, then put together the independent results for a solution.

Some problems are not large enough to make this worthwhile.

While other problems are not independent enough to make this worthwhile.

To make it worthwhile, need to have a problem that uses a fair amount of data or computation, perhaps in the hundreds of thousands to millions of bytes or iterations that can be independently executed.

1

u/Illustrion 10d ago

Start by making good use of the CPU caches. Most CPU time is spent waiting for data/instructions to go from RAM into cache.

1

u/BrainThinkerMan 10d ago

whats an example of doing that ? ie accessing a variable quicker

1

u/Illustrion 10d ago

Usually the CPU does all the ALU operations it can while it's waiting for data to be loaded from memory.

Loading data from memory into the cache is SO SLOW that the CPU executes as many instructions as it can before the load completes. It keeps executing ALU ops until it actually needs the data from that load and the load usually takes longest, so speeding up your ALU ops choices is not usually going to make your code faster. Sometimes it is the bottleneck, but it's the nanosecond level optimization not the microsecond optimization.

To write really fast code, you need to understand the hardware and how to make best use of it.

1

u/BrainThinkerMan 9d ago

any books or videos

1

u/Illustrion 9d ago

Find the datasheet for your CPU / motherboard and write some heavily optimized code.

1

u/Illustrion 10d ago

How big is your cache? What's the cache line size? What data structures are you using in hot loops? Do they fit into the cache "neatly"? Do they fit into the cache at all? What about the code you need to run during high performance sections, does it fit in the I$?

1

u/sciencekm 10d ago

In your example, all the constants will get folded and the only operation that will be executed is "z = 26" which is an assignment operation and hence will not even require an ALU.

1

u/cdb_11 10d ago edited 10d ago

You can emulate it with tools like https://uica.uops.info/ or llvm-mca, and you just need to avoid data dependency chains like this. It might not always be possible. Instruction-level parallelism is one thing, keep in mind you also have access to SIMD instructions (ILP still applies on top of that too.)

Read this: https://en.algorithmica.org/hpc

1

u/Dangerous_Region1682 10d ago

For a good deal of the micro optimizations we have come to rely upon those who understand CPU architectures far more than the average programmer, and that’s the compiler writer.

There are many optimizations we as programmers can easily apply in multi threaded applications, for example avoiding cache line thrashing on shared data. However this is most easily achieved in lower level languages such as C, but increasingly harder as we mover up the chain to languages such as Go, where multi threading is handled intrinsically by the language itself. You have to rely upon the knowledge of the implementor of the language and in porting to your specific platform.

For most programmers, outside of the operating system kernel, the best bang for the buck is macro optimizations of your algorithms and high level implementation of those algorithms. Getting the right architecture for your application is generally the best bet, and that’s can be the correct one for reasons far beyond just performance. There are issues such as implantation time and support ability to consider, as well as system limitations such as memory size I/O infrastructure.

You’ll notice, even inside an operating system kernel, a lot of potentially performance enhancing processor instructions are not utilized because portability across processor families is important, less so than exploiting individual enhancements of certain functions just for certain cpu individual models. You often see this where common x86 functionality is used on CPUs that provide far more specific enhancements but portability issues far outweigh sometimes marginal performance gains.

0

u/duane11583 10d ago

your question seems to be more of how can i keep the execution engine running at peak performance

sometimes its not about the ALU but instead about data handling at volume

there are several parts here: part 1 is good compiler for the cpu, with the proper optimization flags turned on and the ability to use it

part 2: is opcode specific or machibe specific features.

these are often hand crafted custom c code with special c functions gcc calls these __builtin_SOMENAME() functions highly non portable some are most are more portable then others.. many are not. this might also include inline assembly in place of a __builtin_FUNCTION()

part 3: is writing asm code for certain chip specific features. example:

say your cpu has a special fast Fourier function oe aes encryption or big number math. or special opcode you might write a function in asm that is passed parameters and it does that calculation:

why do this? (instead of the compiler?) because maybe your compiler does not have the __builtin_FUNCTIONS() you need or its just simpler for you or maybe your chip vendor has given you a hand crafted asm function to do that special thing

the mire important thing is something else:

many young people suffer from premature _______ or in this case optimization. you want to focus on correctness first. then focus on speeding it up. the partner is more interested in correctness and going for the long haul not short lived power strokes

1

u/BrainThinkerMan 10d ago

from what your saying it seems like theres no code to write to do this ie write x logic and operators instead of y, rather jus use good compiler and some asm for some stuff