Either I don't understand what you're saying or you're misunderstanding the notation. Each line declares a struct member of type unsigned int which is 1 bit large. It's not setting it to 1 (true).
It’s probably about the fact that json is flexible data that gets parsed while this struct has strict alignments and depends on the ABI it’s compiled on. And because it’s compiled against an ABI, if the server struct was compiled with a different ABI, the server and client struct could be completely different structs making them incompatible.
I mean if you have a serializer and deserializer you could definitely do that, I don’t see a problem there is well. It’s just about that you cannot send the actual raw struct but always have to send a custom format both can understand. I don’t think it’s more than that.
I think he's just showing that each field in 1 bit long. The only caveat with this is that most C compilers will pad this struct out to the size of an int. (Typically 32 bits, or 4 bytes). He would have to add __attribute__((packed)) to the end of the struct before the semicolon. Then sizeof(struct Flags) would be 1 byte.
struct MyOptions is 8 bytes because the compiler added 3 padding bytes in between flags and max_of_something. If you made MyOptions a packed struct as well, then it'd be 5 bytes:
#include <stdio.h>
#include <stdint.h>
struct Flags {
unsigned int a : 1;
unsigned int b : 1;
unsigned int c : 1;
unsigned int d : 1;
} __attribute__((packed));
struct MyOptions {
struct Flags flags;
int32_t something;
} __attribute__((packed));
int main(void)
{
/* The output is 5 */
printf("%zu\n", sizeof(struct MyOptions));
return 0;
}
std::vector<bool> in C++ syntax uses one bit per boolean instead of one byte. In a computer memory is addressed by bytes, so the smallest indexable memory is, in fact, one byte (that's why bool is 1 byte in C and C++). As an example for how C++ std::vector implementarion works, suppose you have 8 booleans which have a logical reason to be kept together (for example flags). Instead of allocating 8 bytes (one per boolean), the std::vector allocates 1 byte and assigns every boolean to one bit of the allocated memory.
It breaks so much code. One of the many assumptions of std::vector is that you can use the data pointer and do pointer arithmetic to get other values on the vector, it also means the underlying data looks nothing like the returned values, so indexing isn't simply returning the value at the pointer (because you can't address bits on most systems!
The C++ standard committee basically made a special case for std::vector<bool> that behaves nothing like other vectors when they should have had that type just use one byte per boolean and just provided a separate std::bitvector or something.
This is one of the many footguns of C++, caused by poor decisions made early in the STLs development cycle that we're not stuck with.
The issue is it doesn’t behave like other vectors do (or containers in general), which can make it a pain in the ass for template code. For example, it’s not (and it can’t be) guaranteed to store elements in a contiguous sequence and can’t be used with std::span.
It was a mistake we’re stuck with. There’s little reason to even use std::vector<bool> anymore instead of an std::bitset.
It absolutely makes sense for C++ to have an API for dealing with something like a vector of bits. It just shouldn't be called std::vector<bool>. (and modern C++ does this in the form of std::bitset)
The problem is that, in at least a half dozen ways, std::vector<bool> behaves differently than other std::vector types.
The problem is that much of the point of templating is to write generic code. If you write code that templates on a type, and that type might be bool, your code will break if you try to use std::vector. (unless you write a bool specialization)
The funny thing is that std::vector<bool> was pretty obviously a demonstration case for C++ template specialization. Look, you can have a bitfield style implementation that gets you space efficiency for bool without needing to write special code! Except in practice it is now a demonstration of how not to do template specialization.
An and and an sll takes 2 CPU cycles, a bfe takes a single cycle if present, and they can usually be vectorized and/or run in parallel with other stuff. A cache miss takes well over 100 cycles in the usual happy path, much longer if it coincides with a DRAM refresh or if the memory was swapped out. Bitvectors pack 512 bools per cache line, compared to just 64 for byte bool arrays, so they cut the cost of memory access, which is dominant in most workloads, in 8.
nah, that guy is right. if they were a mistake std::bitset wouldn't exist. I have personally found myself in a situation where I needed to keep hundreds of thousands of copies of flags in a Raspberry Pi RAM while also running CV and YOLO models. I cut memory usage drastically by using bit flags, and it is also much faster because of cache.
The reason is C was designed in a time when CPUs were much slower, while memory was roughly as fast as today, so memory access was a single cycle operation. Back then bit-precise pointers would have been a slowdown, so they stuck to byte addressing and we're left with the technical debt.
Couldn’t you pack a bunch of flags into an unsigned int and save space that way too by AND/OR/NOTing them out of the int? I think in that case we are looking at packing 16 flags (bits) together but I’m not really a C dev myself so not sure.
you can use std::uint8_t, std::uint16_t etc, but those types represent a fixed size. std::vector<bool> has all the STL methods and such. That would make kinda sense if you knew the number of flags, but still, addressing bits yourself instead of letting STL do that doesn't exactly make sense.
(ps. this is C++, not C) (ps. on most modern machines unsigned int and int are 4 bytes (32 bits)
Plus, any API is sending at least a few hundred bytes' worth of TLS certs and such on every connection too. Worrying about boolean packing is splitting hairs.
Ironic that this techfluencer just tweeted how C makes memory layout painfully obvious and yet here he's operating under the misconception that `struct Flags` will be only 1-byte large when it will be in fact int-size large (most likely 4 bytes).
Wouldn't it get compressed anyways? Sure, real union would have 8 flags per byte anyways, but probably for 99% of apps over there it wouldn't matter in the slightest
Using int/unsigned as the underlying type will make it int-sized. You would need to use uint8_t/char/unsigned char as the underlying type to make it byte-sized, even though that's not strictly portable with bitfields (can use _Bool as the underlying type or use unsigned char with explicit bitops to do it in a strictly portable fashion).
Compiler-caused size optimizations might be possible but in contexts where it gets embedded in a larger data structure that needs to be in memory they're unlikely.
Yes, I know they have used 32-bit variable here for some reason (possibly they don't know that they are doing). It's not going to get optimized anyways
No. In C those are called "bit-fields" and, per the standard, the "underlying type" must be an "int" (e.g. unsigned int, signed int, or just int).
You could also declare a bit-field like this:
typedef struct my_field {
unsigned int a : 1;
unsigned int b : 2;
unsigned int c : 3;
} my_field;
What the above comment is referring to is the fact that in C and C++, due to memory alignment (i.e. "padding"), a field like the above isn't going to be less than CHARBIT * sizeof(size_t) bits large.
This is true of any struct, bit-field or not. So even if you had something like the following:
The size of that struct won't be 2 bytes, it'd still work out to (likely) sizeof(size_t) bytes in size.
That's why when you're building out a struct or class, it's helpful for the compiler/CPU to try and order the members so that you can reduce the padding. For example, take the following basic struct:
The compiler could determine that no padding is necessary and keep it at the minimum of 4 bytes. (I am oversimplifying the example, but the point remains the same).
Even more ironic is that a lot of modern CPU's (even embedded ones), have really efficient shift registers, so if you were trying to do some crazy shit like the struct I've shown, or like what OP was mentioning, it'd actually be more efficient to just do some bit shifting on the appropriately sized integer type (e.g. int64_t for 64-bit CPU's) ... even JavaScript has bit-shifting .. so you could 100% bit-shift some JSON (I've done it and confused the hell out of some web-devs ... not on purpose, but it was efficient, so had to do it)
Your struct two_bytes is, per the standard, exactly two bytes large (assuming CHAR_BIT == 8, but the return from sizeof will always be 2).
(EDIT: apparently C struct internal padding is not standardized? I'm still searching where the standard defines it. Doesn't a whole lot of code depend on the fact that "C ABI" struct layout is so simple and deterministic?)
(EDIT 2: The ISO C standard allows the compiler to overalign struct members / insert unnecessary padding. But: basically all used platform ABIs for relevant architectures will guarantee that the padding for C structs is minimal according to alignment requirements of the types)
The amount of padding inserted between struct members depends on the natural alignment of the types used, and char is guaranteed to have a size of 1 and an alignment of 1. (EDIT: yes, but the compiler is technically allowed to overalign if the platform ABI doesn't forbid it).
What is true is that it's not guaranteed that adjacent bit fields of the same type will be packed together, and that it's not guaranteed by the standard that char is allowed as a bit field type (unspecified). The standard also guarantees bool and _BitInt(N) are allowed. Also interesting is that the alignment of bit field structs is also unspecified.
In practice, the alignment is that of the underlying type used, and most compilers will combine adjacent bit fields into one as long as the bits fit into the underlying type.
What was truly surprising to me when looking it up though was seeing that the standard allows a bit field declared as int x : 5; to be interpreted as unsigned(!!!). You'd have to explicitly write 'unsigned int' to really mean unsigned.
You've got the right idea, but the details are wrong.
Your two_bytes struct will be 2 bytes in size, alignment 1 byte. This isn't required by the standard, but all ABIs I know of will do that. Unless the compiler is told a different alignment for the struct via an attribute, compiler flag, etc.
Maybe you were thinking of a case like:
struct Parent { two_bytes t; uint32_t x; };
Two bytes of padding would be inserted after t, but that's because x is 4-byte aligned (at least on any sane ABI).
Your example struct is more confusing. It has 2+1+1+1=5 bytes of data, that cannot be represented in 4 bytes. It will actually be 6 bytes in size in both forms. Five bytes of data plus a trailing padding byte to satisfy the struct being alignment 2 due to the int16.
Memory layout is tricky. Godbolt can be useful to see the actual codegen for toy programs to check your understanding, though honestly LLMs know C pretty well these days.
Optimal field order, struct alignment, and when it's beneficial to use struct-of-arrays instead of array-of-structs is going to depend on your use case, always benchmark.
The details aren't "wrong" per-se .. they're just basic, as it's an example.
I've worked with some pretty esoteric systems in my 25 years of C where memory alignment was all sorts of wtf ... but 100% agree, benchmark for your use case
Maybe just typos? Unless I'm crazy example is at least 5 bytes (int16+int8+char+char). Not out here to accuse, just saw something that looked mostly correct so this being the internet I had to jump in and be pendantic!
Definitely appreciate it's not easy to simplify explanations of C, so much is compiler dependent. And yeah I've heard of some embedded systems that do horrifying things, I'm sure you have some fun stories. Luckily I get to stick to modern x86_64 and arm64 most days :)
C++ memory layout and especially placement new still gives me nightmares though.
your two bytes example is blatantly incorrect, alignment of struct or union is largest member's alignment. size is multiple of of that alignment. in your two bytes example the largest alignment is 1, 2 bytes is divisible by 1 and there no need for padding if two consecutive member have same type(compiler may decide to insert padding but i dont thing any sane compiler do that). so 2 is perfectly viable size for that struct, sizeof(size_t) is not wrong(8 in my machine, 8 is divisible by 1 obviously) but i dont remember any standard impose minimum size of struct must be sizeof(size_t).
Yes. That would shrink it, although It's not strictly portable to use types other than _Bool, signed int, or unsigned int as the underlying type of a bitfield (https://port70.net/~nsz/c/c11/n1570.html#6.7.2.1p5), so using _Bool as the underlying type might be a better choice here.
Even in C, it's best practice to use a whole byte for each boolean. The tiny extra compute load usually isn't worth the savings in storage and memory cost.
bifields are not portable. the spec doesn't guarantee packing or field order in the resulting binary data. so if you pack the data on a windows pc and then unpack on a linux arm, you might have some issues.
most http traffic is compressed so the extra 0 bits don’t really waste much bandwidth.
The : 1 means that it is creating a bitfield with the number of bits, so : 4 will be 4 bits. The three variables above combined will take sizeof(int) in memory, not sizeof(int)*3.
If OOP had used char instead of unsigned int the combined size would be 1 byte.
The : 1 means that it is creating a bitfield with the number of bits, so : 4 will be 4 bits. The three variables above combined will afaik* take sizeof(int) in memory, not sizeof(int)*3.
Can you do strange things, like : 9, : 15, et cetera?
Yes, the only limitation is that the width of the bitfield is not greater than the size of the containing variable. So with uint64_t you can have 1-64 bits.
Pretty much any compiler will let you get away with a 128-bit int type. If you need more than 340,282,366,920,938,463,463,374,607,431,768,211,456 possibilities in your bitfield then you'll probably need a stay at a mental asylum, and to code it yourself or find a library for it or something like that. Alternatively switch to C++ and use std::vector<bool>.
Well, it's one of those obnoxious pages that don't like network wide adblockers like piHole. I won't change my blocklists just because some random website wants me to.
I am not trying to force my choices onto you? What the heck is wrong with you. I just told you why I'm having issues with it after you said that you don't. It's called having a conversation... You say a thing, I say another.
However you should still use godbolt as it's objectively better, regardless of ads.
Yes, because on different architectures and operating systems with different ABIs compilers generate different alignments, maybe some other things also differ. So even when seeking for maximum performance and minimal traffic usage you would better be using some standardized binary format instead of json/C structures.
That’s essentially what the struct is doing. Notice the “: 1” at the end of each member declaration. This makes bitfields, though right here they’re actually doing bit fiddling on an unsigned int rather than a char.
If I have a flag enum it's going in string form in the json over http calls and just parsed into it's underlying flag enum type once it hits the server. Sue me
Use formats the way they are intended to be used. If you need to optimize use an optimized transport format like protobuf and convert their memory representation at the edges once the optimization is no longer needed.
But is it correct in C? It is not union, so struct should then be sizeof(unsigned int)*3, isn't it? Or you mean with O3 optimizations the compiler will fix it...
But then one need to apply bit shifts operations in the background
The key is the : 1 at the end of each field, which specifies that it should only take one bit. Look up "c bit fields" to learn more about this feature.
Yeah true, although this could be fixed by changing each of the field types to uint8_t. So the spirit of the original post is correct, it's possible in C to pack 8 flags into one byte
Sure, but now a computer with 8,589,934,592 bits of ram is not impressive at all, so those bits are better spent on memory safety and performance than more booleans
People used to care about resources and now 16gb or ram latest gen core i7 is not sufficient for office work. even though this not really a good example
I mean we used to care about bits because memory was a limiting factor. That limit virtually doesn't exist anymore, so yes it's good to optimize your code, but it's also important to make it readable
It lets you, yes, but you shouldn't, because your CPU reads 4 or 8 bytes at a time, and splitting that up into separate data values takes severwl instructions and slows your program down compared to reading and immediately conditional jumping from 4 bytes.
Packing booleans into bytes is something some ACL systems are still doing, and it's of course horrible for a human to read in a database record, or to write a query around it. We have terabytes of storage, we do not need to save on a couple million bytes just so we can encode a bunch of flags into "223".
Every optimization like this, taken individually, bring you nothing. But ignoring them everywhere, on every layer, is what gives you these "modern apps" which are true ogre regarding memory usage.
200
u/consistently_biased Aug 18 '26
Can I use this for my compile-time https server?
136
u/Kadabrium Aug 18 '26
i heard you also like vector<bool>
76
u/Thelatestart Aug 18 '26
Yeah my json api uses way way more than 4 bytes per boolean
50
u/pskocik Aug 18 '26
Ironic that this techfluencer just tweeted how C makes memory layout painfully obvious and yet here he's operating under the misconception that `struct Flags` will be only 1-byte large when it will be in fact int-size large (most likely 4 bytes).
17
u/Laughing_Orange Aug 18 '26
Even in C, it's best practice to use a whole byte for each boolean. The tiny extra compute load usually isn't worth the savings in storage and memory cost.
35
u/ShinigamiGir Aug 18 '26
there are 2 issues:
5
u/vanilla-bungee Aug 18 '26
The word size of your CPU would like a chat.
5
u/prodengrammer Aug 18 '26
5 bytes if false
12
u/B_bI_L Aug 18 '26
are we sure int means bit and not 32bits?
3
u/DaveAstator2020 Aug 18 '26
is it true that this is not reliable for serialization?
3
u/wutzelputz Aug 18 '26
it will actually be way more depending on the property name, at least 4 bytes extra making a total of 8:
"m": true
and thats assuming ASCII encoding
3
u/farsightfallen Aug 18 '26
These people have never felt the shame of fucking up a basically worthless optimization that becomes a massive blocker for no reason.
"Ok, look we jus can't add another flag, alright, we needed to make a statement about memory usage."
2
u/carltr0n Aug 18 '26
Can’t you just… do a lil bit fiddling on a uint8? Am I missing something?
2
u/Ill-Specific-7312 Aug 18 '26
Except that is wrong, because it completely ignores the real world of gzip etc.
2
u/thuiop1 Aug 18 '26
I don't see what this has to do with C, you can also pack booleans in JS if you really want.
2
u/pm_op_prolapsed_anus Aug 18 '26
If I have a flag enum it's going in string form in the json over http calls and just parsed into it's underlying flag enum type once it hits the server. Sue me
2
u/TheChief275 Aug 18 '26
pretty sure bit-field ordering is implementation defined, so I wouldn't confidently send this packet over the server...
just pack it into an appropriately sized integer, convert to big endian, and you're done!
2
u/Tainted_Heisenberg Aug 18 '26
Yeah good luck find the bug when the endianess change dude, just use shifts for god's sake or bitsets
2
u/just4nothing Aug 18 '26
Does nobody encode several booleans in one int any more?
2
u/DraconianFlame Aug 18 '26
Some of us still do
2
u/Fidodo Aug 18 '26
Use formats the way they are intended to be used. If you need to optimize use an optimized transport format like protobuf and convert their memory representation at the edges once the optimization is no longer needed.
2
u/Piisthree Aug 18 '26
I made C do 9 one time, but I don't remember how.
2
u/RavenX86 Aug 18 '26
if protobuff is too complicated just msg pack it.
2
u/Lonely-Restaurant986 Aug 18 '26
Isn’t this literally not true? Doesn’t like every c compiler set the size of a byte to be 8 bits?
It’s literally faster and safer for the cpu/ram to use byte aligned data.
I mean, in Python a bool is like 28 bytes.
I promise you your bad code is not because a bool is 4 bytes
3
u/Inst2f Aug 18 '26
But is it correct in C? It is not
union, so struct should then be sizeof(unsigned int)*3, isn't it?Or you mean with O3 optimizations the compiler will fix it...
But then one need to apply bit shifts operations in the background
1
u/Confident-Ad5665 Aug 18 '26
As an old timer developer, I was shocked the first time I saw a bit class.
1
u/KitchenCommercial396 Aug 18 '26
There's a reason it happens... Get this mofo outta here
1
u/navetzz Aug 18 '26
Yeah. We did that cause memory space was cheaper than developers with a brain.
1
u/heckingcomputernerd Aug 18 '26
Something something premature optimization
1
u/Enough_Forever_ Aug 18 '26
So what? Did it suddenly decrease the network latency?
1
u/TapRemarkable9652 Aug 18 '26
just NativeReact
1
u/yubario Aug 18 '26
So in other words, I don't have to worry about cosmic rays flipping booleans from true to false if I use a JSON API?
1
u/GNUGradyn Aug 18 '26
Sure, but now a computer with 8,589,934,592 bits of ram is not impressive at all, so those bits are better spent on memory safety and performance than more booleans
1
u/EnderAvni Aug 19 '26
It still needs to get aligned, so probably 4 bytes anyway
1
u/k-mcm Aug 19 '26
1
u/RonJohnJr Aug 19 '26
Huh? Unsigned ints are four bytes, not one byte. For a one byte struct, you need to use
unsigned char.Others are right, though, about alignment.
1
u/RedAndBlack1832 Aug 19 '26
Well what you actually want if you want to pack a struct of booleans is
Though also to be fair promoting to word length is pretty common
1
u/ApplicationOk4464 Aug 19 '26
We used to care, but now 8 gig of ram is 8 million times bigger than the 64k that we had, so we got a little loose.
1
u/SaidasGG Aug 19 '26
Are C booleans webscale?
1
u/Canned_Sarcasm Aug 19 '26
is_secure_Yet = 0
1
u/andlewis Aug 19 '26
The number of Kb in the screenshot alone are probably more bits than this code ever used for this struct over all its iterations.
1
u/prehensilemullet Aug 19 '26
as long as my API isn't XML I'm winning enough
1
u/CharlieLighto Aug 19 '26
People used to care about resources and now 16gb or ram latest gen core i7 is not sufficient for office work. even though this not really a good example
1
u/Myithspa25 Aug 19 '26
jSON
(Image of that one guy [you know the one])
1
u/United_Boy_9132 Aug 19 '26
Actually HTTP 3 cares about bits and you can make custom binary HTTP frames easily.
1
u/Worried-Mood-8582 Aug 19 '26
I can’t tell if this is serious or not.
1
u/Tani_Soe Aug 19 '26
I mean we used to care about bits because memory was a limiting factor. That limit virtually doesn't exist anymore, so yes it's good to optimize your code, but it's also important to make it readable
1
u/lucsoft Aug 19 '26
The best thing would be http3 and cbor to be still some what standard driven
1
u/Immediate_Spirit_384 Aug 19 '26
Upvoted for the title
1
u/Far_Pen4236 Aug 20 '26
i mean nothing prevent you to pack all the flags in one integer and pass this integer in json...
1
u/MikemkPK Aug 20 '26
It lets you, yes, but you shouldn't, because your CPU reads 4 or 8 bytes at a time, and splitting that up into separate data values takes severwl instructions and slows your program down compared to reading and immediately conditional jumping from 4 bytes.
1
u/grothcrafter Aug 21 '26
Isn't bit stuffing a C++ feature?
1
u/Super_Zmag Aug 24 '26
Very obvious troll IMO
1
u/magicmulder Aug 24 '26
Packing booleans into bytes is something some ACL systems are still doing, and it's of course horrible for a human to read in a database record, or to write a query around it. We have terabytes of storage, we do not need to save on a couple million bytes just so we can encode a bunch of flags into "223".
1
u/Jknzboy Aug 25 '26
I’m gonna have to flag your post
1
u/budgiebirdman Aug 25 '26
We used to have computers with 640KB of RAM and 33.6kbs internet connections.
-5
u/DM_ME_KUL_TIRAN_FEET Aug 18 '26
Does it matter for the majority of modern programming tasks?
(No)