In case anyone wants a background on what Dragonfly vs FreeBSD
> The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. ... While full cache coherency is no longer a top level goal, filesystem coherency is, and that direction continues to guide the project in a number of ways.
> DragonFly BSD was forked from FreeBSD 4.8 in June of 2003, by Matthew Dillon. The project was originally billed as "the logical continuation of the FreeBSD 4.x series", as quoted in the announcement, but this description has long since become obsolete. From a performance perspective DragonFly's only real competitor these days is Linux
Are there any real world success stories with this?
This brings backs memories... I still used FreeBSD by then, and I remember that the 5.x branch really had problems and the drama that surrounded it in the community.
I started using FreeBSD(4.3 or 4.4) in the Uni out of curiosity, but kept it for the massive improvement in performance and stability. Loads that brought Linux to its knees(>10) were handled easily by FreeBSD.
The documentation was also superb... the only drawback was having to compile everything, never got used to it.
Yeah, with all due respect to the developers (because getting as far as they have is a huge accomplishment!) the biggest feature of DFly I can identify is HAMMER, which is most interesting if you don't want to use ZFS for license reasons.
I remember some of the old goals were things like process snapshotting and transferring, as in saving a process state and transferring it to another machine. It seems most of these ambitions ended up discarded, which is a shame, since it's increasingly becoming a "FreeBSD, but with fewer features, more incompatibilities, and marginally worse performance" :(
HAMMER's ability to snapshot an arbitrary directory is still something I deeply deeply deeply respect & wish btrfs or zfs had.
I'm not sure the status but HAMMERFS also supposedly was going to have solid transparent encryption, which is something I wish btrfs did. I'd really like to be able to take a fs snapshot, send it to a friend, have them load the snapshot volume, but be unable to access it. But as soon as I did show up at their place with my keys, we could instantly unlock it & use it. I understand ZFS does a better job of this all, but balancing ZFS pools & adding drives has always sounded impossibly hard compared to btrfs's "it just works" adding-drives to RAID scenarios.
Ahh, that does sound like a nifty feature! I didn't mean to so strongly imply HAMMER was a knock-off ZFS, more that it fills a similar niche, similar enough that ZFS's dominance kind of takes away from the "wow factor" of HAMMER.
It makes more sense (to me anyway) when you consider that at the time HAMMER was first released, the Linux port of ZFS had only just begun, and FreeBSD had just released their first port of ZFS. Btrfs was also still an out-of-tree patch then, only mainlined in 2009. (not to mention ZFS as a whole probably seemed much more shaky around 2010 with Sun's death-by-Oracle and OpenSolaris getting butchered)
I think back then, having a clean and functional "recreation" of ZFS with all the design bloat removed seemed really promising. Now, it's just a nice FS competing with other nice FSes, except with the downside of being on an "exotic OS". Shame :(
I'm thinking of a few Phoronix benchmarks I've seen through the years. I'm not going to pretend like these benchmarks are the be all and end all, but I do think it (at the very minimum) compares run-times for common tasks.
In general, Linux comes out on top, FreeBSD follows, and Dragonfly takes third place. In some instances, Dragonfly beats out FreeBSD, but only just. The overall mean is that Dragonfly is only sightly worse than FreeBSD.
Again, not advocating for using phoronix benchmarks like it proves anything, but I think these results make sense.
It's hard to reliably reproduce a cross-platform benchmark. They use different compilers, different compiler flags and do not show profiles. Also some compilers tend to be quite conservative in less-known environments.
"Single system image" clustering - or in plain language, a single operating system running distributed on top of a bunch of (often commodity) hardware.
Your program runs just like it would on a single PC. The OS is designed to ensure that it functions correctly regardless of which underlying resources are doing the compute.
It's worth noting that Linux is picking up a lot of the basic infrastructure needed to enable this, if only as an unexpected side-development of earlier work wrt. containerization and namespacing of all system resources. Check out the CRIU (Checkpoint and Restore In Userspace) patchset.
I'll preface this with: this isn't a derogatory question, I just don't know a better way to phrase it.
Out of curiosity, who uses this? I assume given how long it's been around there must be a market segment or a geo local where it's popular (like SLES has large penetration in Europe and SAP).
I know multiple closed source vendors that have built products on net/open/freeBSD. Dragonfly is the one distro I've never once run into in the wild.
I have a Dragonfly install as part of my "pet" test farm. The build environment is similar to the other BSD, but from time to time it triggers strange portability bugs in my codes that I love to fix.
I am always fascinated by DragonflyBSD and catch myself peeking everytime some news about it pop up. However I have not had a chance or excuse to fully try it myself yet.
A question on their repository and packaging: How do they handle it? Are they simply compatible with FreeBSD packages so most packages from there are just available on DragonflyBSD as well?
I can't speak to the PlayStation, but this claim doesn't really hold for the Switch. The Switch's OS "Horizon" borrows BSD's networking stack, but then again: which OS doesn't? [0] Nintendo has written their own kernel, which is fully reverse engineered and re-implemented at this point. [1]
The Switch's OS is not based on FreeBSD at all. The Switch's OS is a continued development of the operating system of the Nintendo 2DS/3DS which was likely a custom OS.
Some news sites read the Switch's open source credits crediting the FreeBSD project and ran with the headline "Switch OS based on FreeBSD". By the same measure, people should call Windows "based on FreeBSD" because the kernel uses BSD code or used to at least. It's sub par reporting that turned into an urban legend.
The PS3, PS4 and PS5 operating systems are systems based on the FreeBSD kernel, which is probably why it's tempting to think the Switch would be too.
AIUI Even your Windows claim is wrong on the specifics. Microsoft copy-pasted BSD code for a whole bunch of userspace tools, so you'll find some (but not all) of what you'd expect there, but the kernel socket implementation is their own.
If you go back far enough (prior to NT in particular) the OS doesn't provide sockets, or indeed TCP/IP, and it's true that (some?) popular third party vendors just used BSD code, possibly after filing off the serial numbers, but that's not the Windows kernel.
Also, sockets just aren't that clever. I mean, the idea is pretty clever, but once somebody tells you the idea, it's not exactly rocket surgery to implement.
That does make sense if you're a large corporation. You might not want to develop an OS from scratch, but Windows and WXWorks requires a license. You could go Linux, but what about the GPL? You might be okay, but why risk it. That leaves you with the choice of any of the BSDs.
Yeah, one of my jobs right now involves having to implement some stuff for the playstation 4/5. You should see the dev and test kits for the ps4 and ps5... they look something like weird desktop computers from 2011 or something more than the actual deal.
Seeing if something will work on the playstation is like, is there a port in freebsd of this software and how do we get it to run on this thing. You don't get the ports collection unfortunately but thankfully ports are kind of just like a patch diff that you can apply yourself.
Excuse me what ? If this is really true, why isn't it more widely adopted than Ubuntu for server side deployment ?
then following text is copied from "features" page of official site
>EXTREME SCALING
DragonFly will autotune kernel resources and scaling metrics such as kernel hash-tables based on available memory. The autoscaling has reached a point where essentially all kernel components will scale to extreme levels.
Process and thread components now scale to at least a million user processes or threads, given sufficient physical memory to support that much (around 128GB minimum for one million processes). The PID is currently limited to 6 digits, so discrete user processes are capped at one million, but the (process x thread) matrix can conceivably go much higher. Process creation, basic operation, and destruction have been tested to 900,000 discrete user processes.
File data caching scales indefinitely, based on available memory. A very generous kern.maxvnodes default allows the kernel to scale up to tracking millions of files for caching purposes.
IPI signaling between CPUs has been heavily optimized and will scale nicely up to the maximum hardware thread limit (256 cpu threads, typically in a 128-core/256-thread configuration). Unnecessary IPIs are optimized out, and the signaling of idle cpus can be further optimized via sysctl parameters.
All major kernel resource components are fully SMP-aware and use SMP-friendly algorithms. This means that regular UNIX operations that manipulate PIDs, GIDs, SSIDs, process operations, VM page faults, memory allocation and freeing, pmap updates, VM page sharing, the name cache, most common file operations, process sleep and wakeup, and locks, are all heavily optimized and scale to systems with many cpu cores. In many cases, concurrent functions operate with no locking conflicts or contention.
The network subsystem was rewritten pretty much from the ground-up to fully incorporate packet hashes into the entire stack, allowing connections and network interfaces to operate across available CPUs concurrently with little to no contention. Pipes and Sockets have also been heavily optimized for SMP operation. Given a machine with sufficient capability, hundreds of thousands of concurrent TCP sockets can operate efficiently and packet routing capabilities are very high.
The disk subsystem, particularly AHCI (SATA) and NVMe, are very SMP friendly. NVMe, in particular, will configure enough hardware queues such that it can dispatch requests and handle responses on multiple cpus simultaneously with no contention.
The scheduler uses per-cpu algorithms and scales across any number of cpus. In addition, the scheduler is topology-aware and gains hints from whatever IPC (Inter-Process Communications) occurs to organize active processes within the cpu topology in a way that makes maximum use of cache locality. Load is also taken into account, and can shift how cache locality is handled.
The kernel memory manager is somewhat NUMA aware. Most per-cpu operations use NUMA-local memory allocations. User memory requests are also NUMA aware, at least for short-lived user programs. Generally speaking, the scheduler will try to keep a process on the same cpu socket but ultimately we've determined that load balancing is sometimes more important. CPU caches generally do a very good job of maximizing IPC (Instructions Per Clock). Because memory management is fully SMP-aware, a multi-core system can literally allocate and free memory at a rate in the multiple gigabytes/sec range.
Generally very high concurrency with very low kernel overhead. The kernel can handle just about any load thrown at it and still be completely responsive to other incidental tasks. Systems can run efficiently at well over 100% load.
Supports up to 4 swap devices for paging and up to 55TB (Terabytes) of configured swapspace. Requires 1MB of physical ram per 1GB of configured swap. When multiple swap devices are present, I/O will be interleaved for maximum effectiveness. The paging system is extremely capable under virtually any load conditions, particularly when swap is assigned to NVMe storage. Concurrent page-in across available cpus, in particular, works extremely well. Asynchronous page-out. Extended filesystem data caching via the swapcache mechanism can operate as an extended (huge) disk cache if desired, and/or used to increase the apparent total system memory.
For that to be impressive to me, I also want to know how Linux compares. I suspect that for most non-synthetic workloads, Linux will perform about as well. I may be absolutely wrong -- but my gut feeling is Linux is really good at scaling as well, and has a lot more "real world use" sitting behind it, with a lot more paid engineers fine-tuning it to perform well with actual workloads.
That's why most people aren't going to use Dragonfly over Ubuntu when they're deploying their servers.
> a lot more paid engineers fine-tuning it to perform well with actual workloads.
Just because it is / can be fine-tuned for the workloads of those engineers' employers doesn't mean it is fine-tuned for your loads.
If you're running your servers hard, you probably might considering pay engineers to fine-tune your system. There's certainly room in both FreeBSD and Linux for improvements, depending on what your servers are doing; I'd assume Dragonfly is in the same boat. I may be biased from work experience, but IMHO, the FreeBSD can feel a bit more organized and easier to tinker with; FreeBSD having a kernel debugger certainly helps a lot with some things. If you had infinite resources, you'd run some of your servers on Linux and some on FreeBSD (and maybe some on a purpose built OS just for you, but that requires a large value of infinite resources), and when one does some things better, you can pull ideas back and forth as needed to get all things working better. Of course, some ideas are harder to pull back and forth, and sometimes small differences in ideas make for large differences in results.
For a lot of use cases, the kernel / OS doesn't make a huge contribution to overall performance. As long as you don't use something tragically bad, it'll be fine. There's a pretty narrow set of circumstances where the kernel is the limiting factor, and I say that having spent quite some time working on problems where the kernel was the limiting factor.
> For a lot of use cases, the kernel / OS doesn't make a huge contribution to overall performance. As long as you don't use something tragically bad, it'll be fine.
This is what I was thinking, but did a poor job communicating. I think that Linux, because of all the work poured into it, will usually be good enough. Even though its been less meticulously constructed than DFly and didn't have as much forethought put into picking low overhead SMP algorithms and data structures and what not, in the end, the differences between Linux and DFly in performance will be pretty minimal.
So when faced with using a strange and novel platform with software incompatibilities and possibly a slight performance gain in certain situations, or using the familiar, most people will pick the familiar, and for justifiable reasons IMO.
> So when faced with using a strange and novel platform with software incompatibilities and possibly a slight performance gain in certain situations, or using the familiar, most people will pick the familiar, and for justifiable reasons IMO.
Yep, that's why I stick with FreeBSD :P I don't really want to learn a new tool for ifconfig or netstat. I'd make a joke about three firewalls, but ;)
> Excuse me what ? If this is really true, why isn't it more widely adopted than Ubuntu for server side deployment ?
As great and fun as single image clustering is, it's a pretty niche use-case. Event joyent/smart-os, which went the "other, sensible way", focusing on co-location of storage and processing - turned out to be pretty niche.
Linux had openMosix - didn't see much use outside actual clusters.
If you're not doing the smartos thing, but want full-on ssi - you really (today) need pretty extreme interconnect (ie: infini band or better; high bandwidth, low latency) - every node in you supercomputer needs reasonable access to storage.
The dragonfly people have been working hard on this for years, they've come a long way. But you don't really need it if your workload runs on less than a rack full of servers.
> The kernel can handle just about any load thrown at it and still be completely responsive to other incidental tasks. Systems can run efficiently at well over 100% load.
That's is one really big point especially for DFly but also the other BSD's, having a BSD's at over 150% is not a problem, with linux you try never go over 80-85% if you still want a responsive system. Same with near out of memory, linux handles that really poorly.
i once tried installing and running an openbsd system with 64mb of ram.
It "worked" in the sense that the installation was able to finish, and i was able to run a static webpage on openbsd httpd.
system load was through the roof, but the system was still r esponsive. I started running into issues once i wanted to do anything even slightly more advanced (like for instance, run PHP on a webpage).
Why wouldn't it be true? It's not as if the source and images aren't available for you to look at for yourself.
As for why BSD isn't more popular than Linux, well, that's a much bigger question. It could come down to licensing, project goals (not winning popularity contests), but mostly decades of history and Linux appearing at the right place, at the right time. There is place for alternative operating systems, choice is important.
I have some interest in DragonflyBSD and BSD ethic is close to me, but I'll say that Linux being GPL made it successful. It centralizes development and with every developer joining in it cements it even further. But right place and time above all, if it would start today the license would be a nonstarter for many.
> Linux being GPL made it successful. It centralizes development and with every developer joining in it cements it even further.
Centralization doesn't have much to do with the GPL. With a BSD system, you can take an upstream release and use it as the base of your system and not publish your changes. With a GPL system, you can do the same, but if you distribute your system to others, you have to publish your changes; but you don't have to work with the upstream system unless you want to; your changes might get pushed upstream by someone else, or used as inspiration by upstream, but that's not that common. If you wanted to work with upstream, you can do that with BSD as well.
It may be forgotten or not known by many, but there was a 1991 lawsuit from AT&T over code in BSDi and the code in question was in other BSD distributions at the time including FreeBSD until the 2.0 release in November 1994. Certainly Linux had its moment of legal uncertainty that turned out fine, but it came after it was already well established. BSD had a legal shadow at a much earlier time, and that may have driven some people away.
If PHP supported containers and Haskell didn't -- though we all know Haskell's `jails` feature is absurdly better in all respects, and all white-coated CompSci PhDs know it.
* GPL license (until PHP4)
* evolutionary
* has warts
* pragmatic
* gets shit done
* everyone uses it
* lots tips online (good and bad)
* you have crazy idea, too late, it has already been done
BSD/Haskell
* BSD license
* designed
* elegant
* by the book
* theoretically correct
* nobody uses it
* read the man pages
* you shouldn't do that
probably more, maybe something about globals vs jails/monads. Many of these things stems from evolutionary vs designed.
Because as synthetic benchmark numbers go they are pretty pedestrian. 256 CPU threads? Red Hat claims to support up to 8192 CPU threads in RHEL8. Same for any of the other specific number they call out Linux's synthetic benchmark numbers are much higher.
yes - one of the reasons behind Dragonfly conception was that it's hard to write high-performant code for all architectures at the same time. Ditching ARM64 and others means much cleaner codebase and less effort to maintain all of this.
You are absolutely correct, however DragonFly focused since the beginning on x86, that was the target platform for a cluster and the architecture every dev had. Maybe it's time to rethink that.
DragonflyBSD distinguishes itself by supporting a variant of the Apollo Aegis, and later Domainix, practice of expanding environment variable names in symbolic link text.
Of course it is more clumsy than in Aegis, for the sake of compatibility with other BSDs and systems that might, by long odds, happen to have what look like environment variable references in their symbolic links, but aren't.
> The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. ... While full cache coherency is no longer a top level goal, filesystem coherency is, and that direction continues to guide the project in a number of ways.
> DragonFly BSD was forked from FreeBSD 4.8 in June of 2003, by Matthew Dillon. The project was originally billed as "the logical continuation of the FreeBSD 4.x series", as quoted in the announcement, but this description has long since become obsolete. From a performance perspective DragonFly's only real competitor these days is Linux
Are there any real world success stories with this?