One thing I'm not clear on it if can do, and that I'm interested in, is secure destructors.
Say I'm handling crypto, and I'm carting around an ephemeral key. When this goes out of scope, I definitely no matter what, want this zeroised by its destructor - as opposed to just having it (or a temporary copy made by a compiler optimisation!) zombling around the heap, stack or forgotten unused xmm registers because the compiler figured since I don't reference it again, the memory's contents are no longer important.
Current approaches to this involve explicit_bzero(), or other similar memset(0)-and-I-really-mean-it-don't-optimise-this-out techniques. (And a fair bit of testing and prayer when it comes to potential temporary copies or registers.) But unless you're doing it in assembly language, you don't really know. (The stack beneath you, such as the OS, any hypervisors, SMM, AMT, SGX, µcode etc, aside, of course!)
I'm not quite clear what Rust's behaviour with this scenario is. If it can do this easily, even potentially, I am very interested…?
We had a Bay Area Rust meetup [0] all about cryptography back in December (here[1] is a video if you want to watch the event). One of the subjects of that talk was about a project tars[2] that tries to provide for a secure buffer for keys that is protected against it being leaked using a variety of techniques. It hasn't yet been analyzed, so of course you shouldn't actually trust it yet.
There's a lot of interest in our community to build out a solid foundation and verified foundation of cryptography primitives. If you're interested in helping out, hop in the #rust-crypto channel on irc.mozilla.org.
This keeps coming up, but I think it's a very, very bad idea. It's false security.
If you are running in an environment where you don't trust code running in the same compartment/sandbox/process, then it's futile to zero out memory. The caller could have prepared things such that the memset doesn't work, if the key material went somewhere else.
If you ever find yourself thinking you need to do this, what you instead need is a helper process who's only purpose is to do primitive operations with sensitive key material.
Particularly as Rust is already a "safe" language - it doesn't even make sense to zero memory which by definition another piece of code can't access. Unless there's declared "unsafe" code lying around, but you wouldn't put that in the same process, would you? At which point, what are you even protecting against? If an in-process threat is that advanced, then you're not achieving anything.
Zeroing memory protects against future compromise. Sure, if you were already compromised, you already lost; but if you don't zero cryptographic material and you are compromised in the future (which can even be a physical attack like cold boot), you lose.
It's the same idea as in "forward secrecy": once the key material is discarded, it's gone, and no future compromise can bring it back. Being able to say "after this point in time these values don't exist anymore" is a powerful cryptographic primitive.
This: exactly this. Zeroisation is a fundamental cryptographic primitive, probably the most fundamental. Conceptually simple, but very easy to screw up, and has catastrophic consequences if the assumption is violated.
I want to be able to have an ephemeral secret, and then trust the code to do its very best to get rid of it when it's no longer needed. That doesn't mean just leave it rotting on the heap and promising it doesn't get accesed again, it means burning it to make sure. That's the underpinning of any possible proof of forward security.
Sure, you say, I want a helper process? Fine idea: compartmentation. Now that helper process needs secure zeroisation. And since I want it to be secure, surely I want to write that process in Rust. See where I'm going here?
Whether 'safe' code I trust within my environment can read it again is totally irrelevant, if the machine later gets rooted or booted, nonstopped or whatever.
Zeroing out memory after use is mitigation against security flaws like heart bleed. It's not a security feature in it self. Although you are right that the secure process is probably just the better solution anyway.
It's sort of like the arguments for DRM. Of course pirates will always break it, but we can still mitigate it in a number of ways.
Not false sense of security: suspending VMs to disk. The contents of previously used, free memory that has not been reused by the kernel are written to disk. Now it has a lifetime far longer than RAM.
So you're essentially trying to protect against a hypervisor attack? That's never going to work unless you put the primitives in the hypervisor. Assuming you own it.
No, it wouldn't. For example, the "Heartbleed in Rust" blog post [1] re-used a buffer without freeing it. No destructor runs in between the two uses, so a zeroing destructor could not possibly prevent the bug.
Maybe zeroing destructors make sense as defense-in-depth, but I don't see how they can fix a Heartbleed-style exploit in Rust. In code where the buffer is freed and its destructor runs, Rust's memory safety guarantees already prevent it from being accessed after free. In vulnerable code that just uses the same buffer twice, the destructor never has a chance to run so its behavior doesn't matter.
The real Heartbleed vulnerability (CVE-2014-0160 in OpenSSL) involved reading into uninitialized memory in a newly-allocated buffer, which safe Rust code already prevents [2].
Thanks - that's a good example of what I was trying to convey.
The point is Rust already provides safety guarantees. If you don't trust the runtime, then why would you trust the built-in zero'ing? I get the "defense in depth" argument, but it feels a bit like doing this:
{
int a = secret; // Get secret.
assert(a == secret); // Check "a" is actually that.
a = 0; // Ensure "a" is zero'd on exit.
assert(a == 0); // Just because.
}
And yes, I get that you can build this into the language so it's not quite as ridiculous - you actually wipe tainted stack, for example.
But the point is: the runtime has an ABI and a machine model. Information is allowed to leak across function boundaries, beacuse it doesn't matter. Without using the "unsafe" keyword, there are no methods of getting around the machine model and dipping into the underlying actual machine.
Even if you don't have a "safe" language and runtime, it's still of limited value. It protects against threats involving data or control flow corruption after key usage, and where there isn't sufficient control of the program to perturb the secret-consuming functions. That's more of an annoyance than prevention. On the other hand, it gave the programmer a false sense that it was properly wiping secrets.
It is very probable that a sufficiently smart optimizer could see the assertion was always true and delete it. Then see no one reads "a" and delete it as well. In certain circumstances this can cause a secret to be leaked in, say a register, making our safe function unsafe. You need to be very careful writing secure code, and probably need to go down to the level of writing assembly to be sure the optimizer isn't turning your safe code into unsafe code.
We actually have an interesting project in rust where someone is writing a syntax extension to take rust like code and generate assembly [0]. It's probably unsafe to use right now but if sufficiently well implemented it could be the foundation of a lot of interesting cryptography work.
Sorry, I wasn't clear enough that my code was intended as sarcastic. It's obviously silly to zero variables because the compiler is free to ignore you. The point is, the underlying machine is going to do the same.
There are many ways to dig out stale memory if you're running at sufficient privilege, for example. Direct cache introspection, for example, or bypass. Zero-izing alone is not sufficiently strong to mitigate the threats people imagine it works against.
The heartbleed "issue" in Rust involved code that passed the same array to two functions, making it apparent that the array was never getting destroyed. Since they used a custom memory allocator, secure destructors won't help you since your array might not even get destroyed. Lessons: Don't use custom allocators.
You need to put the key material into an opaque struct, which does store it on the heap. Using the `black_box` function you can zero the key material out in the destructor.
I see. Which would be kind of similar to how you do it in C; the destructor is actually guaranteed to run when it goes out-of-scope?
But what I'm a bit more worried about is how the secret data gets in there, and what happens while I'm working with it: expansion, cipher state, key setup, all the little adds and xors and rots (dammit, why doesn't ROT ever get some real operator love? It's got first-class instructions… :() - all that stuff you'd do in u32 and u64. Temporary copies may still be a problem, if you look at the object that actually comes out of compilers sometimes.
Does using an opaque type actually deal with that issue here?
I know nothing about writing crypto core code. But... Rust supports inline assembly, so after you're done, you could always zeroize every register, right?
Since the compiler can do whatever it wants, you can't be sure it hasn't put any part of the key anywhere on the stack, and there isn't sufficient information for the asm to know which registers contain tainted values.
Unless you can actually prove all parts of the compiler's data transforms going down to assembly I think the safest thing to do is sandbox your key-handling process so nobody else can examine it.
It would be good to have a "secure" type modifier which tells the compiler "once a value with this type is dead, immediately overwrite it with garbage" (instead of the normal behavior of just leaving it around until the register/stack slot/memory area is needed for something else).
Closeout is always difficult. Destructors don't handle errors well. Neither does Go's "defer".
Python's "with" clause seems to do the best job of making sure closeout is handled properly. Take a careful look at the extra arguments to a __exit__ function in Python,s "with". It's one of the few closeout methods where things such as an exception during closeout (an I/O error during file close, for example) is handled in a way that doesn't interfere with other closeouts.
Which is a deficiency of compilers and languages they support: there should be a notion of "verbatim" code, where the compiler doesn't try to apply any code transformation changing the "literal" semantics (as opposed to "intended" semantics).
Of course this would only be possible for languages like C which don't make any advanced abstractions on the hardware arch.
Even though 1.0 isn't out yet, today you can use Rust for many real projects. I'm unsure whether I'd bet my business on it yet, but I'd be open to the idea. And I'm usually a very late adopter.
I've been building a 3d game with Rust and OpenGL, ported from a C++ codebase. So far, my experience has been very positive. Despite Rust's supposed immaturity, it feels more polished than C++ in many ways. Forward progress has been much faster than it was with C++.
Does anyone else have a story (positive or negative) about using Rust in real projects?
I'm betting my business on it. I've written a network search engine in F# and it's in use on the VoIP arm of one of the public telcos. VoIP can generate terabytes of signalling data a day, even while not making much money. (In wholesale, many calls simply don't complete, so you've got a huge amount of data and transactions that pay you $0.)
The challenge with F# is controlling memory usage. Even one extra allocation per packet can make a measureable difference in performance. I ended up doing a ton of unsafe code and manually managing most of the heap. Rust allows me to write fairly high-level code (not as expressive as F# yet but whatever) while getting "best" performance. Inline asm is a bonus, as there's some algorithms for integer compression that can use SIMD for big wins (I can do that in .NET, but it's ugly, and doing it safely means a ~30 instruction thunk). And sometimes in tight loops, I've found it difficult to get .NET to do acceptable codegen, causing double-digit% impacts.
There's also the safety issues writing unsafe code for network-exposed traffic. So Rust is actually more safe than .NET, because I have to toss .NET's safety to gain performance.
The backend management code I can continue to write in F#, and Rust's C-compatibility means it's trivial to interop the code. So I can do "orchestration" of indexing daemons and management APIs and such things in a higher-level language, then for actual indexing and whatnot, just jump over to Rust, seamlessly.
Finally the static compilation means a smoother installation experience for customers. And if I ever ship a closed-source module that executes on the client, I don't need to license Mono for static linking. So that's nice. And the safety guarantees are good, because similar, existing, software in C has put customers at risk before. (I'm not sure if I can effectively market that last part, but hey.)
Rust would appear to have a unique value proposition and I'm very pleased to see it progressing so damn well.
To replace the F# side of things? F# is pretty unique as far as performance/language/tooling goes. It's outclassed in specific cases, but overall it's a great package.
The alternatives you listed aren't known for being able to write top-performance idiomatic code (I've got something _working_ in F#, but it's ugly non-idiotmatic code). The overhead of a GC is just too much to pay when doing linerate networking. Rust allows me to keep nice, high-level, idiomatic style, without paying any overhead. I can account for almost every byte.
Here's a company using Rust to sandbox executable financial contracts: https://codius.org/blog/codius-rust/ . The blog post is unfortunately light on details, but I'm hoping to hear more from them soon. I'd also love to see wycats or carllerche weigh in on Skylight.io's use of Rust from within a Rubygem.
Since you ported from a C++ codebase, how have you handled situations in your game where a lot of inhetitance was used? You may not even use inheritance much in the first place however, just curious about your rust-like solutions for typical c++ game development practices.
I'm very happy to see Rust stabilize, about time we get a systems(ish) programming language with a half decent type system. With that said... I need to get some bikeshedding off my chest:
I hate to let such a triviality lower my enthusiasm for a language so much, but I just can not get over that awful inconsistent closure syntax :/
I don't get it. Most everything else has a nice unique keyword syntax, fn uses (args, in, parenthesis), proc syntax made consistent sense, then lambda is this crazy || linenoise thing that doesn't fit in at all. The "borrow the good ideas from other languages" approach has resulted in a great language, but "cram random syntax from other languages that doesn't fit" doesn't work out so well.
The natural thing to want in a C-like syntax is the "arrow function" closure syntax (like ES6 or C#), but that required too much lookahead to parse. Having a keyword discourages functional style, which would be a shame in a language with a powerful iterator library. So Rust went with the Ruby/Smalltalk-style bars, which are nice, concise, and easy to parse.
It seems like the human parser should be given priority over the computer parser, when considering what is easy and what is hard. The machines work for us!
As far as I know, Rust now has an LL(1) grammar, which means that writing parsers for it can be done by hand (or with the more powerful LALR(1) and LR(1) parser generators). This is very important for humans too, because it means more people are likely to write tools to process Rust code. If you hope to have automatic indentation, auto-completion, refactoring, formatting tools, etc. keeping the syntax simple is really important.
Can't they just expose the parser as a library? Actually, it looks like they did, with the rustc crate.
Hand-writing a parser for some other language leads to madness - just ask the folks who've done SWIG, GDB, or most IDE syntax-checkers. You'll inevitably get some corner-cases wrong, or the language definition will change underneath you long after you've ceased to maintain the tool. Instead, the language should just expose its compiler front-end as a library, and then you can either serialize the AST to some common format for analysis outside the language or build your tools directly on top of that library.
By making the language simple you can easily implement your own parser. This opens up the ability to write native parsers in other languages, say vimscript. By keeping it super simple there -are- no corner-cases.
There are many benefits to this (like the formatters etc that others have alluded to) from things like IDE integration (imagine lifetime elision visualisation, invalid move notifications, etc) static analysis tools and more. None of these tools then need to be written in Rust. It also means it's easier to implement support in pre-existing multi-language tools.
Don't underestimate the necessity of a simple parseable grammar. Besides, people have endured much worse slights in syntax (see here Erlang).
The vim formatters/syntax checkers I've used that actually try to parse the language - other than Lisp, which is the limiting case - are generally terrible. They all miss some corner case that makes them useless for daily work, since they generate too many false-positives on real code.
The ones I actually use all call out to the actual compiler - Python, Go, or Clang for C++.
Just because people write their own parsers doesn't make it a good idea. It may've been necessary when most compilers were proprietary and people didn't have an idea how to make a good API for a parser. But now - just don't do it. You'll save both you and your users a lot of pain.
We are not quite ready to support users of libsyntax and librustc as we are very serious about keeping our stable api stable, and freezing those apis would really impact the future development of the compiler, so it will not be exposed in rust 1.0. We want to eventually get something for this purpose though.
This human parser happens to prefer Rust's closure syntax to any other language's. :) Well, for usage, anyway... the written-out type of a closure for use in function signatures is not nearly as concise.
Parsing a context free grammar in general is O(n^3), but parsing an LL(1) grammar is O(n). That's a pretty huge difference, if you're not careful. Imagine you've got a million lines of code to parse.
> That said, I'm not sure what the exact reason was for choosing the syntax, as that was before my time.
I think the reason is just that it's very concise, and lightweight closure syntax makes things like `Option::map` feel like first-class parts of the language. The closure you pass just sort of seamlessly "blends in".
Note that having especially sugary here is not so uncommon, for example Haskell has `\x -> blah` for Rust's `|x| blah`.
I'm personally very happy that the closure syntax is as concise as it is.
For a brief period, closures had to be annotated in certain cases like |&: args| or |&mut: args| or |: args| to determine whether they captured their environment by (mutable) reference or by value/move.
Now that this is inferred in all cases, closure arguments can just be written as |args| in all cases, just as they were before the current Fn* traits were introduced.
> For a brief period, closures had to be annotated in certain cases like |&: args| or |&mut: args| or |: args| to determine whether they captured their environment by (mutable) reference or by value/move.
That particular annotation controlled the access a closure has to its environment, not how it's captured. |&:|, |&mut: |, and |:| corresponded to the Fn, FnMut, and FnOnce traits, respectively. If you look at the signatures of those traits, you'll see that Fn's method takes self by reference, FnMut takes it by mutable reference, and FnOnce takes it by value. In particular, this means that the body of an FnOnce closure can move values out from the closure (that's why it can only be called once), whereas Fn and FnMut can only access values in the closure by reference and mutable reference, respectively.
The way variables are captured from the environment into the closure is controlled by the "move" keyword. If the "move" keyword precedes a closure expression, then variables from the environment are moved into the closure, which takes ownership of them. "move" is usually associated with FnOnce closures, but it's also needed when returning a boxed Fn or FnMut closure from a function, as you can see below:
fn make_appender(x: String) -> Box<Fn(&str) -> String + 'static> {
Box::new(move |y|
// The closure has & access to its captured variable, but
// it has been moved into the closure so it outlives the
// body of make_appender, thanks to the move
// keyword.
x.clone() + y
)
}
fn main() {
let x = "foo".to_string();
let appender = make_appender(x);
println!("{} {}", appender("bar"), appender("baz"));
}
> The way variables are captured from the environment into the closure is controlled by the "move" keyword. If the "move" keyword precedes a closure expression, then variables from the environment are moved into the closure, which takes ownership of them.
Note: if you don't specify `move`, then the captures are determined in the usual way:
Fortunately with the annotations gone, I think there will be a lot less confusion! The "move" keyword on its own is pretty straightforward, at least once you've learned Rust's ownership model.
|args| expr // upvars captured by reference, can't be called after function has gone out of scope
move |args| expr // upvars moved from function to the closure context (or copied if trivially copyable)
This is simple and good enough for most use cases. If you want more complex schemes, you have to implement them manually, e.g. to reference count the upvars, like Apple blocks do by default, wrap them in Rc and capture that.
Accepting closures is a bit more complicated though.
From the outside looking in, I am mostly impressed with the governance structure of the whole endeavor. It seems to me that it is a great model for other open source projects.
Edit: as someone involved with other, less mature (and less ambitious) open source projects, if you know of pain points in the governance of rust, i'd be interested in learning about them.
There's a governance structure? That's news to me.
I was under the impression that a few primary contributors (mostly/all mozilla employees?) are gatekeepers to merging anything.
Having an "RFC" issue tracker isn't the same as having a governance structure.
Edit: I suppose you could call the above a 'governance structure', but I'm having a hard time seeing anything impressive/different about it from other open source projects
PRs can be reviewed by anyone from a large pool of reviewers. PRs that introduce new features etc. have to come after an RFC is approved, however.
The core team includes Huon Wilson, Yehuda Katz, and Steve Klabnik, none of whom work for Mozilla (though the latter two have done some contracting work). We hope to continue expanding to include other stakeholders.
EDIT: Steve tells me he's currently working as a "seasonal employee" at Mozilla for doing Rust docs, but it's a short term thing.
Maybe a third to a half are Mozilla employees (although it's infamously hard to tell who actually works at Mozilla and is just weirdly into maintaining Rust).
FWIW, the Rust-push team doesn't match the set of reviewers (people to which the integration bot bors will react and merge a PR). Being on that list offers powers like issue tagging and the ability to push to the 'try' branch, but manually merging a PR or pushing straight to master is essentially banned (and would be reverted immediately).
There is an injection from the set of reviews to the rust-push team (Or, at least, there should be), but e.g. people have got privs because they've been doing a lot of triage or need try push, without having review powers.
I have only the vague understanding that there are governance structures in place for open source projects. Can you go into a little detail about what aspects of this project's governance impress you?
aturon's response below (https://news.ycombinator.com/item?id=9046882) outlines most of what I would say - really its the RFC process that I'm most interested with and how that is managed in an open way. There are always pet ideas and desires that complicate processes like this - the volume of RFC's the project has dealt with is impressive with no major community rifts that I have heard of.
I am going to be happy when I can program in rust and not spend the first hour making all my code work with the latest compiler changes. I really do like what rust is offering, but tracking head has been difficult
Me too! I have yet another NES-emulator-in-rust project [1] and I've been hesitating on picking it back up because I only want to convert to "new rust" once.
I saw that integer overflow has been revised: it now defaults to checking overflow in unoptimized builds. I got a little nervous about this when reading the performance-related objections of @thestinger here: https://github.com/rust-lang/rfcs/pull/560
At least optimized builds aren't affected, but it sounds like lots of code (including Rust nightlies) aren't built optimized.
The final draft made an effort to address those concerns -- but also they are completely inapplicable to optimized builds, as kibwen points out. (And if you care how fast your code runs, you really do want optimizations...)
The nightlies are built with optimizations. And trust me, nobody's going to be distributing Rust binaries that weren't built with optimizations because Rust code built without optimizations has performance on par with Ruby rather than C++ (LLVM's optimization passes do a hell of a good job).
Ah sorry, my mistake. I think I had optimized builds confused with "ndebug" builds. But it sounds like only ndebug builds disable overflow checks, and Rust nightlies don't use ndebug.
thestinger seems to have a kernel hacker or embedded systems attitude, which is probably lower than 1% of all use cases; that's not to say it's not important, but he's generated an unproportional amount of, let's say, loud signal.
I'm excited by what Rust will eventually bring to the table if it ever gets popular - a higher level C++ (tools for writing safe native code) replacement without the legacy crap (header files...)
At the same time I don't think I would use Rust 1.0 in production for two reasons :
* the language doesn't seem to be mature enough to be highly productive, for eg. the type system isn't powerful enough to express stuff like Iterable or VectorTN and I'm sure there is plenty of tedious stuff like that along with pains from ownershinp systems
* tools and libs are obviously not there
So I guess I'll wait for early adopters to write the libs and give feedback on their painpoints to the devs.
I've said this before - I think Rust 1.0 is something that I could use (ie. working and stable) but I don't think it's something that I'd want to use yet.
I haven't been tinkering with Rust, merely keep an eye on it until 1.0 lands. For those who are currently more in touch with the situation, does this release date look realistic, without quality suffering? Is the 1.0 release premature, aggressive, or very realistic?
It seems like the standard library has seen a ton of work in the last month or two. I'm surprised at how aggressive the release schedule is, given how things are still churning a good deal.
> For those who are currently more in touch with the situation, does this
> release date look realistic, without quality suffering? Is the 1.0
> release premature, aggressive, or very realistic?
I think it's aggressive, but I don't think it's unrealistic. Personally I had hoped for a late June/early July release to give more time to solidify the docs and to shake out bugs in the compiler.
There will definitely be people who say that the release is premature, but I'm personally not one of them. The core of the language is ready, even if some pieces around the edges could still use some refinement (and will see refinement, backwards-compatibly, in the coming releases).
> I'm surprised at how aggressive the release schedule is,
> given how things are still churning a good deal.
You'd be surprised at how much of a motivator a concrete release date is. :) Churn is happening now because of the impending release, not despite it. The language intends to have a solid compatibility story (via semver) for post-1.0 releases, so everyone who'd been holding off on changes for the past few years has suddenly come out of the woodwork to implement them.
There has been a lot of work on `std`, but it's largely been consistency, cleanup and cutting APIs down -- though the recent path and IO reforms are more substantial, which is why we decided to put out another alpha before finalizing them.
Is it possible to estimate the amount of manpower and time required to develop a new language from scratch till it is stable and reasonably production ready? Adoption of language is different topic, since it depends on users.
Rust and Go are two reasonably new languages. I understand scope and priorities of each language may be different but my idea is to get some approximation/thumb rule for any one before starting similar journey.
As per Github and wikipedia:
Number of contributors for Rust and time taken so far: 840, and 2 years.
Number of contributors for Go language and time taken so far: 424 and 6 years.
Financial details are not known.
It seems developing new language and bringing it to reasonable level is not trivial effort.
1. Is above data correct i.e. are those contributors full time working on those languages i.e. is it full time job of those people?
2. Can we get details like number of developers/number of test engineers/number of documentation writers ...etc?
3. Is it possible to know the total amount of financial resources consumed so far in the effort?
4. Is there any research into resources required for new language development in terms of man power, time, financial resources for various languages?
It is fascinating to see a new language developed in front of us.
To give you an idea, Rust is actually about 8 years old in total, though it was just Graydon for the first four and a half. The language recognizable as today's Rust is about two years old, though. But it still needed that gestation to get to that state, so I'd count all of them.
I think Rust will be especially high because it's pushing hard to synthesize a lot of newish (not invented by Rust, but not often implemented outside of research languages) ideas. And it's has a huge focus on performance, so that makes things harder too.
A scripting language or a JVM language or a PyPy based language (or pick two of those) can make it to a 1.0 much faster.
Both Google and Mozilla have teams dedicated to their languages, but they represent a very small portion of the total number of contributors to the language. I couldn't find exact lists of the team members inside both organizations.
In terms of volume of contributions, for Go, Google employees are by far the most active: https://github.com/golang/go/pulse. In this graph, the 7 top contributors to the project are Google employees. The Rust pulse graph shows the same trend (https://github.com/rust-lang/rust/pulse), with the top 6 contributors being Mozillians (according to a few Google searches).
Something that noteworthy about Go is the "quality" of the team members: Google has Ken Thompson, Rob Pike and Russ Cox working full time on the language. Mozilla may have a few great developers on Rust too, but Google is very serious about Go.
I don't have any information about how financial and human resources are used by Google and Mozilla for the development.
The reason why we recognize the names of the Google people is because of past achievements but when they were working on those we did not know their names either. I think it is a bit much to couple 'quality' to notoriety, for all the same money the Rust people are every bit as good as the Go people and Rust will be their claim to fame (if it already isn't doing that) a couple of years down the line.
In the linked Rust graph, eddyb (the third-highest committer) is a volunteer (an unimaginably prolific one), not a Mozilla employee. kmcallister (the fifth-highest committer) is a Mozilla employee, but not actually on the Rust team (they work primarily on Servo (though there is a fair bit of spillover between the two projects)). Rust has a few other full-time Mozilla employees that aren't represented on that chart for whatever reason (working on feature branches, perhaps?), such as nrc and pcwalton.
pcwalton, at least, has mostly been focusing on making Servo completely amazing. Not a lot on the rustc front these days. Sucks for us, great for Servo. (because pcwalton is fantastic)
Although Graydon did begin playing around with ideas for a programming language in 2006, very little of that language survives beyond the general philosophy of "as fast as C++, but memory-safe and with modern features". There are enormous swaths of the language that are totally unrecognizable if you go back beyond 2012 or so (which is when I personally think "modern" Rust began to emerge). This is about when the borrow checker first appeared, which eventually became the core feature that much of the rest of the language came to be designed around. That said, it still took a lot of iterations on the borrow checker to arrive where we are today.
It depends what you mean by "language" and what you include.
Rust and Go include the compiler, the runtime, some libraries, some package management and some tutorials.
Other things you might want include IDE support, static analysis tools, advanced garbage collectors, GUI toolkit (bindings), slick debugger support, monitoring and profiling engines. These will all add a large amount of time and money to the development of your new platform.
We have not done any scheduling for what comes after 1.0. I've been doing a huge amount of triage to move over issues that used to be in rust-lang/rust to be under that label, too, and to make it more detailed than just 'wishlist.' We'll figure out exactly what's next shortly before the first post-1.0 cycle actually begins.
Since last July or so, Servo has been tracking all of the major features we'd like to see that have been postponed to post-1.0:
https://github.com/servo/servo/issues/2854
A feature that has already seen many proposed rfcs and long discussions is "Efficient code reuse" (a.k.a some kind of inheritance), summarized here: https://github.com/rust-lang/rfcs/issues/349 It was explicitly postponed until after 1.0.
I just wrote a comment on that issue with some half-baked ideas, but I really think that this is one of those "line in the sand" features that will determine (at least for me) whether rust is really staying true to its emerging identity or whether it's on the road to becoming another opinionated kitchen sink language.
The thing about "efficient code reuse" is it probably requires dynamic dispatch. Once you have dynamic dispatch, you suddenly have vtables. But who decides what those vtables look like? Where do they reside in memory? What's the layout of that? If a struct suddenly has an is-a pointer, where is that mentioned in the code? Now my struct isn't just a struct.
I love the rust idea that tons of modern language design still allows for zero-cost abstraction. Inheritance in dispatch starts getting into the land of "putting a lot more stuff in my binary than I asked you to", and I would argue that it's this property more than anything else that keeps embedded programmers and kernel guys safely in the minimalistic land of C.
It would be nice to have a new C, finally. But the more a language has an opinion on runtime layout, behavior, and symbol names, the less C-like it becomes.
Rust got this right when it decided that GC was NOT the correct default behavior for a language. The reason you see people playing with OS kernels in rust and not as much in D is, I think, mainly due to this decision. I think rust should continue carrying this torch.
If not for the ability to have the best of both worlds (a modern language and access to to-the-metal programming with a controllable runtime layout and deterministic performance profile), where exactly is the value in learning how to use the borrow checker?
> The thing about "efficient code reuse" is it probably requires dynamic dispatch. Once you have dynamic dispatch, you suddenly have vtables. But who decides what those vtables look like? Where do they reside in memory? What's the layout of that? If a struct suddenly has an is-a pointer, where is that mentioned in the code? Now my struct isn't just a struct.
1. We already have vtables through trait objects (though not for structs), so this would be nothing new. It's important that we have them, because otherwise common dynamic dispatch would be very annoying to write.
2. Structure layout is already not defined. The compiler is permitted to reorder structure fields as it likes. However, you can force it to adopt your specified in-memory order with the `#[repr(C)]` annotation.
> It would be nice to have a new C, finally. But the more a language has an opinion on runtime layout, behavior, and symbol names, the less C-like it becomes.
The language already has an opinion on runtime layout and symbol names. However, you can specify the layout and symbol names manually if you like (through `#[repr(C)]` in the former case and `#[no_mangle]` in the latter case).
> Rust got this right when it decided that GC was NOT the correct default behavior for a language. The reason you see people playing with OS kernels in rust and not as much in D is, I think, mainly due to this decision. I think rust should continue carrying this torch.
GC has performance costs, while trait objects and symbol names do not, as long as they're opt-in. Garbage collection and virtual dispatch are completely different things; having one in no way moves us closer to the other.
GC has performance costs, while trait objects and symbol names do not, as long as they're opt-in.
Not exactly.
Any program even in a GC'd language can allocate non-GCd data. Even in Java, there is the Unsafe class that can do manual mallocs/frees. C# integrates it with the language. If you do a big pile of work on the non-GC heap then no GC would be triggered and GC is effectively "zero cost" for this code.
And vtable dispatch has a cost for any code that calls a virtual method. Yes, it's "opt in" but this can be misleading. You pay the cost of the virtual method call every time it's invoked. Static languages like Rust and C++ require the programmer to explicitly state which methods can be virtual to try and control this cost, but that isn't the only way.
For example the JVM is capable of eliminating the virtual method dispatch overhead in almost all cases without requiring the programmer to manually specify which methods are virtual. In fact, the JVM can eliminate the vtable overhead for calls that could be virtual, but in fact at that specific call site are not, and even for calls which are only slightly virtual (e.g. there's only really two destinations). So it's possible that in a JIT compiled program you have way more virtual dispatch in theory, but less than the equivalent C++ program would in practice once the code is warmed up.
So vtable and GC performance are very complex topics which no longer reduce neatly down to our intuitions.
>And vtable dispatch has a cost for any code that calls a virtual method. Yes, it's "opt in" but this can be misleading. You pay the cost of the virtual method call every time it's invoked.
So you're basically repeating what he said -- I don't see how the "Not really" you begin with is justified.
Of course you "pay the cost of the virtual method call every time it's invoked".
In languages like C++ or Rust you pay the cost every time the method is invoked. With other types of compiler you may not pay the cost even if the method is marked as virtual, even if it's actually used virtually in other parts of the codebase, due to call site specialisation.
> Garbage collection and virtual dispatch are completely different things; having one in no way moves us closer to the other.
Right, of course not. The comparison was philosophical rather than technical.
My point was that I believe there's a "sweet spot" for a language that is expressive and convenient and modern, but also tries hard not to stray too far from C's spartan abstract machine model (and when it does, it exposes that complexity in a composed pluggable fashion).
I'm beginning to believe rust really has a shot at replacing C and needs to court "bare metal" programmers as well as higher-level programmers to do it; I'm just preemptively registering my wish that rust continue to head down that path.
Rust will not replace c, rust is a more refined c++. If you want something to replace c have: a better type system, raii, better macros, better lifetime management, etc but keep it simple and don't layer it.
The best way to make that wish come true is to use Rust in a project where you would normally use C, and report your experience to help us discover the best ways to support that use case. :)
I think what personally got me excited about this direction were the many "OS kernel in rust" hobby projects [1, 2]. For some reason these strike me as a sort of reverse canary-in-a-coal-mine to judge whether a language is seen as a potential C replacement.
Prior to rust the hobby OS dev community was primarily C / asm with some honorable mentions for other languages. It's also something of a stand-in for the requirements of the professional embedded community.
For these use cases, it's just really cool to be able to use more and more "layers" of the language as you implement more of the underlying abstract machine model.
Rust already supports dynamic dispatch[1], and the 'code re-use' proposals may not look like traditional inheritance. [2] Furthermore, they're being driven _by_ those low-level requirements: it's about being able to efficiently represent things that truly need it, like the DOM.
In the book, you mention that "the std::raw modules contains structs with layouts that are the same as the complicated built-in types". I suppose what I'm asking for is some way for std::raw (plus some contract-like traits and other things) to somehow be the complicated built-in types.
I know that's a hard ask, but the upside would be that anything in the language that's represented by a complex layout + behavior could in principle be replaced by another implementation that preserves the size and behavior contracts. If an implementation is not provided, those features of the language are unavailable. This kind of takes the "you can't use x and y until you give me an allocator" approach and turns it up to 11.
Rust has had dynamic dispatch via since time immemorial, because there are a few things that static dispatch can't express (the original motivating factor was having an array full of different types that all implement the same interface). It is achieved by leveraging a feature of the traits system known as "trait objects": http://doc.rust-lang.org/book/static-and-dynamic-dispatch.ht...
But you are correct in that one of the loudest objections to the potential inclusion of inheritance in Rust is that people don't want two ways of achieving dynamic dispatch, because suddenly then you're in the same boat as C++ where you must decide which incompatible subset of the language to use in your codebase.
Is it safe to assume that anyone wishing to learn Rust can, by alpha2, study The Rust Programming Language and Rust By Example and not need to relearn anything within that scope after 1.0 lands?
The core language is almost completely fixed, and the only real changes will be in unstable areas like associated types. There might be some small API changes but all APIs are marked with stability levels so you should be able to figure out what is stable, and what is not.
Some language guy told me that Rust's ownership system is untenable. He said that researchers tried the same thing years ago and it was concluded to be impossible to work well. I know that's vague, but he claimed to know what he was talking about.
Rust goes a lot farther than academic languages that used regions like Cyclone or the ML Kit. In particular, Rust's regions are only used to enforce stack discipline, and the basic memory management is taken from C++. This way, we avoid the well-known limitations of "classical" region-based memory management.
Besides, consider the fact that we've written hundreds of thousands of code for working, non-toy projects in the language, including the Rust compiler, crates.io, Servo, etc.
Most of the time, Rust's ownership system is helpful. Sometimes, it's a pain though. In these cases, you can actually work around the ownership system if you need too.
For instance, you can use Arc [1] to share memory between threads or Rc [2] to enable thread local GC for a specific variable or RefCell [3] to safely share mutable memory between threads. You can even use raw pointers: it's unsafe (and therefore has to be wrapped in an "unsafe { ... }" block) but possible. Finally, you can call C very easily from Rust [4].
Researchers will often say things like this about techniques they couldn't get working as a way to explain away their failures. Often times some one will come along a couple years later and do exactly what they claimed was impossible or untenable. I've heard people claim that improvements on their techniques are impossible, and that proposed improvements "won't work" when it in fact does (source: I've seen this happen in the research community more than once).
I was one of the people who proposed something like that, back in 2001, as a "strict mode" for C++.[1] I wrote "The basic concept is that pointers and references explicitly declared as auto can't be used in ways that would let the data they contain outlive the scope of the auto variable." This is what Rust calls "borrowing". (That was written before C++ repurposed the "auto" keyword for other languages call "let".)
As a retrofit to C++, that idea wasn't going to work. It would have either been too restrictive or unsafe. It had to be built into the language at a deeper level. That's what Rust does.
One thing I'm not clear on it if can do, and that I'm interested in, is secure destructors.
Say I'm handling crypto, and I'm carting around an ephemeral key. When this goes out of scope, I definitely no matter what, want this zeroised by its destructor - as opposed to just having it (or a temporary copy made by a compiler optimisation!) zombling around the heap, stack or forgotten unused xmm registers because the compiler figured since I don't reference it again, the memory's contents are no longer important.
Current approaches to this involve explicit_bzero(), or other similar memset(0)-and-I-really-mean-it-don't-optimise-this-out techniques. (And a fair bit of testing and prayer when it comes to potential temporary copies or registers.) But unless you're doing it in assembly language, you don't really know. (The stack beneath you, such as the OS, any hypervisors, SMM, AMT, SGX, µcode etc, aside, of course!)
I'm not quite clear what Rust's behaviour with this scenario is. If it can do this easily, even potentially, I am very interested…?