Stuff like this makes me seriously doubt whether Python critiques of Ruby's "blocks and methods" (SO MANY CALLABLES; ONE AND ONLY ONE) are actually made in good faith.
I might misunderstand the point you are making, but this actually stems from a "one and only one" mentality: there is only a single entry for "yield" in the grammar, and it is defined to take a specific kind of expression as its argument.
(edit:) (Additionally, there is the "one and only one" mentality that if yield is valid in a expression for a normal method, it should be valid in an expression for a lambda as well; however, that doesn't cause this syntax issue: statement vs. expression does.)
This level of expression is allowed to contain a comma. As an example, you can do "yield 1, 2", which will yield a 2-tuple containing 1 and 2 as elements.
However, if you wish to embed it as an argument in a function call, you will end up with an ambiguity, as arguments to functions are comma separated. You therefore must embed this expression in parentheses, lest you get a weird grammar conflict.
To be clear, this is identical to the case with tuples themselves, with the one caveat that they decided to make it actually be a syntax error to use it in a way that causes the ambiguity (I will be clear that I do not know if I agree with their decision on that regard).
"return 1, 2" <- valid
"a, b = 1, 2" <- valid
"a((1, 2))" <- valid
"a(1, 2)" <- not what you meant
(edit:) JavaScript has a similar problem, by the way: they have all sorts of special variants of expressions that are not allowed to start with a brace or the keyword "function" due to ambiguities at the top levels of statements.
However, the most similar issue in JavaScript is with relation to the keyword "in": "for (var x = 5 in a" could either mean "loop over x, initialized to 5, as all keys of a" or "initialize x as whether 5 is a key in a".
You can figure out the difference once you get to the next token: ")" (the former) or ";" (the latter), but once you get that far you've already committed to specific parse tree formulations for the code you've already seen.
Ah, ok; that didn't seem related to this specific discussion of yield (yield is valuable to have as an expression even in a multi-statement function), but I can see how it is reminiscent.
It should be noted, though, that when you use the lambda keyword, the result is a function, and has the same properties and behaviors as a function (including that it can contain yield and be a generator).
The lambda syntax is simply a way to declare a function as an expression. You can't have a multi-statement block embedded in an expression due to fundamental limitations of normative whitespace.
There really, though, is only one kind of "callable": "function". There are two syntaxes for declaring it, however: "def", which takes a block of statements, and "lambda", which takes an expression.
This is (to me) drastically different than the behavior difference that I believe is what you are referring to in Ruby, where Proc and lambda have different semantics for embedded usages of the "return" keyword.
(That said, I want to be clear, to both you and any random audience: I personally agree with Ruby's decision to have these two forms of callable, per real-world conversations I've had with wycats.)
(afterword) Someone else might find this interesting: I was going to also make the point that both lambda and def result in the same class (function), but it turns out that Proc.new and lambda do as well (Proc).
