Speaking of "really is just C with some syntactic sugar" — so, is there really a point in putting effort in learning (and using) Cython over C? Reading the code in the article I wasn't completely sure if I should feel like it is "cool" or "confusing". Not that there was something I actually dislike, but it's somewhat awkward to see C written like Python, so I can't "rate" it myself right away.
If you just scroll through parser.pyx, you might get the flavour of why this is better than straight C. The "core", or "skeleton" of the code is all C-typed, but I "fall-back" to Python for work-a-day stuff like reading and writing config files, collecting the set of labels, etc.
There are also places in the algorithmic code where I use Python data structures. In Parser._count_feats, I use a dictionary of dictionaries. This would be quite annoying in C, and efficiency is quite irrelevant here.
I think using Cython in this way has given me a competitive edge in my research. This library has by far the best published results on speech parsing[1], and its speed/accuracy trade-off on written text is as good as any other libraries out there.
> I think using Cython in this way has given me a competitive edge in my research.
May I ask, have you ever considered a possibility using something like OCaml? As far as I heard it's quite convenient to use ML-family languages for writing parsers and that OCaml generates pretty well-performing code while being more like "Python-expressive" than "C-expressive".
> This library has by far the best published results on speech parsing
Ah, yes, I remember it from seeing a while ago, when you just released it. Never looked in the code, though. Didn't recognize it was you, until this post, however. =)
Your heart is in the same place as mine. Mileage may vary, but I found unless it is the case that I am speeding up a legacy application that has already invested in Python and the Python ecosystem, Cython is less valuable. It may be so because I can churn out C or C++ code productively enough. If I do need to add some scripting, I find adding Lua (now) or Tcl (earlier) lighter.
With Cython I have to remember which subset of Python syntax does the version I am using support, the real time-saver that is Numpy is no longer a timesaver. The reason is Cython has to call back into Numpy's C API for Numpy expressions. If you do that in a loop that is a considerable overhead. The alternative is that you have to write the array indexing code at a low level in Cython. This to me is significant, because avoiding that error prone and tedious part is one the reasons I use Python/Numpy in the first place. Then there is the question of debugging C code that a code generation tool has generated. Usually there is gobs and gobs of it. Cython does try to help you with debugging that, but I find myself better of debugging C code that I wrote myself. Not to mention mature tooling that exists around C or C++. For me, usually what happens is that I Cython'ize a small part, then I have to Cythonize the loop containing it for performance reasons and my Cython keeps swallowing the Python application bit by bit, mostly inward out. So usually comes a time for me when I say screw it, I will just write this in C or C++.
As I said, this is a personal take, your experience with it may vary. I still happily use Cython if its a Python code that I have to speed up, or I have to collaborate with someone in Python. I consider Cython the best tool for doing this. If, however, I were to write something from scratch, I typically just go ahead with C++ now a days, and have lately been poking and prodding at Julia. I like what I see there, a lot.
Another example that I worked on last week is for the UI framework Kivy. Most of the core libraries are in Cython, and particularly for the graphics stuff I think it's a nice balance of execution speed and ease of development. In particular, it's trivial to use the Cython objects from pure Python, without having to mess with the CPython API on either side.
Cython is used to write extension modules for Python. If that is something you want to do, it is better than plain C because the code is more compact and less error prone because some things are taken care of behind the scenes. The point of using Cython is to combine the best of both worlds (Python and C).
