I still can't stand Pydantic's API and its approach to non-documentation. I respect the tremendous amount of hard work that goes into it, but fundamentally I don't like the developer experience and I don't think I'll ever feel otherwise. I use it because my coworkers like it and I've learned its advanced features because I had to in order to get things done, not because I like it.
I would love to see a FastAPI alternative still using Starlette internally, but using Attrs + Marshmallow + Cattrs + Apispec instead of Pydantic. It would be a little less "fast" to write a working prototype, but I'd feel much more comfortable working with those APIs, as well as much more comfortable that my dependencies are well-supported and stable.
The problem of course is not that gluing those things together is hard. The problem is that now someone has put untold hundreds of person-hours into FastAPI, and replicating that level of care, polish, bugfixes, feature requests, etc. is difficult without putting in those hundreds of person-hours yourself.
Could you simplify your point? I was an ardent marshmallow user and when I finally switched to pydantic, it felt like I finally sat down in my life after standing forever. The documentation sounds good enough to me, but importantly the interface pydantic provides to define your json schema is the most elegant interface I’ve seen in any language and miles better than the mess marshmallow provided.
For many of us especially in the SaaS side, speed of these operations is a distant third priority compared to ease of writing and understanding the code, and ensuring reliable less buggy code. The actual compute happens on a cluster with spark or snowflake anyway.
There is no reference doc. The docs cover a lot of material in a small amount of space, buying important pieces of information and mixing up a large number of topics under unintuitive headlines. Reading the source code is occasionally necessary just to figure out how it all works.
The API is a little weird, particularly around defining validators. The parameter name-matching is an "interesting" design choice. Accessing "values" as a dict[str,Any] is messy if you care about static typing, although I can understand why they did it.
Furthermore, the behavior of validators and the exact sequence in which they run is not defined by the docs. It's not that hard to figure out, but it also might change at any time because there's no user contract. Attrs is significantly nicer in just about all respects here, especially their attention to detail in their extensive user guide and reference docs.
Speaking of user contract, there's no clear separation between private and public. Without a reference doc it all looks like fair game, but without a reference doc it also might all change at any moment. Either you stick to the examples, or you're off doing a guess-and-check dance and hoping something doesn't break.
Even with the Mypy plugin, I often have to write `if TYPE_CHECKING` all over any nontrivial Pydantic class consuming data from external sources. Variable annotations in Pydantic are fundamentally not PEP 484 type hints. That's fine, but it's confusing that they're almost the same, and, as above, it's almost entirely up to you to figure out how it all works, either by trial and error or by digging around in the issue tracker and StackOverflow.
Ease of writing and reliability is precisely my big area of annoyance and concern. Speed of (de)serialization is comparatively unimportant (although I don't like the huge amount of overhead involved and I avoid using it in hot code paths).
I also don't like using Pydantic-defined classes very much, because the actual init method signature is just *args, **kwargs, which doesn't work well with any tooling. It feels like being back in the Tornado & PyMongo dark ages where everything is dynamic or dynamically-generated and classes are just glorified hash tables.
I agree that the JSONSchema integration is outstanding. BaseSettings is also a tremendous productivity improvement, I love that I can define a class and immediately get a proper app-wide config reading from both env vars and a dotenv file. I also like the default error messages that tell you exactly which field failed validation. I also like the validator system (once I figured out how it worked), respecting the order in which I define the validators as well as supporting validators that run before or after the default set of validators (pre=True and pre=False respectively). I was probably being a little too negative before, but my annoyance level with the developer-facing API and documentation remains high, and I will gladly jump to an Attrs-based alternative as soon as one exists.*
Please please take a look at V2, both the code and the documentation (although I admit, the documentation for V2 isn't finished).
I (the developer of Pydantic) had many of the same frustrations with Pydantic V2 which is why I've spent so long rewriting it to try and fix these concerns.
In particular:
* we now have API documentation [1]
* we have first class support for validating `TypedDict` which gives you a typing-valid dict representation of your data straight out of validation
* we now have strict mode
* we're working hard to define an exact spec for what validates to what [2]
* we have a strict separation between public/private - everything private is in a `pydantic._internal` module, and we have unit tests that everything which can be publicly imported is explicitly public
* we now use `Annotated[]` for defining custom validations/constraints, together with annotated-types [3]
* the protocol for customising validation and serialization has been significantly improved [4]
I'd really love to hear your feedback on V2 and what more we can do to improve it - your feedback seem unusual reasonable for HN ;-) - please email samuel@pydantic.dev or create an issue/discussion if you have any thoughts.
I too have made similar observations regarding pydantic and FastAPI.
I was evaluating various Python async http frameworks and landed on a similar stack:
- attrs/cattrs for models
- starlette+uvicorn for HTTP/websocket
- validation I’m still on the fence about. I’ll see how far I get with the built in validators offered by attrs. I use voluptuous at work and generally like the DX but it’s in maintenance mode.
This is purely personally preference, I’m sure devs using fastapi+pydantic are more productive in the long run. It almost feels like I’m hand rolling my own fastapi implementation but at the same time I don’t want to be too locked in to frameworks like that.
Ive been burnt by magic frameworks that do too much behind the scenes and there’s something nice about fully understanding what’s going on when you hand stitch libraries yourself.
If you like cattrs, you _might_ be interested in trying out my msgspec library [1].
It works out-of-the-box with attrs objects (as well as its own faster `Struct` types), while being ~10-15x faster than cattrs for encoding/decoding/validating JSON. The hope is it's easy to integrate msgspec with other tools (like attrs!) rather than forcing the user to rewrite code to fit the new validation/serialization framework. It may not fit every use case, but if msgspec works for you it should be generally an order-of-magnitude faster than other Python options.
This looks like exactly what I've been looking for. I just want strong typing, json <-> struct and validation. Seems like it ticks all the boxes + speed benefits which is always nice. I especially find it useful that I can use messagepack for internal service chatter but still support json for external stuff and dump astuple to sqlite.
Depending on how far in you are, starlite/litestar has good documentation and offers another "batteries included" framework. Performance wise it's about the same and the stack is about the same. Fastapi suffers from the "one solo dev in Nebraska" paradigm (check out open prs and old tickets). For me the main draw of litestar is the batteries + better docs + more active development with multiple developers vs most other python web frameworks.
+1 for litestar[1]. The higher bus-factor is nice, and I like that they're working to embrace a wider set of technologies than just pydantic. The framework currently lets you model objects using msgspec[2] (they actually use msgspec for all serialization), pydantic, or attrs[3], and the upcoming release adds some new mechanisms for handling additional types. I really appreciate the flexibility in modeling APIs; not everything fits well into a pydantic shaped box.
I haven't heard of Starlite or Litestar before. Is one a fork of the other? Their documentation intro text is identical:
> {Litestar|Starlite} is a powerful, flexible, highly performant, and opinionated ASGI framework, offering first class typing support and a full Pydantic integration.
>
> The {Litestar|Starlite} framework supports Plugins, ships with dependency injection, security primitives, OpenAPI schema generation, MessagePack, middlewares, and much more.
I would love to see a FastAPI alternative still using Starlette internally, but using Attrs + Marshmallow + Cattrs + Apispec instead of Pydantic. It would be a little less "fast" to write a working prototype, but I'd feel much more comfortable working with those APIs, as well as much more comfortable that my dependencies are well-supported and stable.
The problem of course is not that gluing those things together is hard. The problem is that now someone has put untold hundreds of person-hours into FastAPI, and replicating that level of care, polish, bugfixes, feature requests, etc. is difficult without putting in those hundreds of person-hours yourself.