Yep. You can even build the clock oscillator itself with chained NOT (or NAND) gates in a feedback configuration.

It's not a great idea for a production machine but it's much more practical than the software analogy of using a Turing machine for everything.