In general you should prefer crypto constructions which are a result of global competitions.
For example AES and SHA3.
You should avoid at all costs anything that has been standardized by NIST without going through years of reviews by international cryptographers. Dual_EC_DRBG is a clear example of crypto construction which falls into this category.
This is my general rule of thumb.
However knowing which ciphers one should use is not enough! You absolutely need to know HOW to use them.
A basic and superficial example is AES in ECB mode, which is semantically secure as long as you use a key to encrypt one and only one single block.
Another one is, for example, after how many encrypted blocks a key should be rotated, based on the underlying cipher used.
Once you have learnt how to use the basic building blocks of crypto you are then NOT supposed to write your own implementation and instead use existing ones....there is a small problem with this....they are broken or they either not implement all the necessary crypto constructions you need.
OpenSSL is an example of broken crypto implementation, and instead NaCl does not have TLS implemented.
So this is a short summary and my personal opinion of why crypto is hard.
On top of all this there are not enough experts out there which have the time to review crypto implementations or new and old constructions, and we are living a historical period where we desperately need crypto to protect our privacy.
> The judges who chose AES and SHA-3 as the "winners" of the global competitions are the NSA.
Sure, however this process creates alternatives and if the crypto community thinks the winner is backdoored I am pretty sure we will know it and additionally we will have a valid alternative ready to be implemented.
Additionally if the NSA/NIST modifies the specs for the crypto construction there is still the possibility to implement the original one. See SHA-3 for instance. It was about to be weakened, but the crypto community could still implement the original spec.
> That would include AES and SHA-3.
You cut the rest of the sentence and therefore changed completely the whole meaning. My original sentence included: "...without going through years of reviews by international cryptographers."
Take a look at this video of D.J.B.: https://www.youtube.com/watch?v=G-TM9ubxKIg
He makes a great example with the Dual_EC_DRBG, where many cryptographers told NIST that there could be a backdoor. NIST answer basically was: sorry too late, it has already been implemented !
So in other words, in case of Dual_EC_DRBG the standardization process was all in reverse. First NIST standardized it and then the crypto community started to review it and found problems.
You should avoid at all costs anything that has been standardized by NIST without going through years of reviews by international cryptographers. Dual_EC_DRBG is a clear example of crypto construction which falls into this category.
This is my general rule of thumb.
However knowing which ciphers one should use is not enough! You absolutely need to know HOW to use them. A basic and superficial example is AES in ECB mode, which is semantically secure as long as you use a key to encrypt one and only one single block. Another one is, for example, after how many encrypted blocks a key should be rotated, based on the underlying cipher used.
Once you have learnt how to use the basic building blocks of crypto you are then NOT supposed to write your own implementation and instead use existing ones....there is a small problem with this....they are broken or they either not implement all the necessary crypto constructions you need. OpenSSL is an example of broken crypto implementation, and instead NaCl does not have TLS implemented.
So this is a short summary and my personal opinion of why crypto is hard. On top of all this there are not enough experts out there which have the time to review crypto implementations or new and old constructions, and we are living a historical period where we desperately need crypto to protect our privacy.
So my final suggestions is to take some of your spare time and go through Dan Boneh Crypto 1 at Coursera: https://www.coursera.org/course/crypto
It is worth every single minute.
Once you have done that, I would also suggest you to take the Matasano Crypto challenges: http://www.matasano.com/articles/crypto-challenges/
Finally I want to thank everybody who have taken their time to create and maintain both Crypto 1 course and the Matasano challenges.