Building PyCrypto on Amazon EC2

I setup a new AMI Linux instance in the EC2 cloud today primarily for playing around with Python and possibly building some small web apps.  Shortly after firing up the instance, I tried to build and install PyCrypto and ran into some problems.  It was a bit of an adventure.  Here's how I got it working:

Is IDS effective?

Network intrusion detection systems are a popularly considered as a crucial component of network defenses and fit well (in concept) with the idea of defense in depth.  One of the common arguments in favor of IDS, which I first read from Richard Bejtlich, is that "prevention eventually fails."   The argument is persuasive and it seems that we should have some sort of monitoring or detection in place to help us discover when an attack has penetrated or evaded our defenses.  Unfortunately, it's not clear that IDS accomplishes that goal.

In the physical world, the benefit of combining detection with prevention seems more clear.  A fence with razor wire will deter a casual intruder or pedestrian from wandering onto a property, but a fence can be cut or climbed even with razor wire.  We could built a stronger or higher barrier, perhaps a large steel wall, but this is expensive.  It's more cost effective to install motion sensors, alarms, and cameras to alert security staff if someone violates the perimeter.  This does not analogize well to network security.

Blame the management

I'm not the first person to say this, but I really can't stress it enough: security starts with management.  No matter how smart or well-intentioned the employees are, management has to drive security.  Without management support and pressure, individual efforts lack consistency, security measures don't align properly with the business, and, perhaps most important, the incentives are all wrong.  And, when an organization fails at security, it's management's fault.

Optimizing NTLM brute-force

For this post, I'll assume that you're familiar with NT password hashes and/or MD4.  This is based on a paper I wrote in 1998 but did not publish.  I mentioned it in my 2004 article on password protection but it didn't really fit there.

The NT Dialect/NTLM hash in Windows uses the MD4 algorithm.  Password input is treated as Unicode when it is hashed so ASCII characters are converted to 16-bits ('A' = 0x41 = 0x0041).

The MD4 algorithm consists of 48 steps which turn a 512-bit input into a 128-bit output.
MD4 pads all inputs to a multiple of 512 bits, though it can iterate over several
512-bit blocks if necessary.When passwords are 13 characters long or less, the inputs to steps 46-48 are null. An attacker can use this information to reverse the last three steps of any password hash he or she is trying to crack. Subsequently, the attacker
only needs to compute the first 45 steps for each password tried. This results in a
speedup of about 6.25%.

How to fail at cryptography

In my last post, I discussed the number 2¹²⁸ and explained why it’s not possible to brute-force 2¹²⁸ possible keys.  Does this mean that we can use 128-bit cipher like AES with confidence?  Not quite.  Brute-force against AES with 128-bit or larger keys is impossible with any non-quantum computer we will build for the foreseeable future, but that’s only one avenue of attack.  In practice, cryptosystems are broken in a variety of ways.  Sometimes, the algorithm is flawed.  Other times, the algorithm is sound but the implementation is bad.
 

This post attempts to explain, at a high level, some of the technical vulnerabilities that exist in real-world cryptosystems.  I hope that it will help developers, IT and security people gain a basic understanding of the difficulties that exist and give them some ideas of what to look for in code reviews, testing, or product selection.  I also hope to make clear why writing your own implementation is usually a bad idea.  For more information, check out the book Cryptography Engineering and Matthew Green’s blog.  For a look at management/business failures, check out Ross Anderson's  Why Cryptosystems Fail.