Tuesday, June 17, 2014

Week 5: Testing Gnocchi!

I am currently not working on OpenSSL. Looking at old code was very tiring and hard to stay focused on, and I felt that I could be learning more by putting effort towards the Gnocchi side of things.

So with that, I figured the best thing I could do is to write tests for the publisher side of Gnocchi! While we don't have a spec, I can still get experience writing good tests. Honeyman once said that one thing that distinguishes UofM grads in the field is their ability to write good tests, so hopefully I can one day live up to that reputation.

I am using Google's C++ Testing Framework to make a test suite for Gnocchi.

Despite being ran from the command line, the output is neat and organized.

Google's C++ Testing Framework in action
Each one of these tests results in OK if the TEST() function returns without error and if each of the EXPECT*() functions result in true. For example, I can have a test as follows:
Test(basicTest, zero){
    int zero = 0;
    EXPECT_EQ(0, zero);
}
This test, if run, will return OK.

The testing framework also allows for EXPECT_THROW(). Basically, if the tested program throws a signal, we can catch it (if we are expecting a throw). If so, we are still OK.

Gnocchi, in its current iteration, is using Byte Vectors (which I think is fine, especially considering we don't really have a specification yet) to store data in memory. What this means is that for me to make unit tests on individual parts of Gnocchi, I have to EXPECT equality of vectors, which currently isn't supported by Google's code. Fortunately, I found a small extension to the testing framework that allows for EXPECT calls on containers that can be iterated (meaning vectors, of course).

The testing framework allows options such as --gtest_repeat=100 and --gtest_break_on_failure, which allow the user to customize his/her testing experience. For me, these options helped me find a bug in the wind() function in Gnocchi, which was removed in the next patch.

As Gnocchi continues to be modified, the tests created will have to change. That's fine for me, however, because I think I'm learning more about writing good tests that can be easily modified :). Unfortunately, I am still only writing unit tests. Probably this week, I will make higher-level tests that can also test to make sure we have expected behavior on the directory level. Further down the road, perhaps I can benchmark Gnocchi's sign/encrypt/decrypt/etc functions.


Misc:
I noticed Gnocchi was taking an abnormally long time to compile on my system, and I wanted to know why. My laptop's processor isn't the fastest (~2.66ghz Arrandale), but I even tried compiling on a desktop Haswell i7, which didn't result in much of a speedup. Part of the problem could be that I'm developing in a virtual machine (this is actually preferred for me because I have the best keyboard/mouse input that way). I give it 3gb memory (which should be enough?), 3 hyperthreaded cores, and it's stored somewhere on an SSD. But perhaps read/writes are not very fast on virtual machines. Perhaps compiling is more memory intensive than 3gb. Ultimately, however, there are (right now) many dependencies in each part of Gnocchi, which I learned contributes towards long compiling times; hopefully we can cut down on that and reduce compiling times in the future.

Monday, June 9, 2014

Week 4: OpenSSL part 3

In order to test potential exploits I thought of, I setup a private web server using an old computer box. It was my first time doing so, and I learned a bit about routers, networking, ports... (which unfortunately is not the topic of this blog post).

OpenSSL's verify utility can be used to verify certificate chains. In order to do this, one would have to concatenate the chain of certificates into one file, from root-level first. For example, umich was signed by InCommon, which was signed by AdTrust, a root-level Certificate Authority. To have OpenSSL verify it, one would use the commands:
cat adtrust.crt incom.crt umich.crt > combinedCerts.crt
openssl verify -CAfile adtrust.crt combinedCerts.crt
Here, -CAfile specifies a trusted root certificate.

I tried to step through the process of making a fake certificate.
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr
openssl x509 -req -in server.csr -CA incom.crt -CAkey server.key -CAcreateserial -out fake.crt -days 500
Unfortunately, the 3rd command failed because the "private key didn't match the certificate," when the error in OpenSSL was caused by the mismatch in public keys. (Now that I think about it, I was probably misinterpreting private key to mean the (m,d) pair rather than the server.key file.)  To have the public keys match, I modified the first 6 base-64 lines of server.key to contain the public key of the certificate I was signing with (incom.crt). This was called fakeIC.crt. The command,
openssl x509 -req -in server.csr -CA incom.crt -CAkey fakeIC.key -CAcreateserial -out fake.crt -days 500
was ran, which succeeded without error. Verification with OpenSSL in the manner described above succeeded as well! Apparently OpenSSL's verify utility only checks to see that certificates... have the names of their signers on them? I put the certificate chain (adtrust-inCommon-fake.crt) onto the ssl portion of my web server and tried accessing the https port via Chrome. There was an error- fake.crt had an invalid digital signature.

My thoughts at this point were: Perhaps it's because I didn't use x509v3 extensions, which the umich certificate had. Or perhaps I could fake-sign with umich's certificate rather than incom's certificate.

I found out that signing certificates with v3 extensions requred a v3extensions file, which had the data to be placed in as v3 extensions. I made such a file, v3.ext, and wrote it with the same information as the v3 extension information on umich's certificate. Signing with v3 extensions, as well as signing with umich's certificate, both failed the chrome test.

At this point I figured it was because of the actual "signature" portion of the certificate being wrong (silly me). Apparently actual SSL checks the signature, unlike OpenSSL's verify.

I also found that batch requests for OpenSSL's req utility don't actually make/sign multiple certificate requests. Rather, they omit the part of openssl req where the user inputs their certificate subject/credentials via standard input. Optionally, the subject/credentials can be taken from a file. So for someone making many requests using similar subject/credentials, they might use the batch utility as well as a script, but ultimately OpenSSL does not handle requests in bulk. This was further verified by UofM ITS, who said they don't handle multiple certificate requests at the same time either.

Monday, June 2, 2014

Week 3: Using OpenSSL...

...to create signed certificates without the private key!

While dying of confusion from looking at OpenSSL code, I stumbled upon something interesting. The function that checks the private keys of the CA Certificate file and the CA key file actually just checks the public keys. Because of this, it should be possible to use OpenSSL's utilities to create a signed certificate using just the CA Certificate PEM file: Modify the base64 encoding of the CA Certificate PEM file to use the same public key as any random CA key file. (Alternatively, one could modify their CA key file to have the same public key as the CA Certificate PEM file.) This seems like it would work with the UM Web CA Certificate. However, I don't think something like this would work with something root-certified such as the InCommon CA.

For more clarity, on a standard self-signing process, several files are involved:

  • CA/Root Private Key file - rootCA.key
  • CA/Root Certificate file - rootCA.pem
  • Certificate Serial number file - rootCA.srl 
  • Certificate Request file - newcsr.pem
  • Signed Certificate file - newcert.pem
Private keys and root certificates have matching public keys in their PEM encoding, so programs like OpenSSL can verify that the certificate file's modulus and exponent match the private key's. Then, someone (possibly yourself) generates a certificate request and saves it into a file. Various details go into this, such as  Country Name, Locality Name, Organization Name, Common Name, and email. I looked into the code that corresponded to the generation of certificate requests, and luckily, one cannot use a buffer overflow attack to wreck havoc on any online server that creates certificates using openSSL's latest version. Finally, the PEM formats for the CA Certificate file match the format for the Signed Certificate file. 

The PEM format, is, alas, a bit complex. For RSA key files, you have some header information followed by the modulus, public exponent, private exponent, p, and q. This is simple compared to the PEM formats for Certificate Request and Root/Signed Certificate files. They have headers (whose meaning has to be looked up on old documentation websites), followed by all the subject info (Country Name, State Name, etc.) interlaced with even more header information. At some point, the PEM file format describes the encryption info (like 1024 bit RSA Encryption), followed by the modulus, exponent, certificate attributes, and signature algorithm (like sha1WithRSAEncryption). 

Converting to and from base64 to hex takes a bit of effort as well. What we have is actually several lines of 64-character "words." Each line uniquely translates to a 384-bit word. But you might have part of a modulus beginning on the middle of a line and ending in the middle of another line. In order to modify this, special precautions have to be kept. Conversions from hex to base64 don't make sense unless it's a 384-bit (or 96 hex characters, where 0x41 is 2 hex characters) word. Because of this, one would have to find every affected line by a change, modify their hex values, and then convert each line individually from hex to base64. 

OpenSSL is notoriously difficult to read through. There's very little documentation, and function calls literally go everywhere- they jump around different source files in different folders. The certificate signing is done in the apps/x509.c file, whose simple sign function actually has its tree of function calls going to perhaps at least 25 other *.c files, such as crypto/x509/x509_vfy.c, and referencing files like /include/openssl/x509.h. 

I'm also trying to envision the Gnocchi top level structure while I'm at it. I really don't like sifting through OpenSSL code, which I chose to do because I wasn't familiar with the server/javascript involved with the Gnocchi development side. We still have to figure out a good key revocation scheme, meaning our idea of having 4 key portions-- Publisher, High-Bandwidth Content Server, Client, and Low-Bandwidth Key Server-- is not set in stone.

I talked to the guys at ITS, and it looks like we're getting a chance to see how their certificate approval works!