But the environment in this shop was chaotic, and I spent a lot of time fighting fires. Every day brought its own challenges...
Early on, I became aware of a security problem in the product I was supporting. The problem itself was pretty serious, and in my assessment if somebody had managed to exploit this problem the effects would have been significant and widespread.
On numerous occasions, I asked my manager to allocate time into my schedule to fix this problem. Unfortunately, I was spending a huge amount of time fire-fighting other problems, and I simply had no time to put together a proper solution for this security problem. At the time, my manager was not interested in me devoting any time to solving this problem.
This situation went on for a while, until one of our more security-conscious customers started asking some pretty reasonable questions to the effect of "how is this area of your product secured?". As I recall, the official answer (which I wanted no part of) involved some hand-waving.
But I used this incident to stress my point: the security in this area of the product was dangerously flawed, and we needed to fix this problem. And....I stressed a second point too: we could either fix this problem according to our own schedule, or else we could try to fix this problem in a panic because of a public disclosure of this flaw.
I was still in fire-fighting mode -- with very little time to work on this security problem. But, I did manage to write a design specification for how this problem could be solved, and I wrote some proof-of-concept code that showed how the fix could be implemented.
I continued to ask my manager for time to implement the real solution -- time to productize my proof-of-concept, and time to work through a few problems that I hadn't quite solved yet. But this never yielded much.
Instead, what happened was that one day I was told that {very-senior-and-important-engineer} would be implementing a fix for this problem. Apparently it had been decided that this problem was actually important.
{Very-senior-and-important-engineer} had much more experience with the company's products than I did. He had also been involved with the meetings I had held in which I had described the problem, the ramifications of the problem, and my proposed solution. He had easy access to my design document, and my proof-of-concept code too.
Meanwhile, I was still up to neck with other problems.
I was envious that {very-senior-and-important-engineer} was going to get to solve this problem....but, on the whole, I was glad that we'd actually get to solve this problem in a non-emergency mode. This was a pretty complicated problem!..
Also, it seemed somewhat reasonable for {very-senior-and-important-engineer} to fix this problem, since, in my mind, he was the engineer who was responsible for the original flawed scheme.
....
To my great surprise, two days after I was told that {very-senior-and-important-engineer} was going to start working on this problem, he stopped me in the company break-room, rubbed his hands together and in his booming voice proclaimed "IT IS DONE!". I replied "what?". And then he told me that the security problem was fixed.
I was surprised to hear this news, because I estimated that fixing this problem would take....many weeks of uninterrupted effort. I knew that {very-senior-and-important-engineer} could implement code very quickly...and he was far more familiar with the system than I was....but still...
{very-senior-and-important-engineer} told me that he had checked his whole implementation of the security fix into source control a few minutes earlier.
So...I went back to me desk and looked through our source control system. Here I present to you the entirety of {very-senior-and-important-engineer}'s effort. Just to be 100% clear, the {entity} that sits on the other side of this socket.....there is no good reason to have any trust in this entity, and, in fact, you actually have to assume that this entity has been completely compromised by an attacker.
Here's what I saw:
$ svn blame .....
Annotations for /long/path/to/file.java
[...some editing done here....]
***************
public void run()
{
try
{
InetAddress remoteHost = socket.getInetAddress();
String ip = remoteHost.getHostAddress();
System.out.println("\nClient connecting IP = " +
remoteHost.getHostAddress() + " name = " +
remoteHost.getHostName() + "\n");
socket.setSoTimeout(30 * 1000);
InputStream inputStream = socket.getInputStream();
OutputStream outputStream = socket.getOutputStream();
DataInputStream input = new DataInputStream(inputStream);
DataOutputStream output = new DataOutputStream(outputStream);
int time = (int)(System.currentTimeMillis() / 1000);
PublicKey serverPublicKey = serverKeyPair.getPublic();
BASE64Encoder myB64 = new BASE64Encoder();
String b64 = myB64.encode(serverPublicKey.getEncoded());
byte[] bytes = b64.getBytes();
output.writeInt(time);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream outKey = new DataOutputStream(baos);
outKey.write(serverPassPhrase, 0, serverPassPhrase.length);
outKey.writeInt((int)time);
byte[] phrase = baos.toByteArray();
outKey.close();
baos.close();
bytes = encryptAES(phrase, bytes);
output.writeInt(bytes.length);
output.write(bytes, 0, bytes.length);
PrivateKey fooPrivateKey = fooKeyPair.getPrivate();
b64 = myB64.encode(fooPrivateKey.getEncoded());
bytes = b64.getBytes();
bytes = encryptAES(phrase, bytes);
output.writeInt(bytes.length);
output.write(bytes, 0, bytes.length);
output.close();
}
catch (Exception e)
{
e.printStackTrace();
}
This was the entirety of {very-senior-and-important-engineer}'s solution to the security problem.I was pretty gobsmacked when I saw this code. In fact, the solution that I had proposed did in-fact call for some public-private key crypto, but, this is where the similarities between what I had proposed and this "solution" ended. This code didn't look much like my proof-of-concept code either.
Wow....this code... I still shake my head at the thought of it.
I decided to wait a day, to give {very-senior-and-important-engineer} some time. "Perhaps he forgot to check in all of his changes?" I mused. For example, readers who are paying attention will note that there are no changes that correspond to the client side of the connection.
The next day, I concluded that this diff was going to be the entirety of {very-senior-and-important-engineer}'s contribution to the solution to this security problem. That made perfect sense to me, in a twisted sort-of way.
Again....wow...this code.... Wow.
This code amazed me so much that I just HAD TO ask a question about it. So, I stopped by {very-senior-and-important-engineer}'s office and asked him:
Say, I have a question about your code.....the code here goes through the trouble of creating a public/private keypair, and then it passes the public key off to the client....and then right afterwards, it sends the private key to the (untrusted) client as well.....so, my question is this....what was the reasoning behind this?
He answered:
I just wanted to make sure that the client had all of the data that it would need.
Ah....right...
Well, at this point, I knew pretty much exactly what I was dealing with. So, I told {very-senior-and-important-engineer} "say, I think that a few of the points that I made in my design aren't quite captured in this code -- would you mind if I updated it a little bit?". He replied "sure, that would be fine".
....
Over the next {long period of time}, I managed to find small pockets of time, here and there, to implement a fix for this security problem. And, the very first thing that I did was to throw out all of this code....ALL OF IT. When {very-senior-and-important-engineer} proclaimed that this problem was "DONE", he had in-fact implemented around 0.001% of the required fix, and the 0.001% that he did implement was completely and utterly wrong.
Eventually I fixed this security problem. But....it was a really tough problem to solve, one of the more difficult problems I have ever solved in my life.
