One of the biggest differences between folks on the business side and the engineering side is the way they view certain problems. While having tons of data is seen as a serious issue that requires tons of management by developers, business folks tend to see it as a boon, as a way to build a business. Since problems like this only show up after bit of success, these can easily be dubbed luxury problems. A lot of the time, these luxury problems manifest themselves as scaling issues, mostly in the database layer.

As the Behance Network has grown and traffic increased, more and more challenges appeared. We’ve taken these problems as a sign that things are growing. Our MySQL databases are a great example. No matter how poorly the database was designed, when it was brand new and empty, it performed great.  Especially in the incubation stage, when the Behance Network saw very few visitors, and only had a little bit of data to contend with.

Once we put some stress on the server, things fell apart quickly. Looking back, this was a great problem to have, considering the alternative was to never have problems because no one ever puts data into your app. It was also an opportunity for our team to take a look at what happens under load, and find ways to fix it. As we’ve found out, this isn’t just a simple opportunity, but a never ending cycle of struggling to keep up with demand. I hear this is what some business folks call a “viable business model.”

Without problems stemming from growth, there are few opportunities for a team to learn the best way to solve tough problems. These moments should be looked forward to, not dreaded. Notes, knowledge, processes and data the comes from resolving tough problems should be regarded as the most valuable property a company has.

It’s been said a million times that programmers and designers just don’t get along. Just do a Google search and you’ll find plenty of articles attacking the subject from different angles. My favorite reasoning in one article I read was that we can’t get along because one group is right-brained and the other is left-brained. This all seems silly to me, and I don’t think designers and programmers need to clash nearly as often as they do.

There was a SXSW conversation a couple years back about this topic, and our Chief Designer, Matias Corea, said it felt like a therapy session in there. It shouldn’t have to be that way. I believe developers and designers can and must get along to make the best product.

Read more →

Sometimes bugs come along that require significant work to fix. Depending on what project timelines are like at the moment, sometimes fixing the bug isn’t the best option. For example, a race condition in the caching architecture causes pages to be stale. The persistent data store is correct, but the cache is not. To the person who just triggered the update, there’s a bug. The information on the public side is not in sync with the information they just entered.

So, like any other bug, a report will eventually percolate down to the dev team. People scream, fortunes are lost, the svn blame command is used, and the devs who wrote the code pee their pants. Once the chaos dies down, the actual prognosis of this issue can turn out to be extremely grim.
Read more →

Small, obscure optimizations sometimes have the potential to make the greatest impact.  For example, every time a connection is made, MySQL will do a DNS lookup of the host that is trying to connect.  If MySQL is handling many connections, the overhead of an extra DNS lookup can be hefty, simply because of the number of extra operations that have to be performed before MySQL can actually start doing actual work

Thankfully, there is an option in recent versions (4.1+) of MySQL that will instruct MySQL to skip the extra DNS lookup.  It’s a fairly obscure option called skip-name-resolve. The only caveat to using this option is that the users defined the GRANT tables can only use IP addresses as hostnames.  For most MySQL users, this shouldn’t be an issue

While flying to Austin for sxsw, I had a small programming task. Take a string of a few search terms, break it apart and highlight those terms in another string. It’s a straightforward task, and probably a wheel that’s been reinvented thousands of time in the history of computer science. I approached it as an exercise, to see if I could add another squeaky wheel to the pile. My goal was to do it without using any 3rd party code or any resources. I had no access to documentation, google, stack overflow, or any of the other resources I use constantly to get my job done every day.

The code that I produced was bloated, naive, and horribly inefficient (I suspect). While writing it, i knew I wasn’t really on the right path. When I got back to New York, I took a look at it, and more or less decided I had wasted my time. Then I realized I had written it on a plane, and had nothing better to do. I simply got myself into the zone, and wanted to work through a problem until it was solved. After I got over my initial disgust, I wondered what aside from boredom and stubbornness had prompted me to complete the task. Read more →