New York Times opens up code - More Power to Open Source!

XSL Cache is a PHP extension the Times is using to cache stylesheets on its Web site. DBSlayeris a tool the team developed to overcome LAMP scaling limitations thatcaused database replication processes to overwhelm the DB connectionlimits.

New York Times senior software architects Jacob Harris andDerek Gottfrid say they've received a mixed reception from thecommunity, because some people just can't understand why a print mediacompany would jump feet first into the open source philosophy. But opensource software use isn't new to the Times, says Gottfrid. "I've beenhere a number of years, and open source has always played an integralpart in everything we do."

Recently, the team has experienced growth, according to Gottfrid, inthat custom applications developed in-house are "shifting from aproprietary posture. As we were building out and replacing oldinfrastructure, there were some gaps, so we wrote additional code. Andsome of those things we're open-sourcing. It's a small, humble effort."

Gottfrid says that upper management has been supportive. "Opensource has grown so much in terms of people being aware of it, and ourmanagers have grown up in that environment. Five or 10 years ago wewouldn't have had this reception."

To support their community efforts, Harris and Gottfrid launched a blog at open.nytimes.comwhere they and other members of the development team share theirthoughts about code, tools, the New York Times site, and sometimes evenfun little widgetsthey've written. Harris, who join the Times only about a year ago, wasa big influence on the launch of the open source projects. "We have alot of internal data that we could potentially share," he says, "and wehave applications we're interested in exposing."

Harris says he's received some pushback from members of the opensource community. "We went to OSCON, and some people were very excitedthat we were there, and some were like, what the hell are you doinghere. A lot of people feel like we're pretty much irrelevant, and thatthey know what our business model should be. When you talk to geeks,they say 'just stop printing the newspaper,' but that would pretty muchbankrupt us overnight."

Gottfrid says they didn't take it personally. "We've all beeninvolved in the community for a long time. We want to figure out how tohave a meaningful engagement with the community, to try to explain someof our problems so we're not just thought of as a monolith. We want tohave a discussion with the community we're wading into. So we'refiguring out the right way to do outreach; we're trying to find asympathetic ear and engage some feedback."

Gottfrid says the most frequent comment he hears about the New YorkTimes' foray into open source Web development is "'That's surprising.'They general think of us just as a newspaper, so they don't understandwhy we would be doing this. Is there a secret angle? But as soon asthey read some posts, they start to break it down and understand. Nowthere's a bunch of people interested in this."

Gottfrid's first release was DBSlayer. "It's a lightweight databaseconnection-pooling tool that we use internally. We were always saying,'We should open-source this thing.' And that got a positive responsefrom our manager -- they said, 'Go ahead, let's see where you can takethis.' It hasn't taken the world by storm, but it is somewhat of aniche thing. It's a new program, so we haven't developed it fully andit is still in the early stages. But people have submitted patches andwe've gotten good comments. It's been a very positive response. If wewant to open-source other stuff in the future, it makes it a lot easierbecause we've established that we can."

Harris says one of the things they've talked about doing in thefuture is "opening up some of our content in the form of widgets orAPIs, and see what the general public would do with it."

Gottfrid hints at the ultimate goal of NYT's move toward opensource. "The key goes back to the title of the blog: Open. We wanted tohave an open dialogue because we value the conversation with thedevelopment community. We want to be a part of that, and have a placeat the table and listen and give feedback and have that respect. As weopen more and more, it gives us a built-in audience that we can seedthese things with, and a number of people that we already have a goodrelationship with."

Gottfrid says there are things you can do to make it easier to jumpinto the open source community -- among them, "Be humble." He says theydon't try to act like they already know where their place is. "We'venever presented our solution as a definitive solution for all problems.It worked well for us, and hopefully someone else can make it work forthem." And don't forget to be persistent. "Just because you releasesomething today doesn't mean everyone will show up the next day anddownload it and praise you and shower you with accolades."

Harris and Gottfrid stress the importance of trusting yourdevelopers. "This [move toward open source] wasn't a corporatemandate," Gottfrid says. "There were no vice presidents involved inthis at all. This was Jake and me monkeying about and saying, 'Wouldn'tthis be cool?' The fact that they empowered us speaks volumes about theNew York Times entrusting non-senior people to have a voice on theoutside and represent the company. There's a growing awareness thatdevelopers play an integral role in how the medium is evolving."

Harris agrees. "We're no longer just a print company, we're atechnology company. We need to express ourselves in technologicalterms. The best way to do that is to give the developers a voice."