When we first started Chapter Three, I had just relocated to rustic Trinidad California in beautiful Humboldt County, experimenting with a different pace of life from breakneck bike-borne Brooklyn. Being out in the country gave me the focused time and low cost-of-living that were essential in bootstrapping our new business, but with Matt and Zack based in SF, we always knew that’s where the natural HQ would be.

While we’re all more than comfortable with the “virtual office” setting — and we still take advantage of that kind of freedom/flexibility to combine productive work and travel, as well as to meaningfully engage our network of partners and collaborators — there are few channels with more bandwidth than face-to-face, a fact that remains true even in the era of iChat, especially when it comes to understanding client needs. We made a decision early on to go for it, to have a real office, and opened our lovely lofty 3rd Street headquarters within the first six months.

I’ve spent quite a lot of quality time there since. It’s a great space that’s given birth to some great Drupal sites, as well as a San Francsico’s #1 Boutique Fixed-Gear Bike Manufacturer. Still, I always thought there was a lot of potential for a satellite spot up here in Humboldt (the “H.C.” as I like to call it).

There are a lot of other bright tech minds up here who prefer to make a living telecommuting and enjoying the North Coast quality of life rather than joining the fray down in Silicon Valley. Seemed like an opportunity to tap into a whole new pool of talent, maybe even prove a different kind of model.

The idea appealed to my sense of how the internet is shifting economic realities and allowing new opportunities for creative minds. In contrast to the typical model of money evaporating as citizens spend their wages at franchise chain-stores on externally-manufactured goods, which let store-managers and workers draw a paycheck, but move all profits outside the local area, here we’d be positioned to bring new jobs and new capital in from all over the world.

Well I’m proud to say that as of this week, that idea is now a reality.

There are more than a few Drupal modules and 3rd party services on the internet for trying to manage the flood of spam that websites regularly receive. However, many family-friendly websites need to go beyond merely blocking crude ads. They need to also fight off the hordes of forum trolls and instigators.

A few services fill this role, such as the excellent WebPurify web-service (nominal yearly fee). We developed our WebPurify module for Drupal while working with PBS. Being one of THE family-oriented destinations on the Internet, they couldn’t let anything pass that wasn’t “COOKIE MONSTER APPROVED”.

Luckily for us, and Cookie Monster, WebPurify exposes a simple REST API, using the now common XML-over-HTTP paradigm. Their documentation lays out the request and response formats, and even provides a simple PHP example. Our module provides a nice wrapper around the intracies of using this REST interface, and even provides comment filtering right out of the box. The exposed API even lets other Drupal developers use WebPurify in their modules.

There comes a time in the life of every young child when begin to dress yourself. No longer are you content with the dorky collared shirt your mother laid out for you each morning or the matching shoes you and your siblings wore because it was just easier that way. Instead, you yearn for a little class, a little flash, and for the older kids some san francisco style.

In this spirit, for our first year and a half of doing business, Chapter Three has rolled with a functional and utilitarian website design created by, well, Josh’s mother. A great graphic designer in her own right, Josh’s mother helped us with an initial design and it helped kick off our successful business.

However, with no disrespect to Jeremy Bentham or Josh’s mother, we recently put our collective heads together with the inspired graphic artist Monica Katzenell and are launching a new website. The new design is a little sleeker and gives a lot more information throughout the site about who we are as a company, the projects we have done, and the way we have participated in the Drupal community.

To make it all happen, on a technical level we make moderately liberal use of the Drupal block system and created several of our pages with the soon to be dominate Drupal content layout system called Panels module. Much of the image resizing is done through Imagecache and there is some neat integration with the Bio Module to generate our About Page listing and store biographical about everyone who works for the company.

Now that we are all dressed up, anyone want to dance?

Just wanted to post this quick trick I’ve been using lately to automagically hide/show useful default text field values (e.g. “Search” in the search box) using jQuery and the ultra-handy Drupal.settings() object.

Here’s the short and sweet copy/pastable jQuery code:

Basically this quick snippit will add a blank array object (ahh, the joys of moving between js and PHP) to the Drupal.settings object — which is useful for all sorts of great javascript functionality, and is integral to Drupal 6.0’s extended AHAH features; if you don’t already know it, do your self a favor and study up — and automatically fill it with any textarea’s default values when a user clicks/tabs it into focus. This lets us clear the default value, but replace it quickly if the user moves on to another element.

As listed, you probably don’t want this on your site, as it will affect things like editing nodes (e.g. title inputs will go blank when you click on them… not what you necessarily want), but it’s easy to tune this to only hit elments within certain forms since every form in Drupal has a unique #id.

Thinking about this, I decided to tune it up and actually make an extended jQuery function for this so it could be more easily applied to speecific elements like so:

This is a pretty nice little plugin, I think, and it shows just how easy it can be to add nice/reusable UI functionality. Happy Drupaling, and go get ‘em jQuery!

(updated w/slight improvement to jQuery fn)
(updated again w/object style improvements from comments)

I simply cannot recommend highly enough Clay Shirky’s new book, Here Comes Everybody: The Power of Organizing Without Organizations. He’s been the “smartest guy in the room” when it comes to the internets for a while now, resisting the lure of hype while still appreciated the revolutionary changes we’re all living through.

This particular passage struck me as an excellent summation of what Chapter Three is all about:

We are plainly witnessing a restructuring of the media businesses, but their suffering is not unique, it’s prophetic. All businesses are media businesses, because whatever else they do, all businesses rely on the managing of information for two audiences — employees and the world. The increase in the power of both individuals and groups, outside traditional organizational structures, is unprecedented. Many institutions we rely on today will not survive this change without significant alteration, and the more an institution or industry relies on information as its core product, the greater and more complete the change will be.

One of Shirky’s overarching points is that any specific technology is only a small part of the equation. What’s critical is that the technology is ubiquitous to the point of invisibility (as the web is rapidly becoming), and that new and helpful social practices emerge which make advantageous use of the new capabilities the technology offers.

Being a part of the Drupal community is fascinating in this respect because we are at once participating in an astounding phenomena (the peer-based production of an amazingly useful piece of software) and helping others use that tool, and the methods used to make the tool itself to pursue their objectives.

It’s a jungle out here, but I wouldn’t have it any other way.

“They’re going to double in a year. That’s insane, in my opinion” Steve Ballmer (Microsoft, CEO), speaking of Google’s rate of growth as an organization

It’s one thing to see the Drupal community’s exponential growth curve plainly plainly graphed. It’s was quite another to walk into a massive conference hall packed with over 800 Drupal developers last week in Boston. It was honestly all a bit overwhelming.

Since I have been a member of the Drupal project, the incredible rate of growth for the community has been a given. When a relatively small piece of the ‘market’ knew what Drupal was, it’s rate of growth seemed more an interesting aspect of the project than a defining characteristic. But as more and more individuals, organizations, and Drupal specialized companies start betting big on the community, the pressure on the project created by this growth will only continue to increase proportionally.

What does this actually mean? Well, in the past few years it has meant:

• The difference between 20,000 users and 250,000+ on Drupal.org: Setting up the site on Kjarten’s spare server vs. deploying on an expensive cluster tuned and tended to by a dedicated team of highly skilled systems engineers.
• The difference between a volunteer organized DrupalCon for thirty people and producing a four day conference for near a thousand costing over $200,000 and hundreds of hours of heroic organizing effort.
• The difference between Drupal powering The Onion being huge deal and the news of another high profile launch becoming almost old hat.

Despite these changes though, in many ways the Drupal project appears very much the same as it did a few years ago. Yes, every release the software really does get that much better. More patches, better patches, rinse, repeat. But this kind of relatively painless growth is only possible because the open-source method has created an incredible medium of software production that can relatively easily scale to meet the demands of virtually any size project.

But past a certain point the open-source method runs it’s course. The demands of supporting communities, even if they are largely web based, catches up with the code. We clearly reached this point with the Drupal project a little more than a year ago. These new type of challenges the Drupal community has been confronted with in the past year will become the major challenges that the community will face. And as Drupal continues to double, they will double.

But of course, so will Drupal.

This year’s Drupalcon was the biggest Drupal Party to date with over 800 attendees and many different sponsors (including Chapter Three of course). Drupal rocked Boston all week and gave everyone in the community a chance to catch up and keep abreast of some of the cool new stuff going on in the Drupal World. Plus, there was some scrumpcious and tasty vegetarian food on the scene.

In my Drupalcon rockstar moment, I took the stage and gave a session with Drupal Genius Neil Drumm and Digital Newspaper Extraordinaire Ken Rickard on Using External Data Sources with Drupal (slides here). The general idea is that Drupal is marvelous for content management, but becomes even better when it integrates with data from external sources and Drupal allows a lot of different ways to make that happen.

Ken kicked off our session by laying the groundwork for why using external data is important and transitioned into talking about some of the basic tools (drupal_execute, drupal_http_request, database switching) and techniques (lazy instantiation, hook_search) needed to make it all happen. He has a lot of good experience around using external data sources from his work in the newspaper world and provided a detailed orientation to the 150 odd people who attended our session.

Neil talked about his work with a wonderful organization called Map Light that records and analyzes information about campaign contributions and congressional votes. For that project Neil had to import *millions* of pieces of information and developed a couple modules to assist him in his work. Job Queue is his module that helps to import external data in batches (instead of all at once) and Import Manager allows administrators to monitor and manage the overall import process.

For my part, I concluded the session by discussing a few specific case examples of how to integrate external data into a Drupal website. I started with an example, which was dear to my heart as a librarian and admirer of philosophy, was to use the Swish-E Module (which I maintain on Drupal.org) to full text index a folder of philosophical essays in .PDF format and make them available via the Drupal search. The second example (see below) was probably the coolest and involved some data mining magic to use the Data Miner API to take a Drupal user’s MySpace name and automatically download their picture from MySpace for display on their user profile. This general technique I was calling “Data Enrichment” and could be used to enhance the data a Drupal site has around either users or specific pieces of node content. Careful consideration here needs to be made to respect both the terms of service and privacy of your users. The final example I presented had to do with some client work Chapter Three did involving the migration of 20,000+ pieces of raw HTML content through a cool audit system set up as several Views and managed by the Workflow System. This system seemed to have a lot of practical value for people and I got a lot of questions about it after the session.

All in all, Drupalcon was awesome and we are looking forward to Drupalcon Europe later this year and Drupalcon San Francisco in 2009.

Yesterday I ran an impromptu lesson in the Drupal Dojo building on last week’s introduction of Druapl 6.0’s new theme layer enhancements, namely built-in template files and automatic preprocess_functions(). We covered a topic my colleague Matt blogged about a couple weeks ago: using template files to take control of forms, which is a great way to take your UI to the next level by making it much more designer-friendly.

Check the screencast here: Fine-tuning the UI: Theming Forms With Templates In Drupal 6.0. If you’re curious about that technique in version 5, Matt’s blog post is a good place to start.

Also, by popular demand I made a short (6 minute) mini-lesson explaining the virtues of devel.module, again in the 6.0 context with the theme_developer tool featured prominently.

With the release of Drupal 6.0, there have been major steps forward in the theme layer. Two of the most important are the standardization of template files and their associated pre-process functions, and the addition of theme.info files which allow the overriding of whole core stylesheets.

This Sunday I gave a 45 minute overview lesson on these topics for The Drupal Dojo. There will be more later, but in brief I think with this core advance, all that remains for Drupal to be a truly designer-friendly platform is better documentation of best practices.

Check out the screencast for details.

Just a little to the left please. Flip it around. Put that on top of this. Call it by a different name. It is the little changes, that seem trivial and small, that often end up being real headaches to make and support our clients in making. Do we really want to try to build capacity with clients by teaching them to adjust #weight in hook_form_alter?

The Drupal Theming System is pretty powerful and, when done right, can offer a good avenue for our clients and their staff to edit, modify, and change their own website content. Its a lot easier to modify HTML files than Drupal module files.

A good example of where this kind of process is needed is on the user registration page. There are a lot of little bits of language and ordering to change and add, but to do so in Drupal module code can get a little hairy. Observe our technique to abstract the user/register form into a flat template file (while maintaing most of the other good Drupal goodness).

Step One: Create a theme override in your module code for the user/register form that executes a _phptemplate_callback to use a separate template file.

Step Two: Expand the theme override function made in step one to remove the titles and descriptions Drupal provides for the form elements. We do this in the theme function (instead of in a hook_form_alter) to preserve the original field titles so they can be used as part of any error messages coming out of form validation.

Step Three: Create "rendered" versions of each of the form elements and set them as variables that can be passed to the template file.

Note: This can also be done with a generic foreach loop (similar to the one in step two) that renders each form element automatically.

Step Four: Create a template file in your site's theme directory to build the user/register form with the customized variables we defined in step three.

Note: This file needs to be the same name as specified in the _phptemplate_callback (example: user_registration_form.tpl.php).

The drupal magic here is that the user registration form is now uniquely customizable by anyone who can edit the theme template. This allows for customized "prompts" for each profile field element, without changing the site-wide field name in admin/user/profile, and it allows for customization of the username and email titles and descriptions.

This technique will need to be modified to support external modules that modify the user/register form like LoginToboggan. It also needs to take into account things like "required" fieldstates.