Chapter Three LLC

solving obvious problems

Rolling Securely: 5 Tips to Protect Your Personal Data

Matt Cheney

We live in a very insecure digital world where it is very possible for those unsavory elements of our society to spy, sniff, inject, and otherwise interfere with the work we do on our computers.

In real terms the odds of anything particular bad happening to any of us while we build Drupal websites and sell awesome fixies is low, as a matter of best practices and general sanity here are a few things you can do to keep your local workstation or laptop a little more secure.


1.) Make sure to use a secure/encrypted session to the company email.

There are generally two ways you can access your webmail account (see below for examples from our company) and while they look and work the same way there are critical differences. If you use the normal “HTTP” connection, the emails you are sending and receiving can be snooped/hijacked by anyone on your same wifi or cable modem network. You must use “HTTPS” to have a secure/encrypted session to the email provider making it more difficult to snoop on your traffic.

HTTP - http://mail.google.com/a/chapterthreellc.com (bad - dont use this)
HTTPS - https://mail.google.com/a/chapterthreellc.com (good - use this)

2.) Be aware that IRC channels are often *not* secure (same goes for private messages through IRC)

The IRC network as a whole is not encrypted and conversations and passwords that flow through it can be easily snooped and recorded. This is true not only for people on your local network, but also for server operators and people with some smart masking/spoofing skills. IRC is often full of very shady people. In general, IRC should not be used to communicate sensitive information or server passwords.

3.) Securely communicate with your colleagues and clients using AIM *with* the OTR

The best way to discuss sensitive topics or share server/login credentials is through AIM (AOL Instant Messenger) using a plugin called OTR (Off the Record). If both users have this plugin, their communication is encrypted and cannot be read by anyone else. Without this plugin, anyone can read your messages and all data goes through the AOL main network to be relayed out….

Main OTR Page: http://www.cypherpunks.ca/otr/
Adium - http://www.adiumx.com/ (included by default)
iChat - http://chris.milbert.com/AIM_Encryption/#MacOTR
Pidgeon/GAIM - http://www.cypherpunks.ca/otr/README-pidgin-otr-3.2.0.txt and http://www.cypherpunks.ca/otr/binaries/windows/pidgin-otr-3.2.0-1.exe
Trillian - http://trillianotr.kittyfox.net/downloads.php

4.) Encrypt Your Harddrive

There are lots of pieces of sensitive data on your harddrive - from server passwords to saved emails and logged IRC/AIM chats - that should be protected. The best way to protect this data, of course, is to keep your laptop under your control, but if the laptop is stolen or misplaced bad things can happen. A very simple solution is to encrypt your user directory so others cannot read that data (even if they steal your computer).

OSX: http://www.youtube.com/watch?v=VQ9HQwGC_64
Windows: http://www.youtube.com/watch?v=coAMTC7uFxA or http://www.youtube.com/watch?v=-8YTaBVB7f0

5.) Consider Locking Your User Session When You Leave

Along the same lines as encrypting your harddrive, its helpful to also “lock” your computer session when you are not using it. This is less important in the office or a trusted work space, but if you have sketchy friends or are in a public place locking your computer will prevent others from stealing your data or pretending to be you.

OSX: http://technology.cca.edu/support/knowledgebase/index.php?article=24#mac
Windows: http://technology.cca.edu/support/knowledgebase/index.php?article=24#Win...

HOWTO: Upgrade an SVN Managed Drupal Installation (without CVS)

Matt Cheney

At Chapter Three we use the Subversion Version Control System to manage our client and internal Drupal projects. When we kick off a new project we roll out the latest version of Drupal, stick it in the SVN repository, and start developing.

As a matter of best practices it is usually a good idea to check Drupal directly out of the Drupal CVS repository which makes upgrading as simple as "cvs update -dP -r DRUPAL-5--X". However, we have a number of sites that were not checked out in this way and this makes updating Drupal sort of a pain. Our SVN management system wants us to update each file individually (so the changes can be versioned), but individually updating each of Drupal core's 300+ files is a little bit tedious.

However, with a little command line wizardry we can quickly download a copy of Drupal and generate a command to copy the new version of each Drupal core file on top of our installation in a way that can be committed up to SVN. To do this we need to wget a copy of Drupal and then use the all powerful find command to detect and copy the relevant files.

# Download a Copy of Drupal, Extract it, and Enter its Directory
wget http://ftp.drupal.org/files/projects/drupal-5.9.tar.gz
gzip -d drupal-5.9.tar.gz
tar xf drupal-5.9.tar
cd drupal-5.9

# Use Find to Detect Each Local File in Drupal Core and Execute a
# Command to Copy the File to the Live Version of the Website
find * -type f -exec echo cp {} /path/to/www/{} \;   # this just echos the  commands
find * -type f -exec cp {} /path/to/www/{} \;  # this actually runs the commands

To make sure the copy worked we can use the same find command do an individual diff on each file. This will assure us that we successfully updated our website with the version of code we downloaded from Drupal. This command can also be used prior to updating to compare different versions of Drupal to see what core changes are going to be (or have been) made.

find * -type f -exec diff {} /path/to/www/{} \;

The Day You Dress Yourself: The New Chapter Three Website

Matt Cheney

There comes a time in the life of every young child when begin to dress yourself. No longer are you content with the dorky collared shirt your mother laid out for you each morning or the matching shoes you and your siblings wore because it was just easier that way. Instead, you yearn for a little class, a little flash, and for the older kids some san francisco style.

In this spirit, for our first year and a half of doing business, Chapter Three has rolled with a functional and utilitarian website design created by, well, Josh’s mother. A great graphic designer in her own right, Josh’s mother helped us with an initial design and it helped kick off our successful business.

However, with no disrespect to Jeremy Bentham or Josh’s mother, we recently put our collective heads together with the inspired graphic artist Monica Katzenell and are launching a new website. The new design is a little sleeker and gives a lot more information throughout the site about who we are as a company, the projects we have done, and the way we have participated in the Drupal community.


To make it all happen, on a technical level we make moderately liberal use of the Drupal block system and created several of our pages with the soon to be dominate Drupal content layout system called Panels module. Much of the image resizing is done through Imagecache and there is some neat integration with the Bio Module to generate our About Page listing and store biographical about everyone who works for the company.

Now that we are all dressed up, anyone want to dance?

Drupalcon Boston Recap: Bringing Guests to the Drupal External Data Party

Matt Cheney

This year’s Drupalcon was the biggest Drupal Party to date with over 800 attendees and many different sponsors (including Chapter Three of course). Drupal rocked Boston all week and gave everyone in the community a chance to catch up and keep abreast of some of the cool new stuff going on in the Drupal World. Plus, there was some scrumpcious and tasty vegetarian food on the scene.

In my Drupalcon rockstar moment, I took the stage and gave a session with Drupal Genius Neil Drumm and Digital Newspaper Extraordinaire Ken Rickard on Using External Data Sources with Drupal (slides here). The general idea is that Drupal is marvelous for content management, but becomes even better when it integrates with data from external sources and Drupal allows a lot of different ways to make that happen.

Ken kicked off our session by laying the groundwork for why using external data is important and transitioned into talking about some of the basic tools (drupal_execute, drupal_http_request, database switching) and techniques (lazy instantiation, hook_search) needed to make it all happen. He has a lot of good experience around using external data sources from his work in the newspaper world and provided a detailed orientation to the 150 odd people who attended our session.

Neil talked about his work with a wonderful organization called Map Light that records and analyzes information about campaign contributions and congressional votes. For that project Neil had to import *millions* of pieces of information and developed a couple modules to assist him in his work. Job Queue is his module that helps to import external data in batches (instead of all at once) and Import Manager allows administrators to monitor and manage the overall import process.

For my part, I concluded the session by discussing a few specific case examples of how to integrate external data into a Drupal website. I started with an example, which was dear to my heart as a librarian and admirer of philosophy, was to use the Swish-E Module (which I maintain on Drupal.org) to full text index a folder of philosophical essays in .PDF format and make them available via the Drupal search. The second example (see below) was probably the coolest and involved some data mining magic to use the Data Miner API to take a Drupal user’s MySpace name and automatically download their picture from MySpace for display on their user profile. This general technique I was calling “Data Enrichment” and could be used to enhance the data a Drupal site has around either users or specific pieces of node content. Careful consideration here needs to be made to respect both the terms of service and privacy of your users. The final example I presented had to do with some client work Chapter Three did involving the migration of 20,000+ pieces of raw HTML content through a cool audit system set up as several Views and managed by the Workflow System. This system seemed to have a lot of practical value for people and I got a lot of questions about it after the session.

All in all, Drupalcon was awesome and we are looking forward to Drupalcon Europe later this year and Drupalcon San Francisco in 2009.

example of using external data

HOWTO: Fully Theme and Customize the Drupal User Registration Form

Matt Cheney

Just a little to the left please. Flip it around. Put that on top of this. Call it by a different name. It is the little changes, that seem trivial and small, that often end up being real headaches to make and support our clients in making. Do we really want to try to build capacity with clients by teaching them to adjust #weight in hook_form_alter?

The Drupal Theming System is pretty powerful and, when done right, can offer a good avenue for our clients and their staff to edit, modify, and change their own website content. Its a lot easier to modify HTML files than Drupal module files.

A good example of where this kind of process is needed is on the user registration page. There are a lot of little bits of language and ordering to change and add, but to do so in Drupal module code can get a little hairy. Observe our technique to abstract the user/register form into a flat template file (while maintaing most of the other good Drupal goodness).

Step One: Create a theme override in your module code for the user/register form that executes a _phptemplate_callback to use a separate template file.

<?php
function theme_user_register($form) {
 
$vars = array();
 
$output _phptemplate_callback('user_registration_form', $vars);
 
$output .= drupal_render($form);
  return
$output;
}
?>

Step Two: Expand the theme override function made in step one to remove the titles and descriptions Drupal provides for the form elements. We do this in the theme function (instead of in a hook_form_alter) to preserve the original field titles so they can be used as part of any error messages coming out of form validation.

<?php
 
foreach($form as $key => $value) { // loop through top level
   
if (is_array($form[$key])) {
     
$form[$key]['#title'] = '';
     
$form[$key]['#description'] = '';
      foreach(
$form[$key] as $key2 => $value2) { // loop through second level
       
if (is_array($form[$key][$key2])) {
         
$form[$key][$key2]['#title'] = '';
         
$form[$key][$key2]['#description'] = '';
        }
      }
    }
  }
?>

Step Three: Create "rendered" versions of each of the form elements and set them as variables that can be passed to the template file.

Note: This can also be done with a generic foreach loop (similar to the one in step two) that renders each form element automatically.

<?php
 
// Set up the Vars Array
 
$vars = array();

 
// Render Specific Fields You Want on Your Registration Form
  // note - the specific location of the element in the form array varies
 
$vars['name_element'] = drupal_render($form['account']['name']);
 
$vars['mail_element'] = drupal_render($form['account']['mail']);
 
// continue for each field you want...

  // Don't Forget the Submit Button 
 
$vars['submit_button'] = drupal_render($form['submit']);

?>

Step Four: Create a template file in your site's theme directory to build the user/register form with the customized variables we defined in step three.

Note: This file needs to be the same name as specified in the _phptemplate_callback (example: user_registration_form.tpl.php).

<div class="user-register-element">
  <label>Enter a screen name:</label>
  <div class="user-register-element-input">
    <?php print $name_element; ?>
  </div>
  <div class="user-register-element-description">
    Screen names can be up to 13 characters in length.
  </div>
</div>

<div class="user-register-element">
  <label>Enter an Email:</label>
  <div class="user-register-element-input">
    <?php print $mail_element; ?>
  </div>
  <div class="user-register-element-description">
    Emails must be valid.
  </div>
</div>

<?php // continue on for each rendered form element ?>

The drupal magic here is that the user registration form is now uniquely customizable by anyone who can edit the theme template. This allows for customized "prompts" for each profile field element, without changing the site-wide field name in admin/user/profile, and it allows for customization of the username and email titles and descriptions.

This technique will need to be modified to support external modules that modify the user/register form like LoginToboggan. It also needs to take into account things like "required" fieldstates.

Drupal Administration: Keeping it Simple

Matt Cheney

Much like the parent who gives their child the keys to the family car, there comes a time in our client engagements where we set up administrative accounts for our clients to begin managing their own content. At Chapter Three we call this Capacity Building and try to make the process as smooth and intuitive as possible. The problem is the default Drupal administration interface makes this difficult.

In general, the Drupal administration navigation menu (see right) is simply too complex to be really intuitive. There are all sorts of options (read noise) available that have little relevance to the particular administration needs of our clients. Some of this menu can be distilled down through permissioning, but with general permission categories in Drupal like “access administration pages” its not possible to remove all the options. Plus, many of the most relevant options are buried several layers deep in the navigation. Its unlikely our clients will regularly find them on their own.

In the past we have build a static “administration page” where we listed the major administration pages and provided links to each, but with our latest project (that had a number of different editors and content managers who all needed to update the site) we wanted to improve our business practices and give our clients a better way to manage their website. The solution we came up with was to provide a constantly present top menu that would only appear for site administrators and would allow them easy access to their specific administration options in a nice drop down menu. We populated the menu through the Drupal menu system with the 20 or so administration pages that were needed to manage the site. This menu display was accomplished by using the SimpleMenu module and can be seen below.

To manage the individual pieces of content on the site we built one, general purpose “Content Management View” using the wonderful Views module. The idea here is to provide a consistent UI to manage each type of content and to allow site administrators to filter, search, and sort that content for their editorial and review purposes. Setting this interface up as a View allows us a lot of flexibility later to add additional administrative options and support different content types.

The end goal is to make it as easy as possible for our clients to easily work with the Drupal CMS and to manage their specific content. This kind of system is something we intend to refine and deploy in future sites.

Sling that Data: Drupal from 30 miles away

Matt Cheney

Like a fresh cup of coffee with extra soy creamer, a daily morning trip to drupal.org to browse Drupal Planet or see recent activity helps to start the day off right. The current infrastructure issues not withstanding, if I have an internet connection I can get my open source blue alien fix. But what happens if I am a long way from an internet connection? Say, in the middle of a fishing village in rural Mexico or in the woods during a 24 hour bike race. In that case, you need to sling that data.

The general idea behind slinging data is that if you have a stable internet connection somewhere in the general area (within 20 or 30 miles) you can “sling” that data through a wifi bridge to your actual destination. At the ends of the bridge should be a pretty nice wifi antenna. We used 900MHz Yagi antennas which fold down nicely and are easy for transport. The antennas can be mounted on a number of things, but we choose (for ultra portability) to mount them to large wooden rods secured by christmas tree stands. The tricky party in setting this up is to make sure that both antennas point at each other. This is pretty easy if you are just shooting wifi within your immediate line of sight, but for longer connections (like between the chapter three office and the hills of berkeley across the bay) its probably best to use GPS coordinates to set up the connection.

Once you have two antennas pointing at each other, you need to attach the antennas to a couple of generic wireless routers. We used the trusted WRT54G routers with the detachable antennas and connected the routers to the antenna using some RF cabling we found online. On a software level, we immediately replaced the standard Linksys firmware with the industrial strength DD-WRT firmware created by Brainslayer. This allowed the power frequency to be boosted to 84 mW and the routers to be associated and placed into bridge mode (a built in option). Once they were powered on and associated, wow, bam, suprise, a wifi link was activated.

Of course, with that setup the wifi range is weak. Like 1 mile, maybe 2 miles. The real secret sauce is 4 W 915 L Series Antennafier. If you hook that amplifier on both ends of the connection, set its power to full, and start the bridge your wifi signal strength will dramatically increase. Over a two mile link, this setup was able to get a better connection than a normal a normal connection between the two wifi routers in the same room! Under optimal conditions (like in the desert or by from a boat on the ocean) this could send internet 30+ miles. Under non-ideal conditions, a your range will sadly be limited to 15 to 20 miles. So find some remote locations, invite some friends, and get your do fix on a boat, beach, or backyard.

Drupal in Paradise

Matt Cheney

Our office in San Francisco is pretty cool, but to commemorate our first year in business we decided to roll down the coast and do a little Drupal development somewhere even more wonderful than San Francisco. So we got ourselves some airline tickets, got hooked up with a great house with glass walls right on the beach, invited a few friends, and moved the operation to a small fishing village about two hours by dirt roads north of Cabo San Lucas. Why do development with Drupal in an office when you can do it on the beach?

We are equipped with some WIFI internet broadcast from an amped twenty foot omni-directional antenna, a couple bags of vegetarian food brought from rainbow grocery in San Francisco, Zack’s overflowing iPod, and all the fresh oranges, mangoes, avacados, cantaloupes, and strawberries we can eat. It’s a perfect setup to spend the next two weeks building websites, rocking emails, doing a little skype with our clients, and having a two week long Drupal beach party. There are definitely plenty of distractions and we might have some complications from an incoming hurricane, but there is a certain tranquility in coding from a beach in Mexico.

HOWTO: Keeping Drupal on Your Keychain

Matt Cheney

I am sure it’s a common problem. You are a rocking Drupal software engineer and you are talking to a particular cute guy or girl at a pretty awesome party. Naturally, the conversation turns to open source software projects and you boast that you work on Drupal - the coolest content management system out there. Since this boast is bound to be followed up with questions like “what is Drupal?” or “why is Drupal cool?” - wouldn’t it be grand (instead of trying to explain just how cool the garland color picker is) to whip out a portable USB key, stick it in an available computer, and show them the wonders of Drupal right then and there?

Install Ubuntu onto a USB Key

  • Download Ubuntu and mount [mount -o loop ] the image
  • Use fdisk [fdisk /dev/sda] to delete any existing partitions [fdisk: d], add a new partition [fdisk: n,p,1,*enter*,+700M], set its type to FAT16 [fdisk: t,6], and make it active [fdisk: a, 1], add another partition [fdisk: n,p,2,*enter*,*enter*], and save all the changes [fdisk: w]
  • Install the FAT16 filesystem [mkfs.vfat -F 16 -n usb /dev/sda1] and the ext2 filesystem [mkfs.ext2 -b 4096 -L casper-rw /dev/sda2]
  • Install the Linux System Tools [syslinux -sf /dev/sda1]
  • Mount the USB Key [mount /dev/sda1 /mnt] and copy over Ubuntu [cp -rf casper disctree dists install pics pool preseed .disk isolinux/* md5sum.txt README.diskdefines ubuntu.ico casper/vmlinuz casper/initrd.gz install/mt86plus /mnt]
  • Copy this special file to the USB key [tar xf usyslinux.tar ; cp syslinux.cfg /mnt]
  • Run Lilo [lilo -M /dev/sda]

These instructions and system file are adapted from this tutorial. For the purposes of these directions the USB drive is assumed to be at /dev/sda.

Install and Configure the LAMP Stack

  • Install the stack [apt-get install php5-mysql mysql-server php5 apache2]
  • Enable Apache’s mod-rewrite [ln -s /etc/apache2/mods-available/rewrite.load /etc/apache2/mods-enabled/rewrite.load]
  • Allow .htaccess overrides in apache’s configuration file [vim /etc/apache2/sites-available/default] by changing all “AllowOverride None” configurations to “AllowOverride All”
  • Edit your /etc/rc.local file [vim /etc/rc.local] to add the following line [mount /dev/sda2 /mnt], run the command on the command line [mount /dev/sda2 /mnt], and add a web root directory [mkdir /mnt/webroot]
  • Edit the Apache configuration file to point to your new webroot [vim /etc/apache2/sites-available/default] to change all references to /var/www to /mnt/webroot. ** Note - its important to setup a separate webroot on the ext2 system since apache is not a fan of the FAT16 filesystem **

Install and Configure Drupal

  • Create your MySQL database [mysql -u root], set the appropriate permissions, and copy over your Drupal database (or install a new one)
  • Copy over the Drupal code files to your webroot at /mnt/webroot
  • Do any other Drupal configuration needed

Set up “Kiosk Mode” in Linux

  • Create a shell script [vim /home/ubuntu/kiosk.sh] to execute [/usr/bin/firefox —fullscreen http://127.0.0.1] ** Note - sometimes it takes a second for apache to load and the script may need “sleep 30” at the beginning to give apache time to load **
  • Configure Ubuntu at (System —> Preferences —> Sessions) to run the kiosk.sh script at Startup
  • Install the AutoHide Firefox Extension
  • Add a line [rm /home/ubuntu/.ICE*] to your GDM runtime to remove ICE Authority File Locking Issues [vi /etc/gdm/PreSessions/Default]

To boot off of the USB key, insert the USB key into a computer and set its BIOS to boot from the USB key. Wait a couple of minutes for it to boot up and enjoy the Drupal.

Chapter Three on Third Street

Matt Cheney

After spending a few months bouncing around San Francisco coffeehouses and our all too familiar homes, Chapter Three now has a brick and mortar headquarters. Our 1000 square ft office space is located at 20th and 3rd St. in the Dogpatch District of San Francisco. Check us out: Google Maps · Yahoo! Maps

A former cannery, the fruit slicers and can labeling machines have been replaced by laptops, complimentary organic juice and ever present indie rock. Since we enjoy surrounding ourself with genius, we have offered Neil Drumm (of Advomatic) a regular workspace so he can join our daily Drupal party.

This is another step forward for Chapter Three and we are excited about utilizing our office space to be even more productive and collaborative. We have lots of space and tasty snacks so we invite any Drupal contributors in the San Francisco area to stop by for the day and see the office.

Syndicate content