Zaemis

Please God, Give Me Something New!

2011-10-26T00:52:00.000-04:00

"Here... check out this (link). But you have to use the most recent version of Chrome." Sigh. Haven't we been through this already? 15 years have passed and we're headed right back to the same place we started... "Best viewed in <someone's favorite browser>." It's sad, really.

Don't get me wrong. Years of standardization work on HTML, JavaScript, the DOM, CSS, etc. cleaned up a lot of messy lose ends and yes it was indeed necessary, but stringent standardization also stifles creativity. And now that HTML5 and friends have loosened some of the restrictions the pendulum has started to swing back in the opposite direction. People have started to innovate again. This time around it's Firefox vs Chrome instead of Internet Explorer vs Netscape.

There's a systemic problem that goes beyond browser wars, however. Remember AOL? This time it's Facebook trying to be “The Internet.” Remember mainframes and terminals? After pushing everything to the desktop, now we're pushing everything back “to the cloud.” All the fourth and fifth generation programming languages have come and gone and the best we have now is Java and Clojure?! Remember Ajax, er I mean DHTML, er I mean JavaScript? We're all "innovating" but nobody is really doing anything new, exciting, and unique.

Hell, Tesla plugged 200 light bulbs into the ground and lit them up 25-miles away from a power source in 1899 and the best we have now is Solyndra and the Prius? What gives?!

The sine-like cycle of technological advancements wouldn't be bad if each cycle actually gave us something new; you know, an advancement. Maybe that's what has me so jaded. Each cycle seems to rehash the previous cycle and there's nothing really new and exciting anymore. We're moving in circles, not traveling in spirals. The more things change, the more they stay the same. Is Quindlen right, and every story has already been told?

Think about HTML5's canvas element which everyone is saying how great and wonderful it is, an area for 2D drawing that can be manipulated with JavaScript. If browsers had actually implemented decent support for SVG 10 years ago we wouldn't need canvas now. That's right, we've had this “hot new technology” for 10 years already. And sadly, SVG is superior in many ways. And sadly, we've had 3D capability with VRML/X3D since the mid-90s. And sadly, we've settled for something less and are grinning from ear to ear.

Wouldn't you like to have technology from 10 years into the future and have it now? It sounds tempting, but I'm not so sure I really would... because it'd probably be the same as what I've already had for the past 10 years just with a new marketing campaign.

Top-10 PHP String Functions

2011-09-18T00:43:00.003-04:00

By day I work as a programmer at ShoreGroup, Inc. By night I'm a freelance developer and now the managing editor for SitePoint's latest site, PHPMaster.com. Helping out with the site has been pretty fun so far; my Australian counterparts are all pretty cool, and I've met some really great new authors too. If you haven't visited yet, take a moment and check out PHPMaster.com (there's still some wrinkles to iron out on the site, but we're working to identify and fix them all as soon as we can).

Part of my duties as a managing editor include working with authors to make sure the site's content is well balanced. PHPMaster.com is targeting PHP programmers of all skill levels, so there should be a good mix of basic, beginner, intermediate, and advanced content. Planning for a beginner article that demonstrates basic string handling functions, I wondered which function to highlight. I wanted to show ones that would be most relevant, not necessarily ones that were my favorite, so I decided to do some static analysis of popular open-source projects to find out which string functions were used the most. The results were surprising, so I thought I'd share my "research."

I used the source of a closed-source PHP project that I have access to and the following open-source (or open-source-ish) projects as code samples for the analysis:

Drupal (v7.x-dev)
Gallery (v3.0.2)
Joomla (v1.7.0-Stable-Full)
Magento eCommerce (v1.6.1.0-alpha1)
MediaWiki (v1.17.0)
OSCommerce (v3.0.2)
phpBB (v3.0.9)
phpMyAdmin (master-20110914-022001)
WordPress (nightly build, Sept. 14, 2011)
Zend Framework (v1.11.9)
JpGraph (v3.0.7)

Then I ran the following PHP to tally the functions:

#! /usr/bin/env php
<?php
if ($_SERVER["argc"] != 4) {
    $script = basename(__FILE__);
    fprintf(STDERR, "usage: %s directory max exts\n", $script);
    fprintf(STDERR, "\tdirectory - directory to start traversal\n");
    fprintf(STDERR, "\tmax - maximum number of results to return\n");
    fprintf(STDERR, "\texts - comma-separated list of file extensions\n");
    fprintf(STDERR, "example: %s /var/www 20 php,inc\n", $script);
    exit(1);
}
// no error-checking... don't be stupid
$directory = $_SERVER["argv"][1];
$max = $_SERVER["argv"][2];
$extsRegex = "/(" . str_replace(",", "|", $_SERVER["argv"][3]) . ')$/';

$dirIter = new RecursiveDirectoryIterator($directory);
$recIter = new RecursiveIteratorIterator($dirIter);
$iter = new RegexIterator($recIter, $extsRegex);

$funcs = array();
foreach ($iter as $file) {
    $tokens = token_get_all(file_get_contents($file));
    foreach ($tokens as $t) {
        if (is_array($t) && $t[0] == T_STRING && function_exists($t[1])) {
            if (!isset($funcs[$t[1]])) {
                $funcs[$t[1]] = 0;
            }
            $funcs[$t[1]]++;
        }
    }
}
arsort($funcs);

$max = min(count($funcs), $max);
if ($max) {
    list($funcs) = array_chunk($funcs, $max, true);
}
print_r($funcs);

I took the resulting list of functions and extracted the string-specific ones to come up with this top-10 list (sorted in decreasing order of most-used):

substr() - 6,605
sprintf() - 5,604
implode()/join() - 4,829
strlen() - 4,557
chr() - 4,122
str_replace() - 4,009
explode() - 3,401
strpos() - 3,238
htmlspecialchars() - 3,171
trim() - 2,998

I expected functions like substr() and trim() to be on the list, but chr() was a surprise. Before this I probably would have laughed at you if you told me chr() is used almost twice as much as strtolower() (which came in 12th place with 2,267). Interesting results indeed!

Avoid Fetch-Object Abuse

2011-07-10T00:48:00.000-04:00

Lately I'm finding a lot of instances of the mysql_fetch_object() function being used in a particular codebase I help maintain. Unfortunately, I've yet to see it used correctly. It always seems to be used to retrieve a stdClass object from a query result where mysql_fetch_array() or mysql_fetch_assoc() would be the more appropriate choice.

$row = mysql_fetch_object($result);
$kitten = new Kitten();
$kitten->setName($row->name);
$kitten->setColor($row->color);
...

Put aside the argument that the code should be using PDO or the MySQLi extension instead of the legacy MySQL extension. mysqli_result::fetch_object() and PDOStatement::fetchObject() have the same potential for abuse. The above code is wrong because the returned result is an object but treated like it's an array.

The mysql_fetch_array() and mysql_fetch_assoc() functions are used to retrieve a row from the query's result set. I prefer to use mysql_fetch_assoc() so I can access the values using column names.

mysql_fetch_array() - return an array corresponding to the fetched row and move the internal data pointer to the next. The array is indexed numerically if MYSQL_NUM is specified, associatively if MYSQL_ASSOC is specified, or both by default or if MYSQL_BOTH is specified.
mysql_fetch_assoc() - return an associative array corresponding to the fetched row and move the internal data pointer to the next. It is the same as calling mysql_fetch_array() using MYSQL_ASSOC.

mysql_fetch_object() on the other hand was designed to create and populate an instance of an object using the row of data.

$kitten = mysql_fetch_object($result, "Kitten");

If mysql_fetch_object() is called without the second argument, the name of the object type to create, then a stdClass object is returned that has public properties named after the result columns to expose the data. There isn't any benefit to this over an array. It is just an object for the sake of having an object, and not a very useful object at that. Incidentally, an array can always be casted to a stdObject later if, for some odd reason, it's needed:

$obj = (object)$array;

An object is a data structure that encapsulates variables to maintain state and related functions to manipulate the state. A stdClass data object just collects values and then exposes them as public properties. It does not observe proper encapsulation, there is no state, and there are no methods to interact with the object. It is no more than an array that uses object-syntax. My rule of thumb is: if you call mysql_fetch_object() without specifying a class name, you're "doing it wrong."

A related point worth mentioning is even a good programmer may run into some difficulty when trying to use mysql_fetch_object() correctly since the function is intrinsically broken (though some will argue differently). The function first creates an instance of the specified class, then populates its internal variables, and lastly invokes the constructor. The purpose of a constructor is to initialize the object's values and resources, but since the constructor is called last the values set from the result row may be overwritten. The programmer must take this into account and guard against it if the objects might be instantiated by both mysql_fetch_object() and new.

class Kitten
{
    private $name;
    private $color;
    ...
    public function __construct() {
        if (!isset($this->name)) {
            $this->name = "Unknown";
        }
        ...

If you're like me and have a large codebase with the misuse of mysql_fetch_object() deeply entangled throughout, it may not be practical to find and fix each instance. The best advice I can offer is to educate yourself and others how the function should be used so its abuse isn't perpetuated. Then, be cautious when using mysql_fetch_object() correctly and understand the process it follows to create and return an object. If not for yourself, then do it for the kittens.

Smalltalk Challenge: Post 10 - Conclusions

2011-07-04T09:00:00.005-04:00

So here I am, more than 20 hours with Smalltalk and 10 blog posts about my experiences as required by the terms of my challenge. Josh said he chose Smalltalk for me because of the language's history of helping giving rise to object-oriented programming, which he thinks I hate. (For the record I don't hate OO; I hate the awkwardness and complexity its over-zealous application inflicts.) But I think secretly he was hoping it would endear me to Java in the same way as I was hoping OCaml would help him see past Java. Smalltalk gave me a newfound respect, not for OO, but for the language's innovations and lasting influence. If anything in relation to current-day OO practices, my experience confirmed how botched OO adoption has been given what it was intended to be. Regardless, while I don't think I'll be using Smalltalk in the foreseeable future, it was definitely fun to explore and gave me a lot of things to think about.

Smalltalk isn't a popular "mainstream" language like Java or PHP, but it was highly influential on many of them. It's a language that hasn't been afraid to try bold, new ideas. Other languages adopted its innovations; some did so successfully, others not so much. Will new innovations from Smalltalk and its community be just as influential in the future? I can't help but think that if Squeak has successfully abandoned the MVC architecture in favor of Morphic, maybe the days of MVC web frameworks such as Struts, Zend Framework, and Rails are numbered.

Message passing is a key concept in Smalltalk and provides the flexibility of duck typing. But late/dynamic binding prohibits many compiler-time optimizations that could otherwise be performed on statically-compiled code. Is there continued benefit to such an approach now that we're inundated with so many dynamic languages that have (or at least seem to emulate reasonably well) direct invocation, and thus better performance, such as Lua and JavaScript? Perhaps a greater benefit could be realized if message passing was handled asynchronously.

In Smalltalk, everything is an object. Even basic language constructs such as if-statements and while loops have been distilled down to objects and message passing. I respect Kay for intentionally taking an extreme position to force new ways of looking at things, though I believe the "everything is a" mindset can lead to awkward perceptions. The human mind is adept at categorizing things into different classifications exactly because not everything is the same. Ruby claims everything is an object too, which is misleading because it still has basic language constructs. I wonder where the sweet spot between abstraction and reality is, and in what ways can programming languages more accurately reflect how we view the world so we don't end up with with shimmed, over-architectured, spaghetti code.

Overall I found the core concepts of Smalltalk pretty easy to learn (since there are so few), and it was fun to write Smalltalk code and explore objects and Morphs in Squeak's environment. I encourage others who may not be familiar with Smalltalk to spend a little time with it as I did. I can't say it made me a better programmer, but in some sense it made me a better person because I now have a better understanding of history and a lot of things to think about.

Smalltalk Challenge: Post 9 - Koans

2011-07-03T09:00:00.006-04:00

Besides tinkering with turtles and hashes in Squeak, I secretly went back to GNU Smalltalk and went through some of the Smalltalk Koans. Sssh... don't tell Josh!

Programming koans are a series of failing unit tests that a student reads through and corrects. Each test demonstrates a particular concept in the language. They can be a fun way to review one's understanding of a language, and occasionally learn something new. Here's an of a koan:

testSingleCharacterFromString [
    | string |

    string := 'Smalltalk'.

    self expect: (self fillMeIn) toEqual: (string at: 1).
    self expect: (self fillMeIn) toEqual: (string at: 6).
]

When the test suite is run, it displays:

Do not lose hope.  Expected value should equal actual value.

Expected : FILL ME IN
Actual   : $S  (an instance of Character)

TestString#testSingleCharacterFromString has damaged your karma
(in src/koans/TestString.st)

The name of the method indicates it is possible access the characters that make up a string. The test shows how the at: message is passed to a string to obtain a character at the given index. The programmer must replace (self fillMeIn) with the correct value which will allow the test to pass and the student to proceed.

self expect: $S toEqual: (string at: 1).
self expect: $t toEqual: (string at: 6).

This demonstrates that in Smalltalk character instances are preceded by a dollar-sign, and indexes start at 1, not 0 as in many other languages.

Going through the koans, I thought the ones from TestString.st, TestMessage.st, and TestDictionary.st were exceptionally good. My favorite was this one from TestMessage.st, which demonstrates an unintuitive edge-case resulting from Smalltalk's everything-is-an-object and message passing philosophies.

testMessageCascading [
    | value |

    value := 3 + 2; * 100.  "';' separates each message sent to '3'"

    self expect: (self fillMeIn) toEqual: value.

    "Think about it: we are sending multiple messages to '3'."
]

The correct answer is 300. Pretty evil, eh?

Smalltalk Challenge: Post 8 - Virtual Images

2011-07-02T09:00:00.009-04:00

Virtualization is in vogue right now as companies use such technology to run multiple systems on consolidated hardware. Remember, Smalltalk was designed as a language and an environment. No one is about to replace Windows (or Ubuntu, though I'm tempted) with Smalltalk as their operating system, so it makes sense for it to be implemented as a virtual machine.

Similar to products like VMWare Workstation or VirtualBox, most Smalltalk implementations consist of a Virtual machine (VM) application and an image file. The image file contains the definition and state of the Smalltalk environment which is realized by the VM. It may be helpful to think about the image file as a program saved somewhere on your computer's hard drive, and the VM is your computer's processor that executes the code to do something useful.

A Squeak installation consists of four files: a VM executable and three files that make up the image.

The VM is the interpretor that "runs" a Smalltalk image.
The .sources file contains the core Squeak system and base libraries.
The .image file contains the current state of the running system.
The .changes file is contains a log of changes made to the system.

You can save the state of the environment at any time while working in Squeak, which is stored between the .image and .changes files. Such snapshots can be loaded later to go back to previous sessions, or shared with others as a means of distribution.

As with most things in life there are advantages and disadvantages with image-based imperative programming. Apparently though the disadvantages outweigh the advantages. The latest standardization attempt of Smalltalk, ANSI/INCITS 319 1998 (ANSI Smalltalk), does away with the old model and specifies a fully declarative model for writing Smalltalk programs. Some reasons for adopting the new model were:

Programs could be inaccessible if the image becomes out dated or corrupted.
Because the same image is used for development and production, is makes it very difficult to support situations where the development and target execution environments must be different (security restrictions, for example).

I did a quick screen recording to show what writing code in the environment is like. In Squeak, the programmer writes the code for a class and its methods directly in the System Browser window. At the top of the window are selection lists; from left to right, they list class categories, classes, method categories, and methods. Below them is a text area where the code is typed and saved. When the code is saved, Squeak automatically compiles it and makes it available to the rest of the system.

Smalltalk Challenge: Post 7 - Turtles Ahoy!

2011-07-01T09:00:00.011-04:00

I can remember going down to the computer lab while in grade school and using Logo on the then state-of-the-art Apple IIe computers. Supposedly Logo allowed students to explore mathematics in a tangible way, but I don't remember doing much more than drawing shapes by moving the little "turtle" around on the green screen and I still don't have much confidence in my math skills 20 years later. I don't think Logo helped kindle my interest in computers, either; games like The Oregon Trail and Where in the World is Carmen Sandiego and the Commodore 64 computer were far more influential to me. But drawing little green triangles and squares was fun. And while Logo may not have been as influential in my education as Papert would have hoped, it was extremely influential to Smalltalk. In fact, the graphics Pen object is actually very similar to Logo's turtle.

The Pen class is used for drawing. It inherits from BitBlt which allows it to perform raster drawing on any form its bound to. The World is a form itself, so a pen bound to it will appear to draw on top of other windows and elements! The following example draws an octagonal-spiral on top of anything that's visible in the environment, centered around the point of a mouse click:

pen := Pen new.
    pen roundNib: 8;
    color: Color random;
(Sensor waitButton; redButtonPressed)
    ifTrue: [
            pen up;
            goto: Sensor mousePoint;
            down.
        x := 1.
        10 timesRepeat: [
            4 timesRepeat: [pen go: 10 * x; turn: 45].
            x := x + 2]].

A new Pen instance is created, the size of the line it draws is set to 8px by the roundNib: message (what better thickenss to draw octagons with?), and the color of the pen's line is randomly set by the color: message.

Sending a down message is like placing a pen against a sheet of paper. When the pen is against the paper and is moved, a line is drawn. There is also the up message which is like lifting the pen up away from the paper so it can be repositioned without making any marks.

The pen is moved about the screen using the goto:, go: and turn: messages. Like its Logo progenitor, the pen object responds to messages relative to its current position and direction. When asked to move 10px, the pen moves forward in which ever direction it's pointing in. Turning the pen 45 degrees and moving another 10px would result in a 45° angle drawn. The direction of the pen's motion can also be set with north, east, west, south, and direction.

Smalltalk Challenge: Post 6 - Morphic

2011-06-30T09:00:00.006-04:00

As the Dynabook/Smalltalk environment was the first to introduce the windowed user interface, it's no surprise that the Model View Controller (MVC) pattern also made its first appearance in Smalltalk. In Smalltalk, the term "MVC" refers to both the architecture pattern that separates code responsibilities into model, view, and controller objects, and the user interface framework used to develop visual and interactive elements. The MVC framework manages objects in the environment using the MVC pattern.

Model objects are responsible for maintaining the behavior and state of the element.
View objects are responsible for the representation or appearance of the element within the world.
Controller objects are responsible for accepting user input and passing messages to the model and view objects.

But because of the complexity and limitations of the MVC architecture, Squeak has replaced the MVC framework with Morphic, a direct-manipulation user interface toolkit. Unlike MVC, Morphic didn't originate with Smalltalk. Rather, it was written by Randy Smith and John Maloney for Self, another OO programming language influenced by Smalltalk, and ported to Squeak by Maloney who is also one of the core Squeak developers.

All interactive elements that exist within the environment in Morphic are subclasses of Morph, and the majority of code that would normally be required by MVC is encapsulated in the Morph class. For example:

Morph new openInWorld.

will open a blue rectangle that you can move about the screen, resize, rotate, and more. When a custom morph extends Morph, it inherits all of its parent's functionality and the programmer is freed to focus on customizing the morph instead of writing boiler-plate code. As Maloney wrote in An Introduction to Morphic, "In morphic, a little code goes a long way, and there is hardly any wasted effort."

I actually took advantage of Morphic in my Kember Identity search code, though it's not something one might think of as requiring a manipulatable object. The step message is intended for creating animations and morphs that update themselves dynamically over time (adding liveness to the interface). Morphic repeatedly sends step messages to each morph which may respond by executing their step method (the rate at which step messages are sent to an object can be specified by responding to the stepTime message, but by default I believe it's set for every few milliseconds or so). The method actions run concurrently with whatever else is happening in the rest of the environment. This builtin threading was exactly what I needed to ensure the rest of the system remained responsive to the user while the code was searching for a hash that exhibits the Kember Identity.

I subclassed the TextMorph class and supplied the following step method:

Kember » step
    found | (currHash = stopHash)
        ifFalse: [self contents: 'Testing ' , currHash.
            found := self test: currHash.
                found ifTrue: [self contents: 'Found '
                        , currHash , '!'].
            currHash := self nextHash: currHash.
            currHash = stopHash
                ifTrue: [self contents: 'KI was not found.']].

The contents: method is inherited from TextMorph to set the displayed text string, and the rest are instance variables and methods specific to my Kember class.

Morphs can respond to several other messages sent by the framework, too. Responding to the drawOn:message allows a morph to customize its appearance, or how it "draws" itself in the environment. Responding to mouseDown:, mouseUp:, mouseMove:, and keyStroke: events lets the morph interact with the user's mouse and keyboard activities.

Well that's 6 posts down now and only 4 more to go before I complete my challenge!

Smalltalk Challenge: Post 5 - Talking to Objects

2011-06-29T09:00:00.006-04:00

Everything I've read about Smalltalk makes an effort to stress objects are capable of doing three things: maintaining state, receiving messages, and sending messages. The understanding that an object can maintains its own state is common across many languages, but message passing may be unfamiliar to programmers working with other languages (though message passing does play an important role in Objective-C). Alan Kay has said that message passing is the most important concept to understand in Smalltalk, and that objects have been over-emphasized.

Smalltalk's object model follows the classical (class-based) style; the programmer writes a class which is then used by the runtime as a blueprint for instantiating an object. But unlike C++ or Java, the programmer never works with objects by invoking their methods directly in Smalltalk. Instead, messages are sent to the object which receives a message, looks up the appropriate method to perform, and then executes the method. This is more than a conceptual difference; it's a concrete implementation detail as explained by the following quote from Wikipedia's Objective-C article:

In a Simula-style language [like C++, Java, etc.], the method name is in most cases bound to a section of code in the target class by the compiler. In Smalltalk and Objective-C, the target of a message is resolved at runtime, with the receiving object itself interpreting the message.

Perhaps a suitable way of looking at it is that all of the properties and methods of an object are private. The only way to interact with the object is by sending it the appropriate messages.

There are three types of messages in Smalltalk: unary messages that take no arguments, binary messages which have one argument, and keyword messages that can take one or more arguments.

Unary messages appear after the object instance they're sent to. The cr message is an example of a unary message that can be sent to a Transcript object, which responds by displaying a newline character.

Transcript cr.

Binary messages appear with an argument. For example, the binary message = can be sent to foo with the argument 42, and foo responds by comparing its value with the argument and returning a Boolean object set to whether the values were equal.

foo = 42.

Keyword messages end with a colon and can take multiple arguments. The at:put: message is an example of a keyword message which can be sent to an IntegerArray object. The array responds by placing the value 42 at index 5.

anIntegerArray at: 5 put: 42.

Keyword messages are a single message. That is, at:put is not an at: message followed by a put: message. In Java, it might be written like anIntegerArray.atPut(5, 42).

In statements that contain multiple messages, the order of precedence is from left to right with unary messages being sent first, then binary messages, and finally keyword messages, though the order can be influenced by parenthesis. Recall the example 2 + 1 / 4 from Post 3. The expression contains two binary operators, + and /. Following the precedence rules, 2 is sent the message + with the argument 1 and then the message / with the argument 4. Contrast this with how the statement would be evaluated in Java following the mathematical order of operations. To force that execution in Smalltalk, the expression would need to be parenthesized as 2 + (1 / 4).

It seems the biggest benefit of the approach is that its late/dynamic binding affords the flexibility of duck typing in a compiled language... but now that dynamic languages are widely used, and something like an inferred typing system can give the "feel" of dynamic programming in a strongly-typed language, I wonder if this is only historically important now. There is a runtime performance hit since method lookups happens at execution instead of compilation, and some Smalltalk implementations have explored method caching to help mitigate the impact, but it seems there must be a whole class of compile-time optimizations that cannot be performed by the compiler that may could affect performance as well.

I asked about the advantages and drawbacks of message passing on the PiLuD mailing list. Some of the respondents made pretty good points, and I recommend reading the exchange if you're interested in hearing some other opinions from a language-design perspective.

Smalltalk Challenge: Post 4 - Porting the Kember Identity

2011-06-28T09:00:00.005-04:00

There are a few things I find myself tripping up over even after spending some time writing "meaningful" Smalltalk code, like using single quotes to delimit strings (double quotes are used for comments) and remembering the order in which different messages are sent, but the more code I write the easier it is to remember such things. After only a few hours, Smalltalk is still something new and unfamiliar.

The first programs I wrote when looking into Go were solutions to the first two Project Euler problems and a port of the Kember Identity search program. I decided to skip the Euler problems this time and go straight to the Kember Identity port.

The Kember program ultimately boils down to generating and checking MD5 hashes. I didn't find any helpful cryptography related objects or methods in the default image, so I searched Google and eventually found Ron Teitelbaum's Cryptography/Team package. Squeak uses a package management system called Monticello to load code into the image, so getting and installing the package was pretty easy. I copied and pasted the package repository's connection information into the Monticello Browser and loaded Rob Withers' contribution, Cryptography-rww.15.mcz.

Once the package was loaded, I was able to obtain hashes with MD5 » hashStream:, but the returned object was a ByteArray and I needed to interpret it as a 32-character hexadecimal string. At first I took this approach to convert the array to a hex string:

Kember » md5: aString
    "return 32-char MD5 hash of the given text" 
    | hash str |
    hash := MD5 new
                hashStream: (ReadStream on: aString).
    str := ''.
    1 to: hash size do: [:i | 
        str := str, ((hash at: i) radix: 16)].
    ↑ str.

… only to find out later that objects of the ByteArray class were modified by the cryptography package to accept a hex message and will do the conversion for me. Oops! All the bit twiddling I had done could easily be replaced with:

    ↑ (MD5 new hashStream: (ReadStream on: aString)) hex.

Converting between hash representations wasn't the only part of the program I initially over-programmed. I was also doing long-form addition to obtain the next hash value in the sequence when all I really needed was a little bit of type juggling and string padding:

Kember » nextHash: aHashStr
    "increment the MD5 hash"
    | hexHash zeroes |
    zeroes = '00000000000000000000000000000000'.
    hexHash := ((ByteArray fromHexString: aHashStr)
        asInteger + 1) asByteArray hex.
    hexHash size < 32
        ifTrue: [↑ (zeroes copyFrom: 1 to 32 - hexHash size)
            , hexHash].
    ↑ hexHash.

I guess it just proves the saying is true, "learning the libraries is the 20% of learning a new language that takes 80% of the time and effort." If anything, at least I can take comfort in knowing I'm not the first person to over-program a solution while learning a new language.For those that want to check out my Kember code, I've set up my own repository and uploaded a Monticello package to SqueakSource, and a file dump to Github (I suggest looking at it in raw mode if you go to Github because their Markdown chokes and truncates the pretty-print view).

Smalltalk Challenge: Post 3 - Awkward

2011-06-27T09:00:00.006-04:00

After spending a short time in the Smalltalk (Squeak) environment, it's easy to understand how other existing languages at the time were not suited for realizing the full potential of the Dynabook. Working in a visual environment is a far-cry from working at a mainframe terminal. It's ironic though that some of the same issues that plague OO now are the same that held Smalltalk back in the 80's... it consumed a substantial amount of memory and the performance was not always optimal. For me, one of the issues is the awkwardness of Smalltalk's "everything is an object" philosophy.

I believe the "everything is a ___" mindset causes problems, regardless of what fills in the blank. In TCL, everything is a string. In LISP, everything is a list. In Smalltalk, everything is an object. The fact of the matter is not everything in the world is homogeneous. Some things are objects, others are numbers, and yet others are actions. Forcing everything into the same conceptual model forces the programmer to perform cognitive acrobatics.

Understanding the semantics behind a simple statement such as Transcript show: 'Hello World!' is easy; a Transcript window object is sent the message show:, which then responds by displaying the argument. But the semantics behind conditional code execution isn't so intuitive. Smalltalk doesn't have a dedicated if-construct, instead conditional execution is achieved by passing messages to Boolean objects, like so:

x > 0
    ifTrue: [Transcript show: 'Positive']
    ifFalse: [Transcript show 'Negative'].

The object represented by the variable x is sent the > message with the argument 0. x then compares its value with that of the argument and returns a Boolean object in response. If the value of x is a positive number then the resulting truth value of the returned object is true, otherwise the value is false. The ifTrue:ifFalse: message is sent to the implicit Boolean object providing two code block (which are also objects!) as arguments. All basic language constructs such as if-statements and while loops are abstracted as objects! While it does produce an appreciable amount of elegance and flexibility in the language, this thinking takes some getting used to.

There's a lot going on behind the scenes with an expression like 2 + 1 / 4 as well. A language like C will follow the rules of precedence stated by the mathematical order of operations. First 1 is divided by 4 which yields .25, and then 2 is added to .25 resulting in the final value 2.25. In Smalltalk, everything is an object which the programmer interacts with by sending messages. The literals 2, 1, and 4 are actually instances of SmallInteger objects. The + message is sent to 2 with an argument of 1, which responds by adding the value of the argument to its own value resulting in a value of 3. Then, the / message with the argument 4 is sent. The object responds by dividing its value by 4 which results in a final value of .75 for the expression.

Some things can easily be conceptualized as objects, such as windows, streams, and data structures, while it can be difficult to think of others as objects, such as text, integers, colors, and truth values. Over-personification and the perspective of the grammatical predicate/direct object performing actions is backwards of how we speak and how we think.

As awkward as the "everything is a" mindset it, I have to respect Kay for it. He intentionally adopted it so he and his team would be forced to think outside the box. Kay admits "we were actually trying for a qualitative shift in belief structures-- a new Kuhnian paradigm in the same spirit as the invention of the printing press-- and thus took highly extreme positions which almost forced these new styles to be invented." I can respect his effort and appreciate the elegant design it resulted in, but I can't expect an 8-year old to understand passing code blocks as arguments and why the order of operations is different on the computer than in the classroom.

Smalltalk Challenge: Post 2 - Switching to Squeak

2011-06-26T01:46:00.007-04:00

A quick survey of Smalltalk's keywords, constructs, and syntax isn't enough to understand how the language approaches programming. There is only a handful of keywords, no control structures, and the syntax is quirky in spots but is hardly exotic. To really get an appreciation, it's helpful to place the language in its historical context. Smalltalk wasn't intended to be a general purpose systems programming language like C, rather it was designed to allow ordinary people to take full advantage of their computers and to be easy enough for children to learn. The intended environment was immersive, interactive, and most-importantly visual. Smalltalk was more than a programming language, it was an entire operating environment.

Xerox was working on the Dynabook at its Palo Alto Research Center in the early 1970's. The Dynabook was to have many features we now see in personal laptops and tablet devices (such as touch screens, mice, windowed display managers, etc.). Alan Kay argued that existing languages were ill-suited and a new programming language was necessary for the end-user to realize the full potential of the system. He drew inspiration from the work of Jean Piaget and others in developmental psychology and constructionist learning, which set the direction of Smalltalk. The Dynabook project never came to fruition, but Smalltalk lives on, and both had a lasting influence on computing that we still see today.

Given this background, I suspected something like Squeak would really be better suited for learning Smalltalk as it was intended since Squeak is the modern implementation of the language and environment that was developed at Palo Alto. I asked Josh if it was okay to switch from GNU Smalltalk to Squeak. He agreed-- admitting that he only proposed GNU Smalltalk because it's what he already had on his system from Portage. I checked apt and was amazed to find Squeak in Ubuntu's repositories (with bets on the MotU pulling it without telling anyone within the next two releases). After running sudo apt-get install squeak-vm and starting Squeak, I was staring at a pretty gaudy looking graphical environment.

I took the liberty of running a "Hello World" program. The System Browser window is at the top, a Workspace window is in the middle, the Transcript window is at the bottom, and the Objects Library window is to the right. I entered the statement Transcript show: 'Hello World!' in the Workspace, highlighted it, and chose "do it" from the popup menu.

Every element is an object which you interact with by sending it messages. In the above example for instance, I interacted with the Transcript window by sending it the show: message and the argument 'Hello World!'. The window responded by displaying the text.

I did a screen recording of another example to help give you a better idea what interaction in the graphical environment can look like. I placed a Star object on the world (the main "desktop" area) and used its Inspection window to send the messages borderWidth: and rotationDegrees:. The object responded by redrawing itself with a thicker border and different rotation.

Have you been challenged to learn a new language, or have you been learning/working with Smalltalk? Feel free to leave your comments about it below.

Smalltalk Challenge: Post 1 - Installing GNU Smalltalk

2011-06-25T23:44:00.005-04:00

That's right... I'm the one who challenged my coworker Josh to open his mind beyond Java by spending time with a new language and blogging about it. l At first I challenged him to learn Oz, a language that combines the imperative, object-oriented, functional, logic, constraint, distributed, and concurrent programming paradigms (whew!). Unfortunately, apparently the 64-bit version of Mozart on Gentoo is broken at the moment and he didn't want to, as he put it, "build random shit on his box." So I proposed OCaml as an alternative. While it may not combine seven programming paradigms, I'm positive functional programming will be enough to show him there's more to life than what Java offers.

In return, for him to accept my challenge I had to agree to spend some time learning a language of his choosing too, and he set the number of required blog posts to 10. That's not too bad in my opinion since I enjoy looking at different programming languages anyway, but 10 posts seems a bit excessive (I would have been happy with 3). His first suggestion was Shakespeare. Really? I made him pick a more serious language, which turned out to be GNU Smalltalk. Yes, let the record show I rejected his first proposal, but he also rejected mine... and also let the record show that I'm not afraid to use a compiler.

My first attempt to install Smalltalk was through apt-get since I'm using an Ubuntu system right now, and of course I met up with the usual Ubuntu package bullshit...

sudo apt-get install smalltalk
E: Unable to locate package smalltalk

sudo apt-get install gnu-smalltalk
Package gnu-smalltalk is not available, but is referred to by
another package. This may mean that the package is missing,
has been obsoleted, or is only available from another source

E: Package 'gnu-smalltalk' has no installation candidate

It was the SpiderMonkey fiasco all over again. Apparently GNU Smalltalk was removed from Ubuntu back in Lucid Lynx. I had to go to smalltalk.gnu.org instead and download the source. In just a few minutes I had a working binary using the usual configure, make, make install mantra.

There was one minor snag during compilation, though. Initially libtool reported that it couldn't find the name of the link library for libc.la. A quick Google search showed all I had to do was remove libc.la and run make again.

Now it's time to learn some Smalltalk!

Update (6/29/11): I was able to compile GNU Smalltalk without experiencing the libc.la issue by running autoreconf -vi before configure.

Reading Unicode (UTF-8) in C

2011-06-04T18:21:00.001-04:00

In working on scanner code for Kiwi I did a bit of reading up on Unicode. It's not really as difficult as one might think parsing UTF-8 character by character in C. In the end I opted to use ICU so I could take advantage of its character class functions instead of rolling my own, but the by-hand method I thought was still worth sharing.

Functions like getc() read in a byte from an input stream. ASCII was the predominant encoding scheme and encoded characters in 7-8 bits, so reading a byte was effectively the same as reading a character. But you can only represent 255 characters using 8 bits, far too little to represent all the characters of the world's languages. The most common Unicode scheme is UTF-8, is a multi-byte encoding scheme capable of representing over 2 million characters using 4 bytes or less.

The 128 characters of 7-bit ASCII encoding scheme are encoded the same, the most-significant bit is always 0. Other characters can be encoded as multiple bytes but the most-significant bit in such cases is always 1. The bit pattern in the first byte of such multi-byte characters too that make it easy to determine exactly how many bytes are used for the character.

The first bit serves as a flag. If set, the character is encoded using multiple bytes, and if not, the character is encoded using just one. Subsequent bits indicate the number of bytes used. Each byte in a multi-byte encoded character will aways start with the pattern "10".

With that knowledge, one can write a wrapper that reads characters in from a UTF-8 encoded source one at a time. getc() reads a byte at a time, and a quick peek at the significant bits determines if additional bytes must be read to fetch the entire character.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define U_MAX_BYTES 4

unsigned short u_getc(FILE *stream, char *bytes) {
    /* mask values for bit pattern of first byte in multi-byte
     UTF-8 sequences: 
       192 - 110xxxxx - for U+0080 to U+07FF 
       224 - 1110xxxx - for U+0800 to U+FFFF 
       240 - 11110xxx - for U+010000 to U+1FFFFF */
    static unsigned short mask[] = {192, 224, 240}; 

    unsigned short i, j; 

    /* initialize buffer */
    memset(bytes, 0, U_MAX_BYTES); 

    /* read first byte into buffer */
    bytes[0] = getc(stream);
    if (bytes[0] == EOF) { 
        return 0; 
    } 

    /* check how many more bytes need to be read for
     character */
    i = 0;
    if ((bytes[0] & mask[0]) == mask[0]) i++;
    if ((bytes[0] & mask[1]) == mask[1]) i++;
    if ((bytes[0] & mask[2]) == mask[2]) i++;

    /* read subsequent character bytes */
    j = 0;
    while (j < i) {
        j++;
        bytes[j] = getc(stream);
    }

    /* return number of bytes read into buffer */
    return i + 1;
}

The wrapper function u_getc() is passed an open file handle and a buffer to store the character. It returns the number of bytes read into the buffer.

int main(int argc, char **argv) {
    /* allocating +1 for null gives ability to print character
     as string */
    char *bytes = (char*)calloc(U_MAX_BYTES + 1, sizeof(char)); 

    /* read and print until end of file */
    while (u_getc(stdin, bytes)) { 
        printf("%s\n", bytes); 
    }

    return 0; 
}

Grouping characters into tokens such as operators and numeric literals is easy because UTF-8 is backward compatible with 7-bit ASCII, which allows performing character tests like byte[0] == '+' and isdigit(byte[0]). But it was the need to test multi-byte sequences that ultimately lead me to use ICU. Writing and testing a function that properly identifies membership in the set of 14,725+ possible characters that could appear in an identifier name wasn't a tempting prospect.

OO in Language Design

2011-05-30T23:19:00.003-04:00

If you were to create a new programming language, and you wanted to support object oriented programming with it, what would it look like? An object is a set of instance data and various functions that work with it. In Java, they're specified by first grouping the variables and functions together in a class, and then creating an instance from the class definition. In JavaScript, functions are dynamically attached to the instance data (or it's prototype). This is an intentional over-simplification but does highlight the fact that different programming languages approach working with objects differently. I've been thinking about objects and working with structured code in relation to Kiwi and it depresses me how much the object oriented school of thought that started with C++ and came to fruition in Java has influenced our industry.

Of course there are the so-called five concepts of object oriented design, which says a design that meets the following criteria is considered object-oriented (OO):

Object/Class
Information Hiding
Inheritance
Interface
Polymorphism

But I'm not so sure I'm convinced. A coworker accused me of trying to re-define it:

Coworker: So now we're redefining what OO is too... care to define common words as well?

Me: I'm not redefining it per se... just calling bullshit where I see bullshit.

Coworker: Excuse while I go outside and laugh loudly.

Me: Sorry my name isn't Martin Fowler so my opinion probably doesn't matter much, but I still smell bullshit.

If we think about programming in C for a moment, a language that isn't considered to support OO practices, we can still encapsulate our code and provide an interface for data manipulation. Member variables are placed in a struct, public methods are functions listed in a header file, and private methods exist only in the code file. Encapsulating in this case depends on an understanding of variable and file scoping rules of the language, not the existence of some class, public, protected, and private keywords.

We have to pass a pointer to the instance class into the functions, but that still happens in languages that are considered to be OO languages too; it's just syntactic sugar that automatically passes it and assigns it to this. It's difficult to ignore the similarity between foo.doSomething() and doSomething(foo).

Java has developers define the method functions in the same code unit as the member variables. Others allow defining them separately, and then associate them in some way later. C++ does this through file and namespacing, and Go does this with interfaces.

If we were to think of instances of objects in terms as payload structs, then is inheritance really necessary? We don't extend a class for example because we have a protected integer member we want to reuse, we do it for the functionality provided by method functions. Inheritance supposedly helps with code-organization and promotes re-use, but I'd argue not essential for OO. In fact, the proliferation of OO far exceeds the amount of code we actually end up reusing.

Polymorphism probably isn't really needed either for OO. Data typing is all about the underlying semantic meaning of data and what operations are allowed to be performed on it. What it means to "add" two strings together and what it means to "add" two integers together are different even though both are aggregate operations. Typing is less relevant in dynamic languages like PHP and Ruby. We don't care what type an object is so long as we can call a method on it. The concept of polymorphism is only important in statically-typed languages, and promoting it as a core OO concept means a true dynamically typed language could never be object oriented.

The way I see it, encapsulation begets abstraction. Inheritance is about reusability, not objects. Interface is a design pattern that attempts to overcome difficulty for reusing code from different objects. Pure object oriented programming seems really to be about encapsulation and message passing, while the concepts above seem more concerned about structure.

It can be argued that the software industry is in the mess it's in right now because of the over-definition and application of intolerant ideas about OO, and a refusal to acknowledge other schools of thought on the subject. But if I'm taking on the responsibility to create a new language, then I'm going to take the time to think about such things and not just follow the herd. I want to create something better, not just a better Java.

Spaghetti Code Considered Harmful

2011-05-15T14:21:00.000-04:00

To chefs and epicureans, spaghetti is a good thing-- it's inexpensive, easy to make, and very versatile. But spaghetti isn't so appetizing to a programmer. The phrase "spaghetti code" is an all-too-common pejorative we programmers use to describe horrible code, whether it's difficult to understand, poorly organized, or just plain long-winded. And there-in lays the problem. What really constitutes spaghetti code is subjective; I've yet to hear a concrete definition or standard metric that measures it.

I've seen PHP code that amounts to nothing more than a giant switch construct... a hundred lines here to handle this condition over here, another hundred or so lines in the branch for that condition over there, maybe a dozen conditions... you get the idea. But such code isn't necessarily spaghetti. What's wrong with it is not that it's unreadable, I would argue, but rather there is a lack of encapsulation. There's a greater chance of inadvertently modifying the wrong variable 800 lines down in some unknown branch, but scoping isn't spaghetti. The code may otherwise be beautifully formatted with the coding standard du jour, and the logic might be crystal clear.

I've also seen OOP code that has been so meticulously refactored that it would bring a tear of joy to just about any DRY-advocate's eye, but that doesn't always mean the code is well organized or easy to follow. It can be frustrating to trace through deep inheritance chains without the help of a modern IDE with project support. IDEs are tools that should help make working with code easier, not be a requirement to navigate what is essentially a set of text files. New professionals coming out of college now with an OOP perspective would be comfortable with such code. They view the same procedural code an old-time C programmer would be comfortable with as messy, unorganized, sloppy spaghetti.

An informal poll I took asked a handful of developers to try to explain what is spaghetti code. They all agreed that it was a term used to describe bad code, but everyone has his or her own opinion as to what aspects would make them label it spaghetti:

Spaghetti code is code that is written quickly and doesn't follow the accepted standard.
Spaghetti code is any code that uses goto.
Spaghetti code is code written by a beginner who's learning to program in Basic.
Spaghetti code is like spaghetti in that after you've digested it you are fatigued.

It's trivial to find code samples that relegate such metrics as mere preference, because each one relies on a subjective perception of quality.

The best response I received was from a friend and former co-worker, Sean H. "Spaghetti code is what we call other people's code when we think it's a pile of blackened shit that's baked in the sun and has flies swarming about it but we still want to be polite." Spaghetti code really has nothing to do with code, but rather how we choose to interact with one another. Lacking a proper definition but carrying a negative connotation, spaghetti is just complaining about other people's work which is not beneficial to ourselves or to the quality of the code we produce/maintain.

Some programmers are very protective of the code they write and they can react in a variety of ways to people who call their work spaghetti. Depending on their personality, in an extreme case they may view such comments as a personal attack and become overtly confrontational. Few people like to be challenged on their way of doing something. In a more subtle case, the programmer may begin to resent others. He feels misunderstood and begins to draw inward, subconsciously severing communication channels.

But besides negatively affecting others, our complaining about spaghetti code hurts ourselves too. It's easier to motivate ourselves when tasked with fixing a bug when we're not dreading having to trudge through spaghetti. When we label code as spaghetti code, we are actually creating roadblocks that hamper ourselves and artificially increase the difficulty of the task at hand. A better perspective to have is that we have been given the opportunity to use our skills to fix something and to make it better. Constant complaining only reinforces the wrong attitude and makes it easier to become disillusion with our craft; negativity "sucks the life out of you" and doesn't serve to create a better work environment.

It's almost always easier to complain and set oneself up for failure rather than success. Should I stop harping on Java? Probably. Should I not let my skin crawl every time I run into Zend Framework? Maybe. Though I consider myself successful, how much more successful could I be if I didn't put up mental blocks for myself over what amounts to other people's preference? I have no idea. Old habits die hard, but it's a worthy goal to strive for to stop alienating others and making life difficult for myself by complaining about spaghetti code. Feel free to join me in doing so.

Bearing Your Cross

2011-04-12T17:15:00.001-04:00

There's the old saying, "we all have our own crosses to bear." As cliché as old clichés are, they're generally true. Life is full of injustice and hardship. For some it may be an addiction to drugs or alcohol. For others, it may be caring for an elderly relative. The important thing isn't what your cross is, rather how you bear it.

I myself struggle with depression. I've learned to live with it with the help of medication and coping strategies. There are still bad days here and there, but I do my best to avoid complaining about it. I have gone further in life by believing in myself than I have by pitying myself. Depression is my cross to bear, and I focused my energy to learning programming which I can now share with others.

I met an impressive girl the other day who shared with me her struggle to overcome anorexia. She drew the strength from it to go on and start her own business selling custom fetish gear. She wants people to know they can still feel comfortable as whoever they are regardless of how they look, if they're fat, thin, bald, etc.

My best friend's grandmother ended up unexpectedly raising 5 children and several foster children on her own after she was widowed.

I know rich people and poor people, healthy people and sick people, happy people and miserable people. Life dealt each one an unfair hand in one way or another, but each has something different to offer. The next time you feel the urge to complain about life's difficulties, stop for a moment and ask yourself if that's the right thing to do. More often than not you can tap into your struggles as a source of experience and strength to help others. And when you do, your cross won't seem so heavy anymore.

Resuming scp

2011-03-20T23:52:00.003-04:00

I was transferring a file from one system to another at work with scp when the connection dropped. It was a substantially sized file and I didn't want to start the transfer over again, but unfortunately scp doesn't support resume. That's when I remembered rsync. With rsync I was able to continue the transfer where it left off, and afterward I had the idea to set the following alias in my .bashrc file.

alias scp='rsync -vPe ssh'

Now when I transfer files with scp, I'm really using rsync and I don't have to worry about flaky network drops.

Creepy JavaScript Tracking

2011-03-15T23:01:00.004-04:00

I recently began allergy shots so my new Monday morning routine includes me sitting in a doctor's office for 30 minutes (I must wait after receiving the shots and be checked by a nurse to make sure there was no reaction). With nothing else better to do while I waited last week, I started playing around with some JavaScript. This is what I came up with:

<html>
 <head>
  <title>Test</title>
  <script type="text/javascript">
window.onload = function () {
    var mX = 0,  mY = 0,
        sX = 0,  sY = 0,
        queue = [],
        interval = 200,
        recIntv = null,
        playIntv = null,
        b = document.body,
        de = document.documentElement,
        cursor = document.getElementById("cursor"),
        record = document.getElementById("record"),
        play = document.getElementById("play");

    window.onmousemove = function (e) {
        e = e || window.event;
        if (e.pageX || e.pageY) {
            mX = e.pageX;
            mY = e.pageY;
        } else {
            mX = e.clientX + (de.scrollLeft || b.scrollLeft) - 
                (de.clientLeft || 0);
            mY = e.clientY + (de.scrollTop || b.scrollTop) -
                (de.clientTop || 0);
        }
    };

    window.onscroll = function () {
        if (window.pageXOffset || window.pageYOffset) {
            sX = window.pageXOffset;
            sY = window.pageYOffset;
        } else {
            sX = de.scrollLeft || b.scrollLeft;
            sY = de.scrollTop || b.scrollTop;
        }
    };

    record.onclick = function () {
        if (recIntv === null) {
            queue.length = 0;
            recIntv = setInterval(function () {
                queue.push([mX, mY, sX, sY]);
            }, interval);
            this.innerHTML = "Stop";
            cursor.style.display = "none";
            play.disabled = true;
        } else {
            this.innerHTML = "Record";
            clearInterval(recIntv);
            recIntv = null;
            play.disabled = false;
        }
    };

    play.onclick = function () {
        var i = 0;
        if (playIntv === null) {
            cursor.style.display = "inherit";
            play.disabled = record.disabled = true;
            playIntv = setInterval(function () {
                if (i < queue.length) {
                    cursor.style.left = queue[i][0] + "px";
                    cursor.style.top = queue[i][1] + "px";
                    window.scrollTo(queue[i][2], queue[i][3]);
                    i++;
                } else {
                    clearInterval(playIntv);
                    playIntv = null;
                    play.disabled = record.disabled = false;
                }
            }, interval);
        }
    };
};
  </script>
 </head>
 <body>
  <p>Press Record to track mouse movements and scrolling.
Press Play to view.</p>
  <button id="record">Record</button>
  <button id="play" disabled="disabled">Play</button>
  <img id="cursor" src="cursor.png"
   style="position:absolute; display:none;"/>
 </body>
</html>

Essentially the code watches your mouse movements and can play them back for you. A callback function is attached to the window object's onmousemove event to obtain the coordinates of the cursor (mX and mY), and a callback is attached to the onscroll event to obtain the scroll position of the window (sX and sY). To record the input, a simple function is called every 200 milliseconds to push the coordinate values onto a queue. Playback is done by reading from the queue and positioning a sprite and scrolling the window accordingly.

So what's the practical value of this, you may ask? Probably not much beyond spying.

Right from the early days of the commercial Internet, your online activity has been tracked and analyzed. Unfortunately (or fortunately, depending on your perspective) efforts over the years have increased the amount of data that can be collected and improved its accuracy. Gone are the days of relying solely on httpd access logs; today there is Google Analytics, evercookies, and even HBGary's Aaron Barr's social mapping scheme. We are being watched in more ways than we can imagine.

I showed the code to several friends and the consensus was the same. If there was any practical value at all to it, most likely that would be tracking. It's a trivial exercise to adapt it so the coordinates are posted back to the server for storage in a database. Once there, all sorts of analysis could be performed on them. Perhaps a heat map could be constructed to see which products on an e-commerce catalog page shoppers thought about clicking on. Marketers could then fine-tune their images and layout to increase conversion.

I don't have any intention of working on this code anymore; it's served its purpose for me as a brief diversion. The questions I've begun to ponder as a result of it though will endure longer than 20 minutes. Is this sort of tracking already in use, and if so then by whom? I'm not the first to play with this. What would be an easy way to thwart such tracking? Such tracking would probably happen unknowingly and it'd be unlikely there would be any opt-out. Where does one draw the ethical line between tracking and spying? It's understandable someone would want to get a better understanding of how their site is being used, but watching mouse movements seems kind of creepy.

So what are your thoughts? Leave your comments below and let me know what you think.

UPDATE 6/5/2011: I've uploaded this code to my JavaScript Experiments repository at GitHub if anyone is interested in playing around with it.

Geolocation Search

2011-01-08T20:40:00.006-05:00

Services that allow users to identify nearby points of interest continue to grow in popularity. I'm sure we're all familiar with social websites that let you search for the profiles of people near a postal code, or mobile applications that use geolocation to identify Thai restaurants within walking distance. It's surprisingly simple to implement such functionality, and in this post I will discuss how to do so.

The first step is to obtain the latitude and longitude coordinates of any locations you want to make searchable. In the restaurant scenario, you'd want the latitude and longitude of each eatery. In the social website scenario, you'd want to obtain a list of postal codes with their centroid latitude and longitude.

In general, postal code-based geolocation is a bad idea; their boundaries rarely form simple polygons, the area they cover vary in size, and are subject to change based on the whims of the postal service. But many times we find ourselves stuck on a client project that calls for things we know better than to do, so I'll spare you my rant and instead ask that you keep the following in mind:

ZCTAs are not ZIP codes so do not use them as such. Some people recommend using the US Census Bureau's 2000 ZCTA dataset as freely available ZIP code latitude/longitude information for the United States, but the Bureau's own website warns ZCTA codes are designed to label census blocks and do not always correspond to ZIP codes assigned to the same area by the US Postal Service. Unpopulated areas such as lakes and forests may have their own ZCTA code, while smaller ZIP codes may not even "appear in the ZCTA universe."
Postal codes are volatile so make sure you understand your dataset provider's update policies. CD Light, LLC offers a line of commercial datasets which they license from the US Postal Service and Canada Post Corporation and then augment and maintain using a variety of additional sources. New codes are added and obsolete codes are removed from their datasets. Quarterly and monthly updates are made available through reasonably priced subscription plans. federalgovernmentzipcodes.us on the other hand offers a free US ZIP code dataset that is irregularly-maintained using a smaller set of additional sources. Obsolete ZIP codes are included but marked "decommissioned." It's not clear however which company or individual is behind the offering.

Regardless if you're hosting your dataset yourself or using a web service such as GeoCoder.ca, it's still not a bad idea to familiarize yourself with what efforts are taken to build it from current information and to maintain its accuracy. It's better to have more-specific/accurate data than not.

The second step is to develop an appropriate strategy to search your data. You don't want to calculate the distance to all target locations' coordinates from a given origin; this naive, brute-force search is rarely desirable, even if the dataset is small and your application/site traffic is low. My approach is to calculate the latitude and longitude coordinates for a search perimeter's north, east, south, and west bounds (bearings of 0°, 90°, 180°, and -90° or 270° respectively). I can then search the subset of records that have locations which falls within this bounding-box.

I use the following formulas to calculate target coordinates and restrict my dataset search:

where:

d is distance
R is the Earth's radius (WGS84 model semi-major axis is 3,963.19 miles)
ᵩ is latitude
λ is longitude
θ is bearing

Implemented in code, the formulas look like this:

// calculate destination lat/lon given a starting point, bearing, and distance
function destination($lat,$lon, $bearing, $distance,$units="mi") {
    $radius = strcasecmp($units, "km") ? 3963.19 : 6378.137;
    $rLat = deg2rad($lat);
    $rLon = deg2rad($lon);
    $rBearing = deg2rad($bearing);
    $rAngDist = $distance / $radius;

    $rLatB = asin(sin($rLat) * cos($rAngDist) + 
        cos($rLat) * sin($rAngDist) * cos($rBearing));

    $rLonB = $rLon + atan2(sin($rBearing) * sin($rAngDist) * cos($rLat), 
                           cos($rAngDist) - sin($rLat) * sin($rLatB));

    return array("lat" => rad2deg($rLatB), "lon" => rad2deg($rLonB));
}

which a simple wrapper function can then call repeatedly with different bearings to determine the search bounds:

// calculate bounding box
function bound($lat,$lon, $distance,$units="mi") {
return array("N" => destination($lat,$lon,   0, $distance,$units),
             "E" => destination($lat,$lon,  90, $distance,$units),
             "S" => destination($lat,$lon, 180, $distance,$units),
             "W" => destination($lat,$lon, 270, $distance,$units));
}

Records that are located within this search area still need to be filtered, and I use the following formula to calculate the distance between two coordinates:

Implemented in code, the formula look like this:

// calculate distance between two lat/lon coordinates
function distance($latA,$lonA, $latB,$lonB, $units="mi") {
    $radius = strcasecmp($units, "km") ? 3963.19 : 6378.137;
    $rLatA = deg2rad($latA);
    $rLatB = deg2rad($latB);
    $rHalfDeltaLat = deg2rad(($latB - $latA) / 2);
    $rHalfDeltaLon = deg2rad(($lonB - $lonA) / 2);

    return 2 * $radius * asin(sqrt(pow(sin($rHalfDeltaLat), 2) +
        cos($rLatA) * cos($rLatB) * pow(sin($rHalfDeltaLon), 2)));
}

A search using these functions would look like this (assume $latitude and $longitude are given and provide the origin coordinates, and $distance and $units provide the desired search radius):

$b = bound($latitude,$longitude, $distance,$units);
$query = "SELECT location_name, address, city, state, zip_code,
    latitude, longitude FROM locations WHERE
    latitude BETWEEN {$b["S"]["lat"]} AND {$b["N"]["lat"]} AND
    longitude BETWEEN {$b["W"]["lon"]} AND {$b["E"]["lon"]}";
    $result = $db->query($query);

    $locations = array();
    while ($row = $result->fetch(PDO::FETCH_ASSOC)) {
    $dist = distance($latitude,$lonitude, $row["latitude"],$row["longitude"], $units);
    if ($dist <= $distance) {
         $locations[] = array("name"     => $row["location_name"],
                              "address"  => $row["address"],
                              "city"     => $row["city"],
                              "state"    => $row["state"],
                              "zip_code" => $row["zip_code"],
                              "distance" => round($dist, 2));
    }
}

A deep understanding of the mathematics used in navigation is nice to have, but is not truly necessary. We're lucky to live in an age where finding suitable formulas is just a quick Internet search away, and discussions on the trade-offs between their simplicity and accuracy are easy to find. The formulas I used are based on the Haversine Formula which have been accurate enough for my needs. Others such as Vincenty's Formulae take into account the ellipsoidal nature of the Earth to produce more accurate results, but I'm helping people locate near-by friends or businesses... not programming cruise missiles.

The final step is obtaining origin coordinates. In the postal code example from the beginning you can look up the code's centroid latitude and longitude and use those as your search origin. If your target audience is receptive, you may want to consider taking advantage of the JavaScript Geolocation API defined by the W3C. The API exposes the navigator.geolocation object which can obtain the user's current location. A user will rarely know his or her exact coordinates, but many Internet-enabled devices and smartphones now ship with GPS capabilities which the browser can access. navigator.geolocation.getCurrentPosition() causes the brower to request permission from the user to report his or her location, and if permission is granted will invoke a specified callback function.

<form>
 Search <input id="txtDist" maxlength="3" name="distance" size="3" type="text"/>
 <select name="units">
  <option value="mi">mi</option>
  <option value="km">km</option>
 </select> from<br/>
 <input checked="checked" name="searchBy" type="radio" value="zip"/>ZIP code
 <input maxlength="5" name="zipCode" size="5" type="text"/><br/>
 <input id="coords" name="searchBy" type="radio" value="coords"/>coordinates
 <input id="txtLat" maxlength="10" name="latitude" size="6" type="text"/> Lat.
 <input id="txtLon" maxlength="10" name="longitude" size="6" type="text"/> Lon.<br/>
 <input type="submit" value="Search"/>
</form>
<script type="text/javascript">
(function () {
    function setCoords(pos) {
        document.getElementById("coords").checked = true;
        document.getElementById("txtLat").value = pos.coordinates.latitude;
        document.getElementById("txtLon").value = pos.coordinates.longitude;
    }

    if (navigator.geolocation) {
        navigator.geolocation.getCurrentPosition(setCoords);
    }
})();
</script>

Hopefully you now seen how relatively easy it is to support searching nearby points of interest in your web-application. First and foremost it is important to maintain a good dataset of target coordinates, since though users will understand the search results to be approximations, you can never return good results with bad data. Implementing algorithms necessary to search the set efficiently is relatively straightforward and I have shown you the approach I use; other algorithms and formulas can be found and implemented if you need more accurate results. And modern APIs like navigator.geolocation make it possible for you to offer new ways to your users to interact with your data. Feel free to share your thoughts and experiences in the comments section below.

Special thanks to Matthew Turland and Davey Shafik who both helped me out immensely when I first got my feet wet with geolocation.

Ubuntu Packages Please Get Your Act Together

2010-12-05T14:10:00.001-05:00

I didn't intend to write another blog entry so close to the conclusion of my Week with Go series, but my experiences earlier today definitely warranted a rant. Installing the SpiderMonkey JavaScript engine for use as a stand-alone interpreter on Ubuntu 10.04 LTS doesn't sound like outlandish goal. After all, CouchDB ships by default as of 9.10 and it requires a JavaScript run-time. Technologies like node.js and v8 are pretty hot right now so there might even be a few alternatives to choose from, right? WRONG!

sudo apt-get install spidermonkey-bin
E: Couldn't find package spidermonkey-bin

There used to be such a package but apparently SpiderMonkey is "unsupported" now and was removed from the Universe repository.

Sorry Ubuntu. I'm not feeling the love for Rhino. It doesn't do JIT and runs slower than a one-legged sloth. Besides, I don't feel like installing ca-certificates-java, default-jre-headless, icedtea-6-jre-cacao, java-common, libavahi-client3, libavahi-common-data, libavahi-common3, libcups2, libjline-java, liblcms1, libnspr4-0d, libnss3-1d, openjdk-6-jre-headless, openjdk-6-jre-lib, and tzdata-java on my system.

I thought about installing it from source but then I'd lose the benefits all the experts say I get from using packages*. After a bit of searching I found some developer rolled a package that is available from a Launchpad PPA, so all I'd have to do is just add the repository!

$ sudo add-apt-repository ppa:launchpad/ppa
sudo: add-apt-repository: command not found

Sigh. add-apt-repository is provided by the package python-software-properties. The intuitiveness of it all is underwhelming.

It's great that PPAs exist and people volunteer their time to maintain them to fill the gap, but Ubuntu needs to pull its head out of its ass and get its act together. Ubuntu gained popularity because people were pissed at Red Hat, not by emulating OSX with moving its close buttons, adding boring purple artwork, or the Dropbox rip-off called Ubuntu One. It needs to focus on solving real problems, like developing better package identification/search solutions, finding better ways to convey information and their rational when dropping support for packages, and making the entire system user-friendly. If Ubuntu keeps disrespecting the needs of its user base then it's only a matter of time until the popularity-torch is passed to another distro.

* Benefits include unexpected dropped support, irrelevant dependencies and symlinks puked all over the system, inconsistent packaging conventions, headaches, dry-mouth, and an occasional ulcer.

A Week with Go, Day 5

2010-12-03T00:01:00.002-05:00

Concurrent programming in Go makes use of coroutines, which are quaintly called "goroutines", and channels. Coroutines are functions executed asynchronously from the rest of your program, and channels are synchronous pipes through which the routines communicate. I had written a multi-threaded Kember Identity program in C a while back, and decided to rewrite it in Go to gain some exposure working with these features.

The Effective Go primer gives a hint of what goes on under the hood with coroutines in Go.

A goroutine has a simple model: it is a function executing in parallel with other goroutines in the same address space. It is lightweight, costing little more than the allocation of stack space. And the stacks start small, so they are cheap, and grow by allocating (and freeing) heap storage as required.

Goroutines are multiplexed onto multiple OS threads so if one should block, such as while waiting for I/O, others continue to run. Their design hides many of the complexities of thread creation and management.

Indeed, Go does a wonderful job of hiding the complexities of working with threads. Spawning the coroutine is as simple as prefacing the function call with go.

go kemberTest(start, tmp, ch)

Writing and reading values to and from a channel is done with <-. Both are blocking operations, so you don't have to worry about manually fumbling with locks to synchronize routines. The primer has this to say about channels:

Channels combine communication—the exchange of a value—with synchronization—guaranteeing that two calculations (goroutines) are in a known state.

if hash == curr {
    fmt.Println(hash)
    ch <- true
    return
}

No muss, no fuss. Coroutines and channels work together to make concurrent programming a breeze! I sat down expecting to stumble through getting my threads to communicate properly, but to be honest I spent more time working out the correct type castings throughout the program than I spent on the concurrency piece.

The asynchronous nature of coroutines with the synchronous nature of channels make it easy to write generators, too.

func fib(ch chan int64) {
    var a, b int64 = 1, 2
    for {
        ch <- a
        a, b = b, a + b
    }
}

func main() {
    ch := make(chan int64)
    go fib(ch)
    for i := <- ch; i > 0; i = <- ch {
        fmt.Println(i)
    }
}

After spending a week with Go I formed some good opinions and some bad opinions. The language has a dynamic feel with := and garbage collection which is nice, though I would have liked to have seen some more type inference being done by the compiler. An OO model that's not focused on hierarchical inheritance is a welcome change. Go is still rough around the edges and I wouldn't consider it ready to be a full-time development language for production projects just yet, but it has come quite a long way in the past year... and will undoubtedly continue to mature rapidly as an open source project.

Google did a disservice to Go when the language was first released by placing so much emphasis on compilation speed. The tech pundits and skeptics were right in questioning whether the world needed yet another language just because the current ones compiled too slow. Compilation is a function of the compiler after all, so if that was really the problem then a more appropriate solution would be to write a new compiler! But the pundits were ignorant in dismissing Go and believing there was no room for a new language. They failed to see an opportunity to re-examine old beliefs and explore now constructs. Many more would have taken Go seriously out of the gate if Google and early developers had touted its OO model and goroutines/channels instead of its compilation speed.

I can see myself using Go for a few side projects here and there, and perhaps larger projects in the future. Perhaps I'm just not comfortable yet with garbage collection in real-time and system-level programming yet, but I don't see it replacing my use of C; I choose C when a major need of my program is speed or memory and the unpredictable nature of a garbage collector works against those. But Go would definitely be a good choice for problems now solved using Java.

Thanks for spending the week with me (and Go); feel free to share your impressions of Go in the comments below. If you're interested, here's my Kember Identity code.

A Week with Go, Day 4

2010-12-02T00:01:00.002-05:00

Day 4 was spent learning about Go's take on object oriented programming, which I found to be a refreshing change from the likes of Java and C#. It's not type-focused; instead of relying on an object's type to know whether or not it offer certain functionality, the object will generally implement an interface. This mindset is not dissimilar to good JavaScript programming where feature detection is preferred over browser sniffing. The code doesn't care what something is (browser/object), just what it can do (functionality/interface).

The word "object" is probably a bit misleading since Go doesn't really have them in the traditional sense. There's no need to write a class definition as in Java, nor to instantiate an object literal like JavaScript. The programmer specifies a set of functions and Go's compiler deduces the relationships.

type Rectangle struct {
    width  float
    height float
}

func (r *Rectangle) Area() float {
    return r.width * r.height
}

func (r *Rectangle) Perimeter() float {
    return r.width*2 + r.height*2
}

func main() {
    r := new(Rectangle)
}

Here the Rectangle "object" is a pointer to a structure consisting of two floats. The Area() function can be applied as a method against any Rectangle to calculate an area, and the Perimeter() function can be applied to calculate the perimeter.

If I were to later specify the interface Shape which dictated an Area() float method and Perimeter() float method, then any Rectangle instances will automatically implement it. The compiler knows a Shape can have its area and perimeter calculated. A Rectangle can have its area and perimeter calculated; therefore, it must be a Shape.

type Shape interface {
    Area() float
    Perimeter() float
}

I appreciate Go's rejection of Java-style type-masturbation and focus on functionality. After all, code is a liability, functionality is an asset. It looks like Go will let me focus more on writing code which provides functionality instead of code that specifies type hierarchies rivaling Jesus' lineage in the book of Matthew.

Feel free to share your impressions of Go in the comments below and come back tomorrow for the final day's entry in the series. If you're interested, here's my shapes code with with Rectangle and Circle objects.

A Week with Go, Day 3

2010-12-01T00:01:00.003-05:00

The first two days of tinkering and scouring helped me form an opinion of Go based on its syntax. To form a more-informed opinion I would have to write some more code and see how much resistance I experienced along the way. What features were missing? How was typing applied? I wrote a rudimentary version of Deal or No Deal, and slowly some of those meaningless sections in the language spec started taking on more meaning.

I decided to store the case amounts as an integer array (nobody likes the 0.01 anyway!) and came across my first bit of frustration and misunderstanding when it came time to shuffle the amounts.

cases = []int{100, 200, 300, 400, 500, 750, 1000,
    5000, 10000, 50000}
shuffle(cases)

func shuffle(arr []int) {
        rand.Seed(time.Nanoseconds())
        for i := len(arr) - 1; i > 0; i-- {
                j := rand.Intn(i)
                arr[i], arr[j] = arr[j], arr[i]
        }
}

The language spec says arrays are value types and slices are reference types. shuffle was doing what I wanted, but why was Go manipulating the array values if cases was supposed to be passed by value? I posted the question on Stack Overflow and it became clear that cases was a slice. I had written []int{} expecting the behavior of [...]int{}. Hopefully this will be the first and last time I make this mistake, but I have a feeling it won't be even with a correct understanding of what Go is doing. The syntax is just too similar. I don't understand why almost identical syntax was chosen for two different concepts here but different syntax was chosen for identical concepts with var = vs :=.

And I was lucky I only needed to shuffle a list of integer values; Go doesn't have generics, so I wouldn't be able to easily write a general-purpose shuffle function. It feels "dirty" to write shufflei(arr []int), shufflef(arr []float), etc. since the functions would all be identical except for their signatures!

The appeal of clean-looking code, novel looping, and an official formatting utility was waning because of issues and deficiencies more integral to the language and its implementation. Go is still nascent, but we shouldn't be revisiting these problems in a modern programming language.

Feel free to share your impressions of Go in the comments below and come back tomorrow for day 4. If you're interested, here's my Deal or No Deal code.

A Week with Go, Day 2

2010-11-30T00:01:00.001-05:00

After dabbling a little bit on day 1, I dedicated some time on day 2 to skim through Go's language spec and standard libraries. A lot of it didn't have much relevance to me yet because I hadn't begun to play with those parts of the language. What caught my eye though was that Go supports the \v escape (obviously no one at Google has read Stop the Vertical Tab Madness). Welcome to 1963, folks.

In addition to tweaking how loops are written, Go has augmented the traditional syntax of if and switch statements too. I don't see the enhancement providing as much benefit as I do with for. It's almost as if someone decided to let people move the placement of if up a statement earlier just to be different, and it certainly doesn't read well.

x := recover()
if x != nil { ... }

if x := recover(); x != nil { ... }

The list of available packages is rather impressive considering Go has been available for a year. Some packages are pretty standard, like math and cmath, while some other packages like patch, ebnf, and tabwriter look more intriguing. The regexp and rand packages look to me a bit bloated and could stand to be pared down a bit, regexp especially since Go allows multiple return values from functions; maybe someone just got API happy.

There's also a utility named gofmt which is used to format Go code. It serves as the official style guidelines in an attempt to avoid code formatting wars. I don't agree with some of their choices, but that only highlights the usefulness of gofmt. More languages should ship with such a utility officially. gofmt can also be used as a sort of lint application since it can't properly format code with syntax errors.

The selection of library packages for Go will hopefully continue to grow over time, and a tool like gofmt is nice to have in your arsenal. I'm still thinking favorably of Go after the second day.

Feel free to share your impressions of Go in the comments below and come back tomorrow for day 3.

A Week with Go, Day 1

2010-11-29T00:01:00.002-05:00

Go is a general purpose systems programming language developed at Google by Robert Griesemer, Rob Pike, and Ken Thompson. Go has been on my radar since it became publicly available a year ago as an open source project, and since then its documentation has been improving and a small community of users has been forming around the language. Last week I had some time off from work that coincided nicely with the Thanksgiving holiday and I thought it'd be fun to spend some of it looking at Go. Here's the first in a series of five posts that share my thoughts and experiences of spending a week with Go.

My first Go programs were solutions to a couple Project Euler solutions. This was just to get a basic feel for its syntax.

Problem 1

package main

import fmt "fmt"

func main() {
    sum := 0
    for i := 0; i < 1000; i++ {
        if i%3 == 0 || i%5 == 0 {
            sum += i
        }
    }
    fmt.Printf("%d\n", sum)
}

There are a few oddities, but overall there's nothing earth shattering with Go's syntax.

Parentheses aren't used in constructs like if and for as they are in C, though braces are always required, and semicolons are used as separators-- not terminators. The resulting code looks nice and clean. The lack of terminators is a bit disconcerting, though... have we learned nothing from JavaScript?
:= elides variable declarations and the compiler will deduce the variable's type, reminiscent of OCaml's type inference (though it doesn't extend to function declarations and the like). The more verbose way of declaring and assigning sum would be var sum int = 0. It's nice once you get the hang of it, but I think this might be a bit confusing for new programmers.

Problem 2

package main

import fmt "fmt"

func main() {
    a, b, c, sum := 0, 1, 0, 0
    for c < 4000000 {
        c = a + b
        a, b = b, c

        if c%2 == 0 {
            sum += c
        }
    }

    fmt.Printf("%d\n", sum)
}

Whoa, Go doesn't have a while loop? Surely this is madness!

The designers extended the minimal syntax philosophy to common language constructs as well. Different for variations are used to write not only your traditional for loops, but also while loops and foreach/for in loops. I think for c < 4000000 is a bit awkward to read but it becomes second nature to write in short order.

It's nice to have less keywords and the code looks clean and readable. Despite some hesitance, my opinion was favorable overall after the first day.

Feel free to share your impressions of Go in the comments below and come back tomorrow for day 2.

On Gartner's 2011 Top Technologies

2010-11-16T16:50:00.006-05:00

A coworker stumbled across Gartner's top 10 technologies for 2011 and decided to share it with everyone in the office via email. Overlooking the need for a cluebat, and resisting the urge to similarly spam everyone with The Oatmeal's take on email etiquette, I think Gartner missed the mark on some things.

Let's be honest for a moment. Cloud computing has been hyped since '08 when Eucalyptus and Amazon's EC2 Services came out. Social media has been on the radar since since blogs became popular (though admittedly Twitter and Facebook upped the ante). Mobile computing has been slowly making inroads but I think it's still too soon to claim "The PC era is over." All the technologies have been around for years in one form or another; they just happen to be "hot" right now... and by the time someone starts touting them they've probably peaked.

In the next 12 months I predict there'll be a lot of potential for things like IPv4/6 networking, JavaScript, Flash, and whatever comes out of the Facebook v. Google storm that's brewing. I may be wrong... but hindsight is 20/20 so I guess I'll just have to make a few of my own predictions here and see how on the mark I was come next November.

Networking
IPv6 hasn't seen widespread adoption despite being standardized for over 10 years. Supposedly Europe is running thin on the IPv4 front with something like 4% of the address-space left, and getting down to around 1-1.5% might be enough to force the ISPs and TELCOs to start investing more in their infrastructure. Either that, or perhaps someone will devise a creative way to prolong the inevitable either with some new NAT/routing extensions, or a new backwards-compatible protocol obsoleting IPv6 that would overlay nicely on IPv4 to allow backwards compatibility and future growth (along the lines of how 64-bit processor registers extended their 32-bit registers, which in turn extended their 16-bit registers, etc).

JavaScript
Developers are finally starting to fully explore JavaScript's capabilities, are becoming more comfortable with duck-typing, and Google's V8 engine has done pretty well in speeding up JavaScript's execution, to the point where I can see more and more programmers attempting serious server-side programming in JavaScript. Server-Side JavaScript is nothing new, but if one or two start-ups can make it big within the next year or so using a technology stack focused on server-side JavaScript, it may make others in the market comfortable enough to take the plunge themselves... if not in 2011, then possibly 2012. This would be thanks in no small part to ECMA-262's strict mode in an attempt to clean some cruft from the language, and projects like CommonJS and node.js. I only hope it doesn't get bastardized with Java-like public/private syntax in the process.

HTML5/Flash
I think HTML5 is a lot of fluff-- the standard isn't expected to be completed until 2020ish, and technology moves so fast now that by then it'll be irrelevant. Browsers and developers are trying to be early adopters so they can be the cool kids on the block, but let's face it... the multimedia developer is going to prefer something like Flash to create their website experiences in than programing JavaScript code in an IDE; synchronizing timers to manipulate objects and sounds is easier with a click/drag-and-drop interface. JavaScript will be big with programmers/back-end developers, but if Adobe can re-align the Flash product and address some of the memory usage concerns, they can come out on top with the front-end/multimedia folks. Apple won't suffer, but people will see through their arguments for what they are-- self-preservation and vendor lock-in efforts.

Social Media
There's a turf war simmering between Facebook and Google that could have interesting repercussions for the rest of us. Facebook is rolling out new a new email-type feature to make communication with non-facebookers easier and to consolidate/streamline the social media experience, which looks similar to Google's failed Facebook-killer Wave product with a less-crappy interface. Google is suddenly concerned with defending our rights when it comes to our own content. I can imagine a spirited debate on privacy, ownership, and copyrights in the 21st century.

Then there's the question of how can a CEO take advantage of these changes in technology and the tech/social landscape to increase their company's profits. Since we're being honest with one another, I can say I don't know and not feel too bad about it. If I did know, I wouldn't be blogging about it here; I'd be out positioning myself to be the next Gates, Page, or Zuckerberg. Maybe there will be some new startup mash-up type service providers (like qwiki.com only not as annoying?) that will spring up which will eventually turn to targeted advertising for their revenue. Or, wouldn't it be nice if someone came up with a product/service to make parents be responsible and not raise douche-bag children, for it to be cool for kids to put down their cell phones and mp3 players and eat dinner at the table as a family, and maybe spread a little inter-office email etiquette just for good measure?

Just my thoughts...

"Inchworm on a Stick" in PHP

2010-11-10T22:52:00.001-05:00

~- is known as the inchworm on a stick operator in Perl obfuscation circles. It isn't really an operator; it's a term coined to describe negating a value and then taking its compliment, the apparent effect of which is to decrement the value. (To understand why, you need to understand how negative numbers are represented in binary.)

In Perl, ~- works for positive values:

for ($i = 3; $i > -3; $i--) {
    print ~-$i . "\n";
}

$ perl test.pl
2
1
0
4294967295
4294967294
4294967293

Interestingly enough, when the same is tried in PHP the results are slightly different...

2
1
0
-1
-2
-3

PHP treats the value as a signed integer where Perl treats it as an unsigned integer. I don't know which way is "better" or which is "more correct," but it is something I found interesting and I thought would be worth sharing.

8 Tips to Improve Your Golf Game

2010-10-02T22:38:00.014-04:00

I wasted more time golfing this week than I should have. No, not real golf... code golf. Trying to write small programs in the least number of keystrokes can be fun, challenging, and sometimes even addicting. It's not about writing pretty code or code that's easily-understandable. It's all about cramming as much code as you can into the least number of characters. While some languages golf better than others, you can still write impressively small code in your language of choice if you're familiar enough with the intricacies of its behavior. Here are 8 tips to improve your golf score when golfing with PHP:

Use short tags.
Using <? instead of <?php and using <?= instead of <?echo will both save you 3 characters.

Avoid initializing variables if possible.
Uninitialized variables assume the value 0, "", or false depending on the context in which they're referenced. $x=0; is 5 characters too much!

Know your function aliases.
A few functions in PHP are known by multiple names. join() for example is 3 characters shorter than implode().

Don't repeat yourself unnecessarily.
Consider saving the name of an oft-used function to a variable if you use the function more than once, but the function name needs to be more than 4 characters and used at least twice for any savings. $c=chop;$c($a);$c($b); is actually 4 characters more than just calling chop() twice.

Love bare literals.
PHP treats bare literals as strings, so you can save 2 characters by dropping the quotation marks.

Don't love bare literals.*
Use bit negation and binary string literals to save an extra character. ~ß is better than " ", and ~ő is definitely better than "\n".

Take advantage of side effects.
Something like $b=1;$x=80;, where $b is acting as a Boolean, might be written as $b=$x=80;. $x is assigned 80 and then assignment operator returns that value, which is then assigned to $b. Non-zero integers are considered true, so the two are semantically equivalent with a savings of 2 characters. Another example is echo$x--,"bottles of beer";, which is shorter than echo"$x bottles of beer";$x--;.

Reduce incrementally.
Write non-golf code first to make sure you fully understand the problem and your solution, then rework your code smaller and smaller in incremental steps. Don't forget to verify its correctness after each reduction.

Consider different approaches to solving the problem.
You might think to choose an array if you need a sequence of character values accessible by a numeric index, but storing them as a string might be just as effective. Then again, it may not depending on how that choice affects how you work with the data later. Don't hesitate to try both approaches to see where they lead you.

* Don't... negation... get it? Ha ha! Yeah, don't worry. I won't quit my day job.

I Remember...

2010-08-19T22:42:00.024-04:00

I remember when Java first came out surfing the Internet on my 28k dial-up connection and hitting a page with a Java applet; either Netscape would come crashing down or my computer would lock up...

I remember schoolmates telling me Java was "the future" because it ran on a virtual machine which meant it was cross platform, and then being ostracized by them when I pointed out that QBasic would be considered cross platform, too...

I remember reading Java programming books before I understood OOP and wondering why I had to write a hundred lines of code just to output "Hello World"...

I remember reading Java programming books after I understood OOP and still wondering why I had to write a hundred lines of code just to output "Hello World"...

I remember trying to convince myself every time I started Eclipse and watched my P4 run slower than a 486 that I just hadn't given Java a fair shake and I needed to embrace it with an open heart and mind...

I remember working a contract-gig on a government funded Java web-based application and trudging through spaghetti code they claimed was design patterns and infinity-tier architecture they claimed was best practices...

I remember learning unsigned integers are for losers, and Java is a programming language for winners; no one will ever need an unsigned 64-bit integer anyway...

I remember reading licensing clauses that prohibit using Java to build critical software to monitor things like nuclear reactors...

I remember seeing the "Life is too short for Java" t-shirt and feeling I was a horrible person because I sympathized with it...

I remember refusing to apply for a Java programming job, much to my parents' dismay and lecturing because I was unemployed, and instead starting my first business venture, writing my first book, and landing a full-time job using a language I enjoy...

I remember Sun's only way of making a profit off Java was suing business partners they had previously licensed its use to...

With such sweet memories, why can't I remember why I hate Java?

End of Support isn't the End of the World

2010-07-25T23:09:00.005-04:00

The PHP development team released PHP 5.2.14 last week and with it comes the end of active support for the 5.2 branch. A bit of dissent rippled throughout the community... but is it really a big deal? Contrary to popular belief, downloads from php.net don't come with an expiration date.

There is a lot of legacy code running mission-critical applications. These apps work and are stable so the time, effort, and expense required to upgrade them put doing so very low on a companies' priority lists. A few years ago I worked as a System Administrator for a credit union turned bank; the core processing system was written in PL/I and the ATM switching system was written in COBOL. There are probably more applications written in non-OOP PHP 3 code with register globals running atop a Linux 2.4 kernel than any of us want to acknowledge.

But version numbers are just mile-markers that reference a snapshot of the project at a given time. The development team is continually improving PHP so there will always be a newer, better version just around the corner. If your application is running stable on whatever version you have installed, and you're not using features or extensions that are subject to security or bug fixes in newer versions, then what's the problem? Use the version that works for you (and that your company's compliance officer will let you use).

With that said, don't expect the development team to support your favorite branch forever. PHP is open-source; people are free to participate in its development and do so for a variety of reasons. Just as the resources you can allot to refactoring legacy code are limited, the resources the development team have are limited as well. If you need a version 5.2.15, .16, or beyond then get involved and make it happen.

Learning Prolog

2010-07-13T21:47:00.020-04:00

I'm not quite sure exactly I was searching for, but somehow I serendipitously stumbled upon the site learnprolognow.org a few months ago. It's the home for an introductory Prolog programming course. Logic programming offers an interesting way to think about your problems; I've been doing so much procedural and object-oriented programming in the past decade that it really took effort to think at a higher level!

I found the most interesting features to be definite clause grammars (DCG), and unification. Difference lists are very powerful and Prolog's DCG syntax makes it easy to work with them. Specifying a grammar such as:

s(s(NP,VP)) --> np(NP,X,Y,subject), vp(VP,X,Y).

np(np(DET,NBAR,PP),X,Y,_) --> det(DET,X), nbar(NBAR,X,Y),
  pp(PP).
np(np(DET,NBAR),X,Y,_) --> det(DET,X), nbar(NBAR,X,Y).
np(np(PRO),X,Y,Z) --> pro(PRO,X,Y,Z).

vp(vp(V),X,Y) --> v(V,X,Y).
vp(vp(V,NP),X,Y) --> v(V,X,Y), np(NP,_,_,object).

nbar(nbar(JP),X,3) --> jp(JP,X).

pp(pp(PREP,NP)) --> prep(PREP), np(NP,_,_,object).

jp(N,X) --> n(N,X).
jp(jp(ADJ,JP),X) --> adj(ADJ), jp(JP,X).

det(det(DET),X) --> [DET], {lex(DET,det,X)}.

pro(pro(PRO),X,Y,Z) --> [PRO], {lex(PRO,pro,X,Y,Z)}.

v(v(V),X,Y) --> [V], {lex(V,v,X,Y)}.

prep(prep(PREP)) --> [PREP], {lex(PREP,prep)}.

n(n(N),X) --> [N], {lex(N,n,X)}.

adj(adj(ADJ)) --> [ADJ], {lex(ADJ,adj)}.

allows me to parse a statement for grammatical correctness, produce a parse tree, and determine where the point of failure is if it isn't grammatical.

?- s(X,[the,man,shoots,he],Y).
X = s(np(det(the), nbar(n(man))), vp(v(shoots))),
Y = [he]

The power of unification is highlighted by the append/3 predicate, a "function" whose primary purpose is to concatenate two lists. By providing partial lists and pattern structures, append can not only be used to concatenate, but also split a list, double it, and even test an element for membership.

is_member(X,Y) :- append(_,[X|_],Y).
?- is_member(4,[1,2,3,4,5]).
Yes

I have to say working through the course has made me a better programmer. It exposed me to new ways of thinking about old problems which broadened my "programmer toolset." I feel more comfortable working with immutable variables and recursion, too-- both concepts which I knew but subconsciously shied away from.

One of the difficulties of self-study though is the lack of teacher and classmates to ask questions and verify solutions. I purchased the printed version of the course to check my work but was dismayed to see it only contains answers for the chapter exercises. I've decided to make my solutions to the practical sections available for other self-learners.

I definitely recommend setting aside some spare time and working your way through the course if you haven't been exposed to Prolog somewhere in your studies or career already.

On Unobtrusive JavaScript

2010-07-06T18:44:00.000-04:00

Traditionally, unobtrusive JavaScript means the code is kept separate from the page's markup. This separation makes it easier to maintain and reuse the code. However, I view unobtrusive code as more than simple separation. True unobtrusiveness means your JavaScript code will function alongside other code that may be loaded by the browser, and the other code will function alongside yours. Don't pollute the global scope in the execution environment; don't clobber global objects, their properties, or prototypes with your own; and be able to work alongside various frameworks.

Use Discretion when Reviewing Code Metrics

2010-05-01T16:26:00.007-04:00

Code metrics are interesting creatures. Some are just raw numbers, such as depth of inheritance or lines of code, while others are a bit more subjective, like a maintainability index. But ultimately they are all meaningless without broader context and an understanding of the code.

As a brief example, consider the following C# function which accepts a string and returns the 40-character hexadecimal representation of the string's SHA1 hash.

public static String Sha1(String text)
{
    using (SHA1Managed sha1 = new SHA1Managed())
    {
        Byte[] textBytes = Encoding.Unicode.GetBytes(text);
        Byte[] hashBytes = sha1.ComputeHash(textBytes);
        return Convert.ToBase64String(hashBytes);   
    }
}

The function accomplishes one task. Variables are defined closest to their usage. Function calls are clear and do not nest other function calls. Even using ( ) is used so the run-time can automatically dispose of the SHA1Managed resource. Yet a scan using Visual Studio 2010's Code Metrics returns a maintainability index of 71.

The index increases to 73 when the definition of variables move outside of the using ( ) block.

public static String Sha1(String text)
{
    Byte[] textBytes, hashBytes;
    using (SHA1Managed sha1 = new SHA1Managed())
    {
        textBytes = Encoding.Unicode.GetBytes(text);
        hashBytes = sha1.ComputeHash(textBytes);
    }
    return Convert.ToBase64String(hashBytes);  
}

Eliminating the variables altogether will increase the maintainability index to 76.

public static String Sha1(String text)
{
    using (SHA1Managed sha1 = new SHA1Managed())
    {
        return
            Convert.ToBase64String(sha1.ComputeHash(
                Encoding.Unicode.GetBytes(text)));
    }
}

76 is better than 71, but is the latter code really more readable and maintainable?

I posted this example to StackOverflow and asked if anyone knew why the metric would even increase in the first place. The responders agreed it is counter-intuitive, and the consensus is that it is better to focus on writing clean, concise, readable code.

An extremely low-scoring metric can be an indicator that something should be flagged for review, but use your discretion and judgment when reviewing the code. Use the report as a tool to identify possible problems and not a set of requirements to be met.

Signing Assemblies with a Strong Name

2010-04-24T00:05:00.002-04:00

Code Analysis/FXCop warned me that my credit card application was not signed with a Strong Name, which would make it more difficult to determine if the assemblies had been tampered with. For more information on Strong Names and why they're a good thing, see this Tech Republic article.

you first need a cryptographic key before you can sign an assembly. The key is created using the sn.exe tool provided by the Windows SDK.

sn.exe -k sgKey.snk

I added my sgKey.snk file to my project in Visual Studio, and then in the Application's properties I went to the Signing tab, checked the "Sign the assembly" box, and specified my key file.

I had forgotten that I used a 3rd party library to manage logging the user out after a configurable period of inactivity and their assembly was not signed. You can't sign an assembly unless all of its dependencies are signed as well, which makes sense. You need to replace the unsigned assemblies with signed ones first.
If you can compile the 3rd party library from source, you can sign it yourself; otherwise you'll want to ask them to provide you with a signed assembly. I was in an odd situation where I had compiled the library but had not saved the code, and couldn't find the open source project from which I originally gotten the code. My solution was to sign it myself by disassembling the assembly, and re-assembling it using my key.

The ildasm.exe tool is used to disassemble .NET assemblies.

ildasm /output:Timer.il Timer.dll

Then, the ilasm.exe tool let me provide my key file and re-assemble the library so I had a signed assembly.

ilasm /dll /key:sgKey.snk  Timer.il

ildasm.exe is provided by the Windows SDK, and ilasm can be found in your %WINDIR%\Microsoft.NET\Framework\v## directory (where ## is replaced by an appropriate version number of the .NET Framework).

It's not uncommon to have multiple versions of .NET installed on a computer, let alone on a developer's computer, so be sure to use ilasm.exe for the lowest version of .NET you wish to support when you re-assemble your library. You can't assemble it with v4.0.30319\ilasm.exe if you're targeting a .NET 2.0 platform.

Code Analysis in Visual Studio

2010-04-22T21:53:00.000-04:00

Continuing to play around with Visual Studio 2010 Ultimate, I ran the Code Analysis tool on some code I had written for a customer-- a desktop-based application which securely stores credit card information using hardware identifiers as portions of the encryption key. I started with over 300 warnings and have now worked them down to around 120 warnings or so. Those that remain are Globalization related, which I'm not concerned about since it was a one-off project that is unlikely to be internationalized. FxCop, the utility which Code Analysis is based on, is freely available online.

Visual Studio 2010 Released

2010-04-20T22:44:00.005-04:00

If you haven't heard the news already (which I hadn't because I typically don't keep up with such things), Visual Studio 2010 came out earlier this month. I've been playing around with Visual Studio 2010 Ultimate (a 90-day trial version is available) and I must say I'm impressed. Microsoft has finally added common features like block editing and text-zoom, an Extension Manager to extend the IDE à la Eclipse, and support for jQuery. If I were a Fortune 500 company hacking out Windows code all day I could justify a couple of licenses if it really helped my developers efficiently produce a more secure and stable application, but the price tag puts it far out of reach for my needs. I'll stick with either VS2005 or C# Express 2010.

I'll probably post a few follow-up entries as I explore more throughout the next 90 days, so be sure to keep an eye out!

Null Pointer Exception

2010-03-29T21:42:00.004-04:00

This is why I don't program in Java:

import java.util.ArrayList;

class Program
{
   class Person
   {
       public Person partner;
   }

   class Prostitute extends Person { }
   class Eunich extends Person { }

   private Prostitute prostitute;
   private Eunich eunich;

   public static void main(String[] args) {
       Program p = new Program();
       p.run();
   }

   public void run() {
       eunich.partner = prostitute;
   }
}

> Exception in thread "main" java.lang.NullPointerException

But if you do, you should check out the nifty online JXXX Compiler Service.

Subversion Aversion

2010-02-19T20:58:00.005-05:00

There once was an app called Subversion
Who's working was utter perversion;
Whether a check in or check out, I'm always in doubt
If it's preserving my coding excursion.

When Will We Ever Use This?

2010-02-05T22:26:00.009-05:00

My sister's boyfriend showed me a nifty chart which presents the size of by distance from a television before your eyes begin to notice the difference of 1080p resolution. Just out of curiosity, we measured the length of my living room and looked at the chart to see how large of a television I would need. The distance between my couch and television is a bit shy of 20ft... off the chart!

The chart goes up to 130 inches, or approximately 11 feet. But an 11 ft diagonal measurement doesn't really mean anything to me other than "big ass TV"*, so I decided to dig out my high school math skills to help me put it into perspective. Assuming a standard 16:9 wide screen display and some liberal number rounding:

I would need a television that's almost 6 ft tall, and 10 ft wide if I wanted to see Jay Leno's crow's-feet in high definition!

I only have 8 ft ceilings so that gives 2 ft of clearance above it, and the room is 16 ft wide so I'd have 3 ft of clearance on each side (just 6 inches wider than the width of my door casings). That's not a "big ass TV"... that's my wall!

I realized two things after running the numbers: First, that was a great real-life example to answer the question "when will we ever use this stuff?" we were all asking our teachers when we were learning algebra. Secondly, I wonder how many people are wasting money on all the latest and greatest video technology. I'm glad I don't own a Blu-ray player, or I'd now be scratching my head wondering "when will I ever use this stuff?"

* For my friends in metric countries, that's "big arse television"

Writing a URI Regular Expression

2010-01-04T23:15:00.005-05:00

A friend of mine was tasked with writing a regular expression that could recognize a Uniform Resource Identifier (URI) and break apart its primary components. Not a terribly difficult task since there are a lot of sample regexs one can just pluck from the Internet. But he asked my opinion and I deferred to RFC 3986 which outlines the generic syntax for URIs.

The RFC provides this example expression:

^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?

Personally though I think that's rather sloppy. Were I to write the expression from scratch myself then I'd probably be more verbose and restrictive; I'd expressly match patterns specified in the RFC's ABNF. For example, the RFC defines the scheme portion of a URI as:

scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

The portion ^([^:/?#]+): will match characters that are not lexically permitted, such as an underscore or percent-sign. Assuming the i (case-insensitive matching) modifier is used, something like ^([A-Z][A-Z\d\+\-\.]*): would be more correct.

This highlights one of the joys of writing regexs; regexs is one of the few areas of computing in which you don't have to be completely accurate to achieve the desired results. A "close-enough" match will suffice most of the time.

While I'm on the topic of regexs, check out gskinner.com's RegExr: Online Regular Expression Testing Tool at www.gskinner.com/RegExr/ if you haven't already. It’s a great utility!

PHP and SQLite2 on CentOS

2010-01-03T01:29:00.008-05:00

I'm changing shared hosting providers for my websites and the sites I manage for clients as Salt City Tech. Most of the sites are basic websites powered by PHP, share a common code base, and store data in MySQL databases. But I've also been taking advantage of the situation to transition the sites from MySQL to SQLite. There are a lot of things that annoy me about SQLite (such as allowing NULLs in primary-key columns, ignoring foreign key constrains, and incomplete ALTER TABLE support) and a few things that I think are pretty awesome (such as user-defined functions in PHP). Pragmatically speaking, SQLite will be more convenient for deployment to and backup from the shared hosting environment and the sites' simple storage requirements fall exactly in SQLite's sweet-spot.

The transition hasn't been terribly difficult... the biggest obstacle has been configuring a development environment that adequately mirrored the shared hosting deployment environment. I happened to have a cloned virtual CentOS 5.3 image at my disposal so I started with that, but the hosting provider supports SQLite v2 (ancient!), which's storage format is incompatible with the newer version 3 in CentOS. The search engines didn't offer me much help in finding a packaged SQLite2 solution on CentOS that would meet my needs, so I figured I'd post a few notes in case someone else may find them helpful.

Obtain the sqlite-2 RPM for CentOS from RPMForge repository (in case you want to work with the database files directly):

wget http://dag.wieers.com/rpm/packages/sqlite/
sqlite-2.8.17-1.el5.rf.i386.rpm

Install sqlite using the -i and --force options (do not use -U so you do not replace sqlite3):

rpm -i --force sqlite-2.8.17-1.el5.rf.i386.rpm

Install the php-devel package so phpize and phpconfig are available:

yum install php-devel

Obtain the source RPM for PHP (CentOS installs 5.1.6--yuck! Now I remember why I hate using packages for important software):

wget ftp://mirror.switch.ch/pool/3/mirror/centos/5.3/os/SRPMS/
php-5.1.6-23.el5.src.rpm

Extract PHP from the SRPM:

rpm2cpio php-5.1.6-23.el5.src.rpm | cpio -id php-5.1.6.tar.gz

Uncompress the archive:

tar zxvf php-5.1.6.tar.gz

Build the sqlite extension using the standard phpize, ./configure, make routine:

cd php-5.1.6/ext/sqlite
phpize
./configure
make

Install the extension:

cp modules/sqlite.so /usr/lib/php/modules/
echo extension=sqlite.so > /etc/php.d/sqlite2.ini

Even though it's been obsoleted by version 3 for almost 6 years now, version 2 of SQLite works fine for my needs here. And God forbid I have to migrate the sites again or the provider upgrades, upgrading my code base is as simple as this:

sqlite site.db.old .dump | sqlite3 site.db
for f in $(ls *php); do
sed -i 's/new SQLiteDatabase/new SQLite3/g' $f
sed -i 's/fetch\(SQLITE_ASSOC\)/fetchArray\(SQLITE3_ASSOC\)/g' $f
done

Happy 2010!

2010-01-01T00:26:00.001-05:00

Happy New Year! I hope everyone has had a rewarding holiday and will have a prosperous new year! Last year I tentatively designated 2009 as my "Year of Balance." I wanted to make a concerted effort to focus more on what's really important in life. With a busy full-time job, a death in the family, juggling several side-projects, and buying a house this year (2009 was probably my "Year of Expense" in retrospect with the new house and all the joys that come with home ownership)... my head would probably have exploded had I not been making that effort. I'm not sure what my theme for 2010 will be, but I'll still be doing my best live, laugh, and love as much as I can and I hope you will be to!

Pop Quiz

2009-10-12T20:17:00.012-04:00

Here's a little test to separate the serious coders from the cut-and-paste script kiddies. Given the need to generate an arbitrarily long string consisting of random alpha-numeric characters, which solution is best?

Solution A:

function randomString($len) {
    $chars  = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" .
              "abcdefghijklmnopqrstuvwxyz" .
              "0123456789";
    $rndMax = strlen($chars) - 1;
    $str = "";
    while ($len-- != 0) {
        $str .= $chars[rand(0, $rndMax)];
    }
    return $str;
}

$str = randomString(8);
echo "$str\n";

Solution B:

class RandomSequenceIterator implements Iterator
{
    protected $seqMembers;
    protected $key;
    protected $limit;

    public function __construct() {
        $this->setMembers(null)
             ->setLimit(0)
             ->rewind();
    }

    protected function setMembers($strValue) {
        $this->seqMembers = $strValue;
        return $this;
    }

    protected function getMembers() {
        return $this->seqMembers;
    }

    protected function setLimit($intValue) {
        $this->limit = $intValue;
        return $this;
    }

    protected function getLimit() {
        if (empty($this->limit)) {
            return 0;
        }
        else {
            return $this->limit;
        }
    }

    public function current() {
        $index = rand(0, strlen($this->getMembers()) - 1);
        return substr($this->getMembers(), $index, 1);
    }

    public function valid() {
        return $this->key() < $this->getLimit();
    }

    public function key() {
        return $this->key;
    }

    public function next() {
        $this->key++;
    }

    public function reset() {
        $this->rewind();
    }

    public function rewind() {
        $this->key = 0;
    }
}

class RandomCharacterSequenceGenerator extends
RandomSequenceIterator
{
    public function setChars($strValue) {
        $this->setMembers($strValue);
        return $this;
    }

    public function getChars() {
        return $this->getMembers();
    }

    public function generate($limit) {
        $strBuffer = "";
        $this->setLimit($limit);
        foreach ($this as $char) {
            $strBuffer .= $char;
        }
        return $strBuffer;
    }
}

abstract class RandomGeneratorBase
{
    protected static $instance;
    protected static $generator;

    protected function __construct() {
        self::$generator = new
            RandomCharacterSequenceGenerator();
    }

    // abstract public static function getInstance();
    public static function getInstance() {
        if (empty(self::$instance)) {
            self::$instance = new self();
        }
        return self::$instance;
    }

    public function generate($limit) {
        return self::$generator->generate($limit);
    }
}

class RandomNumericStringGenerator extends RandomGeneratorBase
{
    const VALID_MEMBERS = "0123456789";

    protected function __construct() {
        parent::__construct();
        self::$generator->setChars(self::VALID_MEMBERS);
    }

    public static function getInstance() {
        if (empty(self::$instance)) {
            self::$instance = new self();
        }
        return self::$instance;
    }
}

class RandomUpperCaseAlphaStringGenerator extends
RandomGeneratorBase
{
    const VALID_MEMBERS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";

    protected function __construct() {
        parent::__construct();
        self::$generator->setChars(self::VALID_MEMBERS);
    }

    public static function getInstance() {
        if (empty(self::$instance)) {
            self::$instance = new self();
        }
        return self::$instance;
    }
}

class RandomLowerCaseAlphaStringGenerator extends
RandomGeneratorBase
{
    const VALID_MEMBERS = "abcdefghijklmnopqrstuvwxyz";

    protected function __construct() {
        parent::__construct();
        self::$generator->setChars(self::VALID_MEMBERS);
    }

    public static function getInstance() {
        if (empty(self::$instance)) {
            self::$instance = new self();
        }
        return self::$instance;
    }
}

class RandomMixedCaseAlphaStringGenerator extends
RandomGeneratorBase
{
    protected function __construct() {
        parent::__construct();
        self::$generator->setChars(
           RandomUpperCaseAlphaStringGenerator::VALID_MEMBERS .
           RandomLowerCaseAlphaStringGenerator::VALID_MEMBERS);
    }

    public static function getInstance() {
        if (empty(self::$instance)) {
            self::$instance = new self();
        }
        return self::$instance;
    }
}

class RandomMixedCaseAlphaNumericStringGenerator extends
RandomMixedCaseAlphaStringGenerator
{
    protected function __construct() {
        parent::__construct();
        self::$generator->setChars(
            RandomNumericStringGenerator::VALID_MEMBERS .
            self::$generator->getChars());
    }

    public static function getInstance() {
        if (empty(self::$instance)) {
            self::$instance = new self();
        }
        return self::$instance;
    }
}

$str = RandomMixedCaseAlphaNumericStringGenerator
       ::getInstance()
       ->generate(8);
echo "$str\n";

If you managed to scroll down this far, congratulations! You are a true programmer who knows the correct answer is B.

Solution A is a good example of unorganized spaghetti-code. It "works" but it's amateurish. A uses only language-primitives-- none of the advanced, enterprise architecture concepts embodied in Solution B. It is certainly not flexible. (What if one changed the requirements to generate a random string of only digits, for example?)

Solution B is much more organized, encapsulated, and extensible. Object-oriented code models the way things are in the "real world" so it's easier to conceptualize what the code is doing. The problem is broken down into several smaller, more manageable ones, and uses interfaces, abstract classes, and inheritance to eliminate redundant code, facilitate better organization, and foster code-reuse. Understanding existing code is an important part of programming and the liberal use of design patterns such as Singleton and Decorator embody solutions to common problems giving programmers a common vocabulary with higher levels of abstraction to communicate with one another.

Of course, solution B should only serve as an example and not be used as-is in a production environment. Exceptions should be used, but are omitted here for the sake of brevity. And the actual invocation to generate the random string could probably be further encapsulated and abstracted by implementing the Factory design pattern and maybe the Registry pattern-- one would query the registry for a value object to pass to the factory, and in turn the factory would determine which type of generator to return: random uppercase alpha, random numeric, mixed case, etc.

Setting Up SFTP From PHP

2009-10-07T01:18:00.008-04:00

PHP's ftp_ssl_connect() function is for SSL-FTP, where as what I need for a client's application is SFTP. Isn't life grand! Well, it's not really too much trouble... PHP can handle that too with functions from the ssh2 PECL extension. I'm just glad I caught it early on instead of at the 11th hour. I figured though I might as well continue my previous post about setting up this project with a brief description on installing ssh2 and testing it to ensure everything is in working order.

Installation

The ssh2 extension provides bindings to libssh2 which must be installed first on the system. My target platform is CentOS 5.3, so I was able to install libssh2 and libssh2-devel via yum (using the RPMForge repository).

yum install libssh2 libssh2-devel

The ssh2 extension is available from the pecl.php.net website. There is a minor bug in the current version (0.11.0) which prevents it from compiling against PHP 5.3 so I needed to apply this patch.

wget http://remi.fedorapeople.org/ssh2-php53.patch
tar zxf ssh2-0.11.0.tgz
cd ssh2-0.11.0
patch < ../ssh2-php53.patch

I compiled and installed the extension after the code was patched.

phpize
./configure --with-ssh2
make
cp modules/ssh2.so /usr/local/php/lib/php/extensions/

Adding module=ssh2.so to php.ini and restarting Apache completed the installation. I was then ready to move on and test it to make sure it worked as it should.

Testing the Extension

Using the extension to upload and retrieve files programmatically over an SFTP connection is relatively simple. First, a secure shell connection is established with the SFTP server using ssh2_connect(), and then a login is authenticated with ssh2_auth_password(). Both functions return false if they fail:

$sess = @ssh2_connect(SFTP_SERVER, SFTP_PORT);
if ($sess === false) {
   echo "Connection failed.";
   exit(-1);
}
$result = @ssh2_auth_password($sess, SFTP_USERNAME,
   SFTP_PASSWORD);
if ($result === false) {
   echo "Unable to authenticate.";
   exit(-1);
}

Once a session has been established, the ssh2_sftp() function is used to retrieve an SFTP resource.

$sftp = @ssh2_sftp($sess);
if ($sftp === false) {
   echo "Unable to initialize SFTP subsystem.";
   exit(-1);
}

From that point on, the SFTP resource is used with the ssh2.sftp filestream to read and write files.

file_put_contents("ssh2.sftp://" . $sftp . "/tmp/test.txt",
   $data);

In writing my short automated test case, I dumped some bytes from /dev/urandom to generate a test file, wrote the data to the SFTP server, read it back, and compared the results to make sure they matched after the round-trip.

// generate random data for test file
$data = file_get_contents("/dev/urandom", FILE_BINARY, null,
   0, 512);
$data = substr(convert_uuencode($data), 0, 512);

// write data to file on SFTP server
file_put_contents("ssh2.sftp://" . $sftp . "/tmp/test.txt",
   $data);

// retrieve file from SFTP server
$retrieved = file_get_contents("ssh2.sftp://" . $sftp .
   "/tmp/test.txt");

// compare original data with retrieve data
echo ($data == $retrieved) ? "Success!" : "Corruption!";

Creating a CentOS-Based LAMP Virtual Image

2009-09-26T21:30:00.008-04:00

In doing some preliminary research and planning for a client's new project, I determined his current in-house deployment platform would not be sufficient given his requirements. Specifically, the project calls for a moderate amount of URL re-writing and the ability to programmatically FTP files to a remote host. The client is running IIS on Windows Server 2008; I’m not too keen on ISAPI rewrite and IIS Rewrite seems to have fallen off the face of the Internet, and the ftp_ssl_connect() function is only available in PHP if both the ftp module and OpenSSL support are statically built-in so we would have to maintain a build environment for him, too. A LAMP-stack makes more sense. Apache can rewrite URLs with mod_rewrite and compiling PHP is a more supported practice on Linux than it is on Windows.

I discussed the obstacles and some possible solutions with the client and he's okay with LAMP. Instead of bringing in more hardware, though, I suggested taking advantage of virtualization. I assured him I could create a virtual platform that would provide us with everything we need, appear as a new machine on his network, and run directly on top of Windows Server 2008.

Installing CentOS

Originally I wanted to use the new Slackware64, but VMware-Tools proved too much of a struggle to install and I didn't feel comfortable using it for a client's project because of that. I eventually settled on CentOS 5.3 instead.

I fired up the trial version of VMWare Workstation to configure a basic machine image... though I have VMware Workstation 6.5, I chose to set the virtual machine's hardware compatibility for Workstation 5 and compatible with ESX Server. I figured this will give us some flexibility if we need to move the image to bare-metal in the future. CentOS is built from RHEL sources, so I was able to set the Guest Operating System as Red Hat Enterprise Linux 5 and use any Red Hat-specific documentation VMware has.

I tried to keep the installation small, so I unchecked everything in Anaconda-- including the Base packages. I still got packages what I feel are unnecessary dependencies (Requiring wireless-tools on a sever installation, for example. Seriously, Red Hat!), but I guess I can live with it and it won't matter much to the client.

Once CentOS was installed and booted and I was logged in, I needed to install some packages (and their dependencies) with yum that I didn't install during the installation:

autoconf

curl-devel

freetype-devel

gcc

gcc-c++

libjpeg-devel

libpng-devel

libxml2-devel

lynx

make

ncurses-devel

ntp

openssl-devel

patch

perl

sendmail

wget

which

zlib-devel

Notice I didn't install Apache, MySQL, or PHP. That's because I like to compile and install the major software from source. This way I can make sure they're up to date and configure their builds exactly how I need them.

Configuring Mapped Directories

I want to keep the application's data separate from the virtual image so I wouldn't be constrained by the size of the image (trying to explain why he couldn't save more than a gig of data when it was running on a physical server with 100 gigs of free drive space wouldn't be fun). The next task was to create shared data directories on the host and install VMware-Tools so I could map them. I created a directory shared as apache to hold the bulk of the application's code (.php, .html, etc), and mysql to hold the database's tables.

The VMware documentation describes the VMware-Tools installation process in detail, but it's no more difficult than selecting "VM" -> "Install VMware tools..." in VMware Workstation, and then proceeding to install the VMware-Tools RPM in CentOS.

mount /dev/cdrom /media
rpm -Uvh /media/ VMwareTools-7.8.5-156735.i386.rpm
umount /media
vmware-config-tools.pl

VMware adds the following to /etc/fstab:

# Beginning of the block added by the VMware software
.host:/         /mnt/hgfs     vmhgfs   defaults,ttl=5   0 0
# End of the block added by the VMware software

That entry will make the shared folders on from the host operating system accessible as /mnt/hgfs/apache and /mnt/hgfs/mysql. Everything within them owned by root with global read, write, and execute permissions. There's not much that can be done about the lax permissions, but I could at least have the files owned by a more appropriate user than root. I wanted to have them each mounted under /srv instead of /mnt/hgfs as well to be a little more LSB compliant (suck it, /var/www!), so I replaced their entry with my own:

.host:/apache   /srv/apache   vmhgfs   defaults,ttl=5,uid=99,gid=99   0 0
.host:/mysql    /srv/mysql    vmhgfs   defaults,ttl=5,uid=27,gid=27   0 0

It would be nice if future version of VMware will have a more flexible HGFS driver-- but this will be sufficient for the task at hand. At last I could install Apache, MySQL, and PHP.

Compiling

There isn't anything too exciting about installing Apache, MySQL, and PHP from source to talk about, so I'll just share with you my configure options.

MySql Enterprise 5.0.88sp2

./configure \
--prefix=/usr/local/mysql \
--localstatedir=/srv/mysql \
--with-unix-socket-path=/tmp/mysql.sock \
--with-mysqld-user=mysql \
--without-debug \
--with-archive-storage-engine \
--with-csv-storage-engine \
--with-federated-storage-engine \
--disable-maintainer-mode \
--enable-assembler \
--enable-largefile \
--enable-local-infile \
--enable-thread-safe-client

Apache 2.2.13

CFLAGS=-O3 ./configure \
--prefix=/usr/local/apache \
--with-pcre \
--disable-status \
--enable-mods-shared=all \
--enable-so \
--enable-ssl \
--enable-setenvif \
--enable-rewrite

PHP 5.0.3

CFLAGS=-O3 ./configure \
--prefix=/usr/local/php \
--with-apxs2=/usr/local/apache/bin/apxs \
--with-mysql=/usr/local/mysql \
--with-pdo-mysql=/usr/local/mysql \
--with-mysqli=/usr/local/mysql/bin/mysql_config \
--with-gd \
--with-jpeg-dir=/usr/lib \
--with-freetype-dir \
--with-curl \
--with-openssl \
--enable-ftp \
--with-openssl-dir

After that I needed to open CentOS's firewall to allow HTTPS traffic using system-config-securitylevel-tui, and change the security context of the libphp5.so module for Apache because SELinux is enabled.

Final Housekeeping

There were only a few minor housekeeping things to attend to after I had everything installed. I had to add a couple kernel parameters and configure ntp according to VMware's Time Keeping Best Practices for Linux so the time didn't drift. It was also important that I configure logrotate to rotate Apache and MySQL's log files as I did not install them via RPM. Otherwise they could grow unwieldy and use up all the space I had allocated for the virtual image.

So in short order I had not only a sane platform for deployment, but one I could easily clone and use for development as well. The client only needs the free VMware Player software to use the image. The data directories are on the host operating system alongside the image so they are not constrained by the size of the image and can be backed-up independently of the image. When necessary, upgrading the virtual platform can be done independently of the data.

Update 10/04/2009: It appears the above procedure didn't install a cron daemon, though it did install crontab files-- now isn't that interesting!

rpm -qa | grep cron
crontabs-1.10-8

yum install vixie-cron resolved the issue. Don't forget to issue chkconfig crond on so it starts automatically, and /etc/init.d/crond start to start cron for the current session (so you don't have to reboot).

New Feature at Paste Ninja

2009-07-28T23:11:00.005-04:00

Some of you may have noticed last week a new feature was rolled out on Paste Ninja, the premier PHP-powered paste bin-- patches! Here's how it works:

Update an existing paste

Click the Compare link that appears in the updated paste's header

In the Compare dialog that opens, select your desired current and target revisions; a colorized unified diff is displayed so you can verify the changes

Click the dialog's Download button

It’s that simple!

Seamless Error Highlighting

2009-07-21T01:09:00.014-04:00

A lot of output can be generated when you compiling large projects. When it breaks, it can be difficult to identify the particular spot in the build-process where things when wrong. Highlighting the error messages can help them stand out from the rest of the output.

ANSI escape sequences can be used to modify how your terminal window displays its text. For example, outputting the sequence \033[41;37mHello World\033[0m would result in "Hello World" displayed in white text against a red background. Escape sequences begin with an escape character (ASCII 27, octal 033) and bracket. The control values are then given (multiple values are semicolon-separated) and the entire sequence closes with m.

You can highlight certain messages by routing the STDOUT and STDERR streams to sed and performing a replacement.

s,(.*error.*|.*fail.*|.*undef.*),\033[41;37m\1\033[0m,gi

The values you match are of course entirely up to your discretion.

The tricky part is quoting and escaping the expression correctly so various meta-characters aren't intercepted by the shell. And some implementations of sed won't correctly convert \033 to an escape character, so you may need to enter it directly by typing CTRL+V, CTRL+[. Depending on your terminal, an actual escape character may be displayed as ^[ or a special glyph like when you enter it.

make install 2>&1 | sed -e \
's,\(.*error.*\|.*fail.*\|.*undef.*\),[41;37m\1[0m,gi'

If you find yourself using such highlighting often, you may want to define a function to save yourself some typing. For example, with bash you can add something like this to your .bashrc file:

function make() {
/usr/bin/make $@ 2>&1 | sed -e \
's,\(.*error.*\|.*fail.*\|.*undef.*\),[41;37m\1[0m,gi';
}

You can type make install at the prompt like you normally would. bash will call the new make() function, which in turn calls the actual make utility with any arguments (such as install) and colorizes the output. Error highlighting is now seamless and automatic!

OOP in JS

2009-07-14T20:12:00.008-04:00

When people would ask me about Object Oriented Programming in JavaScript, I would always send them to two pages written by Gavin Kistner. The other day when I went to visit them myself to quickly check something, they were gone! What happened, Gavin?!

I promptly dug the pages up from a search engine's cache to mirror them, but all the original copyrights still belong to Gavin.

OOP in JS, Part1: Public/Private Variables and Methods
originally at http://phrogz.net/JS/Classes/OOPinJS.html

OOP in JS, Part 2: Inheritance
originally at http://phrogz.net/JS/Classes/OOPinJS2.html

Currying in PHP

2009-06-28T23:48:00.026-04:00

What happens if you don't have all the arguments handy for a function, but you want to give whatever arguments you do have now and then provide the rest of them to the function later? This is called currying, and is a core concept in functional programming. It's messy, but possible to curry functions in PHP now that closures have been added.

First, let me show you how currying looks in a functional language. Here's a basic example in OCaml/F#:

let do_math op x y =
  match op with
      '+' -> x + y
    | '-' -> x – y
    | _   -> failwith "Invalid op"

let add = do_math '+'

let inc = add 1
let dec = add (-1)
;;

A function named do_math is defined that accepts an operator and two operands. The function's return value will be either the sum or difference of the operands, depending on whether the given operator is + or -. Notice how do_math is then called with a single argument. OCaml doesn't raise an error; it simply returns a function that "remembers" the first argument and accepts the remaining two arguments later (this is an over-simplified and slightly inaccurate statement, but a good enough description for our purpose here). This intermediate function can be used elsewhere, as in the bindings for inc and dec.

Now here's a version of the do_math() function in PHP:

function do_math($op, $x, $y) {
    switch ($op) {
        case '+':
            return $x + $y;

        case '-':
            return $x - $y;

        default:
             throw new Exception("Invalid op");
    }
}

Unfortunately, PHP will throw warnings if you call do_math() without the three arguments it expects.

Warning: Missing argument 2 for do_math(), called in /home/tboronczyk/curry.php on line 16 and defined in /home/tboronczyk/curry.php on line 2 Warning: Missing argument 3 for do_math(), called in /home/tboronczyk/curry.php on line 16 and defined in /home/tboronczyk/curry.php on line 2

Whereas functional languages have currying "built-in," you must explicitly code this ability in an imperative language. Doing so in PHP requires the use of closures:

function do_math($op) {
    return function ($x) use ($op) {
        return function ($y) use ($op, $x) {
            switch ($op) {
                case "+":
                    return $x + $y;

                case "-":
                    return $x - $y;

                default:
                    throw new Exception("Invalid op");
            }
        };
    };
}

It's also possible to extend the function using func_num_args() and func_get_arg() functions, anonymous functions, and closures, so that any number of parameters can be given at a time.

function do_math() {
    if (func_num_args() >= 1) $op = func_get_arg(0);
    if (func_num_args() >= 2) $x  = func_get_arg(1);
    if (func_num_args() == 3) $y  = func_get_arg(2);

    switch (func_num_args()) {
        case 1:
            return function () use ($op) {
                if (func_num_args() >= 1) $x = func_get_arg(0);
                if (func_num_args() == 2) $y = func_get_arg(1);

                switch (func_num_args()) {
                    case 1:
                        return function ($y) use ($op, $x) {
                            return do_math($op, $x, $y);
                        };

                    case 2:
                        return do_math($op, $x, $y);

                    default:
                        trigger_error(
                            "invalid argument count",
                            E_USER_WARNING);
                }
            };

        case 2:
            return function ($y) use ($op, $x) {
                return do_math($op, $x, $y);
            };

        case 3:
            switch ($op) {
                case "+":
                    return $x + $y;

                case "-":
                    return $x - $y;

                default:
                    throw new Exception("Invalid op");
            }

        default:
            trigger_error("invalid argument count",
                E_USER_WARNING);
    }
}

It's messy... but it works! Now you are able to pass one or two arguments to do_math(), capture the intermediate function that's returned, and pass the remaining argument(s) later.

$add = do_math("+");

$inc = $add(1);
$dec = $add(-1);

echo do_math("-", 3, 2);
echo do_math("+", 1, 1);
echo $inc(2);
echo $add(2, 2);
echo $dec(6);
echo $add($inc(4), $dec(2));

The switch statements are rather unmanageable and the spaghettification of code grows exponentially with the addition of each argument. This pattern is straight forward, though. You may want to consider writing a code generator to handle the dirty work of retrofitting a function to one capable of being curried rather than writing them all by hand. Of course, if you know of a better way to curry functions in PHP then let me know by leaving a comment!

Update 06/29/09: Someone asked me what the "real-world use" for all this would be. Currying is used all the time in functional programming, but the hassle of explicitly enabling the behavior in PHP makes that a valid question. My motivation was just to see if it were possible and share my results. Indeed it is. Functions can be curried in any language that supports closures. But for those who want something a little more concrete, let's consider callback functions.

In a previous post I gave the following example to illustrate the use of closures:

$userPercent = 0.5;
$userList = array_filter($percentVowels,
    function($percent) use ($userPercent) {
        return ($percent >= $userPercent); 
    });

It showed an anonymous function being used with array_filter() to filter an array. The array is filtered based on a dynamic value, and a closure is used to "inject" the threshold rather than using a global statement. The same could also be accomplished with currying.

The problem is array_filter() expects a callback function that accepts one argument--the current array element. Currying will allow us to prepare the function with the sorting threshold, and the intermediate function can be used as the callback.

function callback($userPercent) {
    return function($percent) use ($userPercent) {
        return ($percent >= $userPercent);
    };
}
$userList = array_filter($percentVowels, callback(0.5));

Kember Identity

2009-06-14T22:33:00.003-04:00

Ever wonder if there is an MD5 hash the same as the original input? Nope, me neither. But Mr. Kember does and he's asking the world to help him find out if such a thing exists. There's no fame if you find it for him (he's humbly named it the "Kember Identity" already)—but you might make a little cash. Check out his web page for the details. Go ahead and enter his contest if you're feeling ~~gullible~~ lucky!

The MD5 algorithm returns a fixed-length 128-bit hash, so there are 2¹²⁸ possible values. The hash is typically expressed as a series of 32 hexadecimal values. Since the input string and its hash must be the same to reflect the Kember Identity, you wouldn't need to test random strings like "ruby on rails rots your brain"; you only need to test strings that are 32-characters long and contain the numbers 0 though 9 and letters a through f like 8d112b3c68248c12f178188c1b921ec1.

Kember suggests testing values at random because the range of candidates is so large (2¹²⁸ is 34,028,236,692,093,846,346,337,460,743,177). Unfortunately, there're a few problems with this approach:

Deterministic machines cannot generate truly random numbers

Reoccurring values in the sequence will cause an invalid hash to be rested

There are some values that may never be tested

It actually takes less time to test all values sequentially than through random-selection.

Additionally, one has to consider the possibility that such a value doesn't exist. The odds of finding the Kember Identity are actually quite small: 1/((2¹²⁸!)/( 2¹²⁸!)(1-2¹²⁸)!). So how would you know when all possible values have been tested proving the Kember identity doesn't exist if the values are tested randomly? You don't.

The only reliable way to programmatically identify whether the Kember Identity exists and what hashes exhibit it is to test each hashes sequentially and record the results.

The whole thing might not bother me if money wasn't involved. Just send Mr. Kember your $5 entry fee and you're eligible to win the prize pot if your script is first to find the magical hash! But I have a few questions:

How do I contact Mr. Kember to receive my prize when I find a hash that exhibits the Kember Identity?

What happens to my $5 and the rest of the prize money if it is proven the Identity doesn't exist?

At 60-million hashes an hour, it would take over 646,987,670,262,051,588,140,743 millennia to verify them all. How long does Mr. Kember plan on holding on to the prize money?

While it might not be a scam (it says explicitly that it's not a scam somewhere on the irrationally highlighted contest page), it isn't well thought out.

What's Wrong with OOP

2009-06-11T00:02:00.012-04:00

Proponents of Object Oriented Programming feel the paradigm yields code that is better organized, easier to understand and maintain, and reusable. They view procedural programming code as unwieldy spaghetti and embrace OO-centric design patterns as the "right way" to do things. They argue objects are easier to grasp because they model how we view the world. If the popularity of languages like Java and C# is any indication, they may be right. But after almost 20 years of OOP in the mainstream, there's still a large portion of programmers who resist it. If objects truly model the way people think of things in the real world, then why do people have a hard time understanding and working in OOP?

I suspect the problem might be the focus on objects instead of actions. If I may quote from Steve Yegge's Execution in the Kingdom of Nouns:

Verbs in Javaland are responsible for all the work, but as they are held in contempt by all, no Verb is ever permitted to wander about freely. If a Verb is to be seen in public at all, it must be escorted at all times by a Noun.

OOP focuses primarily on the object and expresses actions in terms of the object's abilities. A airplane object can be flown (Airplane.fly()). A door object can be opened (Door.open()). But we really don't view the world in terms of objects and what actions can be done to them. It's backwards. We view the world in terms of ourselves and our abilities. We are the ultimate object. (And no, I don't mean a God object.)

Imagine you are returning from a trip to the local flower garden. Would you say "I smelled the flowers" or "The flowers were smelled by me?" Now you want to tell a friend to go and smell the flowers. Would you say "Go and smell the flowers" or "The flowers must be smelled by you?" When we convey instructions, we give them in terms relative to ourselves. What is programming but conveying instructions to a computer process how some sort of work should be done on our behalf.

How to make a Peanut Butter Sandwich:

Get Jar of Peanut Butter
Get Loaf of Bread
Get Knife
...

All of these things (jar of peanut butter, bread, and knife) can be thought of as objects.

class PeanutButterJar extends Jar ...

Express them as such and they take on methods.

PeanutButterJar.open()
Knife.spread(PeanutButter, Bread)

Whoa! In my world, peanut butter doesn't do anything but sit there and taste yummy. And I'd start looking for a good exorcist the moment a knife starts performing actions all by by itself. A more realistic transcription of the instructions would be:

You.open(PeanutButterJar)
You.spread(Knife, PeanutButter, Bread)

Specifying You as a universal object would seem rather redundant as the scenario progressed, so a good language designer would remove the need to expressly identify it. This would yield

open(PeanutButterJar)
spread(Knife, PeanutButter, Bread)

which starts to look vaguely procedural.

The truth is, procedural and OOP paradigms languages express the complexity of their problem space in different ways. Procedural code is flat and wide with functions. OO code is hierarchical with inheritance. OO-code is not inherently better organized than procedural code merely because it is encapsulated in objects. A reusable library can consist of functions just as easily as a collection of objects.

The way some OOP languages (like Java and C#) force objects on the programmer borders on the absurd. If I'm writing a library of reusable code that needs to maintain its own state, then of course writing classes with proper encapsulation and dating hiding makes sense. On the other hand, If I'm generating a web page with some data stored in a database, then some procedural code and a handful of function calls makes more sense. One of the things I like about PHP so much is that it allows the programmer to decide which paradigm is best suited for the task at hand.

Sadly though, that decision isn't left to the programmer who has been tasked with developing and maintaining a system. Management can tend to focus too much on buzzword compliance. A procedural programming language designed today would never receive wide-spread adoption if it didn't offer some sort of OO construct despite both paradigms having produced successful libraries and applications. And the programmers that don't learn to think beyond themselves will be unfairly left in the dust.

Death Knell for MySQL

2009-04-28T01:14:00.003-04:00

Someone asked me, "What do you think about the Oracle/Sun buyout as it pertains to MySQL?" Well, since you're asking...

I thought it was bad for MySQL when Sun bought them despite what others were saying at the time. It turns out I was right. I think Oracle will be worse, and this time the blogosphere are saying it'll probably be bad. Now the question is, just how bad will it be? Here's my predictions:

I'm sure Oracle realizes they need to tread lightly on the subject of MySQL or else risk the wrath of the open source community. They may integrate some of MySQL to improve Oracle, but they won't promote the continued development of MySQL proper (Berkeley DB anyone?). That is, Oracle won't actively kill MySQL, but they'll let continue to languish the slow and painful death that began before Sun came along. I don't see a financial benefit to Oracle for keeping MySQL healthy. If MySQL does survive, it might be branded as "Oracle Lite."

Core developers will continue work on MySQL in the form of Drizzle, a fork based on MySQL 6.0. Drizzle's focus is on refactoring the MySQL code base and scaling down the feature set-- views, triggers, stored procedures, etc. will be available through modules but not in the core-- to providing a fast and efficient RDBMS for web-based and distributed applications. Drizzle will become very popular as a MySQL alternative for dedicated community members and web developers, and enterprise users who require a larger feature set will migrate to PostgreSQL (and Pythonistas rejoice en masse).

If a commercial company buys control of an open-source project, but then the project's community and core developers fork the codebase and continue development, then the company has effectively only purchased rights to a particular branch. It's legal, but it's not a palatable situation for commercial corporations who might be looking to buy up open source applications. I doubt we'll see Oracle starting a SCO-like court battle over MySQL... but we sure are living in interesting times. Welcome to the era of new law.

I'm primarily a PHP developer so I'll most likely migrate to Drizzle if and when that time comes. A lot of what I do could probably be done with SQLite, but I don't particularly care for the way SQLite does some things. That's another story for another day...

Certification Failure

2009-04-01T17:06:00.013-04:00

Some employers look favorably on certifications, or even require them; other employers could care less. Some people are certified in something but clueless when it comes to actually using the technology. Some people get certifications like they're going out of style just because they can. Some people cheat on the exam. So how much stock should one put in certifications? I'm not sure I know the answer to that. I guess it depends on the certification, what the testing environment is like, who runs the certification program, etc.

Today I ran across PHP-Rocks during my daily web-surfing. It's a small site that offers a set of tutorials ranging from beginner up to advanced, and a PHP "certification" exam. The exam piqued my interest. It was free to take, and I was curious as to what type of questions it asked, so I signed up. Of course I often sign up a dummy account and fake email address when I do such things because I don't intend on becoming a regular visitor to the site, nor do I care to be placed on some spam mailing list. I chose "Joe Biteme" as my name for this excursion.

I answered randomly, not taking the exam seriously (like I said, I was more interested in what type of questions they were asking rather than actually getting their "certification"). I utterly failed it with a miserable 26.6667%! But I figure if they don't feel guilty about offering me the opportunity to pay them $5 to email me the certificate for a failed exam, then I probably shouldn't feel guilty about making a mockery of their exam process (and perhaps even the exam itself) by registering a fake identity and answering randomly.

Click on the image below to enlarge it and you'll see I successfully completed the PHP developer exam with a fail!

In full disclosure, yes I took (and passed) the Zend Certified Engineer exam for PHP5 offered by Zend, and yes I took it much more seriously than I did PHP-Rock's exam. Also, it's not my purpose to single out a particular web site... I just found their snafu too humorous not to share.

Goto and Exceptions

2009-03-19T21:41:00.010-04:00

It slipped quietly under the radar for some. For others, it raised quite a stir. No, I'm not talking about PHP's implementation of namespaces (a battle that's finally done and over much to everyone's relief). I'm referring to the infamous goto statement.

Many programmers have had the "goto is evil" mantra drilled into them from an early start. The basis of this can probably be traced back to Edsger Dijkstra's March 1968 letter to the editor in Communications of the ACM (though he wasn't the first to argue against goto). Dijkstra felt the proliferation of goto at the time was producing unmaintainable "spaghetti" code. Fast-forward more than 40 years later and the controversial feature is still alive and well... and about to make its debut appearance in PHP.

I found myself discussing some of the new features in PHP 5.3 with a friend a few days ago after he read my previous post about anonymous functions and closures. Our conversation eventually turned towards goto and whether it was possible or not to use it responsibly.

goto re-routes the execution flow of a program from one section of its code to another. Even the most basic control structures like if wouldn't exist in high-level languages like C, Java, and even PHP, if it weren't for some type of goto-like operation (high-level languages are either implemented in low-level languages like x86 Assembler which has the equivalent operation JMP, or are implemented in other high-level languages which in turn are implemented in low-level languages). Constructs like conditional statements, loops, and functions all ultimately reduce down to a goto operation.

goto itself isn't much of a necessity in high-level languages because dedicated constructs like conditional statements, loops, etc. are available to the programmer, and these are much easier for humans to work with than goto. Yet goto can still be useful under certain circumstances. For example, some programmers use goto to direct the execution flow to dedicated error-handling logic elsewhere in a program in languages that lack exception handling (such as C).

So is it possible to use goto responsibly in high-level languages, particularly in PHP? I've drawn the conclusions that it is indeed possible:

to emulate exception handling in a procedural environment (otherwise good, but not if the language supports offers real exception handling which PHP does)

to overcome some of the perceived limitations in the try/catch exception handling model often found in an OOP environment (useful and arguably good because the alternatives can lead to fragile code)

Let's start right in by comparing some sample code to decide if using goto to emulate exceptions is responsible use. (Everyone loves code, right?) Here's a function with the error-handling code interspersed throughout it:

// return an array of employees, else -1 on error
function retrieveEmployees()
{
    // assume DB_HOSTNAME et al are defined elsewhere as 
    // constants
    $db = mysql_connect(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD);
    if ($db === false) {
        echo "Unable to connect to database server.";
        return -1;
    }

    $success = mysql_select_db(DB_SCHEMA);
    if (!$success) {
        mysql_close($db);
        echo "Unable to select database.";
        return -1;
    }

    $query = "SELECT id, last_name, first_name FROM employees
        ORDER BY last_name ASC, first_name ASC";
    $result = mysql_query($query, $db);
    if ($result === false) {
        mysql_close($db);
        echo "Unable to execute query.";
        return -1;
    }

    $employees = array();
    while ($row = mysql_fetch_assoc($result)) {
        $employees[] = $row;
    }

    mysql_close($db);
    return $employees;
}

Now, here's the same function refactored to use goto. The actual error-handling code has been moved to the end of the function, where it is out of the way of a programmer trying to read through the function's code and to understand its purpose.

function retrieveEmployees()
{
    $db = mysql_connect(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD);
    if ($db === false) goto CONNECT_ERROR;

    $success = mysql_select_db(DB_SCHEMA);
    if (!$success) goto SCHEMA_ERROR;

    $query = "SELECT id, last_name, first_name FROM employees
        ORDER BY last_name ASC, first_name ASC";
    $result = mysql_query($query, $db);
    if ($result === false) goto QUERY_ERROR;

    $employees = array();
    while ($row = mysql_fetch_assoc($result)) {
        $employees[] = $row;
    }

    mysql_close($db);
    return $employees;

    // possible errors
    CONNECT_ERROR:
        echo "Unable to connect to database server.";
        return -1;

    SCHEMA_ERROR:
        mysql_close($db);
        echo "Unable to select database.";
        return -1;

    QUERY_ERROR:
        mysql_close($db);
        echo "Unable to execute query.";
        return -1;
}

The use of goto in this case makes it arguably easier to follow the logic and understand the purpose of retrieveEmployees() because you no longer have to visually sift through code chafe to find the proverbial wheat.

Incidentally, this is the same goal of exceptions-- move error handling to make the code easier to understand. PHP has supported exceptions since the nascent days of PHP 5, so let's refactor the code again. This time I'll make use of exceptions.

// extend Exception class
class ConnectErrorException extends Exception { }
class SchemaErrorException extends Exception { }
class QueryErrorException extends Exception { }

function retrieveEmployees()
{
    try {
        $db = mysql_connect(DB_HOSTNAME, DB_USERNAME, 
            DB_PASSWORD);
        if ($db === false) throw new ConnectErrorException();

        $success = mysql_select_db(DB_SCHEMA);
        if (!$success) throw new SchemaErrorException();

        $query = "SELECT id, last_name, first_name FROM
            employees ORDER BY last_name ASC, first_name ASC";
        $result = mysql_query($query, $db);
        if ($result === false) throw new QueryErrorException();

        $employees = array();
        while ($row = mysql_fetch_assoc($result)) {
            $employees[] = $row;
        }

        mysql_close($db);
        return $employees;
    }
    catch (ConnectErrorException $e) {
        echo "Unable to connect to database server.";
        return -1;
    }
    catch (SchemaErrorException $e) {
        mysql_close($db);
        echo "Unable to select database.";
        return -1;
    }
    catch (QueryErrorException $e) {
        mysql_close($db);
        echo "Unable to execute query.";
        return -1;
    }
}

Comparing the second and third iterations of retrieveEmployees(), one could argue that using goto results in a cleaner syntax than actual exception handling. With goto, the programmer doesn't need to extend the base Exception class, position the code in question within a try/catch blocks, and make sure all of the braces were matched correctly. Instead, the programmer only has to provide a target label and the actual goto call.

Without the overhead of instantiating an Exceptionobject, there's also a slight performance boost. The results of a highly (un-)scientific benchmark I ran to compare goto error-handling against exceptions using the above examples showed goto is a little over 4% faster.

Unfortunately, those benefits are meager when you take a closer look and see the drawbacks of using goto. Exceptional events allow the programmer to signal than an error occurred, but goto is a direct jump to the error handling logic. Each label must be unique. The programmer may soon find himself duplicating code and devising creative label names to handle the same type of errors in slightly different ways. Another drawback is that PHP requires the label to be within the same scope as the goto call. This means the following code will generate a fatal error:

function generateError()
{
    goto GENERATED_ERROR;
}

GENERATED_ERROR:
    die("Oops!");

Exceptions can offer more flexibility since the thrown exception bubbles up the call stack until it is caught by a suitable catch block:

class GeneratedErrorException extends Exception { }

function generateError()
{
    throw new GeneratedErrorException();
}

try {
    generateError();
}
catch (GeneratedErrorException $e) {
    die("Oops!");
}

Exceptions were designed explicitly for the purpose of error-handling. There's only one way to get into the error-handling code of a catch block-- by a throw call. The code that follows a goto label on the other hand can be executed as part of the normal flow of execution, or as a backwards redirect. Exceptions have their own dissidents, but even so they don't have nearly the bad rap that goto does. Using goto will probably draw the wrath of many anti-goto programmers upon you faster than a speeding photon, while using exceptions will win you job interviews.

The conclusion for my first talking point is that using goto to emulate exceptions is useful, but it's better to use exceptions instead of faking things if the language provides them (which PHP does).

Exceptions are well suited to the event-driven execution model where the execution flow is determined by events (such as the user clicking on a graphical interface component or pressing a key on the computer). The runtime loop waits for something to happen, possibly for an infinite amount of time, and then executes the logic assigned by the programmer to an appropriate action when it detects an event. If there is a problem that will prevent the action from being completed successfully then an exception can handle it, terminate that thread, and return the flow to the main wait-state. Users can fix the problem and try again without the program terminating.

Unfortunately, PHP doesn't enjoy this execution model, and it's been my experience that exceptions are just as easy to abuse when used in batch/imperative models as goto is. All too often a programmer will catch the exception and then simply terminate the program.

class FileAccessException extends Exception { }

$filename = "test.txt";

try {
    $fp = @fopen($filename, "r");
    if ($fp === false) {
        throw new FileAccessException("cannot open $filename");
    }
}
catch (FileAccessException $e) {
    die("Error: " . $e->getMessage());
}

Is this really effective error handling? It might be if the script is processing a batch job or generating a web page. There is nothing the requestor can do in those cases to address whatever it is that caused the error, and the only reasonable course of action is to gracefully terminate the script. But what if the script were running interactively from a command prompt? PHP is primarily used to generate web content, but as more and more people realize its flexibility and the benefits of reducing the number of programming language across all parts of an application, CLI scripting with PHP is becoming more popular. In that case, a more proper action for the script to take would be to inform its users what the error was and suggest some possible steps they can take to resolve it.

// return whether the script is allowed read access ("r"),
// or write access ("w") a file
function testPermission($filename, $access = "r")
{
    clearstatcache();
    list(,,$mode,,$uid,$gid) = stat($filename);
    $perms = array("u_r" => (bool)($mode & 0400),
                   "u_w" => (bool)($mode & 0200),
                   "g_r" => (bool)($mode & 0040),
                   "g_w" => (bool)($mode & 0020),
                   "o_r" => (bool)($mode & 0004),
                   "o_w" => (bool)($mode & 0002));
    list($user) = posix_getpwuid($uid);
    $group = posix_getgrgid($gid);

    list($eUser) = posix_getpwuid(posix_geteuid());
    $isUser = ($user == $eUser);
    $isGroup = in_array($eUser, $group["members"]);
    $isOther = !($isUser || $isGroup);

    return (($isUser  && $perms["u_" . $access]) ||
            ($isGroup && $perms["g_" . $access]) ||
            ($isOther && $perms["o_" . $access]));
}

try {
    $fp = @fopen($filename, "r");
    if ($fp === false) {
        throw new FileAccessException();
    }
}
catch (FileAccessException $e) {
    if (!file_exists($filename)) {
        die("$filename does not exist. Please create the " .
            "file.\n");
    }
    else {
        if (!testPermission($filename, "r")) {
            die("Please check read permissions on " .
                "$filename.\n");
        }
        else {
            die("Unknown error attempting to access " .
                "$filename.\n");
        }
    }
}

The code is a bit smarter now about the error and offers users some guidance as to what needs to be done to resolve it, but the program still terminates. Not all exceptions should be fatal. In this case, it would be better for the code to give the users an opportunity to fix the error and then try to re-read the file instead of forcing them to start the program over again.

Whereas in event-driven execution users can simply retry the action, there is no clean way to retry the code that triggered the error procedurally. One possibility is to surround the action in a do/while loop.

// prompt the user whether to retry an action
function promptRetry()
{
    do {
        echo "Type 'R' to retry or 'Q' to quit: ";
        $retry = strtoupper(trim(fread(STDIN, 2)));
        if ($retry == "R") {
            return true;
        }
        else if ($retry == "Q") {
            return false;
        }
        else {
            echo "Invalid entry. ";
        }
    }
    while (true);
}

do {
    $retry = false;
    try {
        $fp = @fopen($filename, "r");
        if ($fp === false) {
            throw new FileAccessException();
        }
    }
    catch (FileAccessException $e) {
        if (!file_exists($filename)) {
            echo "$filename does not exist. Please create " .
                "the file.\n";
        }
        else {
            if (!testPermission($filename, "r")) {
                echo "Please check read permissions on " .
                    "$filename.\n";
            }
            else {
                echo "Unknown error attempting to access " .
                    "$filename.\n";
            }
        }
        if (!promptRetry()) exit();
        $retry = true;
    }
}
while ($retry);

I don't like how the $retry variable is set within the catch block to trigger the reiteration of the surrounding do/while loop. It seems a bit fragile to set variables within catch blocks in order to influence the behavior of code outside the block. Instead, the example can be refactored to use goto. This eliminates having to keep track of $retry altogether and just redirects the execution flow itself.

TRY_OPEN_FILE:
try {
    $fp = @fopen($filename, "r");
    if ($fp === false) {
        throw new FileAccessException();
    }
}
catch (FileAccessException $e) {
    if (!file_exists($filename)) {
        echo "$filename does not exist. Please create " .
            "the file.\n";
    }
    else if (!testPermission($filename, "r")){
        echo "Please check write permissions on " .
             "$filename.\n";
    }
    else {
        echo "Unknown error attempting to access " .
            "$filename.\n";
    }
    if (!promptRetry()) exit();
    goto TRY_OPEN_FILE;
}

The conclusion on my second talking point was that goto can be used to overcome some of the perceived limitations of working with exceptions in a batch/procedural execution model. Without the support of an event-driven execution model, and without a dedicated retry-type statement, programmers need to resort to looping constructs. The code that results can be brittle and difficult to maintain over time. goto offers an eloquent and succinct alternative.

Anonymous Functions and Closures

2009-03-07T00:40:00.008-05:00

Of all the new goodies that are promised in PHP 5.3, the two which I think I am the most excited about are anonymous functions and closures.

Anonymous functions are functions that are defined without being bound to a proper name. Typically, anonymous functions are used only a limited number of times and for a specific purpose; you could think of them as "throw-away" functions if you'd like.

Let's consider the following example which illustrates a standard function used as a callback:

function percentVowels_callback($word) {
    $word = strtolower($word);
    $chars = count_chars($word);
    $numVowels = 0;
    foreach (array("a", "e", "i", "o", "u") as $vowel) {
        $numVowels += $chars[ord($vowel)];
    }
    return $numVowels / strlen($word);
}

$animals = array("aardvark", "elephant", "iguana", "orangutan",
    "urchin");
$percentVowels = array_map("percentVowels_callback", $animals);

The array_map() function accepts the name of a function and an array, and produces a new array resulting from applying the callback function to each element of the input array. The callback function I've defined accepts a string and returns the percentage of which is made up of vowels. For the sake of simplicity, I am only considering the letters A, E, I, O and U to be vowels, even though sometimes Y and rarely even W can be treated as vowels as well.

Since the function is only used once by this array_map() statement, it may make sense to refactor percentVowels_callback() as an anonymous function. Typically the purpose of a function is to eliminate repetitive code and build reusable components... but the purpose of anonymous functions is different. Anonymous functions group together a related set of statements.

Earlier versions of PHP (starting at version 4.0.1) provided limited support for defining anonymous functions with create_function(). Here's an example which shows the call to array_map() with percentVowels_callback() refactored as an anonymous function using this create_function().

$percentVowels_callback = create_function('$word', '
    $word = strtolower($word);
    $chars = count_chars($word);
    $numVowels = 0;
    foreach (array("a", "e", "i", "o", "u") as $vowel) {
        $numVowels += $chars[ord($vowel)];
    }
    return $numVowels / strlen($word);');

$percentVowels = array_map($percentVowels_callback, $animals);

Depending on your preference and coding style, the callback can be defined inline, as well. Depending on how the application is organized, you may need to jump to a different section of the code file or to a different file all together to inspect the contents of the function when you are tracing through code, and then find your way back afterwards to the calling location. The function's body can be available visually where it is most pertinent.

$percentVowels = array_map(create_function('$word', '
    $word = strtolower($word);
    $chars = count_chars($word);
    $numVowels = 0;
    foreach (array("a", "e", "i", "o", "u") as $vowel) {
        $numVowels += $chars[ord($vowel)];
    }
    return $numVowels / strlen($word);'), $animals);

The create_function() function accepts two strings--the first listing the variable names to serve as the anonymous function's arguments, the second containing the code for the function's body--and returns a unique string which can be used to identify the function.

create_function() does have a few drawbacks, though. Because the argument list and function body are provided as strings, you must be careful to make sure certain characters within the string are escaped properly. You need to escape any single-quotation marks that appear if your strings are single-quoted, or you need to escape double-quotation marks and dollar signs if your strings are double-quoted. Moreover, you also lose the benefits of any syntax highlighting your IDE may provide since it highlights the strings as... well, strings! Overall, the approach is cumbersome and clunky.

As of version 5.3, PHP will offer better support for anonymous functions and a new syntax which supports closures. The new syntax for anonymous functions is more similar to the manner in which JavaScript and other event-driven languages define them.

$percentVowels_callback = function($word) {
    $word = strtolower($word);
    $chars = count_chars($word);
    $numVowels = 0;
    foreach (array("a", "e", "i", "o", "u") as $vowel) {
        $numVowels += $chars[ord($vowel)];
    }
    return $numVowels / strlen($word);
};

$percentVowels = array_map($percentVowels_callback, $animals);

Gone is the clumsiness of create_function() and its string arguments. The anonymous function's arguments and body are provided as PHP code which can be highlighted correctly by your IDE. The one caveat to the new syntax is that there MUST be a trailing semi-colon after the function's closing brace, as $percentVowels_callback = some value is a regular assignment statement.

One is also able to define the anonymous function using the new syntax inline as well.

$percentVowels = array_map(function ($word) {
    $word = strtolower($word);
    $chars = count_chars($word);
    $numVowels = 0;
    foreach (array("a", "e", "i", "o", "u") as $vowel) {
        $numVowels += $chars[ord($vowel)];
    }
    return $numVowels / strlen($word);}, $animals);

So far I've shown you a rather uninspired example using an anonymous function, so let's expand it a bit and make things more interesting. Suppose the application should reduce the list of percentages to those that are equal to or greater than a user-specified value. For this we can use array_filter().

The array_filter() function accepts an input array and the name of a function, and produces a new array resulting from applying the callback function to each element of the input array. If the callback returns true then the element tested is included in the final output array. We have a slight problem, though-- the documentation for array_filter() shows it only passes one value to the callback (the current array element to be examined). We need access to user-specified value as well. Closures will allow us to "reach out" outside the scope of the percentFilter_callback() function to see the value of $userPercent.

$userPercent = 0.5;
$userList = array_filter($percentVowels,
    function($percent) use ($userPercent) {
        return ($percent >= $userPercent); 
    });

Closures allow you controlled access to values to the parent scope of a function. The new syntax introduces the use keyword to specify which variables should be imported. The anonymous function is passed the current element of the $percentVowels array by array_filter(), but it has access to the value of $userPercent as well which is needed for the comparison.

Closures in PHP import the variable from the parent scope the same as if it were passed as an argument to the function and manipulating the variable does not have an effect outside the function (unless it was passed by reference).

However, it is important to keep in mind that closure is not the same thing as using global. global reference variables within the global scope of the script. Closures on the other hand can only bind to variables in the parent scope of a function, which make using them considerably safer than using global. I'll leave you with this final example which highlights this difference between them:

$x = 42;

function foo() {
    function fizz() {
        global $x;
        echo $x;
    }
    bar();
}

function bar() {
    $bizz = function () use ($x) { echo $x; };
    $bizz();
}

For more information on anonymous functions and closures, check out these pages:

Evil Access

2009-01-20T23:22:00.019-05:00

I was thinking today about database APIs when inspiration struck. I ended up hacking out the following class, which I think demonstrates a rather interesting approach to interfacing with a database (interesting enough at least to post here).

class DBQuery implements Iterator
{
    protected $_db;
    protected $_query;
    protected $_result;
    protected $_index;
    protected $_num_rows;
    
    public function __construct($host, $dbname, $username,
        $password) {
        $this->_db = new PDO("mysql:dbname=$dbname;host=$host",
        $username, $password);
    }
    
    public function __get($query) {
        $this->_query = $query;
        $this->_result = $this->_db->query($query);
        return $this->_num_rows = $this->_result->rowCount();
    }
    
    public function quote($value) {
        return PDO::quote($value);
    }
    
    public function __call($query, $values) {
        $this->_query = $query;
        $this->_result = $this->_db->prepare($this->_query);
        $this->_result->execute($values[0]);
        return $this->_num_rows = $this->_result->rowCount();
    }
    
    public function clear() {
        $this->_index = 0;
        $this->_num_rows = 0;
        $this->_query = '';
        $this->_result->closeCursor();
    }
    
    public function rewind() {
        $this->_index = 0;
    }
    
    public function current() {
        return $this->_result->fetch(PDO::FETCH_ASSOC,
            PDO::FETCH_ORI_ABS, $this->_index);
    }
    
    public function key() {
        return $this->_index;
    }
    
    public function next() {
        $this->_index++;
    }
    
    public function valid() {
        return ($this->_index < $this->_num_rows);
    }
    
    public function __toString() {
        return $this->_query;
    }
}

DBQuery isn't your typical database access class. In fact, I would suggest it's slightly evil since it distorts traditional PHP syntax by abusing taking advantage of three specific PHP features.

PHP allows special characters in an identifier if the string is quoted and is enclosed by {}. So, $myValue and ${"my value"} are both equally valid variable identifiers.

The magic overloading methods allow you handle undefined properties and methods in your class. Specifically, I've made use of __get() and __call().

A class that implements the Iterator interface can be traversed using a foreach loop.

Here's a look at how DBQuery would be used:

// connect to the database
$dbq = new DBQuery("localhost", "test", "dbuser",
    "dbpassword");

// query the database if the user is authorized
$username = "administrator";
$password = sha1("password");
if (!$dbq->{"SELECT * FROM admin_user WHERE username=? " .
    "AND password=?"}(array($username, $password))) {
    die("Unauthorized.");
}

// query the database and display some records
$dbq->{"SELECT id, first_name, last_name FROM employee"};
foreach ($dbq as $result) {
    print_r($result);
}

// casting the object as a string yields the query string
echo "Query: $dbq";

Don't try this at home, though, my friends. Just because you can write code like this doesn't mean you should.

Motivational Posters

2009-01-14T23:41:00.008-05:00

I've been having a hell of a time fighting with some Perl code at work lately. I tossed together a humorous Perl demotivational poster to release some of my frustration, and thought others might appreciate it.

But why stop there? There are other languages worth making fun of besides Perl! So, I tossed together some more snarky posters to be fair.

7 Things You Didn't Care to Know About Me

2009-01-04T23:26:00.019-05:00

I've been tagged by Chris Cornutt and Jeff Jones to share some things you probably don't know about me. (It's nice to see chain mails flow through the blogosphere too, eh?) So, here's seven little-known facts about yours truly...

My first career choice was to be a diocesan priest, and I briefly attended Seminary after I graduated from high school. My family isn't overly religious, and my parents didn't push me to go; I think I just like to help people and felt drawn to the rich history of the Roman Catholic Church. Life has a funny way of playing out differently than we intend for it, doesn't it?
After I left seminary, I attended a state university and majored in elementary education with a concentration in music history and literature. I'm not overly fond of children, though, so I didn't complete the program. I would only need to take two more classes to be a state-certified elementary teacher.
I studied French for 5 years in high school, a year of Latin in the seminary, and then learned Esperanto independently.
Someday I'd like to run for Governor of New York, though I'm certain nobody would vote for me. My biggest push would be to reform of the state's tax laws, though I'd probably legalize prostitution... people are going to do it anyway so the state might as well make money off it.
I try to make sure I have a bunch of $1 bills in my pocket when I go to parties; you never know what you can pay a drunk person to do for your own personal amusement. I once had a guy prancing about in a blue speedo for $5. Another time, the same guy chewed on a ball of tinfoil for 30 seconds for $1. I think that officially makes me an asshole.
I crawled on top of the dining room table when I was a baby, fell off, and tore the frenulum under my tongue. I didn't gain any super-human abilities with my tongue... just reduced frenula tissue.
I don't use shaving cream. I would inevitably get it in an ear or up a nostril and it would bother me. Instead, I shave in the shower after the warm water has softened my skin and use a disposable razor that has an aloe strip.

So there you have it. I'm supposed to tag other people and have them share seven things about themselves as well, though everyone to whom I would have sent it has already participated. Sorry... That's all, folks!

If you're curious about the deep, dark secrets of others in the PHP Community, you may want to checkout the list Cornutt compiled.

Happy New Year

2009-01-01T04:39:00.007-05:00

I've never been one to make New Year's resolutions. If I really want to change something about myself or my life, why what until January 1st to set goals for myself? It doesn't matter to me if it's January 1st, April 15th, July 4th, or December 26th... I just set goals and go for them!

Instead, I prefer to choose an over-reaching theme for the year. This suits me better because there's no success or failure benchmarks to meet; it's something I keep in mind to help guide me as I tackle the challenges and opportunities that come my way.

2006 was my "Year of Adventure."

2007 was my "Year of Change."

2008 was my "Year of Success."

What will 2009 be?

I've chosen to designate 2009 as a "Year of Balance" for myself. Like everyone, there's good things and bad things in my life; putting the balance back into my life again will empower me to better appreciate the good and deal with the bad more productively.

So, it was very fitting the other day when I saw this beautiful video featured on the Astronomy Picture of the Day website. The site's goal is to present images, videos and descriptions to inspire people to learn more about astronomy, though I found this particular video inspiring in a different way.

In the short 4-minutes or so that it took to watch the video, I found myself pondering the beauty of nature that surrounds us, and how sad it is that so many of us never appreciate it because we're too busy going about our daily lives. I then found myself thinking about what things in life are truly important, and what I should focus on as I live my life.

Each day I will work and be productive, I will play and relax, I will laugh, love, and enjoy the company of friends, and I will find new things in my life deserving of my appreciation.

Happy New Year; I wish you and all your loved ones the best in 2009.

Paste Ninja Goes Multilingual

2008-12-19T15:15:00.013-05:00

Chris Cornutt was nice enough to give Paste Ninja (the premier PHP-powered pastebin app) some exposure by highlighting it on PHPDeveloper.org a week or so ago. Thanks, Chris! As a result of the extra publicity, the site saw an increase in traffic as people came to check it out. I hope they liked what they saw... but stay tuned, because there's more features to come!

It was interesting to look through the access logs afterward. There are a lot of pastebins available on the Internet, but the feature set and usability of the application is what will help differentiate it from the rest. The list of countries from which various people visited and the values of their browser's Language-Accept headers peeked my interest, and I realized that I needed to offer Paste Ninja's interface in multiple languages so people could use my service in whichever language they may find the most comfortable. So, Paste Ninja is now multilingual!

The goal is to translate PasteNinja's interface into the top 10 or so languages. I'm obviously not not going to be able to accurately translate all of the necessary text into each language myself, so I've started enlisting some help.

Years ago I studied French and Esperanto, so those were the first two languages I targeted. I then searched out people who were willing to review my work and correct any grammar deficiencies and the like. With the help of Babelfish, Google Translate, and some very creative Internet searching, I was able to produce rough translations in other languages.

If you ever need a professional French translator, Sophie Vialaneix is unbelievably good in every way and comes with my highest recommendation. She was so friendly and professional that it made me wish I had more text for her to review (thanks, Sophie)! Remi Woler from #phpc was wonderful and offered to correct my Dutch (thanks, Remi). My friend and co-worker Bobby Mladenov graciously provided a Bulgarian translation (thanks, Bobby), and Dr. Wing Ming Chan is also graciously providing a Chinese translation (thanks, Wing). Keep their efforts in mind as well when you see text that doesn't read "Grandmother your password here."

The interface is available in other languages as well, such as German, Italian, Polish, Portuguese, Russian, and Spanish. These were produced with translation software and some creative Internet searching, so expect them to be less-than-perfect. I will revise the translations as I find people proficient in them who are willing to review them. If you'd like to volunteer your skills, just leave a comment and let me know!

Oh, and the interface is also available in Pig Latin just for fun. I'm sure Ben Ramsey will be happy to take all the credit for that idea. ;)

Update 12/21/08: David Zülke was kind enough to review the German translation. Thanks, David!

Update 12/23/08: Sacha Poznyak was kind enough to review the Russian translation, Ali Curtis was kind enough to review the Spanish translation, and David Cole was kind enough to review the Portuguese. Thanks, Sacha, Ali and David!

Update 12/31/08: Shahar Fisher was gracious enough to donate his time to review the Hebrew translation. Thank you, Shahar!

PHP != Perl

2008-12-12T15:19:00.006-05:00

People irk me when they expect PHP to behave like <insert your favorite language here>. Most recently, I ran across this blog entry Perl vs PHP: Variable Scoping via the help of PHPDeveloper.org. I'm sure Leonid is a great fellow and really smart... but he just happened to trip my irk-wire.

The post is an innocent enough "here's a problem that I ran into, so I'll blog it in case someone else has the same issue." Reading between the lines though, I get the feeling his underlying belief is that because PHP and Perl are similar that they should have the same variable scoping rules, and that since PHP's doesn't correspond to Perl's that PHP is wrong and this should be documented as such.

Each language is different. If PHP were so similar to Perl, it wouldn't be called PHP... it'd be called Perl. Or maybe a dialect, like PHPerl. I really wish people would stop comparing the two. Quite frankly, I don't see much similarity between the two languages at all.

Maybe I can get away with saying PHP and Lisp are similar because I can do the same thing in both languages!

$a = 2 + 2 * 4;
echo $a;

(setq a (* (+ 2 2) 4))
a

Update 12/20/2008: Luke Wellington wrote a brief article entitled PHP Is Not Java for the PHP Advent 2008 project. I'm glad to see I'm not the only one who gets irked by such things (though he is much more eloquent about it than I am).

JavaScript Keyboard Widget

2008-12-07T23:25:00.006-05:00

Have you ever done something stupid just so you don't have to do whatever it is you're supposed to be doing because you feel like procrastinating? I found myself doing that this evening by programming a keyboard widget in JavaScript.

Right now it's just sloppy code without any real functionality, but feel free to take a look at it at http://www.saltcitytech.com/projects/keyboard.

Theoretically, one could add a couple of state variables to monitor the ctrl, shift, alt and caps lock keys, check those variables in the keys' onclick callback, and have a fully functional keyboard widget. Since the key characters are defined in a mapping object, it should be relatively easy to internationalize it just by changing the mapping definitions.

I may clean up the code in the future depending if someone actually sees a viable use for it or if I find myself needing to procrastinate again.

Dialog and Paste Ninja

2008-12-04T21:31:00.011-05:00

I'd like to share with everyone two projects that I've been working on in what little spare time I've had lately-- Dialog and Paste Ninja.

Dialog
Dialog is a lightweight JavaScript-based dialog window. Why does the world need another dialog widget, you ask? It probably doesn't, but I couldn't find a freely available one that was as flexible as I wanted so I wrote my own. Flexibility and portability were key design features, so Dialog is written completely in JavaScript and CSS, and its appearance is fully CSS driven.

Dialog supports multiple dialog types by default, such as confirmation, information, warning, and error, and can further be customized with a custom dialog type. It can even display modal dialogs and block input to the page until the prompt is dismissed!

You can learn more about it and get the code ~~at its project page, http://www.saltcitytech.com/projects/dialog~~ from my JavaScript Experiments repository at GitHub.

Paste Ninja
Paste Ninja is a pastebin application that lets you to share code snippets with others. Instead of flooding an IRC channel or your Instant Messenger conversations with lines of source code, you can paste it online for public viewing.

The world probably doesn't need another pastebin app, either, but there are several features in store that other pastebins don't have. The goal is to revolutionize how people use pastebins and make internet debugging a truly collaborative experience.

The back-end of Paste Ninja is written in PHP and uses a MySQL database to store pastes, comments, and the application's configuration information. The front-end is written in JavaScript.

I haven't decided yet if I'm going to opensource the code for Paste Ninja-- or under which license it would be released under if I did so-- but I'm not adverse to the idea if there is enough demand. Regardless, it's free to use, so be the "master of your pastebin" and give Paste Ninja a try at http://www.pasteninja.com.

Choosing an Assignment Operator

2008-11-28T21:21:00.005-05:00

If you were to write your own programming language (as I still intend to do someday even if only for the learning experience it provides me), what symbol (or symbols) would you use to represent the assignment operator?

Even though it would looks like BNF or old-school Pascal, I would lean towards using := as the assignment operator. That would leave me free to use = for equality and there wouldn't be the issue of = vs. == tripping up new programmers. Besides, performing assignments is more common than making comparisons, so perhaps assignment should have the more distinct operator. Going one step further, if the language I wrote was not strongly typed, I would use == as the identity operator (as like PHP's ===). := and = to == seems a more logical progression to me than = and == to ===.

I suspect I wouldn't use = as both assignment and comparison as Basic does because of the ambiguity it causes. For example:

x = y = 0

Does this mean "assign 0 to both x and y", or "assign the boolean comparison whether the value of y is 0 to x?" Statements such as these:

x := y = 0
x := y := 0

are then both clear in their meaning.

I think my second choice would be just : and have something like:

x: 2 + 2

The lvalue appears as if it were a label, giving the visual representation that x means 2 + 2. Plus, it would be one (of many) syntaxtic differences that would separate my language from the others. I don't know of any languages that currently use : as an assignment operator.

Of course, this all presumes the elements of a statement are written in a certain order. If you were to use a keyword such as set:

x set 2 + 2

just appears awkward to me. It would have to be:

set x 2 + 2

But if you always require the assignment target on the left-hand side of your operator, then is an explicit assignment operator really required as all? The implied assignment operation of something like:

x 2 + 2

is clean and succinct.

JavaScript Frameworks Suck

2008-11-10T12:51:00.012-05:00

There's been an interesting discussion off and on around the office this past couple of weeks about JavaScript frameworks, specifically which framework is the best so we can standardize on one. Of course I have to be difficult... my answer is none of them.

As a general rule of thumb, "frameworks are evil." There are exceptions, but frameworks seem to cause a lot of unnecessary bloat, make tasks difficult to accomplish if they fall outside the intended scope of the framework, create obstacles to efficient debugging, and adversely affect page load times causing the application to appear slow and sluggish. The real question ought not be what framework is best, but rather be what exactly are you trying to accomplish with client-side scripting in the first place.

Creating a rich user experience with standard JavaScript is not difficult. Many of the niceties the frameworks provide aren't magic... for example, $() is just function(x) document.getElementById(x);}. And AJAX is easy if you forgo XML in favor of JSON as your transfer format. If you don't understand your goals then you might as well just shoot yourself in the foot.

Once you have identified exactly what your needs are, and if those needs suggest you use a framework, then those needs will also dictate which framework would be suitable for use. If you need widgets, for example, then YUI! would stand out more as the best choice. If you need modularity and flexibility instead, then MooTools might be the way to go.

To further illustrate my point to a coworker that frameworks don't always make things easier, I implemented a basic Accordion widget in MooTools and straight-up plain old JavaScript. The development time in JavaScript proper was half-that of developing with MooTools because I didn't have to learn any special APIs, scrounge the documentation for a list of dependency files, etc. My implementation weighs in at 50 lines vs. MooTools' 3,100+ lines and 21 dependency files.

function $$(className) {
    var classElements = new Array();
    var els = document.getElementsByTagName("*");
    var pattern = new RegExp('(^|\\s)' + className + '(\\s|$)');

    for (var i = 0, j = 0; i < els.length; i++) {
        if (pattern.test(els[i].className)) {
            classElements[j] = els[i];
            j++;
        }
    }

    return classElements;
}

function Accordion(headerClass, panelClass, showIndex) {
    this.headers = $$(headerClass);
    this.panels  = $$(panelClass);

    for (var i = 0; i < this.headers.length; i++) {
        this.headers[i].args = {
            index: i,
            headers: this.headers,
            panels : this.panels
        };
        this.headers[i].onclick = function () {
            var a = this.args;
            for (var i = 0; i < a.panels.length; i++) {
                a.panels[i].style.display = ( i == a.index) ?
                    "" : "none";
            }
        };
    }

    (showIndex === undefined ? this.headers[0] : 
        this.headers[showIndex]).onclick();

    return true;
}

window.onload = function() {
    new Accordion("a_header", "a_body");
};

Sure it's not as "feature rich" as MooTools' Accordion, but any additional features can easily be added when the time comes, and they certainly wouldn't require 3,050 more lines of code.

People want fast front-ends. They're impatient and don't want to wait. Bloated code slows down the front-end and gives them impression of a slow back-end. Half the development time and 98.4% less code? Now that sounds good to me!