the psychology of computer programming

is an excellent book by Jerry Weinberg. As usual I'm going to quote from a few pages:
I've read this book twice before, once here, and again here.
We must deal with one other problem, which is important because of a seldom questioned view of programming - a view which this book will spend a great deal of time questioning. That view is that programming is an individual activity.
Now that hardware has grown cheaper whilst labor has grown more expensive, group members are much less likely to share a machine and system than there were years ago.
The requirement to develop capability cannot be met adequately by a single person. We learn much faster and much better with the active cooperation of others.
Has anyone ever thought of asking appplicants whether or not they like programmming?
With adults, however, the barriers to learning have usually become internalized, and the average adult learns very little of left to his own devices.
It is a well-known psychological principle that in order to maximize the rate of learning, the subject must be fed back information on how well or poorly he is doing.
Such companies are sitting ducks for anyone who comes along with a fancy package of promises - and with lots of sitting ducks, can the hunters be far behind?
An increase in salary only motivates for a short time it is the raise, not the salary level which is a symbol of current value.
To a surprising degree, the only time we fail to learn is when there are negative forces set up against it.
Because the machines are rigid, the people who use them must, if they are to be successful, supply more than their share of flexibility.
We are trying to make the machine help people take advantage of the immense psychological resources they have in overcoming their immense psychological shortcomings.


the universal computer

is an excellent book by Martin Davis (isbn 978-1-4665-0519-3). As usual I'm going to quote from a few pages:
Liebnitz's involvement with the Harz Mountain mining project ultimately proved to be a fiasco. In his optimism, he had not forseen the natural hostility of the expert mining engineers towards a novice proposing to teach them their trade. Nor had he allowed for the inevitable break-in period a novel piece of machinery requires or for the unreliability of the winds.
Unlike the usual experience with a new untried gadget, Turing's Bombes, built from his design, worked correctly as soon as they were made.
"There are several theorems which say almost exactly that ... if a machine is expected to be infallible, it cannot also be intelligent... But these theorems say nothing about how much intelligence may be displayed if a machine makes no pretence at infallibility." [Turing]
There is nothing in Godel's theorem to preclude the mathematical powers of a human mind being equivalent to an algorithmic process that produces false as well as true statements.
It is interesting to contrast von Neumann's view of computer programming as an activity with Turing's; von Neumann called it "coding" and made it clear that he thought of it as a clerical task requiring little intellect. A revealing anecdote tells of a practice at the Institute for Advanced Study computer facility of using students to translate by hand, computer instructions written using human-readable mnemonics into machine language. A young hot-shot programmer proposed to write an assembler that would do this conversion automatically. Von Neumann is said to have responded angrily that it would be wasteful to use a valuable scientific tool to do a mere clerical job. In his ACE report, Turing said that the process of computer programming "should be very fascinating. There need be no real danger of it ever becoming a drudge, for any processes that are quite mechanical may be turned over to the machine itself."
There is no reason to think that a full scale ACE-style computer would not have worked well if the organization and resources to build one had been there. The issue is best understood in the more general context of the question of which computer functions should be supplied by the hardware and which by software. Turing had proposed a relatively simple machine in which a lot was left to be supplied by software, but where, in compensation, the programmer had very substantial control of underlying machine operations.
"I expect that digital computing machines will eventually stimulate a considerable interest in symbolic logic... The language in which on communicates with these machines ... forms a sort of symbolic logic." [Turing]
Searle tells us that Deep Blue "has a bunch of meaningless symbols." Well, if you could look inside Deep Blue when it was in operation, you wouldn't see any symbols, meaningful or not. At the level of circuits, electrons are moving around. Just as, if you look inside Kasparov's skull while he is playing, you wouldn't see any chess pieces, you'd see neurons firing.
Our consciousness is a principal way in which each of us experiences his or her unique individuality. But we know it only from the inside. We experience our own consciousness but not that of anyone else.


print "squashed-circle" diamond

There's been a bit of a buzz about the Print-Diamond practice recently. I recall doing this a couple of years ago with Johannes Brodwall. In cyber-dojo naturally. We took a wrong turn and were making a thorough mess of it. I vividly recall Johannes saying:
This is too difficult. We're doing it wrong.
I love that. If it's difficult you're probably doing it wrong. We gave up and took a break. We got a really nice Indian take away. Then we went back to Print-Diamond. Very quickly we came up with a new idea. We imagined the diamond lying in the center of an x,y axis. The Print-Diamond of 'C' would therefore look like this:

       -2 -1  0 +1 +2

  -2    -  -  A  -  -    
  -1    -  B  -  B  -
   0    C  -  -  -  C
  +1    -  B  -  B  -
  +2    -  -  A  -  -


Viewed like this you can think of the Diamond as a sort of squashed circle with the A,B,C characters all lying on the circumference. From here it was a short step to this (Ruby):
(-2..+2).map{|row| 
  (-2..+2).map{|col| 
    row.abs + col.abs == 2 ? 'X' : '-'
  }.join
}

which, when puts'd gives:
--X--
-X-X-
X---X
-X-X-
--X--

And we knew we were on our way.
Let's hear it for Curry Driven Development!

yet another interesting TDD episode

In the previous episode I described how a custom assert function declared a local array which I did not initialize.
static void assert_fizz_buzz(const char * expected, int n)
{
    char actual[16];
    ...
}

The effect of not initializing the array was that state from one test leaked into another test and I got an unexpectedly passing test. I fixed the problem by initializing the array.
static void assert_fizz_buzz(const char * expected, int n)
{
    char actual[16] = "";
    ...
}

The most recent episode (in cyber-dojo naturally) revolved around this same issue. This time I was redoing the roman numerals exercise (1 → "I", 2 → "II", etc) in C. My custom assert function started like this.
...
#define PRINT(s) print_string(#s, s)

static void print_string(const char * name, const char * s)
{
    printf("%10s: \"%s\"\n", name, s);
}

static void assert_to_roman(const char * expected, int n)
{
    char actual[32];
    to_roman(actual, sizeof actual, n);
    if (strcmp(expected, actual) != 0)
    {
        printf("to_roman(%d) FAILED\n", n);
        PRINT(expected);
        PRINT(actual);
        assert(false);
    }
}

Once again I failed to initialize the array. I'm a slow learner. This time the test failed unexpectedly! And the diagnostic was even more unexpected:
to_roman(1) FAILED
  expected: "I"
    actual: "I"

This is a less than ideal diagnostic! It seemed that strcmp and printf had differing opinions on what a string is! I fixed it by adding initialization.
...
static void assert_to_roman(const char * expected, int n)
{
    char actual[32] = "";
    to_roman(actual, sizeof actual, n);
    if (strcmp(expected, actual) != 0)
    {
        printf("to_roman(%d) FAILED\n", n);
        PRINT(expected);
        PRINT(actual);
        assert(false);
    }
}

After this the tests passed. This addressed the immediate problem but it did not address to the root cause. So I removed the initialization and (with a hat tip to Mr Jonathon Wakely) I reworked print_string to display the length of the string as well as an indication of the (un)printability of each character:
static void print_string(const char * name, const char * s)
{
    printf("%10s: \"%s\" %d ", name, s, (int)strlen(s));
    for (size_t i = 0; i != strlen(s); i++)
    {
         putchar(isprint(s[i]) ? 'P' : 'U');   
    }
    putchar('\n');
}

With this change the diagnostic became:
to_roman(1) FAILED
  expected: "I" 1 P
    actual: "I" 4 UUUP

Much better. Then I thought about initializing the array a bit more. I realized that initializing the array to the empty string was a poor choice since it masked a fault in the implementation of to_roman which did not start like this:
void to_roman(char buffer[], size_t size, int n)
{
    buffer[0] = '\0';
    ...
}

So I added that and reworked print_string as follows:
static void assert_to_roman(const char * expected, int n)
{
    char actual[32];
    memset(actual, '!', sizeof actual);
    to_roman(actual, sizeof actual, n);
    ...
}

And I was back to green :-)

commas in C - can you pass the puzzle ?

In a recent Deep-C training course in Bangalore I was discussing sequence points with some C programmers. I explained that the comma operator is one the very few operators that creates a sequence point. I like the comma. It's the name of a beautiful butterfly. K&R's famous Hello World program had a comma between hello and world. I should really put a comma into my blog picture!

During several cyber-dojos it became clear to me that many of the programmers (despite having programmed in C for several years), did not understand that in C, not all commas are the same. I've created a small piece of C code to try help C programmers understand the humble comma...

Have a look at the following 5 lines of code. Do you know what each line does?
int x = (1,2,3);
int x =  1,2,3;

   x =   1,2,3;
   x =  (1,2,3);
   x = f(1,2,3); 
.
.
.
.
.
.
Scroll down for my answers once you've decided...
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
int x = (1,2,3);

This declares an int called x and initializes it to the result of the expression (1,2,3). The commas inside this expression are operators. 1 is evaluated and its value (1) discarded, then a comma provides a sequence point, then 2 is evaluated and its value (2) discarded, then a comma provides a sequence point, then 3 is evaluted and its value is the value of the expression (1,2,3). So x is initialized to 3. You'll probably get warnings saying there are no side-effects in the expressions 1 and 2.
int x =  1,2,3;

This is different. If this compiled it would declare an int called x and initialize it to 1 and then declare two more ints called 2 and 3. It has the same structure as int x = 1,y,z; which declares three ints called x, y, and z. The commas are not operators, they are punctuators/separators. You can't declare variables called 2 or 3. It does not compile.
   x =   1,2,3;

In this fragment x is assumed to have already been declared. It is not a declaration. The commas are operators again. Assignment has higher precedence than the comma operator so this binds as (x = 1),2,3;. So 1 is assigned to x, and the result of this assignment expression (1) is discarded, then there is a sequence point, then 2 is evaluated and its value (2) is discarded, then there is a sequence point, then 3 is evaluated and its value (3) is discarded. You'll probaby get warnings saying there are no side-effects in the expressions 2 and 3.
   x =  (1,2,3);

Again, x is assumed to have already been declared. It is not a declaration. The commas are operators again. This is the same as the first fragment except it is not a declaration. x is assigned the value of the expression (1,2,3) which is 3. Again you'll probably get warnings saying there are no side-effects in the expressions 1 and 2.
   x = f(1,2,3);

Again, x is assumed to have already been declared. As has a function called f which accepts three int arguments. These commas are not operators. They are punctuators/separators. They separate the three expressions forming the three arguments to f. But they do not introduce any sequence points.

How did you do?

another interesting TDD episode

The classic TDD cycle says that you should start with a test for new functionality and see it fail.
There is real value in not skipping this step; not jumping straight to writing code to try to make it pass.
  • One reason is improving the diagnostic. Without care and attention diagnostics are unlikely to diagnose much.
  • A second reason is to be sure the test is actually running! Suppose for example, you're using JUnit and you forget its @Test annotation? Or the public specifier?
  • A third reason is because sometimes, as we saw last time, you get an unexpected green! Here's another nice example of exactly this which happened to me during a cyber-dojo demo today.
I was doing the fizz-buzz practice in C.
I started by writing my first test, like this:
static void assert_fizz_buzz(const char * expected, int n)
{
    char actual[16];
    fizz_buzz(actual, sizeof actual, n);
    if (strcmp(expected, actual) != 0)
    {
        printf("fizz_buzz(%d)\n", n);
        printf("expected: \"%s\"\n", expected);
        printf("  actual: \"%s\"\n", actual);
        assert(false);
    }
}

static void numbers_divisible_by_three_are_Fizz(void)
{
    assert_fizz_buzz("Fizz", 3);
}

I made this fail by writing the initial code as follows (the (void)n is to momentarily avoid the "n is unused" warning which my makefile promotes to an error using the -Werror option):
void fizz_buzz(char * result, size_t size, int n)
{
    (void)n;
    strncpy(result, "Hello", size);
}

which gave me the diagnostic:
...: assert_fizz_buzz: Assertion `0' failed.
fizz_buzz(3)
expected: "Fizz"
  actual: "Hello"

I made this pass with the following slime
void fizz_buzz(char * result, size_t size, int n)
{
    if (n == 3)
        strncpy(result, "Fizz", size);
}

Next, I returned to the test and added a test for 6:
static void numbers_divisible_by_three_are_Fizz(void)
{
    assert_fizz_buzz("Fizz", 3);
    assert_fizz_buzz("Fizz", 6);
}

I ran the test, fully expecting it to fail, but it passed!
Can you see the problem?
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
The problem is in assert_fizz_buzz which starts like this:
static void assert_fizz_buzz(const char * expected, int n)
{
    char actual[16];
    ...
}

Here's what's happening:
  • assert_fizz_buzz("Fizz", 3) is called
  • char actual[16] is defined
  • fizz_buzz(actual, sizeof actual, 3) is called
  • if (n == 3) is true
  • "Fizz" is strncpy'd into actual
  • fizz_buzz(actual, sizeof actual, 3) returns
  • strcmp says that expected equals actual
  • ...
  • assert_fizz_buzz("Fizz", 6) is called
  • char actual[16] is defined
  • actual exactly overlays its previous location so its first 5 bytes are still 'F','i','z','z','\0'
  • fizz_buzz(actual, sizeof actual, 6) is called
  • if (n == 3) is false
  • fizz_buzz(actual, sizeof actual, 6) returns
  • strcmp says that expected equals actual

My mistake was in the test; actual has automatic storage duration so does not get initialized. It's initial value is indeterminate. The first call to assert_fizz_buzz is accidentally interfering with the second call. Tests should be isolated from each other. I tweaked the test as follows:
static void assert_fizz_buzz(const char * expected, int n)
{
    char actual[16] = { '\0' };
    ...
}

I ran the test again and this time it failed :-)
...: assert_fizz_buzz: Assertion `0' failed.
fizz_buzz(6)
expected: "Fizz"
  actual: ""

I made the test pass:
void fizz_buzz(char * result, size_t size, int n)
{
    if (n % 3 == 0)
        strncpy(result, "Fizz", size);
}

Let's hear it for starting with a test for new functionality and seeing it fail.

an interesting TDD episode

I'm doing the roman-numerals kata in C.
I write a test as follows:
#include "to_roman.hpp"
#include <assert.h>
#include <string.h>

int main(void)
{
    char actual[32] = { '\0' };
    to_roman(actual, 111);
    assert(strcmp("CXI", actual) == 0);
}

I write a do-nothing implementation of to_roman.
I run the tests and I get (I kid you not) this:
...
test: to_roman.tests.c:26: main: Assertion `__extension__ ({ size_t __s1_len, __s2_len; (__builtin_constant_p ("CXI") && __builtin_constant_p (actual) && (__s1_len = __builtin_strlen ("CXI"), __s2_len = __builtin_strlen (actual), (!((size_t)(const void *)(("CXI") + 1) - (size_t)(const void *)("CXI") == 1) || __s1_len >= 4) && (!((size_t)(const void *)((actual) + 1) - (size_t)(const void *)(actual) == 1) || __s2_len >= 4)) ? __builtin_strcmp ("CXI", actual) : (__builtin_constant_p ("CXI") && ((size_t)(const void *)(("CXI") + 1) - (size_t)(const void *)("CXI") == 1) && (__s1_len = __builtin_strlen ("CXI"), __s1_len < 4) ? (__builtin_constant_p (actual) && ((size_t)(const void *)((actual) + 1) - (size_t)(const void *)(actual) == 1) ? __builtin_strcmp ("CXI", actual) : (__extension__ ({ const unsigned char *__s2 = (const unsigned char *) (const char *) (actual); register int __result = (((const unsigned char *) (const char *) ("CXI"))[0] - __s2[0]); if (__s1_len > 0 && __result == 0) { __result = (((const unsigned char *) (const char *) ("CXI"))[1] - __s2[1]); if (__s1_len > 1 && __result == 0) { __result = (((const unsigned char *) (const char *) ("CXI"))[2] - __s2[2]); if (__s1_len > 2 && __result == 0) __result = (((const unsigned char *) (const char *) ("CXI"))[3] - __s2[3]); } } __result; }))) : (__builtin_constant_p (actual) && ((size_t)(const void *)((actual) + 1) - (size_t)(const void *)(actual) == 1) && (__s2_len = __builtin_strlen (actual), __s2_len < 4) ? (__builtin_constant_p ("CXI") && ((size_t)(const void *)(("CXI") + 1) - (size_t)(const void *)("CXI") == 1) ? __builtin_strcmp ("CXI", actual) : (__extension__ ({ const unsigned char *__s1 = (const unsigned char *) (const char *) ("CXI"); register int __result = __s1[0] - ((const unsigned char *) (const char *) (actual))[0]; if (__s2_len > 0 && __result == 0) { __result = (__s1[1] - ((const unsigned char *) (const char *) (actual))[1]); if (__s2_len > 1 && __result == 0) { __result = (__s1[2] - ((const unsigned char *) (const char *) (actual))[2]); if (__s2_len > 2 && __result == 0) __result = (__s1[3] - ((const unsigned char *) (const char *) (actual))[3]); } } __result; }))) : __builtin_strcmp ("CXI", actual)))); }) == 0' failed.
...

So I work towards improving the diagnostic with a custom assert, as follows:
#include "to_roman.h"
#include <assert.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>

static void assert_roman(const char * expected, int n)
{
    char actual[32] = { '\0' };
    to_roman(actual, n);
    if (strcmp(expected, actual) != 0)
    {
        printf("to_roman(%d)\n", n);
        printf("expected: \"%s\"\n", expected);
        printf("  actual: \"%s\"\n", actual);
        assert(false);
    }
}

int main(void)
{
    assert_roman("CXI", 111);
}

I run this and my diagnostic is as follows:
test: to_roman.tests.c:16: assert_roman: Assertion `0' failed.
to_roman(111)
expected: "CXI"
  actual: ""
...

Much better :-)
Now I start to implement to_roman
#include "to_roman.h"
#include <string.h>

void to_roman(char * roman, int n)
{
    roman[0] = '\0';
    strcat(roman, "CXI");
}

And I'm at green.
I refactor to this:
#include "to_roman.h"
#include <string.h>

void to_roman(char * roman, int n)
{
    roman[0] = '\0';
    strcat(roman, "C");
    strcat(roman, "X");
    strcat(roman, "I");
}

I refactor to this:
#include "to_roman.h"
#include <string.h>

void to_roman(char * roman, int n)
{
    const char * hundreds[] = { "C" };
    const char * tens[]     = { "X" };
    const char * units[]    = { "I" };

    roman[0] = '\0';
    strcat(roman, hundreds[0]);
    strcat(roman, tens[0]);
    strcat(roman, units[0]);
}

Remembering that in my test, n is one-hundred-and-eleven, I refactor to this:
#include "to_roman.h"
#include <string.h>

void to_roman(char * roman, int n)
{
    const char * hundreds[] = { "", "C" };
    const char * tens[]     = { "", "X" };
    const char * units[]    = { "", "I" };

    roman[0] = '\0';
    strcat(roman, hundreds[1]);
    strcat(roman, tens[1]);
    strcat(roman, units[1]);
}

I refactor to this:
#include "to_roman.h"
#include <string.h>

void to_roman(char * roman, int n)
{
    const char * hundreds[] = { "", "C" };
    const char * tens[]     = { "", "X" };
    const char * units[]    = { "", "I" };

    roman[0] = '\0';
    strcat(roman, hundreds[n / 100]);
    n %= 100;
    strcat(roman, tens[n / 10]);
    n %= 10;
    strcat(roman, units[n]);
}

And I'm still at green. Now I add a new test:
int main(void)
{
    assert_roman("CXI", 111);
    assert_roman("CCXXII", 222);
}

I run it and am amazed to see it pass.
It takes me a little while to figure out what is going on.
I'll take it line by line.
When n == 222 this line:
    strcat(roman, hundreds[n / 100]);

is this
    strcat(roman, hundreds[2]);

and hundreds[2] is an out-of-bounds index. However, hundreds[2] just happens to evaluate to the same as tens[0] which is the empty string. So at this point roman is still the empty string. The next lines are these:
    n %= 100;
    strcat(roman, tens[n / 10]);

which is this:
    strcat(roman, tens[2]);

And tens[2] is also an out-of-bounds index. And tens[2] just happens to evaluate to the same as units[0] which is also the empty string. So at this point roman is still the empty string. The next lines are these:
    n %= 10;
    strcat(roman, units[n]);

which is this:
    strcat(roman, units[2]);

And yet again units[2] is an out-of-bounds index. This time units[2] just happens to evaluate to "CCXXII" from the test! So after this roman is "CCXXII" and the test passes! Amazing!

I edit the code to this:
void to_roman(char * roman, int n)
{
    const char * hundreds[] = { "", "C", "CC" };
    const char * tens[]     = { "", "X", "XX" };
    const char * units[]    = { "", "I", "II" };

    roman[0] = '\0';
    strcat(roman, hundreds[n / 100]);
    n %= 100;
    strcat(roman, tens[n / 10]);
    n %= 10;
    strcat(roman, units[n]);
}

And I'm still at green.

So now I'm wondering if there are any lessons I can learn from this episode. It was not a good idea to run the tests (to try and get an initial red) when doing so would knowingly cause the (unfinished) program to exhihibit undefined behaviour. In cyber-dojo terms an amber traffic-light is not the same as a red traffic-light. After adding the second test I should have edited to_roman as follows:
void to_roman(char * roman, int n)
{
    const char * hundreds[] = { "", "C", "" };
    const char * tens[]     = { "", "X", "" };
    const char * units[]    = { "", "I", "" };

    roman[0] = '\0';
    strcat(roman, hundreds[n / 100]);
    n %= 100;
    strcat(roman, tens[n / 10]);
    n %= 10;
    strcat(roman, units[n]);
}

Then I would have got a proper red:
test: to_roman.tests.c:17: assert_roman: Assertion `0' failed.
to_roman(222)
expected: "CCXXII"
  actual: ""
...


lessons from testing

I have run hundreds of test-driven coding dojos using cyber-dojo.
I see the same test anti-patterns time after time after time.
Do some of your tests exhibit the same same anti-patterns?



are you missing a TDD step?

Here's a TDD state diagram.
  • start by writing a test for new functionality
  • see it fail
  • make it pass
  • refactor
  • round and round you go
It looks a bit like an animal. Let's give it some eyes!



But there's something not right!
There's no red-to-red self-transition.
My animal is missing an ear!
I'll add the missing ear.




What is this new ear?
It's for changes made at red.
I see the test fail.
I read the diagnostic.
Then I stay at red and improve the diagnostic.
When I'm happy with diagnostic I get rid of it by making the test pass.



This was part of my lessons from testing presentation which reviews common test anti-patterns I see on cyber-dojo.

Note: I'm being careful not to call this red-to-red transition a refactoring since refactoring is for changes made at green.


pro git

Is an excellent book by Scott Chacon (isbn 978-1-4302-1833-3). As usual I'm going to quote from a few pages:
Git as a content-addressable filesystem is a very powerful tool that you can easily use as more than just a VCS.
In a DVCS, clients don't just check out the latest snapshot of the files: they fully mirror the repository... Every checkout is really a full backup of all the data.
Conceptually, most other systems store information as a list of file-based changes. These systems think of the information they keep as a set of files and the changes made to each file over time... Git doesn't think of or store its data in this way. Instead, Git thinks of its data more like a set of snapshots of a mini filesystem... This makes Git more like a mini filesystem with some incredibly powerful tools built on top of it, rather than simply a VCS.
It is important to note that the fetch command pulls the data to your local repository - it doesn't automatically merge it with any of your work or modify what you're currently working on. You have to merge it manually into your work when you're ready.
Running git pull generally fetches data from the server you originally cloned from and automatically tries to merge it into the code you're currently working on.
The way Git branches is incredibly lightweight, making branching operations nearly instantaneous and switching back and forth between branches generally just as fast. Unlike make other VCSs, Git encourages a workflow that branches and merges often, even multiple times a day.
A branch in Git is simply a lightweight moveable pointer to one of these commits.
To switch to an existing branch, you run the git checkout command.
Git determines the best common ancestor to use for its merge base; this is different than CVS or Subversion (before version 1.5), where the developer doing the merge has to figure out the best merge base for themselves.
In Git it's common to create, work on, merge, and delete branches several times a day.
In Git there are two main ways to integrate changes from one branch into another; the merge and the rebase.
At the core of Git is a simple key-value data store. You can insert any kind of content into it, and it will give you back a key that you can use to retrieve the content again at any time.
If someone at any point in the history of your project added a single huge file, every clone for all time will be forced to download that large file, even if it was removed from the project in the very next commit. Because it's reachable from the history it will always be there.


management 3.0

is an excellent book by Jurgen Appelo, subtitled Leading agile developers, Developing agile leaders (isbn 978-0-321-71247-9). As usual I'm going to quote from a few pages:
The hierarchy is needed for authorization; the network is needed for communication.
Big species consume more and breed slower.
The Red Queen's Race is an evolutionary hypothesis describing that a complex system needs continuous improvement to simply maintain its current fitness, relative to the systems it is co-evolving with. Some scientists claim that the Red Queen's Race, or the principle of co-evolving species, is an even more important driver of evolution that any other kind of environmental change.
We can consider the internal structure of each system to be a code for the environment and the other species that it is evolving with.
There is no accurate (or rather, perfect) representation of a system which is simpler than the system itself.
We can figure out why the human heart fails (reductionism) but we can never create a heart that won't fail (constructionism).
Managers must learn that they are "in charge" but not "in control".
Recent research has shown that the copying of ideas is the most successful of all strategies.
Uncertainty results in a bias towards self-interest.
Feedback is only feedback when there is a purpose behind it.
Research shows that self-discipline is twice as important as IQ for final grades of students. Effort matters more than talent.
Focus on delivering value.
We need continuous business improvement.


scrum

is an excellent book by Jeff Sutherland, subtitled A revolutionary approach to building teams, beating deadlines, and boosting productivity (isbn 978-847-94108-4). As usual I'm going to quote from a few pages:
The most powerful part of Scrum from his [Jeff Johnson] point of view? Demos.
Scrum is not about the developers. It's about the customers and stakeholders.
I learned a lot about systems theory and how a system only has certain stable states. As a cell evolves, it moves from one stable state to another. Figuring out the rules to move a complex adaptive system from one state to another, and how to make the next state a positive one rather than a negative one, was something I spent nearly a decade on. To change a cell, you first inject energy into the system. At first there's chaos, there seem to be no rules, everything is in flux...
"How many gantt charts have you seen in your career?" I asked.
"Hundreds," he replied.
"How many of them were right?"
He paused. "None."
In business we all too often focus solely on individuals, even if production is a team effort.
It's the system that surrounds us, rather than any intrinsic quality, that accounts for the vast majority of our behaviour.
Every three weeks each team had to demonstrate to their colleagues what it was working on. This was an open demonstration; anyone could come. And if that demo wasn't both working and cool, [MediaLab] directors killed the project.
"Sprints." We called them that because the name evokes a quality of intensity.
Nothing gets moved to Done unless it can be used by the customer.
After engaging for a while in Sprints and Stand-ups, you stop seeing time as a linear arrow into the future but, rather, as something that is fundamentally cyclical.
People think in narratives, in stories.
The Product Owner has to be available to the team, to explain what needs to be done and why. While the Product Owner is ultimately accountable for the Backlog, there needs to be a constant dialogue with the team.
Orientation isn't just a state you're in; it's a process. You're always orienting... [OODA]


the genius in all of us

is an excellent book by David Shenk, subtitled why everything you've been told about genetics, talent and intelligence is wrong (isbn 978-184831218-0). As usual I'm going to quote from a few pages:
We're better at stuff because we've figured out how to become better. Talent is not a thing; it's a process.
We do not inherit traits directly from our genes. Instead we develop traits through the dynamic process of gene-environment interaction.
In truth, the [word] 'intelligence' has become a mere vocal sound, a word with so many meanings that finally it has none. [Charles Spearman]
Stability does not imply unchangeability. [Michael Howe]
In 1932, psychologists Mandel Sherman and Cora B. Key discovered that IQ scores correlated inversely with a community's degree of isolation.
Talent is not the cause but the result of something.
People make a great mistake who think that my art has come easily to me. Nobody has devoted so much time and thought to composition as I. [Mozart]
Heritability is a population average, meaningless for any individual person. When someone says that heritability of height is 90 per cent, he does not and cannot mean that 90 percent of my inches come from genes and 10 percent from my food. He means that variation in a particular sample is attributable to 90 percent genes and 10 per cent environment.
Genes don't directly cause traits; they only influence the system.
Genes are probabilistic rather than deterministic. [Michael Rutter]
We have far more control over our genes - and far less control over our environment - than we think.
What would be really interesting for people to see is how beautiful things grow out of shit. [Brian Eno]
The brain circuits than moderate a person's level of persistence are plastic - they can be altered. They key is intermittent reinforcement.


the reason I jump

is an excellent book by Naoki Higashida, subtitled One boy's voice from the silence of autism (isbn 978-1-4447-7677-5). As usual I'm going to quote from a few pages:
The Reason I Jump unwittingly discredits the doomiest idea of received wisdom about autism - that people with autism are anti-social loners who lack empathy with others. (Foreword)
I very quickly forget what it is I've just heard. Inside my head there really isn't such a big difference between what I was told just now, and what I heard a long, long time ago.
What makes us smile from the inside is seeing something beautiful, or a memory that makes us laugh. This generally happens when there's nobody watching us. And at night, on our own, we might burst out laughing underneath the duvet.
When I see I've made a mistake, my mind shuts down. However tiny the mistake, for me, it's a massive deal. Once I've made a mistake, the fact of it starts rushing towards me like a tsunami. And then, like trees or houses being destroyed by the tsunami, I get destroyed by the shock.
There are times when I can't act, even though I really, badly want to. This is when my body is beyond my control.
When I'm jumping, I can feel my body parts really well... and that makes me feel so, so good. By jumping up and down, it's as if I'm shaking loose the ropes that are tying up my body. When I jump I feel lighter.
It's not quite that the noises grate on our nerves. It's more to do with a fear that if we keep listening, we'll lose all sense of where we are.
My guess is that the despair we're feeling has nowhere to go and fills up our entire bodies, makes our senses more and more confused.
When you see and object, it seems that you see it as an entire thing first, and only afterwards do its details follow on. But for people with autism, the details jump straight out at us first of all, and then only gradually, detail by detail, does the whole image sort of float up into focus.
Numbers are fixed, unchanging things. That simplicity, that clearness, it's so comforting to us. Invisible things like human relationships and ambiguous expressions, however, these are difficult for us people with autism to get our heads around.
I understand that any plan is only a plan, and is never definite, but I just cannot take it when a fixed arrangement doesn't proceed as per the visual schedule. Visual schedules create such a strong impression on us that if a change occurs, we get flustered and panicky.
We can put up with our own hardships okay, but the thought that our lives are the source of other people's unhappiness, that's plain unbearable.

some cyber-dojo measurements

cyber-dojo has hosted about 13,000 practice sessions so far. I've written a short ruby script to extract some measurements from a sample of 500 sessions. I was looking at transitions between red, amber, and green traffic-lights:
  • red means one or more tests failed
  • amber means the tests did not run (eg syntax error)
  • green means the tests ran and all passed
The first column is average number of lines added/deleted.
The second column is colour → colour transition.
The third column is sample size.

3.94 ambergreen 447
4.65 amberred 379
4.67 amberamber 1462
5.39 redgreen 607
6.01 redred 604
7.52 greenred 420
13.65 greenamber 436
17.67 redamber 432
22.18 greengreen 598

Here's how I interpret the results:
  • If you're at red or green and you make a small change (5.39,6.01,7.52) you're likely to stay at red or green.
  • If you're at red or green and you make a large change (13.65,17.67) you're likely to transition to amber.
  • There is a big spike in the number of amberamber transitions (1462). I speculate that long sequences of these transitions are occuring after a large 13.65 greenamber or 17.67 redamber transition.
  • I think the greengreen value of 22.18 is larger than it should be because it's including plain file renames.


deliberate duplication TDD at XP Days Kiev

Back in October I did a 90 minute TDD master-class at XP-Days Kiev using cyber-dojo. You can watch the video of the talk which is now online. In it I code a simple exercise (Print Diamond) using sliming and deliberate duplication with refactoring.
  • I discuss ‘sliming’ – the technique of hard-coding magic-numbers to make tests pass.
  • How can sliming help guide your choice of what test to write next?
  • How can sliming be combined with deliberate duplication?
  • With micro-refactoring?
  • How do you know you’ve slimed too much?
  • How and when should you unslime?
  • I look at the the word “unit” in Unit Testing to understand why the definition is useful.
  • I consider some very important differences between “real” code and “test” code – they are not the same.


systems-thinking at NorDevCon

Today I'm in Norwich to run a cyber-dojo at NorDevCon.
Last night I presented a short talk on Systems Thinking. Here are the slides:

improving

Back to quotes table-of-contents

From The Mind of War
He also came to appreciate the routine practice and repetition that was required to become really good at something and to overcome the boredom by focusing on minute improvements.

From Taiichi Ohno's workplace management
Once he asked me how the terms kaizen and kairyo (reform) were differentiated in the West. I said that while kaizen means to make improvements by using brains, kairyo means to make improvements by using money, and that in the West, most managers only think of improvement in terms of money. [Massaki Imai]

From Nudge
The best way to help Humans improve their performance is to provide feedback.

From Becoming a Technical Leader
People improve their performance not by amputating their old behaviors, but by adding new ones.

From The Toyota Way
Extra inventory hides problems... Ohno considered the fundamental waste to be overproduction, since it causes most of the other wastes… big buffers (inventory between processes) lead to other suboptimal behaviour, like reducing your motivation to continuously improve your operation.

From Smart Swarm
If individuals in a group are prompted to make small changes to a shared structure that inspires others to improve it even further, the structure becomes an active player in the creative process.

From Peopleware
The more you improve the way you go about your work, the harder the work will be.

The paradox of the CMM is that process improvement is good, but process improvement programs aren't, or at least they often aren't.

From Implementing Lean Software Development
The paradox is that in our zeal to improve the predictability of software development, we have institutionalized practices that have had the opposite effect. We create a plan, and then we act on that plan as if it embodies an accurate prediction of the future.

From Dr Deming
To improve output, production, sales, profit, quality, or any other important factor, every part of the organization had to improve.

From Toyota Kata
The improvement kata does not come to life in an organisation simply because it is a good idea.

In many cases the normal operating condition of an organisation - its nature - is not improving.

At Toyota, improving and managing are one and the same.

From Certain to Win
If a just-in-time production line had to wait for a formal decision process to work, it would hardly move at all, and it would never improve.

From Toyota Production System
Improvement is eternal and infinite.

From Quality Software Management: Vol 4. Anticipating Change
You need stability in order to make improvements.

Testing to improve, not to prove.

If leadership improves then the culture must improve.


effort

Back to quotes table-of-contents

From Thinking Fast and Slow
It is the mark of effortful activities that they interfere with each other.

From Wabi Sabi - the japanese art of impermanence
More than any learned ideas, it was the effort and attitude of the gardener that would decide the outcome of the garden.

From Maverick
In business, effort is too often confused with result.

From Wooden on Leadership
Effort is the ultimate measure of your success.

From Drive
Effort is one of the things that gives meaning to life. Effort means you care about something, that something is important to you and you are willing to work for it. [Carol Dweck]

From All I need to know about manufacturing I learned in Joe's garage
Fanatic: A person who redoubles his effort after having lost his direction.

From Zen in the Art of Archery
You had to suffer shipwreck though your own efforts before you were ready to seize the lifebelt he threw you.

From Mindset
It's startling to see the degree to which people with the fixed mindset do not believe in effort.

From The Conquest of Happiness
To like many people spontaneously and without effort is perhaps the greatest of all sources of personal happiness.

From The Mythical Man Month
Our estimating techniques fallaciously confuse effort with progress.

From The Aesthetics of Change
Cybernetics therefore suggests that 'all change can be understood as the effort to maintain some constancy and all constancy as maintained through change'.

From Simplicity
Simplicity means focused effort.

FromThe Principles and Practice of Fly and Bait Casting
One certain index of efficiency is the absence of effort.

Style is synonymous with efficiency, and style and effort do not go together.

Excessive effort is not only uncalled for, but if practised defeats itself.

From Zen Bow, Zen Arrow
One day of effort is one day of bliss; One day of sloth is a hundred years of regret.

illusion

Back to quotes table-of-contents

From The Secrets of Consulting
A crisis is the end of an illusion.

People create illusions, which they build to replace the lost reality.

We soon find ourselves spending all our energy maintaining the illusions.

When you create an illusion to prevent change... the change becomes more likely and harder to take.

From The Alchemist
"Tomorrow, sell your camel and buy a horse. Camels are traitorous: they walk thousands of paces and they never seem to tire. Then suddenly, they kneel and die. But horses tire bit by bit. You always know how much you can ask of them, and when it is that they are about to die."

From How to use Conscious Purpose Without Wrecking Everything
Ignoring feedback merely means that the system will eventually experience a massive unpleasant surprise rather than a small unpleasant surprise.

From An Introduction to General Systems Thinking
They know "better" - which is to say that their illusion is stronger.

The more sure we are the more likely we are to suffer an illusion.

From Thinking Fast and Slow
This quality of pastness is an illusion. The truth is, as Jacoby and many followers have shown, that the name David Stenbill will look more familiar when you see it because you will see it more clearly.

From The Logic of Failure
If we never look at the consequences of our behaviour, we can always maintain the illusion of our competence.

The results also support the idea that activity may foster an illusion of competence.

From Existentialism and Humanism
I should be without illusion and I should do what I can.

From Switch
Positive illusions pose an enormous problem with regard to change.

From Mind and Nature
In Ames experiments, you are always made to observe the truth before being subjected to the illusions.

From Quality Software Management. Vol 4. Anticipating Change
It is easy to look at this diagram and believe that you're seeing a defined process. You're not. What you're seeing is an optical illusion.

paradox

Back to quotes table-of-contents

From Implementing Lean Software Development
The paradox is that in our zeal to improve the predictability of software development, we have institutionalized practices that have had the opposite effect. We create a plan, and then we act on that plan as if it embodies an accurate prediction of the future.

From Slack
When the new automation is in place, there is less total work to be done by the human worker, but what work is left is harder. That is the paradox of automation: It makes the work harder, not easier.

From Peopleware
The paradox of the CMM is that process improvement is good, but process improvement programs aren't, or at least they often aren't.

From The Lean Startup
The paradoxical Toyota proverb "Stop production so production never had to stop."

From Experiential Learning 3: Simulation
Paradoxically, realism often interferes directly with learning from a simulation.

From Free
Paradoxes are the opposite of contradictions. Contradictions shut themselves down, but paradoxes keep themselves going, because every time you acknowledge the truth of one side you're going to get caught from behind by the truth on the other side.

From The Road Less Travelled and Beyond
If a concept is paradoxical, that in itself should suggest that it smacks of integrity and has a ring of truth.

When you get to the root of things, virtually all truth is paradoxical.

From Zen Soup
The words of truth are always paradoxical [Lao Tzu]

From The Aesthetics of Change
A paradox in ecology is that the most flexible species are the dullest. When a flexible species is not controlled by its ecosystem, ecological climax breaks down and a system of weeds remains.

From Management of the Absurd
The real strength of a leader is the ability to elicit the strength of the group. This paradox is another way of saying that leadership is less the property of a person than the property of a group.

From Adapt - Why Success Always Starts With Failure
Cognitive dissonance describes the mind's difficulty in holding two apparently contradictory thoughts simultaneously.

From The Book of Five Rings
The practice of all the arts is for the purpose of clearing away what is on your mind. In the beginning, you do not know anything, so paradoxically you do not have any questions on your mind. Then, when you get into studies, there is something on your mind and you are obstructed by that. This makes everything difficult to do.

From Mastery
Those we know as masters are dedicated to the fundamentals of their calling. They are zealots of practice, connoisseurs of the small, incremental step. At the same time - and here's the paradox - these people, these masters, are precisely the ones who are likely to challenge previous limits.

kanban push-me pull-you

The video of the kanban push-me pull-you presentation I gave at David Anderson's Lean Kanban UK conference is now up on youtube. Here's the slide-deck too.

listening

Back to quotes table-of-contents

From Zend Mind, Beginner's Mind
Usually when you listen to some statement, you hear it as a kind of echo of yourself. You are actually listening to your own opinion. If it agrees with your opinion you may accept it, but if it does not, you will reject it or you may not even really hear it.

From Quality Software Management: Vol 1. Systems Thinking
People's language often reveals when they believe that they are victims of events, rather than having a choice of reactions to the event. Learn to listen for falsely deterministic key words.

From Quality Software Management: Vol 2. First-Order Measurement
No other observational skill may be more important to software engineering than precision listening.

From Authentic Happiness
The overarching principle of good listening is validation.

From Agile Coaching
We find the hardest part of listening is resisting the temptation to jump in too early with advice or to switch the conversation to a similar story that happened to you.

From The Way of the Leader
Listen carefully. Observe closely.

From The Fifth Discipline
Listening is not easy.

From Management of the Absurb
When we really listen, so that we understand the other person's perspective, we risk being changed ourselves.

From Surfing The Edge of Chaos
Because we thought our job was to persuade, too often we forgot to listen.

From Non Violent Communcation
When the faculties are empty, then the whole being listens.

From Cat Stevens
From the moment I could talk, I was ordered to listen

From The Road Less Travelled and Beyond
Listening well also requires total concentration upon another and is a manifestation of love in the broadest sense of the word.

From Understanding the Professional Programmer
It's sometimes hard to know when someone is listening - rather than merely waiting to seize control of the conversation. One way everyone knew Mack wasn't listening was by noticing how seldom he allowed other people to finish what they were saying.

From Siddhartha
He learned more from the river than Vasudeva could teach him. He learned from it continually. Above all, he learned from it how to listen, to listen with a still heart, with a waiting, open soul, without passion, without desire, without judgement, without opinions.