(photo by Computer Based Math)
I recently attended Conrad Wolfram’s conference on computer based math (http://www.computerbasedmath.org/).  People from all over the world gathered to give examples and discuss how to improve math education.  Conrad was quite actively involved, not just giving the keynote and closing remarks but asking questions and actively participating in the room the full two days.  He clearly cares deeply and was listening carefully to decide what to do next.
 
Being a bit of a “gamifier” these days, I made up my own imaginary game for this conference: A contest, where each invited participant had to create a plan for what Conrad Wolfram should do next, given his goals and beliefs and resources.  Note, as far as I know no one else is playing this game, nor is there a prize, but if someone else wants to play please post a link to your entry in the comments. :)
 
Conrad Wolfram does an eloquent job of laying out his goals in his TED talk, which has been viewed by millions of people.  If you haven’t seen it, please watch it now before reading any further. (http://www.computerbasedmath.org/resources/reforming-math-curriculum-with-computers.html)
 
What do I think Conrad Wolfram should do?

I suggest Wolfram should focus on disrupting high stakes assessment by creating a new assessment.  In addition, he should initially aim for the use of his new assessment by corporations hiring young adults for entry level jobs.
 
Why Assessment?
 
In the conference, both the presenters in front of the room and the front line educators I talked to during break agreed that assessment was the tail wagging the dog of math education.  They teach to the test and the students are motivated to learn because they need to pass high stakes tests. Parents see test scores as critical for their child’s success and administrators and governments use test scores as a measure of success.   People may say they hate the tests but all the reward and motivation processes are tied to the tests.  Change the assessment and people will change how they teach and learn.

How can the Assessments be changed?
 
The conference was attended by about 150 people. I would say almost everyone agreed with the majority of Conrad’s TED talk.  However, even in this friendly room, by the end of the first day, many people were noticing that in a conference about how important it was for people to be able to interpret and manipulate real world data, no data was being presented, only anecdotes.  This was explicitly stated in some of the talks the second day.
 
Thus one of the conclusions I draw from this conference was that Wolfram needs to present data to be taken seriously.  Because of who he is and what he is trying to promote he will be held to a higher standard on having data to support his thesis, and the world of education in general is working to become more data driven.
 
During one of the breaks I asked Conrad why he didn’t mine the vast wealth of existing assessment data to support his proposals.  He said that the current assessments are so thoroughly focused on the wrong thing that he is worried that using that data will result in the wrong conclusions.  That is, if the tests are built to test computational skill, and you teach people how to use math to solve real world problems with computers, the results of the test of computational skills may quite likely go down.  Thus he is concerned that using existing test data may damn his curriculum ideas.
 
How can good Data be collected?

There were quite a few people focused on assessment attending the invitational conference, so clearly Conrad and the organizers know that assessment is a hugely important piece of the puzzle.  But if you believe all the current high stakes tests are rubbish, how do you disrupt the mature, highly bureaucratic, regulated existing big business that is high stakes tests?  No one will accept Wolfram’s new assessment without good data.  What organization or country will take the risk to both give and teach to these new unproven assessment instruments so that data can be collected?
 
Disrupting High Stakes Testing  

Let’s take a look at high stakes tests through the lens of Christensen’s theory of Disruptive Technology.  Christensen says that disruptive technologies are first adopted by people who are not consumers of the current dominant technology.  The new technology does not start out better for everyone; it starts out the better, or often for some minor reason, the only choice for some subsegment of consumers who are not currently using the dominant technology.
 
Who are the consumers of high stakes math tests? The students?  They take the tests, and in some cases pay for them, but they aren’t paying because they want the scores, they are paying as part of the cost of applying for educational opportunities.   Thus arguably the consumers, and certainly the users, of the data generated by high stakes testing are currently educational institutions and governments.  Educational institutions use the scores to decide who to admit and government agencies at various levels use them to attempt to evaluate their government funded schools.  There are probably a few other uses but this seems to me to be most of the uses of the test scores.   
 
There is another group that, like colleges, expends significant resources to sort young adults -- Employers.  They try to use grades, diplomas and certificates, interviews and sometimes test scores to decide who they are going to hire.  

Conrad’s thesis is (and I agree) that his wider definition of Math skills matches what employers need better than the educational system’s focus on computation.  He is also CEO of a corporation and has better access to other CEOs then most educational reformers.  I suggest he can use these connections to find partners who are large global employers and convince them to invest their resources into giving this new assessment to people before they hire them.  This would allow large scale data collection and allow the assessment creators to work on fine tuning an assessment that correlates to work place success.

With compelling data sets and the incentive of having large global employers saying this a test they care about the results of when hiring, I think schools and governments would have the political cover to teach to these tests, and change the laws to allow the existing high stakes testing to be changed.

Not an easy task and a plan that will take years, but I do think this is a plan consistent with Conrad Wolfram's goals and beliefs and one that he is in a uniquely good position to implement.

Good luck! 

Hear, hear!

Thank you for your thoughts on the assessment thread of the discussion. This was definitely one of the key themes of the conference, and the issues you identified will help people focus, going forward.

Some powerful entities like Pearson are moving into the assessment business. Then there are efforts such as OERu, in different directions altogether. We are, most definitely, living in interesting times.

by Maria Droujkova on 12/22/11

Add comment