D520 Week One – 2010

There are not too many changes from last year, and those are mostly for the better.  The overall format for the course is the same – 4 hours a week (all in the lab) for 15 weeks, with a two-week break about two-thirds of the way through.

I did make a slight change to how I’ll use the 4 hours – the course used to be 6 hours per week, which was roughly divided into two hours of lectures, two hours of tutorials (i.e. going over code as a class), and two hours of lab exercises (i.e. students working on code alone, with one-on-one help from me).  With the change to four hours, my plan was to keep the two hours of labs, and work the tutorial content (considerably reduced) into the lecture hours.  This didn’t work particularly well – it took the students longer than I anticipated to grasp the new material and IronPython in general, and so we spent a lot of time working through the lab exercises together (either the week after they attempted it alone, or as the first attempt) – essentially the lab hours were reduced (nearly eliminated, although they still worked on the projects alone) and replaced by tutorial time.

For 2010, I’m keeping it this way: around 2 hours of lecture-style delivery, followed by around 2 hours of tutorials (e.g. we’ll walk through building MultiDoc from IronPython in Action together).  The students still have weekly lab exercises, but they will need to complete these in their own time (an added benefit is that this forces them to spend time out of class on IronPython, rather than hoping they would work through the textbook alone).  I expect we’ll spend 20 or 30 minutes at the start of several classes going over the previous week’s lab exercises.

I again gave the students a set of revision exercises [PDF] (example answers next week), a course outline [PDF], and some brief notes [PDF].  The revision exercises are pure Python, and there are some additional ones from last year – I hope that all the students can complete the first 6 or so fairly easily, the next 3 or 4 with some effort, and will probably find the final two somewhat tricky.  The notes have only three sections this year: which chapters of the textbook (again the excellent IronPython in Action) are covered (and a couple of sentences that summarise them or point out which parts are important to us), key points, and example code.  Last year the notes had a “Tools” section each week as well – as outlined below, this is simpler this year, and so I walked everyone through the first install, and will make a one-file summary of tools available in the next couple of weeks (e.g. including alternative editors).  Just as with last year, none of the students had obtained a copy of the textbook in advance, so we started completely fresh.

I’m still providing recommended reading (two links each week – a lot of changed material from last year – particularly shorter content, and I’ve tried to group it together in logical sections more this year).  This week it was “How I Hire Programmers” (Aaron Swartz) and “Hardware is Cheap, Programmers are Expensive” (Jeff Atwood). Continue reading “D520 Week One – 2010” →

What happened to the 2009 IPy Notes?

Around this time last year, I started posting weekly reports on the IronPython classes I was teaching at Northtec, including copies of the material that I was giving the students.  These stopped abruptly around the middle of the course.  The course did complete (and was fairly successful – I think the change to IronPython was definitely for the better), and I did continue making notes as I taught.  However, I didn’t manage to get them online.

On the 10th of October 2009, my father passed away (very unexpectedly, of a heart attack the day before).  As a result, I didn’t have as much extra time as I had anticipated, and I didn’t really have the energy/motivation to post the notes.  (The classes continued, except the week of the funeral, where the hours were halved).

The course began again last week (21st July 2010), and I’m going to try this again.  As I go, I’ll complete the 2009 notes as well (some/all of the links in the 2009 ones are broken, which I’ll fix, and when I get to where the notes stopped, I’ll post the notes that I took last year, although these will be less exhaustive than if I had done them at the time).

No promises, but I expect that I’ll be able to post throughout the entire course this year, and I’ll again make the material that I use available (you may use it under a Creative Commons license if you wish, although much of the material is references to IronPython in Action).

I had planned to present a short summary of the experience of using IronPython in this way at PyCon NZ, but I’m again unable to make it this year (for happier reasons – a close friend is getting married that day); perhaps next year!  I’m still interested in hearing from anyone else that’s using IronPython in the classroom.

Latest MySky (HDi) update

The latest (and heavily publicised) MySKY (HDi only?) update was released today.  Oddly, our update was applied around 2 p.m. – the currently showing program was replaced by a very ugly notice that the system was being updated.  I can’t understand why the update wasn’t deployed during the night (e.g. 4 a.m.), when the number of viewers must be significantly lower.  What if I had really cared about the program that was showing?

My initial reaction was very negative.  After it was pointed out (via Twitter) that many of SkyTV’s mistakes are correctable via new options in the settings back, I’m slightly less negative, but it’s still not an improvement.  This is especially frustrating considering that the UI is so terrible, and has been ever since MySKY was introduced about five years ago (I’ve had MySKY since just after the launch).  Being told that they know about the many problems and fixing them isn’t possible until the hardware is revved doesn’t inspire any confidence at all, since the problems existed with the original MySKY and certainly were not fixed when the hardware was revved (with HDi).  Someone at SkyTV is responsible for selecting the hardware – if the platform doesn’t support creating a decent UI, then they should choose something else.

(Has anyone on the MySKY team used an Apple product?  Or even Windows 7 or a Zune product?  Would they recognise good UI when they saw it?  Do they have anyone that has any UI talent or experience?  If so, perhaps they could give that person a bit of power to fix things?)

The “what’s new” page has a feedback form where you can comment on the recent changes.  I encourage you to do so.  My comments were:

Planner Search

Sorting by time is a pointless way to organise the planner – it caters to people that don’t understand how to use a DVR properly (e.g. who are using it like they would have used a VCR). Sorting by A-Z is vastly more useful, because the typical planner task is either finding a specific program to watch (I always know the first letter of the name, I rarely know or care when it was broadcast/recorded), or browsing through programs to find something to watch (I don’t care what’s oldest/newest, I care what the program is).

The changes in this update make things worse in this respect. Although it’s possible to turn off the video and set the “search” interface to A-Z sort by default, you’re then in an interface where you have to go right/left to get to delete/keep. The planner should default to A-Z search, or at least have that as an option.

Confirm Delete

Although it’s fantastic that after 5 years the “tiny button right next to ‘don’t delete this’ deletes irrevocably” problem has been addressed, a confirmation dialog is (although common) generally the wrong UI decision to make. Users (a) get into habits of just ok’ing such dialogs, decreasing their value, and (b) are less willing to try things out. It is almost always better (and certainly would be here) to simply make any action reversible (i.e. have an undo delete).

Preview

The preview in search seems completely pointless. I almost certainly know what the program is based on the title, and if I don’t, then starting to show the program isn’t going to help. This is exacerbated by the fact that with the “automatic” buffer setting the preview generally starts with the end of the program that was on before the program that you’re looking at.

It *would* be useful to have an episode name & number (where applicable) in the main planner screen (e.g. to distinguish between several episodes of a kid’s program that is played out-of-order), or to only show each series once (and selecting it goes into a submenu where specific episodes can be selected; this is common in other interfaces where screen space is limited). Showing the synopsis in search helps with this, but takes up a vast amount of space (only five rows of text – even on a huge screen!), and comes with the annoying and loud video.

Positive

After five years, the guide finally shows an indication of which programs are scheduled to be recorded.  This feature was present in the old Sky Digital system (which didn’t record, but did allow you to schedule reminders), and it’s amazing that it’s taken this long to restore it.  However, it ignores series-link, so it only highlights the next episode.  It would be nice to get the feature right.

Dr? No.

I finally withdrew from my PhD today (probably many people thought that this had happened some time ago).

In the beginning…

The story really starts in 1999.  I started working consistently while studying, and also got rather bored with the study that I was doing.  I also got involved in the Students’ Association – first lightly, then pretty heavily.  As a result, at the end of 2000, I was about 1.5 papers short of finishing my BSc and BBS.  I needed to do half a semester of work, and so I decided to go those at the same time as a Postgraduate Diploma in Science (these went along with running for ASA President).  The ASA job didn’t work out, but I did finish the last undergrad stuff I needed to do, and rather unexpectedly found that I really enjoyed the postgrad study.

I was able to do all the papers for the postgrad diploma along with the undergrad work, except for the double-paper research project, in that year.  That meant that the next year started off with finishing off the research project – normally half of a semester’s load, but since I didn’t have anything else to be doing, I poured in a whole semester’s effort into it.  That left me at the middle of the year with everything complete.

To be Dr T…

At this point, I was offered a $20k scholarship to do a PhD at Massey.  A lot of thought went into this at the time – including a lot of talking with Olyvia (this was not that long before we got engaged).  It was clearly going to be financially terrible for three years, but money has never been my focus.  On the positive side, I was assured (and believed) that there was quite a lot of demand for lecturers, and I had found that I really enjoyed teaching, research, and administration, which are basically the three components of a (NZ) lecturing position.  It seemed at the time like it was a good career choice (especially the flexibility that it offered), and so I took up the choice.

I chose to continue on with the work that I’d done for my research project (what I ended up calling synthetic actors).  There were a lot of reasons that this was a good choice – it was AI, which I particularly enjoyed, it involved theatre, which I was spending a lot of time doing, and it was a very clear field (i.e. few other researchers in the area).

Early errors…

In retrospect, both of these were poor choices.  I should never have enrolled with Massey to do a PhD.  The Computer Science department at Albany is just a poor relation of the strong Maths/Statistics department, and so it’s not really a good choice, even though at the time (and possibly now) they had some really good staff.  I should have gone somewhere where Computer Science was a department with real funding and support, and where I could potentially have got some work once I was done.

Although synthetic actors were fascinating and I got the work done (other than a bit of writing) in the end, it was a bad choice.  For a start, a topic like this, which bridges disciplines, needs to be done somewhere where there is more history of cross-discipline study.  To do a topic like this I should also have been somewhere that had a strong theatrical department (i.e. I really needed to be in Wellington).  It’s really very important to pick a topic that matches the institution (or an institution that matches the topic).

The other problem with the topic, exacerbated by doing it at Massey, where there’s no history of this type of research, is that it is really hard to measure.  Computer science is really a lot like mathematics and statistics in that what you’re typically after is something that you can do and collect numbers and then have some sort of graph that shows that things are improved.  A semi-art topic like synthetic actors doesn’t suit that well at all.  I ended up finding things to measure, but that did mean that a lot of the effort that the project needed (e.g. putting on the two productions) wasn’t central to the real research.

Somewhat ironically, a much more suitable topic was at hand a little later.  I started working on SpamBayes, and a computer science/statistics project would have been a great fit with Massey and also very measurable.  It would also have been much more translatable into work (in fact, my work on the project, outside of the PhD work, has contributed much more).

Good times…

Despite these two things, everything went pretty well for the first 2.5 years.  Money, as expected, was pretty tight – $20k is not a lot, especially with a wedding and living a ‘grown up’ life (i.e. not poor student flatting or living at home with parents).  In fact, our rent was about $20k at this time.  This meant that I had to work a lot as well as study full time, but even though I had little spare time, I was enjoying both, so it seemed ok.  The English department at Massey was hugely helpful here, because they paid me decent wages to teach a paper each semester (whereas in Computer Science you got less than working at a service station).

Towards the end of the three-year period, everything was basically on track.  I had about a third to a half of my thesis complete (about half of this was later rewritten from scratch) with about 6 months to go.  My plan was that I would spend a reasonable amount of time time looking for (lecturing) work to move into as I finished up the thesis – I even gave up the lucrative English teaching in order to do this.

Stalling…

This is where everything stalled.  I vastly underestimated how difficult finding lecturing work would be (and how unhelpful and insulting Massey would be).  With a couple of months of stipend to go, I disparately needed to find work (Olyvia wasn’t working at this time either, so the need was particularly urgent).  Although there wasn’t any lecturing work to be found (or at least, anyone interested in me), there was a lot of other work.

In particular, SpamExperts approached me about doing a short project (nearly 5 years later, I’m still working for them!), Lifeway College approached me about teaching a module (in Business Modelling with Spreadsheets, of all things), and Northtec (indirectly) approached me about filling in for a few weeks (I’m still doing this – and another paper – too) for a programming paper.

All of this was enough to pay the bills, but combined with still looking for lecturing work, left me no time to actually finish off the thesis.  The two teaching jobs in particular (which I valued a lot because they seemed like good experience for lecturing work) required me to create the courses from scratch (and I had to learn Business Modelling, since I had no prior knowledge).  Several months passed without any thesis work getting done.

This was really my biggest error – underestimating how difficult it would be to get work afterwards.  I should have come up with some plan that wouldn’t have resulting in losing the last (essential) few months of full-time study that I had.  I’m not sure what that plan would have been, but it was what was missing.

Stuck…

The plan around this time was that we’d have our first child during the next year – we didn’t want to be old parents (and if we have two children want to have a break between them), and it seemed like a good time (with me going into the real work force and so on).  Samuel came along very quickly (planning’s fine, but there’s still some luck involved) – I rather naively assumed that finishing the PhD and working would fit into the pregnancy year without any problem (after all, it was Olyvia that was pregnant, not me, right?).

I probably could have managed to get work done during that time.  However, I was enjoying a break of just normal working life, I was working a lot of hours (I had many jobs, but all were – in theory – temporary, so I was scared that they’d end and I’d have nothing, so I needed savings).  Finally, supporting Olyvia during the pregnancy took a lot more time that I expected, with antenatal classes, midwife visits, scans, shopping, and so forth.  I could have skipped this, of course, but they took priority for me over the PhD.

Limbo #1

Once Samuel really arrived (28/8/6), there really was no time to spare.  The first year, in particular, was quite tricky, and I spent all my time parenting and working.  If I had put thesis work into the mix, I would have produced rubbish or would have burnt out.  A year or so later, things did start to quieten down a bit, but I was in an odd limbo where I had had such a long break that I wasn’t sure where I was.

Going back to the work after a long break was difficult, and since I’d used up the 4 years I was meant to have, I wasn’t sure whether the work would be accepted anyway.  Massey was extremely unhelpful here (not my supervisor – he was great), in that I was continually asking for time and never hearing about whether I was granted it or not – the only time I heard from them was the ridiculous requests for 6-monthly reports and queries about when it would be done.

(The reports infuriate me.  In theory, they’re meant to track how people are going.  However, I submitted years of reports saying “I failed to get anything done”, and nothing came of them.  So obviously they are ignored, and are pointless.  I could have spent the time actually getting work done instead).

I had about a year of this limbo.  This was the second big mistake, really.  I could have managed to get the work complete during this time – I was busy with work and parenting, sure, but there was also a bit of time here and there where I could have knuckled down and got the work done without going crazy.  The PhD seemed very distant and less and less important as time progressed and so motivation was very difficult.

Nearly unbelievably, progress again…

About 18 months ago, the limbo finally ended.  I finally got some clear answers (thanks to my supervisor I believe) from Massey about whether I was able to continue working on it or not.  The original plan was to finish the thesis (other than some editing) by the end of 2008.  I made a private plan with my supervisor to get a certain number of words done every week (emailing fresh copies weekly) and if I failed to get that done any week, I’d withdraw.  I worked very hard at this, and although it was tricky, I did manage to meet each deadline.

At the end of the year, I was a little short (about 1.5 chapters and a bit of tidying up), although I’d done more editing than originally planned.  Limbo came back.  Eventually (some time in early 2009) I did hear that I had an official (after years!) extension until 29/11/09.  This was clearly enough time, at least in theory.

Limbo #2

However, the last 12 months have been a disaster for me personally.  2009, without doubt, has been the worst year of my life so far (hopefully, ever).  I had no idea that things could go so badly.  The details don’t really belong here, but I honestly don’t believe that I had any chance to get the work done (to an acceptable standard) this year.  I really needed to get it done the previous year (when in honesty I should have been able to find time) – there was just no way I could work on it this year.

The end

Supposedly Massey have been trying to talk to me about the status recently (I don’t know for how long) and haven’t been able to get hold of me.  I find this difficult to believe considering how easily an email address for me (or even this blog!) can be found via Google.  Olyvia got a message on the phone today querying the status (predictable given that the deadline is 9 days away), and I sent the IIMS HoD a message indicating that I was withdrawing (in reality, there’s no choice – and if I’d known what the year would be like, I would have just done that a year ago).

Retrospective thoughts

This is obviously disappointing.  The PhD was the biggest project I’d worked on at the time, and I poured a lot of time, money, and effort into it.  It’s also nearly done – I have tens of thousands of words of a thesis complete.  It’s frustrating quitting something when it’s so close to completion.

I’ve also always been mindful of the support that others gave me, particularly in those first three (productive, active) years.  Olyvia, Mum and Dad, Jacs, Jo, Geoff and the other MADSODS, Chris, Jenny, and so forth.  It always seemed like I owed it to them to complete the damn thing as well.

However, it has been hanging on my neck like the proverbial millstone for years now.  Few days go by without at least one guilty stressful thought about how I haven’t done anything towards it that day.  It will be nice to leave that behind.

In reality, there’s no practical benefit to getting the PhD.  I long since lost any desire to work in academia (thanks, Massey!), and even if I got the PhD, I’ve spent too many years without getting anything published to be a strong candidate for any job (I was a stronger candidate, in many ways, back when I failed to find lecturing work years ago).  In NZ, which is where I want to continue working, there’s little career value to a PhD outside of academia.  I still have plenty of work (although the exchange rate is hurting a lot) and I’m certain that I could find more if I had to (although I’ve clearly been wrong about that before!).

On the other side of the equation, there are things that really are important to me.  In particular, family.  If given the choice between spending more time with Samuel and writing a thesis that’s only of sentimental worth, I’ll pick Samuel without any regrets.

Looking back, I can see where I made mistakes – the poor topic choice, failing to see how difficult finding work would be, not utilising time in 2007/8.  However, I don’t regret any of the decisions I made at the time.  Most of the time, I was choosing things that are still more important to me than the PhD.

To those reading this that did help – thanks.  I really do wholeheartedly appreciate all the support.  Also: my apologies – I’m sorry if I have let you down by not completing everything.  I hope you understand that there are just things that are more important to me (I know some people do, since they’ve said that to me in the past – Dad did this a while ago).

So that’s that.  I’ll probably put the thesis up at some point – maybe another eager kid will want to work with synthetic actors (despite Claudio giving it up and then me giving it up) and will find it useful.  I might have more thoughts then.  For now, I’m going to go back to things that I care more about.

p.s. Yes, I did partly choose the title of this post to counter the Google juice that the James Bond Tony Meyer seems to have.

D520 – Week Six

Chapter 6 of IronPython in Action covers “properties, dialogs, and Visual Studio”.  This seemed an obvious place to insert the material on user-interface design that is normally covered in the course, and to look a bit more deeply than the textbook does at Visual Studio itself (and the Windows Forms controls and their properties).  I only scheduled a single week to cover this, but I suspected that it might take more than one (I left an empty slot in the schedule to cover one such over-run), and that was, indeed, the case.  The students received notes [PDF], slightly longer this week (covering the UI design material not in the textbook, as well as the usual chapter summary, key points, and examples, and the steps required to install IronPython support in the ‘Experimental Hive’ Visual Studio SDK), and a fairly simple lab exercise [PDF]. Continue reading “D520 – Week Six” →

Mailbox flattening messages

The mailbox.py code does this to dump a message (e.g. when adding to a Maildir file):

def _dump_message(self, message, target, mangle_from_=False):
    # Most files are opened in binary mode to allow predictable seeking.
    # To get native line endings on disk, the user-friendly \n line endings
    # used in strings and by email.Message are translated here.
    """Dump message contents to target file."""
    if isinstance(message, email.Message.Message):
        buffer = StringIO.StringIO()
        gen = email.Generator.Generator(buffer, mangle_from_, 0)
        gen.flatten(message)
        buffer.seek(0)
        target.write(buffer.read().replace('\n', os.linesep))
    elif isinstance(message, str):
        if mangle_from_:
            message = message.replace('\nFrom ', '\n>From ')
        message = message.replace('\n', os.linesep)
        target.write(message)

(There’s a bit more that deals with other types of message).  Unfortunately, with some messages containing Unicode characters, this breaks with a UnicodeDecodeError:

Traceback (most recent call last):
  File "", line 1, in
  File "/usr/lib/python2.5/mailbox.py", line 245, in add
    self._dump_message(message, tmp_file)
  File "/usr/lib/python2.5/mailbox.py", line 203, in _dump_message
    buffer.seek(0)
  File "/usr/lib/python2.5/StringIO.py", line 106, in seek
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd6 in position 106: ordinal not in range(128)

The message does flatten with message.as_string() – or str(message) – without any problems.  I’m not really sure why mailbox.py doesn’t just use as_string(), rather than create a generator itself.  I’m also not totally sure where the error is coming from, especially since it happens inside of seek().

For now, I can just call mailbox.add() with msg.as_string(), rather than with msg directly, and it’ll work fine.  At some point, if I have time, I’ll revisit this and try and figure out if it’s a Python bug that should be reported.

Forced Materialism

Fewer people than normal (is there a “normal” after only three?) asked what to get Samuel for his birthday this year – perhaps they all asked Olyvia and she didn’t mention it, or perhaps people know him better, or perhaps it’s just easier to buy for a three-year-old.  My stock response (which I tried, fairly successfully, to get Olyvia to use as well) was as follows (this didn’t apply to parents or grandparents):

Sam has so many toys that he just doesn’t have time to play with everything.  He has toys in his toy box that he hasn’t even played with yet.  He really doesn’t need any more at the moment.  He also has heaps of books and clothes.  If you want to buy him a present, then you are, of course, welcome to do so.  In that case, our suggestions are:

* Make him something.  He’s a bit of a fussy eater these days, unfortunately, so food might not be a good choice.  If you can think of something else to make, then that would fine.
* Something to do.  e.g. an IOU for a trip with you to somewhere. This could be as simple as a picnic at a park somewhere – he loves being outside.
* A donation to a charity and a card (he likes opening mail) saying what you did.  He’s not going to understand it now, but when he’s older, he’ll hopefully think that was really cool, and right now he already has so much stuff.

My hope was really for the latter – we actually do quite a few things already, so there isn’t a pressing need for the middle choice.  Making him something is great, but it seemed unlikely to appeal to many givers.  We did actually try this once before – suggesting a donation to a charity (Starship, I think) as a Christening present.  If I recall correctly, we didn’t get a single bite that time. Continue reading “Forced Materialism” →

D520 – Week Five

Chapter 5 of IronPython in Action deals with XML, although it starts out covering some of the more advanced things you can do with functions.  I considered skipping this chapter (the function material is perhaps a bit advanced, and covering XML isn’t a necessity), but decided that it was worth learning about XML in .NET (since it’s so common) and that it would make using the MultiDoc example tricky (since the file format is XML) and I really wanted to use MultiDoc.  I gave the students notes [PDF], again covering the textbook material that we could look at, the tools (unchanged), key points, and a link to the MultiDoc example code.  We had a new lab exercise [PDF] this week (implementing Conway’s Game of Life), as well as two new recommended reading articles: a Lukas Mathis post about preferences and another Spolsky post, this one about “architecture astronauts”. Continue reading “D520 – Week Five” →