Work on Tony Andrew Meyer

GitHub "sync fork" silently failing

Sun, 03 Mar 2024 10:40:26 +1200

(Post theme: Oops! I Did It Again by Britney Spears)

Posting because I’ve hit this three times now and each time not remembered the solution until I figured it out again, in the hopes that writing it down will prompt my memory next time and avoid a fourth “figure it out fresh” cycle.

At work (in the team I’m on), we develop in the open using public Github repositories. To introduce a change, we fork the repo, create a branch, and submit pull requests to the upstream main branch.

Writing as a metaphor for coding

Sun, 25 Feb 2024 23:59:25 +1200

At work, our book club is reading Code Complete, and had our first discussion session today. I first read this in early 2011 - roughly 6 years into professional software development - and remember thinking quite positively of it at the time. We’re only three chapters in, but I’m decidedly more mixed on it this time around - I’ve had a lot more varied experience with development projects than in those early years, and a lot of the book lands quite differently (I do suspect that the later chapters will have stood the test of time better).

Python return styles

Tue, 19 Sep 2023 18:15:20 +1200

I was reading through the charmed tech ops code today and noticed a bunch of code like:

def f():
 if check:
 return val
 else:
 return None

I don’t particularly love this (although I do feel you should match the style of an existing code base and should avoid changes that don’t meet a reasonable level of value-add). I would generally leave out the else (and dedent the second return). It seemed likely that these generate the same opcodes so this is purely about readability, but it was worth checking that:

Kiwi PyCon 2013

Mon, 09 Sep 2013 11:21:21 +1200

Kiwi PyCon 2013 was held last weekend at AUT in Auckland. This was the first time I’ve been able to attend; although I’ve planned to in previous years, the timing never worked out (even this year was very tight). I missed Michael’s keynote, unfortunately, and also the first session, because it’s also the end of the winter sports season, and so I had to attend prizegiving that morning. If the conference had been somewhere else, I probably couldn’t have attended at all, but since it was only ~65km from where I live, I could still attend most of the two days. The first talk I attended was Integrating Javascript with Python Web Frameworks. This is an interesting topic, and was well presented, but seemed more novice than intermediate to me. Unfortunately, most of my interest in this topic is more on the side of light Javascript integration (i.e. where the bulk of the work is still done on the Python side), and the talk focused on the other area. However, that wasn’t a flaw in the talk, just a limit in what I got out of it (it seems likely that by the time that I ever do something that’s Javascript-heavy, if I ever do, everything will have changed considerably by then). Next up was Testing and OpenStack. I’m only vaguely familiar with OpenStack, and the abstract is very poor, but I don’t like changing tracks mid-session, and testing is of interest, so this was the talk to attend. Again the presentation was fine, but the talk seemed more novice and less intermediate (some familiarity with OpenStack was assumed, but I don’t consider that means that the talk itself was at an intermediate level). I didn’t really take much away from this. The final talk in the session was Python and Creative Data Analysis. This was a novice talk, and for someone at novice level, probably well worth attending (it was presented quite well). For someone not a novice, even someone with nearly no matplotlib experience (like me), there wasn’t much to be gained. For the final session of the day, I started with Dissecting Memory Mysteries of Python. I had a lot of hope for this presentation, because this is an area of interest for me, and practical advice would be really useful for work that I do. It was labelled as an intermediate talk, and I would say that was accurate (but I was probably hoping for more of an experienced level talk). The presentation was ok, but suffered from having two speakers, one of whom spoke too quietly. The talk did cover some of how memory management works in Python, but concentrated too much on reference-cycles for my taste (which I find in practice is never the problem when short on memory), and their practical suggestions for improving memory usage weren’t really that good. For someone with no previous understanding of how Python manages memory, this would probably have been a good talk. The final talk of the day, Using Cython for distributed-multiprocess steganographic md5sum-collision generation. For… reasons, was the best of the conference. The speaker was excellent - far superior to most of those at the conference, and the talk was entertaining and somewhat informative (mostly entertaining, but that’s exactly what it claimed to be, and those are often the best talks at a conference: pure information is better found elsewhere). The only negative was that the abstract was pretty poor - it doesn’t really reflect what the talk was about at all. If the conference had 5 talks of this quality, I would have no hesitation towards attending again. The second (or third, if you count the tutorials) day was opened with a keynote by Jessica McKellar. Jessica is a very good speaker, and the topic was suitable for a keynote, especially from someone with her background. Essentially, she was advocating increasing involvement with and usage of Python, and going over what she saw as the key barriers to entry at the moment, and therefore what should be targeted in order to help Python remain popular (her brief explanation of why popular is important was well done). Although I certainly agree with her central tenet (everyone needs to do a couple of small things and that’s enough to make a significant difference, and that difference is important), I don’t agree with all of the examples she gave. Her four points were Windows, games, mobile, and science (the latter being an example of something that is working well; i.e. where we can copy ideas from and remain strong). I started with Python on Windows, although I almost never use it any more. There are some difficulties, although I think that if you’re the sort of person for whom programming is a good match, you can work through the issues without much difficulty - the challenge would be more with ‘casual’ programmers - e.g. a scientist that could use Python to do something, but isn’t really a developer. Most of the challenges here I think can be addressed by documentation - e.g. the Python Tools for Visual Studio are excellent, fairly simple to use, and familiar to a lot of Windows developers. All that they need is more people pointed towards them. I don’t know enough about the casual programmer to know how they get started, but it seems like better documentation would help them too. However, the core of my disagreement here is that I’m convinced that Windows use is imploding - this is going to happen more rapidly than people realise, and so putting a lot of effort in here is not a good use of resources. It would be far better to target mobile instead. Although Python isn’t necessarily the first choice one would make for games, I think the examples that Jessica gave were really just a re-hash of the Windows issues, and those will also just go away as fewer games are developed on/for Windows, in favour of mobile devices and possibly consoles. I would argue here that again the resources would be better spent on making sure that developing games on the web, iOS, and Android is as straightforward and well-supported as possible. Her last two points, mobile, and science, I do generally agree with, although I think she undersold the mobile issue. For example, there are many people that will only be using a mobile device - they need to be able to actually develop on iOS or Android (or maybe Chrome OS), not just build for it. There are some tools for this - some that are really pretty good, but more would be better. The first proper talk of the day was Exploring Science on Twitter with IPython Notebook and Python Pandas. Again, this was labelled as intermediate, but really was novice. It was an ok introduction to IPython Notebook in particular, but not a lot more than that. For a novice, or someone that had never heard of IPython or Pandas, probably valuable. For others, not so much. The second talk of the session was From nothing to daily use: How Python changed my work life. This was correctly labelled as novice, but was by another excellent speaker. There wasn’t much to take away in terms of Python development, but it was a great insight into non-programmers (sort-of) coming into Python, and how Python is in use at a central New Zealand organisation. Again, if there were five talks of this quality, then PyCon NZ would be an easy recommendation. The final talk of the session was Advanced methods for creating decorators. This was correctly labelled as experienced (one of only two at the conference), and was exactly what it claimed to be. The speaker read the presentation, which is never great (I’m not sure what I got from it that I wouldn’t get from reading an essay), but otherwise it was informative and well done. More talks like this, especially if they were better presented, would be excellent. After lunch I attended Publishing Theology with Python, again labelled as intermediate but actually novice. For a novice talk, it was fine. I was hoping for more practicalities, and more focus on getting Word documents to ReST (when actually that step is done by a raw copy and paste and manual markup), but it was moderately interesting. Prior to afternoon tea were the lightning talks (around 12, I think). These were generally very low quality - there were a couple that were worth listening to, but many that I would rather have skipped, and some where the speaker was both not good at speaking, and didn’t really appear to have anything to say. Too many people used it as a “share what you are doing” opportunity, rather than something that would be of interest to the group. The final session of the day started with Computational Advertising, Billions of records, and AWS - Lessons Learned. This was listed as experienced, but was really intermediate (at best). A poor abstract (just throwing together a lot of words, really), but an ok talk. Either more practicalities (exactly how were things done), or being much more general would have improved this, rather than the middle ground that was taken. The last session of the conference was the third-best: Testing for Graceful Failure with Vaurien and Marteau (actually Loads). This was accurately intermediate (although on the novice side rather than the experienced side), and very well presented. The two tools sound interesting, and definitely something that I’ll be checking out (these are the only two tools that I learned about at the conference that sound worth investigating). It would be great to see more talks like this. There’s not much to say about the rest of the conference - it was fairly typical (venue, food, etc - although the swag-bag was very light). There were a lot of giveaways (t-shirts, books, licenses, etc) that took a lot of time and were a bit odd. I don’t really understand why people would applaud someone who was just randomly selected to get some free stuff, or why the giveaways needed to be drawn out so much. In general the organisers did a good job, though - it’s not really their fault that the quality of talks (and speakers) wasn’t as high as it could be (unless they turned down the good talks, which is unlikely). It’s possible that I missed some great talks - I’ll be watching the videos once they are available (most of them, if not all). Two issues that should be addressed for next year are the “audience level”. Of the talks I saw, there was only one that was actually experienced, four that were actually intermediate, and six that were actually novice. Many of the novice talks (and one of the intermediates) claimed to be higher. The first problem is that the level need to be accurate, so that people can go into a talk with a realistic expectation of what’s required. The second problem is that this skews far too far towards novices - there needs to be more experienced talks in particular, and more truly intermediate ones as well. Overall, I wasn’t particularly impressed. Since the conference was close to home, it only cost me around $400 (travel, registration, food), which is very cheap as far as conferences go, but could instead pay for a new iPad Mini (which I’d use for much more than a weekend), or several books, or licenses, or various other things where I think I would get better value. If I was looking for work (or workers), or for collaborators on an open-source project, then I can absolutely see value. However, I’m not doing any of those right now. If the conference had more experienced-level talks, more inspiring talks, and more great speakers, then I would absolutely attend again. If I was a novice, or low-level ‘intermediate’, then I would certainly see value. The venue for 2014 wasn’t announced (Dunedin was two years ago, and they tend to alternate, so I would guess that Christchurch and Wellington are the two most likely, although Hamilton might be a contender), but as things are I’m not really sold on attending again, unless the speaker line-up looks particularly good, or something else changes (e.g. if I am looking for workers). An option that I would be interested in, but I doubt will be done, is a “green” option for attending. I would pay a discounted registration (90%?) to get access to live streaming video of the presentations and a posted t-shirt (or even the whole conference bag). I can still interact with other attendees in real time (via Twitter, ADN, etc - questions can also be handled this way), if not to quite the same level, get much the same out of the conference, and save a lot of the cost (e.g. even this year with it so close to where I live, I would still save 75% of the overall cost), and be much more environmentally friendly (i.e. reducing wasteful travel).

Textastic

Tue, 15 Nov 2011 20:02:44 +1200

Textastic is a programmers editor for iOS. It’s not perfect, but it’s very close to the app that I was waiting for ever since the first iPad was released. Even before I had an iPad, I’d mostly stopped using my laptop. The only time I did use it was when travelling overnight (I’d take it if I had to work while away, or in case there was an emergency and I had to work on something). I don’t need a full computer for this - typing on a software keyboard on a 9.7 inch screen is perfectly adequate for emergency work, and when combined with a bluetooth keyboard (which is small enough to easily pack, and I already own) and a monitor (now a TV via AirPort mirroring) it’s quite useable as a work-while-travelling device. The only missing component was the software: for a very long time there was nothing available. There are oodles of word processing applications, note takes, and there are even a few HTML editors - but nothing designed for real programming work. I’m not sure why this is, but I suspect that most professional programmers already have something along the lines of a small MacBook Pro or Air and find it convenient to travel with that, so wouldn’t use an iPad, even in these situations. The best available was, unfortunately, using SSH (this was before Prompt, too) and doing the word in a command line on a server somewhere. At $14, Textastic is expensive for an iOS app, but cheap for a programmers tool (it’s definitely worth the price). This is a text editor, not an IDE, so you don’t get a debugger or interactive prompt or anything like that - it would be difficult to do that anyway given Apple’s restrictions, and I prefer a dedicated programmers text editor to an IDE anyway. (You can easily switch to other apps that together provide most of what an IDE does). It’s a good editor - it does syntax highlighting for many languages (including Python and everything else I’m interested in), shows line numbers, and you can switch the file’s encoding and line ending choice. You can quickly jump to “symbols” (e.g. method), although I generally just scroll since I know where I’m going. There’s a decent find (and replace), and you can choose from DejaVu Sans Mono, Courier New, Luxi Mono, Monofur or Inconsolata for the font. There are a variety of colour themes, there’s control over tab settings (including soft tabs) and word-wrap. Although you can set the font size, you can also just pinch & zoom to get the size that you’re after, which is far superior to manually picking a size. Generally the app does what it should: it gets out of the way and lets you get on with writing/editing the code. There’s an additional top row in the keyboard that offers symbols that you’re likely to need (this oddly scrolls horizontally to fit everything in) so that you don’t have to toggle between keyboards often (except for numbers) if you’re not using an external keyboard. Where the app falls down for me is where iOS in general performs badly: moving data between applications. You can share files to the app (e.g. from Dropbox), and there are actually many ways of getting files in and out (via WebDAV, iTunes, creating local files), but you’re left with the same issue that lots of iOS apps have: you end up with duplicates. To its credit, the app tries to address this (e.g. it can know that a file you’re editing belongs in a particular Dropbox location and sync the changes back), but at the end of the day it’s clearly a hack. The app is still useable, but this does mean that I more often read code in it than write it (it’s far better for reading code than the Dropbox app is). It seems obvious to me how to fix this problem (I don’t understand why it isn’t already the case). Surely nearly every programmer that would use this app has their code stored in some sort of repository (CVS, SVN, Git, Mercurial, Bazaar, etc). The repository is the file system. The app has a working copy checked out, and you can commit changes back to the repository. The app doesn’t need to support every action that the VCS/DVCS offers - really just (in SVN terms) add/remove/update/checkout (and perhaps propset) - if you need to do something else, then you use a dedicated SVN/Git/etc app). This completely solves the issue of multiple copies of files, because that’s exactly what these tools are designed to handle. Unfortunately, the app would need to support at least SVN, Git, Mercurial, and Bazaar since there’s a lot of diversity at the moment. However, it seems likely that there are libraries that can be used (even in iOS) that would handle a lot of the grunt work. If the app let me check out a SVN working copy, make changes, and then commit them back, it would be absolutely perfect (I’d pay ten times the price if the app supported this). If you need a programmers text editor on iOS, then I highly recommend Textastic.

Prompt

Mon, 14 Nov 2011 19:53:19 +1200

Panic’s Prompt is the SSH iOS application that I (and I’m sure many others) were waiting for from the release of the first iPad. My only complaint is that it took over a year (a year of using considerably inferior alternatives) before the app was finally available. SSH isn’t pretty, but somehow Prompt is. Although there’s a lot of space required for the keyboard, connections, and settings, Prompt somehow makes the remote display quite large enough to be readable (and if you use an external keyboard, then it’s perfectly sized, and you get the control keys and arrow keys working as you’d expect). Connection management is very simple, and all the settings you’d expect (initial command, prompt string, etc) are available. The keyboard includes an additional top row that has the keys that you need to use most frequently in SSH (escape, control, tab, /, -, |, @, arrow keys) and are missing from the standard iOS keyboard. It even autocompletes using the shell history, saving typing long paths and commands repeatedly. There’s very little I can say about the app other than if you will ever need/want to SSH from your iPad, then this is, without any doubt, the application to have. It’s $11, but worth easily five times that much; I’d grumble and probably try and convince work to contribute to it, but I’d pay $100 for it. If you don’t know what SSH is, then this is not the app for you. (Before Prompt, I used iSSH which also does VNC - I use VNC a lot less frequently, but still need it occasionally, so now I need to figure out which is the best VNC app, which are unfortunately all fairly pricey. If you have suggestions, let me know!)

PyPad

Fri, 11 Nov 2011 19:22:15 +1200

PyPad is a Python interpreter for iOS. This sounds incredibly exciting, right - finally I can do proper development and run Python programs on iOS! Unfortunately, that’s not really the case (mostly due to Apple’s restrictions). PyPad lets you create multiple modules and execute each of them. However, only a subset of the standard library is available, and there’s nothing iOS-specific available (so you can’t access the camera, or touch information, and so on). Getting code in and out of the app is done via copy and paste. The standard keyboard is provided, with start/pause/stop buttons. I keep the app installed so that I can (via AirPlay mirroring) demonstrate simple Python snippets. However, if I have an Internet connection available, then I can do that in Prompt (ssh’d to a server that has Python installed) much more elegantly. The app is clearly limited by Apple’s restrictions as to what’s acceptable for iOS. However, it does seem like it could do much more (e.g. see Codea) if more of the standard library was available (this would mean rewriting chunks, I presume) and if there were special iOS-specific modules available for accessing things specific to the device (especially for accessing touch and graphical output). It could accept .py and text files from other applications, making it easy to get code in (e.g. from Dropbox) and share files (as text) - although perhaps that crosses Apple’s boundary for what’s ok. It would be nice to include the Python documentation, too (I have a separate app for this, but it makes sense to have it in once place). The app is only $2, so if you’ve any interest in Python on iOS, then I’d recommend buying it to have a look and to encourage more development. You probably won’t end up using it that much, however.

Possible gmail break-in

Tue, 15 Mar 2011 08:09:49 +1200

When I logged into gmail this morning, I saw the message that I dread the most - detection of an unusual access. There was a connection (two days ago) from a server in Malaysia, although it’s actually an AWS server (Amazon web services). There are two possibilities: the good one is that this is something that I’ve previously given access to my account, accessing it via an alternate method (e.g. Backupify can access my mail to back it up, and they use AWS) so that it showed up an unusual. The bad one is that someone was using AWS to bulk-attack accounts and got in. In favour of the good one, as far as I can tell, no email was sent - I can’t see anything amiss at all. The email account is the central lockbox for everything, of course, so it’s possible that it was just used to break into other things, or the email content was retrieved. My password (changed now, of course) was a random 8-character string of lower-case alphanumeric characters, so not particularly simple to break (although not difficult either, given sufficient resources). I never give out the password to anything that I do not completely trust, and nor do I give out access via other methods (e.g. oauth, openid) unless I trust those services too. I had intended to turn on two-factor identification, but hadn’t got to it yet. I’ve done that now, for the main account at least. My password is now over 30 characters long, including upper and lower case and punctuation - I probably should have changed this a while ago too. For now, I’m leaning towards the good possibility, so I won’t be completely resetting everything that can send a password reminder to my gmail account. I’ll be keeping an eye on things as closely as I can in the next week or so, though. If you see anything suspicious come from me, please let me know.

Trac to iCal

Thu, 13 Jan 2011 12:05:38 +1200

One project I work on uses Trac and has a custom “due date” field (it doesn’t really have milestones - updates are more granular). While this is useful, one problem is that I don’t check this Trac instance every day, and so sometimes I’ve missed deadlines because I haven’t noticed that they are due. However, I do check my calendar every day (multiple times a day). It seems like exporting this “due date” value into my calendar will help with this. This little script creates a calendar that can be subscribed to. I run it once a day (due dates don’t change very often), and have iCal set to update once a day, so it should work fine. If it’s of use to you, use it (no restrictions). Let me know if there are things that can be improved! It requires the Python iCalendar module, which I already use for parsing public holiday data.

Displaying PDFs via the iPad

Tue, 24 Aug 2010 12:57:40 +1200

I used the iPad’s external screen output for the second time today (the first was trying out Chopper 2 with the TV as the screen and iPhone as controller) - this time not just as an experiment. I have a set of revision worksheets (all PDFs that I inherited many years ago - I might have Word documents somewhere, but I’m not sure where). Going over the questions in class, I can simply read the question out loud (but people don’t listen well enough and can’t ‘go back’ to it), or write it on the board (slow, handwriting code is problematic when you’ve got messy writing), but ideally it’s projected. The classroom does have a projector - in previous years I’ve hooked up my aging laptop and put the PDFs up to see. My hope was that I could use the iPad this year (since it’s meant to be replacing the laptop in most circumstances). The first problem is that I don’t have an application that shows PDFs that will use the external screen (e.g. iBooks ignores the external display). This is pretty annoying, and I hope it gets changed with an update in the near future. It’s not hard to imagine a business meeting where everyone has iPads and wanting to show a PDF that has been created (an advertisement, a manual, etc) to everyone at the meeting. Basically the only application I do have that will use the external display to show a document is Keynote. So the next step was creating a Keynote document with the same content as the PDFs. The next problem that arises is that I can’t copy the text from the PDFs on the iPad (not in iBooks, and not in Stanza, which is the only other PDF reader I have installed). I could see it perfectly well, but there was no way to extract the text. I hope this changes too. Just enabling select+copy in iBooks would be great. It’s not hard to imagine use cases here, either: select text, copy it, and tweet a quote from a document. (Ok, there are DRM issues with books, but just do something like leave out “select all” - even just enabling this for PDF would be ok). I resorted to using the laptop. I copied the PDF content to a PowerPoint (no Mac here to use OS X Keynote) document, and then (via Dropbox, since I don’t have a computer to sync with here) and opened it in (iOS) Keynote. The final steps, which did work, were very nice. I formatted the slides using Keynote (very simple, didn’t miss a mouse or keyboard at all), including inserting a few images via Safari. When the time came to use the slides, the projector accepted the iPad’s output without problems (although the position and location of the cable meant that the iPad was in an awkward location - buying the Keynote Remote application and using an iPhone/iPad Touch would have improved that). (For some reason, I couldn’t get the “laser pointer” functionality working at the time, although this has worked fine when I’ve tested before. I didn’t really need that). Overall, it’s a pretty good experience if you already have Keynote (or PowerPoint) material. It’s a pretty terrible experience if you have something else (e.g. code!) that you want to display. Thankfully, that can be addressed in software - even in third-party software.

D520 Week Two – 2010

Mon, 09 Aug 2010 18:04:16 +1200

No radical changes from either last year’s week two or last week. In a way, this is the real first week - in the previous week we learn about the course and about what IronPython is (and remember how to program in Python), but we don’t do much more than that. In the second week, we really get into doing some actual IronPython programming. I gave the students [PDF], and the [PDF], and the recommended reading was two Joel Spolsky posts: one on (IT) customer service and one on how hardware improvements impact software design. The notes are again in three sections: textbook chapters (this week is Chapter Three, a fairly essential introduction to .NET and IronPython), key points, and example code (from Chapter Three). The lab is essentially the same as in previous years. We started out by quickly going over the . There isn’t really much time to spare for this (and it ended up being about 45 minutes anyway) - ideally the students already could manage these (except perhaps the last couple), but that isn’t really the case. It felt like they were getting more up to speed after going through these - I’ll need to be careful that this segment doesn’t end up being too long, otherwise we’re essentially dropping the lecture and putting the lab back in (just in the other order and a week late), and the students will end up waiting for me rather than attempting themselves. (Although this first non-assessed lab is a slightly different case). I simplified the textbook material again, ignoring .NET structures, enumerations, delegates, and so forth: the focus was really on events and Windows Forms. From there, I hope that we’ll build in to using other aspects of the .NET framework. (The GUI choice is again complicated this year - last year it was obviously Windows Forms, but unclear on the best way to design; this year Visual Studio is the best way to design, but the IronPython integration only offers a WPF graphical form designer, and I don’t really want to completely switch to WPF. For the non-graphical design that we do at first, we’re sticking with WinForms). I think this simplification helped fit the material into a single lecture (where it felt a bit rushed last year), although part of that is probably that I’m much more used to the 4-hour format now. I stuck to the new plan (other than going over lab 0 first) where we didn’t start work on lab 1 in class at all, and the second half of the class I worked through the Windows Forms examples. It felt like there was more time for this, too - I think I did more of the examples (i.e. did them more bit-by-bit) than last year. However, I don’t think anyone is confident enough with Windows Forms to actually create a GUI quiz application for the lab - although with a different group of students I suspect that might not be true. In general, the course seems to be progressing well (although it’s early stages yet) - like last year much smoother than when transitioning from Python to Visual Basic, and better than last year in that I’m more familiar with the 4-hour block, with IronPython, and with how parts of the course work.

What happened to the 2009 IPy Notes?

Mon, 26 Jul 2010 16:51:13 +1200

Around this time last year, I started posting weekly reports on the IronPython classes I was teaching at Northtec, including copies of the material that I was giving the students. These stopped abruptly around the middle of the course. The course did complete (and was fairly successful - I think the change to IronPython was definitely for the better), and I did continue making notes as I taught. However, I didn’t manage to get them online. On the 10th of October 2009, my father passed away (very unexpectedly, of a heart attack the day before). As a result, I didn’t have as much extra time as I had anticipated, and I didn’t really have the energy/motivation to post the notes. (The classes continued, except the week of the funeral, where the hours were halved). The course began again last week (21st July 2010), and I’m going to try this again. As I go, I’ll complete the 2009 notes as well (some/all of the links in the 2009 ones are broken, which I’ll fix, and when I get to where the notes stopped, I’ll post the notes that I took last year, although these will be less exhaustive than if I had done them at the time). No promises, but I expect that I’ll be able to post throughout the entire course this year, and I’ll again make the material that I use available (you may use it under a Creative Commons license if you wish, although much of the material is references to IronPython in Action). I had planned to present a short summary of the experience of using IronPython in this way at PyCon NZ, but I’m again unable to make it this year (for happier reasons - a close friend is getting married that day); perhaps next year! I’m still interested in hearing from anyone else that’s using IronPython in the classroom.

Dr? No.

Thu, 19 Nov 2009 20:10:21 +1200

I finally withdrew from my PhD today (probably many people thought that this had happened some time ago).

In the beginning…

The story really starts in 1999. I started working consistently while studying, and also got rather bored with the study that I was doing. I also got involved in the Students’ Association - first lightly, then pretty heavily. As a result, at the end of 2000, I was about 1.5 papers short of finishing my BSc and BBS. I needed to do half a semester of work, and so I decided to go those at the same time as a Postgraduate Diploma in Science (these went along with running for ASA President). The ASA job didn’t work out, but I did finish the last undergrad stuff I needed to do, and rather unexpectedly found that I really enjoyed the postgrad study. I was able to do all the papers for the postgrad diploma along with the undergrad work, except for the double-paper research project, in that year. That meant that the next year started off with finishing off the research project - normally half of a semester’s load, but since I didn’t have anything else to be doing, I poured in a whole semester’s effort into it. That left me at the middle of the year with everything complete.

More notes coming soon

Mon, 21 Sep 2009 12:40:52 +1200

I am still putting together the weekly D520 notes. However, the last couple of weeks have been a bit busy and so they’re waiting to be cleaned up and have links added. The intra-semester break starts next week so I’ll catch up then.

D520 - Week Six

Mon, 07 Sep 2009 21:31:20 +1200

Chapter 6 of IronPython in Action covers “properties, dialogs, and Visual Studio”. This seemed an obvious place to insert the material on user-interface design that is normally covered in the course, and to look a bit more deeply than the textbook does at Visual Studio itself (and the Windows Forms controls and their properties). I only scheduled a single week to cover this, but I suspected that it might take more than one (I left an empty slot in the schedule to cover one such over-run), and that was, indeed, the case. The students received notes [PDF], slightly longer this week (covering the UI design material not in the textbook, as well as the usual chapter summary, key points, and examples, and the steps required to install IronPython support in the ‘Experimental Hive’ Visual Studio SDK), and a fairly simple lab exercise [PDF]. The recommended reading for this week (which I didn’t get a chance to mention, but is listed in the course material) is a very light-hearted comparison of different programming languages (somewhat old, so the students probably don’t actually know many of these, but I like the underlying truths - I guess I should have tried to add languages like Ruby, PHP and Javascript myself) and a Wil Shipley article about the fast/good/cheap trade-off. I really enjoy Shipley’s style of writing, but even if I didn’t, I’d include this article: the point about there being many dimensions (brevity, features, speed, time, robustness, flexibility, …) and that you can’t have them all is really important for starting developers to understand. However, even more than that, I really agree with Shipley’s contention that you should always start with brevity. The first half of the class is straight from previous offerings of the course (although a little condensed), and is basically just a 90 minute talk about user-interface design, aimed at non-designers. As previously, I use the points from Tog’s First Principles of Design as the outline, although I talk about more than just what’s in the article (and only skim over some parts of it). The course isn’t really about UI/UX design, but it is an important element, and I try to emphasise it in some way each week (and many of the recommended reading articles are about some sort of UI/UX design). For the two projects, the students will have to complete not just a development design document, but also a UI design, including an explanation of why they made the decisions they did. Historically, this is notoriously badly done for the first project, with most students (despite my emphasis when outlining the project, and giving them a marking schedule) submitting nothing more than a screenshot of a Visual Studio mock-up. For the first project, all I really need to see is that they’ve considered something, rather than just thrown things together - by the end of the course, I expect that they should be able to put something together that is a reasonable UI. There has historically been an examination question on this topic as well - something along the lines of “outline three important considerations in user-interface design”. The biggest issue was in installation - again! I carefully tested running through the installation on a clean copy of the Windows image they have. It was slow (so I recommended doing it before coming to class, and planned to kick it off at the start and only start using it a couple of hours later, after the UI talk), but worked fine. I forgot, however, about the Northtec proxy server that blocks so much Internet functionality. There were basically three steps required to get the “Experimental Hive” version of Visual Studio running with IronPython (1, unfortunately) support: install Visual Studio 2008 SP1, install Visual Studio 2008 SDK, compile the IronPython sample. The Visual Studio SP1 installer is one of the “stub” installers that are popular in some circles - i.e. it’s a very small installer that then downloads whatever is necessary to complete the installation. While there are some merits in this approach, it doesn’t work well in all situations: this was one of those. It seems that the service pack installer only support proxies that provide auto-discovery, which the Northtec proxy does not. That left two alternatives: force the students to download and install the service pack (many hundreds of MB) outside of the lab, or use the .iso version of the SP. I downloaded the .iso (830 MB!), and that installed without problem in the lab. However, I don’t really have a convenient way to distribute files of that size among the class (generally the files I provide are quite small) - if I had realised this in advance I could have had it added to the file server, but that wasn’t really feasible in the time available. I had to fall back to copying the files on to a USB drive (1GB, so max’ing it out) and distributing them via sneakernet. Figuring out all of this (around the actual teaching) took most of the class, so when I left I had only got this far (i.e. halfway through step 1 of 3). The SP installation should be trouble-free (but will probably take 30-60 minutes), so the students should have that done by next class. We’ll continue with the SDK installation (~120 MB standard installer) and IronPython compilation at that point. Next year, I’ll make sure that SP1 is installed on the image, side-stepping this whole problem (I hadn’t got to testing software installation when the images were created this year, because I knew that I wanted the students to do installation themselves). Of course, I could be really lucky, and Visual Studio 2010 might include built-in IronPython support. We were able to use Visual Studio in the method described in IronPython in Action, however (creating class libraries in Visual Studio and subclassing in IronPython). I think the students found this less appealing that I do - moving the generated .dll around was problematic for some of the students, for example. I suspect that even though using IronPython directly in Visual Studio is problematic (e.g. stuck with IronPython 1) many students will elect to do that. To me, properties seem a bit like decorators and lambdas, in that they are slightly advanced Python, and perhaps a bit ahead of where the students generally are in D520. However, we certainly covered properties in Visual Basic, and the fundamentals are the same, really. The MultiDoc example was very useful here, in that it uses properties to implement observers, so there was a practical use for properties, rather than the toy examples that are often used in explaining why you’d want to hide your attributes behind getters and setters. I think that the students understood the general idea. As part of this, I briefly covered the observer pattern, as outlined in the textbook - perhaps it’s just the work that I’ve done, but I come across observers all the time, so I tried to get across the importance of this idea. My plan at this stage is to find somewhere in the schedule where I can insert an “Advanced Python Refresher”, where the students can take another look at decorators, properties, lambdas, and anything else that comes up along these lines. I doubt many (maybe any) students will use these techniques in their projects, but I do expect that they’ll feature in the final examination - and will probably be a good indicator of the good programmers (those that really understand programming and might end up doing it professionally) versus the good students (those that put in enough effort to get through, but are really suited for other IT jobs - sysadmin, testing, managing, etc). Chapter 6 has nicely thorough coverage of the very versatile MessageBox, but it’s not really overly complicated to use, if you have a list of the icon and button options available (as in the book, or on MSDN). Since we’ve seen MessageBox a few times already, I skipped over this material, focusing instead on the custom dialogs. The chapter also has a section on serialising in .NET - the students are already familiar with the pickle module, so this is really just a .NET version of something they already know. I originally intended to cover this next week, but if time is short again, I might just skip it completely (since they can easily use pickle instead). We ran out of time before I could really go over the MultiDoc code from Chapter 6, so I left that for the next week. The lab exercise was just redesigning the Airline application used in earlier labs. There were two aims here - firstly to get the students to try out the various controls that are available (it’s easier to explore in Visual Studio, even if you end up using the controls in hand-written code). Secondly, the students must design their projects, including the UI, and submit that design. This doesn’t just mean ‘sketch the UI’ - it means consider and explain the reasons behind the UI decisions - hopefully the lab started students on the road to doing this properly in the project.

Mailbox flattening messages

Sun, 30 Aug 2009 21:35:44 +1200

The mailbox.py code does this to dump a message (e.g. when adding to a Maildir file):

def _dump_message(self, message, target, mangle_from_=False):
 # Most files are opened in binary mode to allow predictable seeking.
 # To get native line endings on disk, the user-friendly \n line endings
 # used in strings and by email.Message are translated here.
 """Dump message contents to target file."""
 if isinstance(message, email.Message.Message):
 buffer = StringIO.StringIO()
 gen = email.Generator.Generator(buffer, mangle_from_, 0)
 gen.flatten(message)
 buffer.seek(0)
 target.write(buffer.read().replace('\n', os.linesep))
 elif isinstance(message, str):
 if mangle_from_:
 message = message.replace('\nFrom ', '\n>From ')
 message = message.replace('\n', os.linesep)
 target.write(message)

(There’s a bit more that deals with other types of message). Unfortunately, with some messages containing Unicode characters, this breaks with a UnicodeDecodeError:

D520 - Week Five

Mon, 24 Aug 2009 17:57:44 +1200

Chapter 5 of IronPython in Action deals with XML, although it starts out covering some of the more advanced things you can do with functions. I considered skipping this chapter (the function material is perhaps a bit advanced, and covering XML isn’t a necessity), but decided that it was worth learning about XML in .NET (since it’s so common) and that it would make using the MultiDoc example tricky (since the file format is XML) and I really wanted to use MultiDoc. I gave the students notes [PDF], again covering the textbook material that we could look at, the tools (unchanged), key points, and a link to the MultiDoc example code. We had a new lab exercise [PDF] this week (implementing Conway’s Game of Life), as well as two new recommended reading articles: a Lukas Mathis post about preferences and another Spolsky post, this one about “architecture astronauts”. Students again read the recommended reading (again, before class!). There was some debate about the Mathis article about preferences. The points I hoped they took away were the difference between configuration and preferences and that preferences don’t suit ‘casual’ users. I think some students were unconvinced, but I hope that I managed to get across the point that they are (by definition, as students in this course) more than casual users, and so not the sort of user that preferences don’t suit. (However, even though I’m far from a casual user, I too generally dislike preferences - one reason I use a lot of Apple products). I got the feeling that the material on functions was a little advanced, as I suspected. This was, however, the ideal place for me to cover lambdas, as I proposed doing last week, so I added that in. I talked about functions defined in other functions, passing functions as arguments, anonymous functions (lambdas), and decorators. These are important things to learn if you’re going to be a Python developer (or a developer in other languages that let you treat functions in this way), but perhaps belong in a more advanced class. Next year I might skip over this (i.e. assign it as recommended reading), and just rewrite any of the textbook examples to use regular functions. On the other hand, it’s great exam question material! Last week, I stepped through the MultiDoc code, typing it in as I went. I felt there wasn’t sufficient time for that this week, so instead opened up the source that Michael provides and talked through it. I don’t feel this was as successful as the previous week, so I’ll go back to typing as much as I can. I didn’t have time to comment the code as I went, but did so after class (for the version of MultiDoc as it stands at the end of the chapter). The commented version is available under the same license as the original. I completely skipped going over xmldocumentreader.py since it’s fairly generic and well covered in the textbook, and concentrated on documentreader.py and documentwriter.py instead. Implementing Conway’s Game of Life wasn’t an exercise that was in the course in previous years (although it would be essentially the same task to do it in Visual Basic), although I have used it some time ago in other courses (and I think I had to do it myself back in my undergraduate years). It seemed like an exercise that would concentrate on Python programming in general, and just happen to use some .NET classes. Two of the classes that they would need (Timer and PictureBox) I expect they’ll want to use in their projects later on. I particularly like to cover Timer (whether it’s Visual Basic or IronPython) because the students at this point are still working in a procedural fashion, rather than an event-driven one. That means that their natural instinct when needing to do something like update the grid every X seconds is to have a loop with a time.sleep() call in it. They then end up with an unresponsive GUI, because they are blocking the main thread. A timer isn’t the only way to deal with this, but it’s a good default choice, and covering it explicitly saves me explaining it to many students individually when they come across it (e.g. in project work). However, most of the class found that they didn’t know where to start (and were either so stuck that they really had no idea, or just didn’t feel like taking a crack on this particular day). So I ended up working through the exercise with them (rather like I ended up doing the Quiz the week after that exercise). I don’t mind doing this occasionally, but the students really need to start on their own eventually. I could perhaps provide a detailed design next year, that clearly showed what functions you would use and how they would be tied together. I wrote three versions of Life in the end (all extremely similar). I wrote my first one simply to check how difficult it would be for the students, intending it to be an example answer. The second was a small modification of the first, which added a bit more colour to make the display more interesting. The third was the one I wrote during class - it’s considerably simpler (i.e. not as nice code) than the other version.

D520 - Week Four

Sat, 15 Aug 2009 16:54:19 +1200

This week continued from the previous one, covering Chapter 4 of IronPython in Action. That meant no new notes, and no new lab exercise. We basically did two things: worked through the MultiDoc example in Chapter 4, and worked on implementing the Airline lab designed in the previous week. The first recommended reading for the week was Part 1 of Joel Spolsky’s “Talk at Yale”, wherein he tries to relate his study to his career - the part I hoped they would find interesting was the discussion of “geeks” versus “suits”. The second recommended article was Steve Yegge’s “Code’s Worst Enemy”, which is mostly about code bloat. In retrospect, these might not be the best pairing, since Yegge is always long, and this particular Spolsky article is very long (if you read all three parts). However, I was again pleasantly surprised to hear that students were actually reading these. I was right to set aside a whole ’lecture’ section of a week for introducing MultiDoc - I used all two hours (with breaks and a late start, actually about 90 minutes) going over the code, and didn’t quite manage to finish - I got up to the very last section, but didn’t have time to go through adding the menubar/menu items (so I did that in the break and showed the working example afterwards).

D520 - Week Three

Sun, 09 Aug 2009 23:01:31 +1200

When planning the semester’s schedule for D520, I choose a few topics that seemed large and gave them a two-week time-slot. One of these was chapter 4 of IronPython in Action, which covers duck typing, design patterns, and introduces the MultiDoc example that’s used throughout the middle section of the book. One of the concepts that the course has always (at least, as long as I have known it) tried to push is the importance of design - not just user-interface design (although that’s both important and covered), but the importance of doing at least some planning before starting to write large amounts of code. In the last couple of years, I’ve moved the course away from focusing on extensive formal design to also cover design patterns and testing (particularly automated testing, like unit tests). Since this is such a major issue for the course, and since I planned on using MultiDoc as an example in class (I try to always have an example that continues on from week to week), this seemed like an obvious point for a two-week session. As such, the [PDF], the textbook chapter (4), and the [PDF] (somewhat) are shared over week three and week four. The notes follow the standard layout, although there’s an extra section that condenses some of the material from previous years about emphasising design; the tools remain the same as the previous week. The lab is outlined in more detail below. The recommended (but completely optional) reading was a 2005 Gamasutra post about rapid game design (many of the students are often into gaming, so I like to include this article) and one that attempts to explain why ease-of-use is often neglected by Scott Berkun (from 2002! Oldies but goodies this week). I’d noticed in the previous week that the students weren’t doing a particularly elegant job of completing the “Quiz” lab exercise, which was causing difficulties in converting the console version to a GUI version (apart from the expected difficulties of building a GUI with Windows Forms for the first time). While their programs did the task, there was a great deal of repetition - basically a print statement, a raw_input and an if, repeated for each question in the quiz (i.e. no loop, no data structure). I decided that rather than just provide an example application, I’d walk through the lab myself, showing the students how I would progress from the console version through to a GUI. I deliberately did this without any preparation (although I have done a Visual Basic version of this application in previous years, and I did do a partial GUI version in class in week two), so that the students would see how I had to go back and change things (emphasising the importance of design), and how mistakes were a natural part of developing (although thankfully I didn’t make too many - it’s always a little tricky writing code when you need to restrict yourself to writing something that a beginner can easily understand). I started with a console version that was basically the same as the ones I saw being created, then altered it so that it stored the data in a dictionary and looped through it. We then worked through changing that application into a Windows Forms application. The resulting program isn’t great (and the UI is very ugly), but it demonstrated the progression that I wanted to show. I made [zip]. I talked for a while about duck typing - I did this ‘freehand’ rather than via the material in the textbook, so that the students would have multiple perspectives to use to absorb this. I’m not sure how well they understood the importance - it’s a fundamental aspect of Python programming, but they haven’t yet come across anything significant enough that typing is important, so it’s a little abstract. I think that since they started in Python, rather than a strictly typed language like C or Java, that they don’t understand quite how lucky they are (when they get to doing templates in C++, they’ll wish they had duck typing there!). Hopefully it did sink in a little, especially for the better students, and they’ll see the importance as we work through more advanced programs. It’s also an obvious exam question! (Any of the students smart enough to have found this via Google will have figured that out already, I expect). For the section on design, I mostly talked ‘freehand’ as well, condensing material that I’ve covered in a few classes in the previous couple of years. I talked a little about the MVC style that the textbook introduces, but also talked a lot more about the high-level, overall, importance of design, how students tend to work, how real-life projects often work, and tried to convey the important of some sort of planning. I left introducing MultiDoc for the next week (that will be the main focus of the first half of the lesson). That meant that we didn’t use the textbook extensively this week - not because it was lacking in any way, but rather because it’s a large topic and I wanted to use it more in the second part. I hope that (at least some of) the students will read the chapter in the days between week three and week four (to follow through with what we covered in week three) and so will be more prepared for week four. (I’m quite looking forward to starting the MultiDoc example next week). The lab exercise is based on an exercise that has been in the course for a long time (I think it was perhaps from a textbook that was once used). In the past, the students have been given a working (although very buggy) application used to do simple seat booking for an airline. Their task has been to write a formal design document for the application, using the way that the application worked as the basis. In the following week, the students would then implement the application themselves, based on their designs. I’ve never loved this exercise - partly because the example application was so buggy (although I fixed it up somewhat), and partly because it’s doing things backwards (design after implementation), which is exactly what I’m telling them not to do. I did like the concept itself (no coding - just design) and the example (airline booking) was good enough. I threw away the ’example application’ part of the exercise, and simply gave the students a description of what the application was meant to do. It was then up to them to figure out how it should work (including an idea of what the interface would be, although I emphasised that wasn’t the focus of the task). Next week I’ll give them an example design, and they can implement the application, either based on their own design, or on my example one. I’ll probably build a GUI for them as well, so that they can concentrate more on improving their coding skills than figuring out how to make Windows Forms look nice. The students accepted this much more readily than in the past (there’s usually grumbling about the lack of coding). I’m not sure what that signifies, and I haven’t seen any submissions yet, so I’m not sure how well they did. It did seem like the designs (as in the past) were very shallow, failing to consider many aspects (data structures, starting up, shutting down, and so forth). In that, the results seem quite similar to previous years, which is pleasing (since I’m trying to keep as much continuity as possible, given the huge change from Visual Basic to IronPython).

D520 Week Two

Fri, 31 Jul 2009 11:01:39 +1200

Here’s my material from the second week of “D520: Programming” (in IronPython). The students got [PDF] and the first proper [PDF]. The recommended reading this week was a post by Lukas Mathis about poor hardware design (and lessons to be learnt), and a post by Wil Shipley about tracking down a Delicious Library bug. The notes are again in four sections: textbook chapters (this week chapter 3, which is fairly essential reading), tools (same as last week, although I also recommended IronPython 2.6b2), key points, and example code (from chapter 3 of the textbook). The lab exercise is a modification of one from last year (when it was in Visual Basic) - I’m trying to keep as many of the previous lab exercises as possible, so that there is still a tiny bit of continuity between 2008 and 2009. Thoughts on how the class went (again, mostly for me when preparing the 2010 course) - as promised, much shorter than last week (less than half the words!: