GitHub “sync fork” silently failing

(Post theme: Oops! I Did It Again by Britney Spears)

Posting because I’ve hit this three times now and each time not remembered the solution until I figured it out again, in the hopes that writing it down will prompt my memory next time and avoid a fourth “figure it out fresh” cycle.

At work (in the team I’m on), we develop in the open using public Github repositories. To introduce a change, we fork the repo, create a branch, and submit pull requests to the upstream main branch.

As changes make their way into main, it’s handy to be branching off an up-to-date fork, and GitHub has a convenient “sync fork” tool in the web UI to do this. Most of the time, this works perfectly. Unfortunately, a few times I’ve tried to sync and it silently fails (the cause is user error, but the silent nature of the fail is very poor UX from GitHub, and I feel like they could actually make the whole process work, which would be ideal UX).

The reason it has failed (each time it has happened to me) is that branch protection is enabled for the main branch on the fork. I’ve enabled this to avoid accidentally pushing changes to main (forgetting to create a branch first), which I never want to do – except for pulling in upstream changes.

There might be a simpler way to work around this (other than opening a PR from upstream main to the fork main) but I just disable the protection, sync, then enable it again.

Writing as a metaphor for coding

At work, our book club is reading Code Complete, and had our first discussion session today. I first read this in early 2011 – roughly 6 years into professional software development – and remember thinking quite positively of it at the time. We’re only three chapters in, but I’m decidedly more mixed on it this time around – I’ve had a lot more varied experience with development projects than in those early years, and a lot of the book lands quite differently (I do suspect that the later chapters will have stood the test of time better).

The second chapter of the book is focused on the importance of choosing a metaphor for software development – it ends strongly suggesting that building construction is an elegant metaphor. I agree that metaphor can be useful (but is also easily overused, and people get fixated with the imperfection of the chosen metaphor – “But if you extend them too far or in the wrong direction, they’ll mislead you”, as the author says) and that construction can work as one for software development.

However, the author, says:

For an individual’s work or for small-scale projects, the letter-writing metaphor works adequately, but for other purposes it leaves the party early—it doesn’t describe software development fully or adequately.
Steve McConnell, Code Complete, Chapter 2 “Software Penmanship: Writing Code”

This struck me as deeply ironic. Writing is an excellent metaphor for software development, and the author of a book – especially a large technical one – should be extremely aware of this.

The author seems to only consider the analogy of the metaphor to be of “writing a casual letter”, rather than writing in general. This is unfair to the metaphor and not applied to that of construction (which covers all sizes, not just flat-pack assembly). Writing ranges from a succinct social media post to a collection of textbooks or connected story that ranges over dozens over books by multiple authors (Wild Cards, Cosmere, Marvel, Riftwar, to name a few). This is hardly unique to writing – construction, farming, and other metaphors the chapter explore all range in sizes.

Writing is usually a one-person activity, whereas a software project will most likely involve many people with many different responsibilities.
Steve McConnell, Code Complete, Chapter 2

This quote was particularly interesting, as one of my colleagues brought it up in our discussion as an example of how McConnell was correct about writing being a poor metaphor, whereas I read it as an example of how wrong he was.

The “many people with many different responsibilities” absolutely applies to (non letter!) writing – you have editors, alpha and beta readers, illustrators, publicists, and many more. A lot of these work well in the software development metaphor – early readers are code reviewers, illustrators are designers, publicists are, well, publicists, and so on. This is the aspect that I was focused on when I read it.

My colleague, however, saw the truth in the “one-person activity”. It’s true that a lot of writing is done by one person. There are a lot of examples where this is not the case (co-authoring, anthologies, shared universes, ghostwriting, a lot of journalism, etc), but it’s probably safe to say that there’s less cowriting (particularly of the same block of text, however you define “block”) in code than other writing. I think where we differ is that I see this as one area where the metaphor is weak, but the rest is so strong that it outweighs that.

I also do suspect that that there’s an awful lot of software that is written by a single person. This isn’t just sole-developer products, but also components of large, multi-person, systems where there’s a single person that does nearly all the writing and maintenance of that one component (or module or library or micro-service or …). This isn’t good software development, but in my experience it is, unfortunately, fairly common. Often there are other people involved, but only lightly, in the same way that an editor or sub-editor might be with other writing.

In writing, a high premium is placed on originality. In software construction, trying to create truly original work is often less effective than focusing on the reuse of design ideas, code, and test cases from previous projects.
Steve McConnell, Code Complete, Chapter 2

I’m not sure that originality is that much of a premium in letter writing – but then I never did much of that. More broadly, originality has a “premium” in some forms of writing, but not in others – people are generally looking for the same broad style in a new romance novel, they find it easiest to read news articles that are structured in the same way, the latest Reacher novel is not that different from the previous one (or the last Will Robie, or the last Alex Cross, and so on)¹.

There’s a concept in software development that you get three “innovation tokens”. Similarly, when writing a novel or series of novels, you can have a truly original character, or world, or plot, or theme, but you almost always need to have the majority of the work be fairly familiar to the reader (particularly a reader who is familiar with the genre). Just as with software development, you can go wild with innovation and originality, but it’s high risk and makes it harder for readers/users.

When you finish writing a letter, you stuff it into an envelope and mail it. You can’t change it anymore, and for all intents and purposes it’s complete.
Steve McConnell, Code Complete, Chapter 2

This is true of a letter, but untrue of most modern writing. Pushing the irony more, this is a quote from the second edition of Code Complete. Writing that ends up in print often is in a final form – but there’s also a lot of software out there that just runs without any updates.

To be fair to McConnell, 30 years ago there was a lot less revision of writing – no editing Wikipedia, pushing out ebook updates, rapidly revised news articles, or tweaked blog posts. On the other hand, there was also a lot of “finalised” software back then too, where you’d buy a box with a disk or cartridge and, unless you did buy an updated version, you’d have that same version forever.

But extending the metaphor of “writing” software to a plan to throw one away is poor advice for software development, where a major system already costs as much as a 10-story office building or an ocean liner.
Steve McConnell, Code Complete, Chapter 2

I’m very much in favour of a reasonable level of well-executed planning and design, but beginning a software project without being willing to throw away any dead ends would strike me as a huge red flag. Defining the role of a software architect is complicated (see also this great Charity Majors post) but I maintain that the core task of an architect is working with people to minimise the number of decisions that are regretted in the future. That might be a wonderful design where almost nothing is thrown away. It might also be settings things up so that throwing things away is inexpensive and experimentation leads to market success.

I’d argue that with the really huge projects, we have plenty of examples (Novopay, INCIS, and others just in Aotearoa New Zealand) that show that big upfront planning with a reluctance to throw anything away leads to terrible cost overruns and missed deadlines.

Building software implies various stages of planning, preparation, and execution that vary in kind and degree depending on what’s being built.
Steve McConnell, Code Complete, Chapter 2

This is exactly what writing is like. Just like when writing a small “throw-away” software script, almost no planning goes into a social media post. When writing a novel, there’s a huge amount of planning, preparation, and execution (here’s a series going into it from one author – there are many, many, others – books on how to write books are almost as common as podcasts about making podcasts!). I find it difficult to believe that this was not the case for writing Code Complete itself.

McConnell extends that argument to compare the cost of making a mistake on a small project with one on a large project (ignoring any learning that may have come from making the mistake) – this equally applies in writing. If I end up rewriting a section of a blog post, that’s fairly trivial. If I have to restructure the entire third act of a novel, that’s a bigger issue, and if I have already finished two books in a trilogy and need to make changes to them to make the final volume complete, that’s very expensive to fix.

In building a house, you won’t try to build things you can buy already built. You’ll buy a washer and dryer, dishwasher, refrigerator, and freezer. Unless you’re a mechanical wizard, you won’t consider building them yourself. You’ll also buy prefabricated cabinets, counters, windows, doors, and bathroom fixtures. If you’re building a software system, you’ll do the same thing. You’ll make extensive use of high-level language features rather than writing your own operating-system-level code.
Steve McConnell, Code Complete, Chapter 2

When writing a novel, you don’t start from first principles. You’ll probably have a three-act structure, your story probably fits – at least somewhat – into one of seven basic plots, and you’ll build on top of whatever tropes are appropriate to the world you’re writing about. You don’t need to explain enemies-to-lovers or the misdirects in a mystery story, and you can call your Hobbits “halflings” and most readers will immediately have the right characteristics to mind. Just as in McConnell’s example of luxury housing having some of these fittings custom-made, the highest quality novels often deviate from the norms in a few specific areas (an imaginative magic system, a innovative take on local history, an unusual prediction about the future, and so on).

Finally, the construction analogy provides insight into extremely large software projects. … [Builders] build in margins of safety; it’s better to pay 10 percent more for stronger material than to have a skyscraper fall over. A great deal of attention is paid to timing. When the Empire State Building was built, each delivery truck had a 15-minute margin in which to make its delivery. If a truck wasn’t in place at the right time, the whole project was delayed.
Steve McConnell, Code Complete, Chapter 2

Cost and timing are important in extremely large writing projects too. Delays in editing or proofing or illustration or printing or many other things can delay an entire book launch (the Potter books have apparently reached nearly US$8B in sales, the Empire State Building cost under US$1B to build, adjusted for inflation). When a book is very high profile, it’s worth spending extra to have additional early readers and editors, just as you build with stronger material (obviously an unsuccessful book is less significant than a collapsing building).

Overall, I do agree with some of the core takeaways of the second chapter – that there’s value in metaphor, that they are “a little sloppy”, and that you should use the best (mix) of metaphors that work in the context. I just can’t agree that construction is a better metaphor for writing code than writing (non-code) is.

This chapter might have hit the sweet spot for me – the combination of reading/writing and software development – but I’m hopeful that the upcoming ones will also be as thought-provoking and enjoyable to discuss with my colleagues. If you don’t have a work book club, I highly recommend starting one! (Non-work book clubs are great too!).

To be clear: I really enjoy a good romance novel and all of the book series listed there. The familiarity has the same comfort as a good TV procedural or film sequel. ↩︎

Python return styles

I was reading through the charmed tech ops code today and noticed a bunch of code like:

def f():
    if check:
        return val
    else:
        return None

I don’t particularly love this (although I do feel you should match the style of an existing code base and should avoid changes that don’t meet a reasonable level of value-add). I would generally leave out the else (and dedent the second return). It seemed likely that these generate the same opcodes so this is purely about readability, but it was worth checking that:

>>> def f1(a):
...   if a == 1:
...     return 'A'
...   else:
...     return None
... 
>>> def f2(a):
...   if a == 1:
...     return 'A'
...   return None
... 
>>> def f3(a):
...   return 'A' if a == 1 else None
... 
>>> def f4(a):
...   if a == 1:
...     return 'A'
... 
>>> dis.dis(f1)
  1           0 RESUME                   0
  2           2 LOAD_FAST                0 (a)
              4 LOAD_CONST               1 (1)
              6 COMPARE_OP               2 (==)
             12 POP_JUMP_FORWARD_IF_FALSE     2 (to 18)
  3          14 LOAD_CONST               2 ('A')
             16 RETURN_VALUE
  5     >>   18 LOAD_CONST               0 (None)
             20 RETURN_VALUE
>>> dis.dis(f2)
  1           0 RESUME                   0
  2           2 LOAD_FAST                0 (a)
              4 LOAD_CONST               1 (1)
              6 COMPARE_OP               2 (==)
             12 POP_JUMP_FORWARD_IF_FALSE     2 (to 18)
  3          14 LOAD_CONST               2 ('A')
             16 RETURN_VALUE
  4     >>   18 LOAD_CONST               0 (None)
             20 RETURN_VALUE
>>> dis.dis(f3)
  1           0 RESUME                   0
  2           2 LOAD_FAST                0 (a)
              4 LOAD_CONST               1 (1)
              6 COMPARE_OP               2 (==)
             12 POP_JUMP_FORWARD_IF_FALSE     2 (to 18)
             14 LOAD_CONST               2 ('A')
             16 JUMP_FORWARD             1 (to 20)
        >>   18 LOAD_CONST               0 (None)
        >>   20 RETURN_VALUE
>>> dis.dis(f4)
  1           0 RESUME                   0
  2           2 LOAD_FAST                0 (a)
              4 LOAD_CONST               1 (1)
              6 COMPARE_OP               2 (==)
             12 POP_JUMP_FORWARD_IF_FALSE     2 (to 18)
  3          14 LOAD_CONST               2 ('A')
             16 RETURN_VALUE
  2     >>   18 LOAD_CONST               0 (None)
             20 RETURN_VALUE
>>>

Interestingly, there is one difference out of those four variants, where instead of a second RETURN_VALUE op, there’s a JUMP_FORWARD to the single RETURN_VALUE. I assume that means that it’s slower in a way that’s so imperceptible that it’s not worth caring about, but let’s check:

>>> timeit.Timer("f(random.random())", setup="import random\ndef f(x): 'a' if x > 0.5 else 'b'").timeit(10000000)
1.4031157046556473
>>> timeit.Timer("f(random.random())", setup="import random\ndef f(x):\n if x > 0.5:\n  return 'a'\n else:\n  return 'b'").timeit(10000000)
1.326032117009163

I feel this does land in the expected place, where it’s about style & readability. I suppose f1() explicitly shows that it’s making a choice between two cases, but I feel that’s countered by f2() and f3() showing that the function returns a value. In general, it’s nice to have the smallest amount of indentation, so I would not choose to use f1().

Between f2() and f3(), I would generally choose the single-line approach of f3() if the line didn’t wrap (or exceed whatever the wrapping length is for the project, e.g. 100 characters). I would not do this:

def f3_alt():
    return (
        really_really_really_really_really_really_really_long_val
        if another_very_very_very_very_long_condition
        else also_quite_long_here_and_now_we_wrap
    )

If you’re doing this, then there’s no benefit over the f2() style (and, as it turns out, there is a very tiny performance penalty).

Python has an implicit return None at the end of a method that doesn’t otherwise return. This means that for this specific case, where one of the values is None, there’s also the choice between f2() and f4(). In reality, f4() would be slightly better than it is here, in that it would probably have a return type (with |None) and a docstring that explains that None is a possible return value. However, I still (weakly) prefer f2() where it’s called out explicitly rather than f4() where you need to know that this is what Python does.

Kiwi PyCon 2013

Kiwi PyCon 2013 was held last weekend at AUT in Auckland. This was the first time I’ve been able to attend; although I’ve planned to in previous years, the timing never worked out (even this year was very tight).

I missed Michael’s keynote, unfortunately, and also the first session, because it’s also the end of the winter sports season, and so I had to attend prizegiving that morning. If the conference had been somewhere else, I probably couldn’t have attended at all, but since it was only ~65km from where I live, I could still attend most of the two days.

The first talk I attended was Integrating Javascript with Python Web Frameworks. This is an interesting topic, and was well presented, but seemed more novice than intermediate to me. Unfortunately, most of my interest in this topic is more on the side of light Javascript integration (i.e. where the bulk of the work is still done on the Python side), and the talk focused on the other area. However, that wasn’t a flaw in the talk, just a limit in what I got out of it (it seems likely that by the time that I ever do something that’s Javascript-heavy, if I ever do, everything will have changed considerably by then).

Next up was Testing and OpenStack. I’m only vaguely familiar with OpenStack, and the abstract is very poor, but I don’t like changing tracks mid-session, and testing is of interest, so this was the talk to attend. Again the presentation was fine, but the talk seemed more novice and less intermediate (some familiarity with OpenStack was assumed, but I don’t consider that means that the talk itself was at an intermediate level). I didn’t really take much away from this.

The final talk in the session was Python and Creative Data Analysis. This was a novice talk, and for someone at novice level, probably well worth attending (it was presented quite well). For someone not a novice, even someone with nearly no matplotlib experience (like me), there wasn’t much to be gained.

For the final session of the day, I started with Dissecting Memory Mysteries of Python. I had a lot of hope for this presentation, because this is an area of interest for me, and practical advice would be really useful for work that I do. It was labelled as an intermediate talk, and I would say that was accurate (but I was probably hoping for more of an experienced level talk). The presentation was ok, but suffered from having two speakers, one of whom spoke too quietly. The talk did cover some of how memory management works in Python, but concentrated too much on reference-cycles for my taste (which I find in practice is never the problem when short on memory), and their practical suggestions for improving memory usage weren’t really that good. For someone with no previous understanding of how Python manages memory, this would probably have been a good talk.

The final talk of the day, Using Cython for distributed-multiprocess steganographic md5sum-collision generation. For… reasons, was the best of the conference. The speaker was excellent – far superior to most of those at the conference, and the talk was entertaining and somewhat informative (mostly entertaining, but that’s exactly what it claimed to be, and those are often the best talks at a conference: pure information is better found elsewhere). The only negative was that the abstract was pretty poor – it doesn’t really reflect what the talk was about at all. If the conference had 5 talks of this quality, I would have no hesitation towards attending again.

The second (or third, if you count the tutorials) day was opened with a keynote by Jessica McKellar. Jessica is a very good speaker, and the topic was suitable for a keynote, especially from someone with her background. Essentially, she was advocating increasing involvement with and usage of Python, and going over what she saw as the key barriers to entry at the moment, and therefore what should be targeted in order to help Python remain popular (her brief explanation of why popular is important was well done). Although I certainly agree with her central tenet (everyone needs to do a couple of small things and that’s enough to make a significant difference, and that difference is important), I don’t agree with all of the examples she gave. Her four points were Windows, games, mobile, and science (the latter being an example of something that is working well; i.e. where we can copy ideas from and remain strong).

I started with Python on Windows, although I almost never use it any more. There are some difficulties, although I think that if you’re the sort of person for whom programming is a good match, you can work through the issues without much difficulty – the challenge would be more with ‘casual’ programmers – e.g. a scientist that could use Python to do something, but isn’t really a developer. Most of the challenges here I think can be addressed by documentation – e.g. the Python Tools for Visual Studio are excellent, fairly simple to use, and familiar to a lot of Windows developers. All that they need is more people pointed towards them. I don’t know enough about the casual programmer to know how they get started, but it seems like better documentation would help them too. However, the core of my disagreement here is that I’m convinced that Windows use is imploding – this is going to happen more rapidly than people realise, and so putting a lot of effort in here is not a good use of resources. It would be far better to target mobile instead.

Although Python isn’t necessarily the first choice one would make for games, I think the examples that Jessica gave were really just a re-hash of the Windows issues, and those will also just go away as fewer games are developed on/for Windows, in favour of mobile devices and possibly consoles. I would argue here that again the resources would be better spent on making sure that developing games on the web, iOS, and Android is as straightforward and well-supported as possible.

Her last two points, mobile, and science, I do generally agree with, although I think she undersold the mobile issue. For example, there are many people that will only be using a mobile device – they need to be able to actually develop on iOS or Android (or maybe Chrome OS), not just build for it. There are some tools for this – some that are really pretty good, but more would be better.

The first proper talk of the day was Exploring Science on Twitter with IPython Notebook and Python Pandas. Again, this was labelled as intermediate, but really was novice. It was an ok introduction to IPython Notebook in particular, but not a lot more than that. For a novice, or someone that had never heard of IPython or Pandas, probably valuable. For others, not so much.

The second talk of the session was From nothing to daily use: How Python changed my work life. This was correctly labelled as novice, but was by another excellent speaker. There wasn’t much to take away in terms of Python development, but it was a great insight into non-programmers (sort-of) coming into Python, and how Python is in use at a central New Zealand organisation. Again, if there were five talks of this quality, then PyCon NZ would be an easy recommendation.

The final talk of the session was Advanced methods for creating decorators. This was correctly labelled as experienced (one of only two at the conference), and was exactly what it claimed to be. The speaker read the presentation, which is never great (I’m not sure what I got from it that I wouldn’t get from reading an essay), but otherwise it was informative and well done. More talks like this, especially if they were better presented, would be excellent.

After lunch I attended Publishing Theology with Python, again labelled as intermediate but actually novice. For a novice talk, it was fine. I was hoping for more practicalities, and more focus on getting Word documents to ReST (when actually that step is done by a raw copy and paste and manual markup), but it was moderately interesting.

Prior to afternoon tea were the lightning talks (around 12, I think). These were generally very low quality – there were a couple that were worth listening to, but many that I would rather have skipped, and some where the speaker was both not good at speaking, and didn’t really appear to have anything to say. Too many people used it as a “share what you are doing” opportunity, rather than something that would be of interest to the group.

The final session of the day started with Computational Advertising, Billions of records, and AWS – Lessons Learned. This was listed as experienced, but was really intermediate (at best). A poor abstract (just throwing together a lot of words, really), but an ok talk. Either more practicalities (exactly how were things done), or being much more general would have improved this, rather than the middle ground that was taken.

The last session of the conference was the third-best: Testing for Graceful Failure with Vaurien and Marteau (actually Loads). This was accurately intermediate (although on the novice side rather than the experienced side), and very well presented. The two tools sound interesting, and definitely something that I’ll be checking out (these are the only two tools that I learned about at the conference that sound worth investigating). It would be great to see more talks like this.

There’s not much to say about the rest of the conference – it was fairly typical (venue, food, etc – although the swag-bag was very light). There were a lot of giveaways (t-shirts, books, licenses, etc) that took a lot of time and were a bit odd. I don’t really understand why people would applaud someone who was just randomly selected to get some free stuff, or why the giveaways needed to be drawn out so much. In general the organisers did a good job, though – it’s not really their fault that the quality of talks (and speakers) wasn’t as high as it could be (unless they turned down the good talks, which is unlikely). It’s possible that I missed some great talks – I’ll be watching the videos once they are available (most of them, if not all).

Two issues that should be addressed for next year are the “audience level”. Of the talks I saw, there was only one that was actually experienced, four that were actually intermediate, and six that were actually novice. Many of the novice talks (and one of the intermediates) claimed to be higher. The first problem is that the level need to be accurate, so that people can go into a talk with a realistic expectation of what’s required. The second problem is that this skews far too far towards novices – there needs to be more experienced talks in particular, and more truly intermediate ones as well.

Overall, I wasn’t particularly impressed. Since the conference was close to home, it only cost me around $400 (travel, registration, food), which is very cheap as far as conferences go, but could instead pay for a new iPad Mini (which I’d use for much more than a weekend), or several books, or licenses, or various other things where I think I would get better value. If I was looking for work (or workers), or for collaborators on an open-source project, then I can absolutely see value. However, I’m not doing any of those right now. If the conference had more experienced-level talks, more inspiring talks, and more great speakers, then I would absolutely attend again. If I was a novice, or low-level ‘intermediate’, then I would certainly see value. The venue for 2014 wasn’t announced (Dunedin was two years ago, and they tend to alternate, so I would guess that Christchurch and Wellington are the two most likely, although Hamilton might be a contender), but as things are I’m not really sold on attending again, unless the speaker line-up looks particularly good, or something else changes (e.g. if I am looking for workers).

An option that I would be interested in, but I doubt will be done, is a “green” option for attending. I would pay a discounted registration (90%?) to get access to live streaming video of the presentations and a posted t-shirt (or even the whole conference bag). I can still interact with other attendees in real time (via Twitter, ADN, etc – questions can also be handled this way), if not to quite the same level, get much the same out of the conference, and save a lot of the cost (e.g. even this year with it so close to where I live, I would still save 75% of the overall cost), and be much more environmentally friendly (i.e. reducing wasteful travel).

Textastic

Textastic is a programmers editor for iOS. It’s not perfect, but it’s very close to the app that I was waiting for ever since the first iPad was released.

Even before I had an iPad, I’d mostly stopped using my laptop. The only time I did use it was when travelling overnight (I’d take it if I had to work while away, or in case there was an emergency and I had to work on something). I don’t need a full computer for this – typing on a software keyboard on a 9.7 inch screen is perfectly adequate for emergency work, and when combined with a bluetooth keyboard (which is small enough to easily pack, and I already own) and a monitor (now a TV via AirPort mirroring) it’s quite useable as a work-while-travelling device.

The only missing component was the software: for a very long time there was nothing available. There are oodles of word processing applications, note takes, and there are even a few HTML editors – but nothing designed for real programming work. I’m not sure why this is, but I suspect that most professional programmers already have something along the lines of a small MacBook Pro or Air and find it convenient to travel with that, so wouldn’t use an iPad, even in these situations. The best available was, unfortunately, using SSH (this was before Prompt, too) and doing the word in a command line on a server somewhere.

At $14, Textastic is expensive for an iOS app, but cheap for a programmers tool (it’s definitely worth the price). This is a text editor, not an IDE, so you don’t get a debugger or interactive prompt or anything like that – it would be difficult to do that anyway given Apple’s restrictions, and I prefer a dedicated programmers text editor to an IDE anyway. (You can easily switch to other apps that together provide most of what an IDE does).

It’s a good editor – it does syntax highlighting for many languages (including Python and everything else I’m interested in), shows line numbers, and you can switch the file’s encoding and line ending choice. You can quickly jump to “symbols” (e.g. method), although I generally just scroll since I know where I’m going. There’s a decent find (and replace), and you can choose from DejaVu Sans Mono, Courier New, Luxi Mono, Monofur or Inconsolata for the font. There are a variety of colour themes, there’s control over tab settings (including soft tabs) and word-wrap. Although you can set the font size, you can also just pinch & zoom to get the size that you’re after, which is far superior to manually picking a size.

Generally the app does what it should: it gets out of the way and lets you get on with writing/editing the code. There’s an additional top row in the keyboard that offers symbols that you’re likely to need (this oddly scrolls horizontally to fit everything in) so that you don’t have to toggle between keyboards often (except for numbers) if you’re not using an external keyboard.

Where the app falls down for me is where iOS in general performs badly: moving data between applications. You can share files to the app (e.g. from Dropbox), and there are actually many ways of getting files in and out (via WebDAV, iTunes, creating local files), but you’re left with the same issue that lots of iOS apps have: you end up with duplicates. To its credit, the app tries to address this (e.g. it can know that a file you’re editing belongs in a particular Dropbox location and sync the changes back), but at the end of the day it’s clearly a hack. The app is still useable, but this does mean that I more often read code in it than write it (it’s far better for reading code than the Dropbox app is).

It seems obvious to me how to fix this problem (I don’t understand why it isn’t already the case). Surely nearly every programmer that would use this app has their code stored in some sort of repository (CVS, SVN, Git, Mercurial, Bazaar, etc). The repository is the file system. The app has a working copy checked out, and you can commit changes back to the repository. The app doesn’t need to support every action that the VCS/DVCS offers – really just (in SVN terms) add/remove/update/checkout (and perhaps propset) – if you need to do something else, then you use a dedicated SVN/Git/etc app). This completely solves the issue of multiple copies of files, because that’s exactly what these tools are designed to handle.

Unfortunately, the app would need to support at least SVN, Git, Mercurial, and Bazaar since there’s a lot of diversity at the moment. However, it seems likely that there are libraries that can be used (even in iOS) that would handle a lot of the grunt work. If the app let me check out a SVN working copy, make changes, and then commit them back, it would be absolutely perfect (I’d pay ten times the price if the app supported this).

If you need a programmers text editor on iOS, then I highly recommend Textastic.

Prompt

Panic’s Prompt is the SSH iOS application that I (and I’m sure many others) were waiting for from the release of the first iPad. My only complaint is that it took over a year (a year of using considerably inferior alternatives) before the app was finally available.

SSH isn’t pretty, but somehow Prompt is. Although there’s a lot of space required for the keyboard, connections, and settings, Prompt somehow makes the remote display quite large enough to be readable (and if you use an external keyboard, then it’s perfectly sized, and you get the control keys and arrow keys working as you’d expect). Connection management is very simple, and all the settings you’d expect (initial command, prompt string, etc) are available.

The keyboard includes an additional top row that has the keys that you need to use most frequently in SSH (escape, control, tab, /, -, |, @, arrow keys) and are missing from the standard iOS keyboard. It even autocompletes using the shell history, saving typing long paths and commands repeatedly.

There’s very little I can say about the app other than if you will ever need/want to SSH from your iPad, then this is, without any doubt, the application to have. It’s $11, but worth easily five times that much; I’d grumble and probably try and convince work to contribute to it, but I’d pay $100 for it. If you don’t know what SSH is, then this is not the app for you.

(Before Prompt, I used iSSH which also does VNC – I use VNC a lot less frequently, but still need it occasionally, so now I need to figure out which is the best VNC app, which are unfortunately all fairly pricey. If you have suggestions, let me know!)

PyPad

PyPad is a Python interpreter for iOS. This sounds incredibly exciting, right – finally I can do proper development and run Python programs on iOS! Unfortunately, that’s not really the case (mostly due to Apple’s restrictions).

PyPad lets you create multiple modules and execute each of them. However, only a subset of the standard library is available, and there’s nothing iOS-specific available (so you can’t access the camera, or touch information, and so on). Getting code in and out of the app is done via copy and paste. The standard keyboard is provided, with start/pause/stop buttons.

I keep the app installed so that I can (via AirPlay mirroring) demonstrate simple Python snippets. However, if I have an Internet connection available, then I can do that in Prompt (ssh’d to a server that has Python installed) much more elegantly.

The app is clearly limited by Apple’s restrictions as to what’s acceptable for iOS. However, it does seem like it could do much more (e.g. see Codea) if more of the standard library was available (this would mean rewriting chunks, I presume) and if there were special iOS-specific modules available for accessing things specific to the device (especially for accessing touch and graphical output). It could accept .py and text files from other applications, making it easy to get code in (e.g. from Dropbox) and share files (as text) – although perhaps that crosses Apple’s boundary for what’s ok. It would be nice to include the Python documentation, too (I have a separate app for this, but it makes sense to have it in once place).

The app is only $2, so if you’ve any interest in Python on iOS, then I’d recommend buying it to have a look and to encourage more development. You probably won’t end up using it that much, however.

Possible gmail break-in

When I logged into gmail this morning, I saw the message that I dread the most – detection of an unusual access. There was a connection (two days ago) from a server in Malaysia, although it’s actually an AWS server (Amazon web services).

There are two possibilities: the good one is that this is something that I’ve previously given access to my account, accessing it via an alternate method (e.g. Backupify can access my mail to back it up, and they use AWS) so that it showed up an unusual. The bad one is that someone was using AWS to bulk-attack accounts and got in.

In favour of the good one, as far as I can tell, no email was sent – I can’t see anything amiss at all. The email account is the central lockbox for everything, of course, so it’s possible that it was just used to break into other things, or the email content was retrieved. My password (changed now, of course) was a random 8-character string of lower-case alphanumeric characters, so not particularly simple to break (although not difficult either, given sufficient resources). I never give out the password to anything that I do not completely trust, and nor do I give out access via other methods (e.g. oauth, openid) unless I trust those services too.

I had intended to turn on two-factor identification, but hadn’t got to it yet. I’ve done that now, for the main account at least. My password is now over 30 characters long, including upper and lower case and punctuation – I probably should have changed this a while ago too.

For now, I’m leaning towards the good possibility, so I won’t be completely resetting everything that can send a password reminder to my gmail account. I’ll be keeping an eye on things as closely as I can in the next week or so, though. If you see anything suspicious come from me, please let me know.

Trac to iCal

One project I work on uses Trac and has a custom “due date” field (it doesn’t really have milestones – updates are more granular). While this is useful, one problem is that I don’t check this Trac instance every day, and so sometimes I’ve missed deadlines because I haven’t noticed that they are due. However, I do check my calendar every day (multiple times a day). It seems like exporting this “due date” value into my calendar will help with this.

This little script creates a calendar that can be subscribed to. I run it once a day (due dates don’t change very often), and have iCal set to update once a day, so it should work fine. If it’s of use to you, use it (no restrictions). Let me know if there are things that can be improved! It requires the Python iCalendar module, which I already use for parsing public holiday data.

#! /usr/bin/env python

import sqlite3
import datetime

import icalendar

cal = icalendar.Calendar()
cal.add("prodid", "-//PROJECT NAME Trac Deadlines//trac.tonyandrewmeyer.com//")
cal.add("version", "2.0")
cal.add("method", "publish")
cal.add("x-wr-calname", "PROJECT NAME Ticket Due Dates")
cal.add("x-wr-caldesc", "Due dates for PROJECT NAME tickets")
db = sqlite3.connect("/trac_location/db/trac.db")
c = db.cursor()
c.execute("select t.id, t.owner, t.summary, c.value from ticket t, "
           "ticket_custom c where t.id=c.ticket and t.status!='closed'")
for ticket_id, owner, summary, due_date in c.fetchall():
    if not due_date:
        continue
    due_date = datetime.datetime.strptime(due_date, '%d/%m/%Y')
    due_date = datetime.date(due_date.year, due_date.month, due_date.day)
    event = icalendar.Event()
    event.add("summary", "PROJECT NAME #%s (%s): %s" % (ticket_id, owner, summary))
    event.add("dtstart", due_date)
    event.add("dtend", due_date)
    event.add("dtstamp", datetime.datetime.now())
    event.add("url", "https://trac.tonyandrewmeyer.com/ticket/%s" % ticket_id)
    event["sequence"] = datetime.datetime.now().strftime("%Y%m%d%H%M")
    event["uid"] = "ticket_due_%s@trac.tonyandrewmeyer.com" % ticket_id
    cal.add_component(event)
c.close()
db.close()
open("/var/www/subfolder/project-name-trac-due.ics", "wb").write(cal.as_string())

Displaying PDFs via the iPad

I used the iPad’s external screen output for the second time today (the first was trying out Chopper 2 with the TV as the screen and iPhone as controller) – this time not just as an experiment.

I have a set of revision worksheets (all PDFs that I inherited many years ago – I might have Word documents somewhere, but I’m not sure where). Going over the questions in class, I can simply read the question out loud (but people don’t listen well enough and can’t ‘go back’ to it), or write it on the board (slow, handwriting code is problematic when you’ve got messy writing), but ideally it’s projected. Continue reading “Displaying PDFs via the iPad” →