William Edwards, Coder

Can you quote Life of Brian?

It was a defining attribute of the kind of people we were.  It wasn’t that programmers would be made to learn it from other programmers, but rather that when programmers got together they discovered they all already knew it.

I was surprised today by a programmer who had never seen the “MongoDB is web scale” video.  In turn, he said he’d talked to a programmer just the other day who didn’t know what Skynet was.

What is the programming world coming to?

WillCity update

Previously, previously.

When run on local-host, the game runs four sims at once, in lock-step.  Three of them are test patterns, and the top left is for playing around.

This makes it very easy to try out and tweak algorithms!

The top-left layout - residential is red, industrial is pink and commercial blue - is supposed to be fairly optimal.  I want the algorithm to favour it, which you can see at the moment it does!

The bottom left has more space between the zones, and road only on one side.  The algorithm should favour growth near the road, and at the moment it does!

The bottom right is a very sub-optimal layout and I want the algorithm to punish it.  Commercial and industrial don’t like each other (although the commercial cares more than the industrial).

The early sims are quite pleasing, but after a while - just a few years since my whole timeline thing is not quite set up - things get out of whack and success breeds success and it becomes a run-away chain reaction.  I need to think about levelling and damping and so on.

I’ve been talking to the OpenCity devs and the way their algorithm works is:

1. each sim step, pick a random residential, commercial or industrial zone to re-score.  It doesn’t recompute the scores for all zones each time step.  Change is gradual.
2. in a NxN grid, see if there are any of the other two zone types
3. depending on the zone type, reward or penalise nearby other zone types
4. if the reward is positive, there’s a small chance that the zone will level up
5. if the reward is negative, there’s a small chance that the zone will level down

I have a slightly more computationally-expensive approach, although far from the simple balance in OpenCity:

1. each sim step, pick between 1 and N% of the RCI zones are re-scored
2. for each re-scoring zone,
3. perform a best-first search to determine all the other zones within reach
4. the transport cost uses one of three transport cost layers - car_and_foot, bus and train.  The best-first search can jump between these layers at bus stops and train stations
5. at this point I have a list of nearby zones for each type and how far it is away and its score and so on; a quite complicated and nuanced algorithm could use all this data.  For now, we do something rather simple:
6. average the distance*score of each nearby zone type
7. if the zone is residential, re-score = commerical_score + industrial_score + residential_score*0.1
8. if the zone is commercial, re-score = residential_score - industrial_score + commercial_score*0.1
9. and so on
10. then, if the re-score is more than the current score, there’s a 25% probability that 5% of the difference is added to the zone’s score
11. and inverse for re-scoring down
12. with some basic clamping

The kid’s computer

I now have a ‘spare’ old laptop and I’m thinking about setting it up so my oldest daughter can use it.

My daughters have no shortage of ready access to tech already:

1. an old stationary computer with Qimo Linux and no internet access
2. an old eee pc netbook with Doudou Linux and no internet access
3. an ipad 2 they borrow all the time, and are very adept at using - especially spotify, youtube and a hundred make-up games
4. an intimate knowledge of DVD players and all that kind of thing

Remember my tech-savvy generation?  That was a year ago now!  Considering that they can’t read nor write English yet I am often flabbergasted at their proficiency.

I think its time to give them a computer with internet access and perhaps start trying to send them down a programming path too.

My eldest daughter, at 7, is very inventive and is currently planning a flying car.  For real.  Of course I’m sure she understands its just fun to plan, but plan she does.  Its full of little innovations like a bin between the seats so you can throw away sandwiches and things you don’t like and then the motor itself will be driven by rubbish.  This is after a discussion at school about recycling, of course.  You can tell I’m a very proud dad :)  We’ve just spent a week collecting pictures of double-decker buses and things getting stuck under bridges in an attempt to determine how high they can be…

She is already giving me lots of advice on my city building game, and has recently petitioned for bridges and tunnels.  Its really exciting to listen to her describe the cities we could build…

I want to foster this curiosity by leading her towards programming too.  I think of programming as lego without gravity, and want her to build wonderful virtual worlds.  Its not that I imagine there’s a career ahead, but rather that I want her to have a boundless world to explore and conquer, rather like books provided our generations with.

So the meat of my question:

1. what OS or such do I put on this laptop?
2. how do I set it up for safe browsing etc?
3. are there any Incredible-Machine -type games these days?
4. what kind of programming environments should I start her on?

My first mac

If I could have a cookie for every person who’s asked me how I’m getting along with my new works macbook, I’d be feeling very queasy right now!  Its like they are taking bets on how soon I burn and build the laptop with Xubuntu!

Its a 15” retina quad-i7 16GB RAM SSD thingy.  The only down on the hardware that I can spot is its got integrated Intel HD 4000 graphics.  So maybe my dreams of playing with GPGPU are dashed?

I’ve never used a mac before so I’m really struggling with the keyboard.  I had to ask around the mac users in the office for various key incantations to do things like tildes and curly brackets.  Even the setting up of my email account in the first-boot wizard was mired by not knowing to select that I had a ‘pro’ keyboard, and consequently not having any @ key!

I haven’t really worked out why mac is different.  I can’t see how having all these extra ctrl-but-different keys is useful.  Sure it might be historic but its hardly as intuitive as I imagined Apple products to be.  I regularly have to ask people where keys are, because they aren’t marked and there’s so many ctrl-alike modifiers to press to find things!  Infuriating.  I keep failing to cut-n-paste stuff, I haven’t yet found the backslash and how do I get right-click menus?  Other new mac users in the office - many of us got macs in the latest round of retreading - are equally frustrated.

I’ve had to give Apple my address and let them associate that with my email address; I just didn’t have the energy to deny them it really, with the whole needing an account for the app store and stuff.

I’ve had to pretty quickly find out how to install stuff outside the app store and get homebrew, git, virtualenv and eclipse installed.  The whole dragging things to the applications folder seems annoyingly manual really too.

Something that irritated me in the app store was showing me updates that depend upon me updating OSX itself and therefore not applicable.  If there’s a dependency order, the installer ought to be clever enough to cope with it!

And the size of the installs and updates is staggering!  XCode is several GB!  What on earth is in it?  iPhoto update was hundreds of MB - or was it over a GB, I forget? - and that was an update.  What happened to deltas?  Apple really could stop wasting bandwidth, and stop wasting user’s time!

I had to find out how to enable webGL in Safari which was disappointing.  My Javascript game prints web worker ‘ticks’ per second and is basically 2x faster in Chrome than Safari on the same machine.  I see a similar 2x or more factor comparing Chrome to Firefox too.

There’s lots of puzzling little things like that after you close windows the app bar at the top is still that app, and the traffic-light window controls on the left jar with the fullscreen arrow on the right.  And what is with that fullscreen arrow thing on the right?  Why do some apps not have it and why is it not maximise?  Truth is, I find the windows-style window controls and control-bar in the window much more objectively better.  I know there’s a history thing, but I think it better (in the same way I find Unity annoying I guess).  The OSX window decoration really feels like its accreted with duplication and confusion and inconsistency everywhere.

There’s just so many ways to move between windows, the docker tracks apps not top-level-windows and the list of pain points goes on and on.

Its not polished like iOS.  Its a cluttered mess really.

So am I going to keep OSX?  I don’t think I’m going to burn-n-build with Xubuntu just yet…  I have to master this keyboard, find the backslash key, and bypass the whole iCloud thing.  But on my terms, I can use OSX and try and learn to like it…

Tech recruiters are a dirty business

Just like estate agents, salesman and middle-men everywhere, you get a dirty, used feeling when working with them.

I have brushed with a handful of professional, scrupulous head hunters, but they are the rare exception.  If you know a recruiter who seems genuine, just appreciate how lucky you are and think how you’d describe the rest…

I have heard a first-hand account of a drunk recruiter at a party spilling the beans and giggling about how they ask their candidates to tell them all about other opportunities the candidate is pursuing, playing the lets-be-fair card to get the candidate to open up.  And what does the recruiter do with such information?  They call those other companies up to try and place another candidate and thus earn a commission!

The most unscrupulous of them seem to flourish most, so their behaviour is generally rewarded.

Got to vent, just read some of the most naive advice ever.

What I’m working on

My blog has been rather quiet recently… due mainly to renovation projects and that my hobby coding time - an hour here, a quarter there - has been on a special project:

I’ve been trying to work out a lot of the concepts behind city-building games.  Here’s my hobby simulator so far:

Is Javascript fast enough to write 3D games yet?

There’s a lot of hype about Javascript performance recently.

Mozilla’s asm.js comes to within spitting distance of C performance in recent benchmarks.

And Unreal 3 has been ported to Javascript using emscripten+asm.js.

So is Javascript fast enough to write 3D games in yet?

I’ve lots of empirically-tuned opinion on this!  I’ve been playing with webGL and Javascript (and NaCL) for a while now.

I put Javascript+webGL, running in the browser of a high-end machine with monster graphics card, at about Doom 3 level.

All the tiny inefficiencies - from the bounds checking of index arrays, from the struggle of deferred rendering, from the cost of draw batches and so on add up.

The number of times requestAnimationFrame is called and you emit your draw-ops and the number of times the window actually updates are completely disconnected, because your browser streams the draw-ops asynchronously to the GPU and skips whole frames as necessary!

You can’t profile anything; half the times firebug stops to debug an error its on completely the wrong line, half the times there are problems loading something because of a syntax error in a file the line of the file that loaded that file is shown, and web-worker performance is inscrutable.

There’s a really big disconnect in performance between OS and browser too.  My tight inner loops computing city simulations go at over 3x the speed on Chrome, for example, and my webGL scenes often render at a slightly higher rate in Angle on win32 than with real OpenGL on Linux!  Go figure.

The biggest problem is that everything is so variable.  I have an onscreen FPS ticker that puts a vertical bar for each draw call, so you can clearly see the very variable callback interval even when drawing the exact same scene again and again.  Sometimes there are half-second or more inexplicable pauses.  The whole user-experience is horrible.

Maybe on the very fastest desktop gaming rigs all these performance problems are inconsequential but on anything mid-range or below webGL really isn’t going to replace native code any time for a year or two more.

In a couple of years mainstream laptops and high-end tablets will be able to run Doom3 games in Javascript+webGL and the whole distribution advantages start to kick in.  But not yet :(

Pfft went the programmer’s dream

TokuDB is now open-source.  It is basically so fast it propels MySQL to the forefront of DB performance.  Everyone go use it.

But, sadly, nobody will pay for it.

With it open-sourcing so dies a little programmer dream we’ve all got.  Its basically not possible to make a successful closed-source product company any more.  Perhaps its not possible to make a successful software company any more.

We’ve all dreamed of making some complicated yet essential program that we can enjoy making and make a nice million on making, right?  The idea that being a software wizard can make your fortune.  The idea that the future was ours for the taking… that’d we’d all be our own little Bill Gates.

Go buy some support.

barebones.js updated

It now includes a simple terrain demo.  Hopefully its an ideal skeleton for turning into your RTS game!  Just add units…

I will soon be blogging about how I did the terrain and the navigation of it and things like working out the tile under the mouse cursor.  Stay tuned.

Browser fire and move

Google have announced they are no longer using WebKit.  They will fork WebKit, clean out the bits they don’t use, and move forward calling it Blink.

WebKit was the KHTML renderer adapted for Apple’s Safari browser.  Apple couldn’t really get the KHTML developers to take monolithic patches and ended up forking KHTML and calling it WebKit in about 2001.  There was a bit of kissing and making up but Apple had asserted its authority.

Google’s Chrome browser used WebKit from the get-go but used its own Javascript VM - V8 - and a multi-process architecture.  They have also made exciting strides in practical native code sandboxing and plugin architectures and I’ve been a big fan of NaCL even if my use and hope for it wanes.  But that crucially enables Flash.

Apple responded by making multi-process extensions called WebKit2.  But really its generally said to be far inferior to the Chrome approach.

The key thing is Apple vs Google for innovation and differentiation.

As Chrome is now the biggest browser - over 50% share on its own - Apple is going to fall behind.  Its not really a technical fight; Apple aren’t going to grow their 4% share by out-innovating Google.  In the last few weeks I’ve been looking at various car-selling websites using an iPad and most of them just don’t work.  Real bricks+mortar companies don’t rush out to make their websites compatible with new walled-garden tech like the iPad’s Safari browser no-matter how obvious nerds may think surfing for a new car on-line with a tablet would be.

Prediction: the Android browser also moves to Blink, or is simply replaced by Chrome proper.

Of course Opera only the other day announced they would adopt WebKit!  I wondered where this left them, but luckily Opera have now said they’ll follow Google.  Good, don’t want Opera to disappear completely.

hand-written asm.js

asm.js leaves me divided.  I agree to a large extent with this old V8 coder.

I care a lot about pure Javascript performance.  A recent project had me making a web worker that had to do lots of path finding.  And no matter how fast your priority queue implementation is, it isn’t fast enough.

I’ve been basing my priority queue on this one.  The hottest function is the scoring function; its not a big function - function(a,b) { return a[0] < b[0]; } - but its called a lot.

In my problem domain the maximum size of the queue is bounded by the size of the board and the objects I want to prioritise can be expressed as a list of integers and the prioritisation can be based on the comparison of the first element in each slot.

So you could imagine an asm.js priority queue that used an Int32Array view (tidier if you could use an Int16Array in my case but coercion to ints is a 32bit thing) of its heap and exposed push and pop methods.

But the problem became how the push and pop signatures would work and how to generalise it.  The functions have to take numbers and return numbers.  I ended up imagining having to reserve some part of the heap for passing in and out the values and writing a non-asm.js wrapper that keeps the heap and the priority queue code internally and does the wrapping unwrapping in non-asm.js code at non-asm.js speed.

Altogether I wish that asm.js wasn’t so extraordinarily narrow.  I want it to read and write objects, and be more of a Cython-for-Javascript.

And then I want actually for a tool that highlights your type-determined-local-variable code that the Javascript runtime can make optimise, so I can move more and more code in the fast path into this implicitly-but-statically-typed subset.  And this is tooling, and doesn’t need asm.js.

Being told the |0 and such tricks is just frustrating.  Its saying that if you write code thats hard and verbose for humans to understand and correct, the compiler has it easy.  Its pushing for me to write my stuff in something I can emscripten into an unreadable tangle of Javascript, rather than writing fast Javascript by hand.

Javascript has lots of issues.  From a performance aspect it has the albatross of dynamic typing and late binding meaning lots of checks and hash tables (although lack of threads can help make things better than Python blah blah).  Asm.js isn’t really addressing any of this.

I want colour-coding in Firebug so I can see profiling information in the sourcecode rather than in profiler reports and I want coloured annotations so I can see all the inferred properties - constness, data types, cacheability etc - of my code and strive to optimise as humans can.

I want tools and a generic Javascript runtime that can infer what asm.js wants the coder to specify.

ChromeOS is lovely

Just because I was curious, I installed ChromeOS on my Ubuntu box.  You can run the whole OS as though it was just another app!  (Instructions)

Ok I admit I wanted to poke around and think through web-intents-kind-of-things and content embedding ideas that never die.

I’ve recently turned on both a new iPad and a new iPhone for my wife and consider them great first-time user experiences.  If anything, I’d push for less choices and more automaticness in the Apple first-time wizard, though.  And so it was the Apple experience I had in mind when I turned on ChromeOS.

(After leaving Ubuntu because of Unity and more recently going through the new phone experience, blog followers will know that I get easily frustrated at clutter and confusion.)

And ChromeOS is none of this!  ChromeOS is lovely.

The initial login phase was gorgeous.  The wizard was super-simple and only wanted my Google account.  And it coped admirably with the two-channel authentication I have enabled on my Google account (take that, Samsung Galaxy S3 bug!  But so did cyanogenmod, so clearly at least Google test this stuff even if Samsung don’t).

Now what I have is ChromiumOS which is the open-source and not-bundled-with-hardware version of ChromeOS, so I don’t know exactly how the experience differs. 

The whole UX is a great example of the new trend in Flat Design (great article).

Read More

giving.code: Learning Spaces

The Learning Spaces project has a very pretty prototype online here.

SSL is broken. For real.

Remember the BEAST and other attacks that got everyone using the RC4 stream cipher instead?

Well, RC4 is now broken too.

China can read dissident emails etc.  End game.

At some point TLS 1.1/1.2 might come along.  One day.  But you needed it yesterday.